How to design, run and analyze questionnaires

A questionnaire is a fixed set of questions that every respondent answers in the same order, so the answers can be compared across hundreds or thousands of people. Use questionnaires when you already know the likely answers and need to count them. To design one that holds up, start from the decision, choose the mode, order and pilot the questions, then analyze closed and open answers separately.

When a questionnaire is the right research tool

A questionnaire is the right tool when the decision needs a number and you can already list the answers people are likely to give. Interviews are the right tool when you cannot yet write that list, or when the decision turns on the reason behind a behavior. Many enterprise decisions need both, in that order or the reverse.

What the decision needs	Best tool	Why
How many customers do X, by segment	Questionnaire	Same question for everyone, so counts compare across segments
Which of several known options people prefer	Questionnaire	A closed list or ranking gives a clear priority
Why customers do X	Interviews	The reason is not on any list yet
What options exist that the team has not thought of	Interviews	Open conversation surfaces answers no one wrote down
How many hold a view, and why they hold it	Both	The questionnaire sizes it; interviews explain it

The test is simple to run before any question is written. Draft the option list for your most important question. If the team cannot agree on the options, the study needs interviews first. This overview of qualitative and quantitative research methods sets out where each method fits.

Start from the decision, then write the objective

Every question in a questionnaire should trace back to one information need, and every information need back to one business decision. Write the decision first, then list what the team must know to make it, then write the questions. A question that serves no information need is cut, however interesting its answer might be.

Hold a cut list for stakeholder requests

In a large company, the questionnaire attracts requests. The brand team wants an awareness question, the regional lead wants two questions about a local competitor, and finance wants willingness to pay. Keep a cut list and one rule: a question stays only if a named person will act on its answer. Everything else goes on the list for a future study.

Reuse validated instruments where they fit

If a validated scale already measures what you need, such as an established satisfaction or usability measure, use it as published. A new scale written in-house has no track record, and its results cannot be compared with anyone else's.

Name your reporting cuts now

Decide now which cuts the report will show: business unit, market, customer segment, tenure. Those cuts decide which demographic and firmographic questions you need, and how many responses each group needs before its results can be reported on their own. A sample size calculator gives a first figure for each cut.

Choose how the questionnaire will be administered

There are four common modes: online self-administered, paper or mail self-administered, telephone interviewer-administered and in-person interviewer-administered. Self-administered modes remove the interviewer's influence and cost less per response. Interviewer-administered modes let someone clarify a question and probe a vague answer, at a higher cost per response.

Mode	Reach	Cost per response	Depth of answers	Risk of socially desirable answers
Online, self-administered	Wide, wherever you hold contact details	Low	Short, especially for open questions	Lower
Paper or mail, self-administered	Reaches people who are rarely online	Moderate	Short	Lower
Telephone, interviewer-administered	Wide, slower to field	Higher	Deeper, with clarification	Higher
In person, interviewer-administered	Narrow, by location	Highest	Deepest, with probing	Higher

The interviewer modes carry a known trade-off. A respondent speaking to a person may give the answer that sounds better, while a respondent alone with a form gives no reason at all unless the question asks for one. Telephone interviews allow clarification without the cost of travel. For enterprise customer studies where the company holds contact details, online self-administration is the usual starting point, with interviewer modes kept for groups that will not answer online or for questions that need probing.

Order, length and flow decide whether people finish

Order questions in a funnel. Screening questions come first, easy factual questions next, the core questions in the middle, and sensitive and demographic questions last. Keep each respondent's route as short as the decision allows, and keep the format consistent so people spend their effort on the answers.

Use branching to keep each route short

Branching, or skip logic, sends each respondent only the questions that apply to them. A merchant who has never contacted support skips the support block. A long enterprise questionnaire can serve five business units and still feel short to each person who takes it.

Keep scales consistent

Use the same number of points and the same direction for every rating scale. A respondent who learns that 5 means best on the first page will misread a scale on page three where 1 means best. Choosing between rating, ranking, open and other question types is its own topic, and the decision behind each question should drive that choice too.

Group by topic

Group questions by topic and put a one-line transition between groups, such as "The next questions are about getting paid." The transition tells respondents what the next block covers.

A worked example from a payments business

This example is illustrative. A large payments company must decide whether to change merchant onboarding in two markets. The insights team builds the questionnaire from that decision down, and the steps below show where each choice came from.

From decision to questions

The decision is whether to redesign onboarding in both markets or only one. The team lists three information needs: where in onboarding merchants stall, how satisfied recently onboarded merchants are with each step, and whether the pattern differs between the two markets. Those needs produce about nine questions: a screener on onboarding date, a stall-point question with a known list of steps, satisfaction ratings for four steps, a market question, and one open question on what nearly made the merchant give up.

The question that was cut

The product team asked for a question on interest in a new lending feature. It served none of the three information needs, and no one on the onboarding decision would act on the answer. It went on the cut list for the lending team's own study.

Mode and the open question

The questionnaire went online to merchants on the company's own list, because every merchant has an email address on file. The one open question stayed in because the steering committee wanted reasons beside the ratings.

Where it fell short

The ratings showed that one onboarding step scored lower in one market than in the other. The open answers came back short, mostly a few words each, such as "too slow" and "confusing". The team could see where the problem was and could not yet see what to change. That gap is the job of the follow-up step in the analysis section below.

Pilot it before it reaches thousands of people

A pilot finds the questions people misread before they cost you a full sample. Run it in two stages: a think-aloud pretest with a few people from the target audience, then a soft launch to a small share of the sample. Fix what each stage shows before the full launch.

Run a think-aloud pretest

Ask a few people from the target audience to read each question aloud and say what they think it is asking. Every hesitation and every reinterpretation is a question to rewrite.

Soft launch and check the data

Send the questionnaire to a small share of the sample and read the results before anyone else does. Look for drop-off at one question, identical answers down a grid, and open boxes left empty. A high completion rate on its own proves little, as this post on why completion rates do not mean better answers explains.

Get approvals before launch

Legal, data protection and brand reviews take time in a large organization. Book them before the pilot, so the version that passed the pilot is the version that gets approved.

Plan for nonresponse from the start

Plan for the people who will not answer, because in European national surveys that group grew for two decades. An analysis of 753 national probability-sample surveys in 36 European countries from 1999 to 2018, published in Quality & Quantity, found a consistent decline in survey outcome rates, regardless of country or project.

The same analysis found that neither the sampling frame nor the data collection mode moderated the decline. That leaves the parts of the design a team controls: the length of the questionnaire, its relevance to the person receiving it, and the invitation.

Before reporting, compare who responded with who you invited, on the cuts you planned at the start. Say in the report which groups are thin. Where the gap matters, weighting can adjust for it, provided you know how the invited population breaks down.

Analyze closed and open answers differently

Closed answers are counted and open answers are coded. Report every closed-answer figure with its base, meaning the number of people who answered that question, and cross-tab by the cuts you planned. Code open answers into themes, keep the quote that supports each theme, and follow up with interviews when the open answers are too thin to act on.

Closed answers

Report counts and percentages with their base beside every figure, then break them down by business unit, market or segment. A percentage from a small base needs a caution beside it, or it should not be reported on its own.

Open answers

Read the open answers, group them into themes, and keep at least one quote for each theme so a stakeholder can check it against the raw data. This guide to coding open answers into themes covers the coding step in detail.

When open answers are too thin

When the open answers stop at "too slow", the next step is a conversation that asks "slower than what?". That is the gap the payments team hit, and it is the limit this post on how fixed questionnaires cap what you can learn describes. Pulse runs surveys alongside AI-moderated voice interviews, so the same study can collect the counts and then the reasons. Its AI moderator asks a follow-up when a reply is vague, and every theme links to the transcript it came from.

Questions readers ask

Short answers to the questions people ask most about questionnaires.

What are the four types of questionnaire?

Questionnaires are usually grouped by how they are administered: online self-administered, paper or mail self-administered, telephone interviewer-administered and in-person interviewer-administered. Self-administered questionnaires are cheaper per response and free of interviewer influence. Interviewer-administered questionnaires cost more and allow clarification and probing.

When is a questionnaire the right method?

Use a questionnaire when the decision needs a number and you can already list the likely answers. Use interviews when you need reasons or cannot yet write the options. When the decision needs both, run the two together; Pulse runs AI-moderated interviews with your customers beside a survey in one study.

What information should a questionnaire collect?

Only the information a decision needs. Write the decision, list what the team must know to make it, and write questions that serve each need. Add the demographic or firmographic questions your reporting cuts require, and nothing else. A question no one will act on belongs on the cut list.

Is it spelled questionnaire or questionaire, and what is the plural?

The correct spelling is questionnaire, with a double n, and the plural is questionnaires. "Questionaire" with one n is a common misspelling. Check the spelling in the invitation and the questionnaire title before launch, since both are the first thing a respondent reads.