How to Write and Design Effective Surveys

Contents

In computer science, the phrase “garbage in, garbage out” is used to express the idea that poor quality or flawed input data will produce faulty output data. This same principle applies to market research survey design. The data collected from a survey is only as good as the writing and organization of the questions asked.

Designing surveys capable of generating quality data and actionable insights can be a complex process. In this post we will discuss the steps and strategies involved in effective survey design.

Defining the Research Objectives

The first step in survey design is to clearly establish the objectives of the survey. Researchers need to know what they want to learn before they can figure out how to learn it. The objectives should be attainable, specific (rather than general), and represent the most important takeaways the researcher hopes to receive from the study. Once the objectives are set, they will help guide what topics are included in the survey, what questions are asked, and even how questions are worded.

Choosing Survey Question Types

Before writing the survey questions, researchers first have to decide the types of questions to ask. Survey questions can be divided into two distinct categories: closed-ended questions and open-ended questions.

Closed-ended questions: In closed-ended survey questions, respondents select from a finite set of pre-defined responses. Examples of closed-ended questions include simple yes/no questions, multiple choice questions, and Likert Scale or ratings questions.
Open-ended questions: In open-ended survey questions, respondents are asked to reply to each question in their own words in a free-form text box, rather than selecting from a set list of options.

Although open-ended questions provide deep insight into the feelings and thought processes of respondents, they are not as objective or easily quantifiable as closed-ended questions. They also take more time and effort for respondents to answer which can lead to survey fatigue. With these factors in mind, most survey designs primarily consist of closed-ended questions followed by one or two open-ended questions near the end of the survey.

Writing Survey Questions

Taking the time to write clear, concise, well-worded survey questions helps to ensure reliable responses and improves response rates. Although writing well-worded questions is as much an art as a science, the list below offers general guidelines for effective question design:

Keep wording simple. Use simple words and simple sentence structure. Clear, concise wording asks respondents to do less guesswork and increases the accuracy of the resulting data.
Keep questions neutral. Avoid opinion, bias or loaded words that might lead the respondent toward a particular response.

How wonderful was your experience at our hotel? How would you rate your experience at our hotel?

Avoid double-barreled questions. Questions that ask for feedback on two different things within the same question are difficult to answer and analyze.

Do you think the president should lower taxes and raise the minimum wage? In your opinion, should the president lower taxes? Should the president raise the minimum wage?

Do not use absolutes. Terms such as “always,” “all,” “every,” etc., force respondents to agree or disagree completely without nuance or scale.

Do you always go to the gym after work? How many times per week do you go to the gym after work?

Use reference frames. Make sure respondents are all considering the same time and place when answering a question.

How often do you shop for groceries? How many times in the past week did you visit the grocery store?

Avoid negative and double-negative questions. Using negative words like “not” or “prevent” and double-negative questions confuse respondents about whether to answer “yes” or “no”.

The government should not provide health care to its citizens, yes or no? Should the government provide health care to its citizens?

Avoid jargon, technical terms and acronyms. Make sure the entire target audience understands the language level and what the question is asking.

How has upgrading to the A14 Bionic CPU impacted your processing performance? Does your iPhone 12 open apps slower, the same or faster than your iPhone 11?

Anticipate all answer choices. Make sure the answer choices are unique (do not overlap) and include all possible options. If an exhaustive list of options is not known, include an “Other-Specify” option.
Explain sensitive or unexpected questions. Provide a short explanation to justify why a potentially sensitive question is important to the research and reassure respondents that their responses will remain confidential.

Determining Survey Length and Question Sequence

Even with clear, concise, well-worded questions, respondents will eventually lose interest in or abandon surveys that are too long or disorganized.

Optimal Survey Length

To avoid survey fatigue, keep the questionnaire as short as possible. GeoPoll recommends that from start to finish, surveys take no more than 10 minutes to complete, which usually equates to a maximum of 30 questions. Open-ended questions take longer to complete than close-ended questions, allowing for fewer questions overall. By testing the survey both internally and with a pilot group of respondents, researchers can see if the drop-off rates for their survey are higher than expected and adjust before sending the survey to the full sample.

Question Sequence Guidelines

The order in which questions are asked in a survey can influence how they are answered. It can also impact completion rates. General guidelines to consider when organizing the sequence of questions in a survey include:

Similar to a natural conversation, start the survey with general impersonal questions that will be easy for the respondent to answer. This helps to warm-up respondents and get them interested and involved in the survey.
Gradually increase question complexity and specificity but avoid overwhelming respondents by asking multiple difficult questions one after the other.
Group questions by topic and in a logical sequence (again, similar to the natural flow of a conversation). Questions asked out of context can frustrate and confuse respondents.
Save more sensitive questions until later in the survey. Research suggests respondents may be more willing to answer sensitive questions after already putting in the effort to answer earlier questions, and if they are offended by a sensitive question late in the survey, it will not influence their responses to previous questions.
Conclude the survey with more general easy to answer questions such as demographics as a warm-down to leave respondents with a positive final impression of the experience.

Conduct Surveys Around the World with GeoPoll

GeoPoll has experience designing and administering surveys all over the world for clients ranging from global brands and international development organizations to local media stations and NGOs. Survey design can be a difficult task and due to its impact on the outcome of a study, GeoPoll’s research experts offer assistance at varying levels based on each client’s needs. We understand that every project is unique and are committed to using our expertise to guide our clients through key decisions to produce the most accurate insights possible. To learn more about GeoPoll’s survey development processes and various research modes, please contact us here.