Frequently Asked Questions around Mobile Phone Surveys

For researchers around the globe, it is clear that the coronavirus outbreak will alter how business is done for years to come; Many data collection firms have paused in-person research for the safety of enumerators, but now more than ever, accurate, on-the-ground data is needed. There will be dramatic shifts in both humanitarian needs and consumer habits, and both development organizations and consumer brands must stay on top of these changes in order to deliver aid to those who need it and keep their own businesses alive. After years of being used as an experimental research method or to supplement traditional research modes, remote mobile surveys are suddenly in the spotlight.

Despite mobile-based methodologies being the safest and most effective way to gather data during a crisis such as COVID-19, there are still unknown factors when using mobile to collect data. Who can be reached, what modes are best suited to each project, and how questionnaires should be designed are just a few of the questions that come up when organizations are looking to transition projects from face-to-face methodologies to mobile surveys.

While formal research on the usage of mobile surveys is sparse, below is an overview of the research available and anecdotal evidence from GeoPoll’s 8+ years conducting remote survey work in Africa, Asia, and Latin America.

What can mobile phone surveys be used for?

Mobile surveys have been used as a tool by reputable organizations including the World Bank, The United Nations World Food Programme, Unilever, GIZ, and Insight2Impact for several years. There is some debate over if mobile surveys can yet be used to fully replace face-to-face studies, however, there is agreement that mobile surveys excel at collecting rapid data during crises. USAID, FHI360, Keystone Accountability, and others have utilized remote methodologies to identify vulnerable populations and shifting trends during crises including the 2014-2015 West Africa Ebola outbreak. Organizations such as the World Food Programme have spearheaded the usage of mobile surveys in multiple regions to gather food security data, finding that mobile was able to correctly identify trends and seasonal shifts in food security.

Additionally, there is evidence that mobile is better at gathering data on sensitive questions than in-person modes. Research GeoPoll conducted with Kantar TNS found that SMS respondents were more likely to indicate that they felt unsafe in their homes and that they have gone without food than respondents from the compared face-to-face survey.

Who can be reached with remote mobile surveys?

The mobile methodologies that are feasible during COVID-19 (CATI, SMS, IVR, and mobile web or mobile application) require respondents to have access to a mobile phone through which they will answer survey questions. While some studies, such as the World Bank’s Listening To Africa project, provide mobile phones to respondents through initial face-to-face outreach, at present organizations can only reach respondents through mobile phones they already have. This reduces the sampling frame of any given study to those with access to mobile devices, but mobile penetration in Africa and other regions is increasing each year; a 2017 study by Pew research found over 80% of the population in 6 African countries had access to some sort of mobile phone.

Studies have found that the mobile population overrepresents those who are more educated, male, and younger age groups in many countries, but it’s still possible to target those who are older or less educated – it simply may require a larger sample base to draw from to find these respondents. GeoPoll has often targeted very specific groups, from farmers of certain crops to mothers of young children, and is able to do so through careful sampling methods and screening questions. We can also create samples which are nationally representative by key demographics through a stratified random sampling approach and use of quotas to reach the desired sample size within each demographic group.

Finally, GeoPoll uses mobile-friendly Living Standard Measures questions to calculate the LSM or SEC group of respondents. GeoPoll’s recent studies on coronavirus and other studies have included respondents from the lowest SEC groups, who typically do not have running water or electricity in their homes. Certain modes may be better for targeting certain groups; CATI has been found to be better at reaching older age groups, while IVR may reach those less educated.

Who you will target also depends on the sample source; GeoPoll has access to mobile subscriber databases in over 60 countries which we can draw from, or we can gather sample through an enhanced Random Digit Dialing (RDD) process, or recruitment through online, radio, or TV advertisements. GeoPoll can also send surveys to provided lists of contacts for those looking to reach a pre-recruited group.

What mobile survey mode should I use?

Current available modes from GeoPoll include:

• SMS: Surveys conducted via a 2-way SMS chat
• CATI: Voice calls placed by trained interviewers working remotely during COVID-19
• IVR: Automated voice calls with a recording
• Mobile web: A mobile web link that is sent via SMS or another mode and opens a basic web browser
• Mobile application: Mobile applications which administer surveys with those with access to smartphones.

The ideal mode for conducting research will depend on a variety of factors, including questionnaire length, budget, and target respondent groups. It has been found that CATI and IVR are generally more expensive than SMS, however, cost varies widely by mode and is also dependent on countries studied, screening criteria, number of questions, and other factors. In order to get a true estimate of cost, you should contact a research firm such as GeoPoll who can provide price quotes by mode for your specific project. 

How should I design questionnaires for mobile?

As many remote mobile methodologies are self-administered (with the exception of CATI) and have other limitations such as character limits, questionnaires for mobile must keep the mode of survey research in mind. For example, while one study found that individual question length didn’t affect response rates, studies GeoPoll and others have done have found that longer SMS studies have lower completion rates. Additional studies have demonstrated that certain question types may work better in certain modes; for example, a GeoPoll study found that SMS select-all-that-apply questions yield fewer answers than forced-choice questions.

What other factors should I consider when conducting mobile research?

Other factors to consider when embarking on a mobile research project include:

• Incentives: Findings are mixed on the use of incentives; A study in Ghana and Tanzania found small incentives increased completion rates, but that higher incentives had similar effects to lower amounts, and other studies have found lesser effects. Some surveys may also cost respondents airtime; GeoPoll and other services often use zero-rated shortcodes to send messages so that they can be received and replied to even when respondents do not have airtime.
• Language: Many countries speak multiple languages, and surveys should be offered in more than one language – for SMS, services such as GeoPoll offer multi-language surveys, and for CATI, research providers should have multiple interviewers with different language skills. GeoPoll’s call centers are staffed with interviewers who often speak up to 6 languages, depending on the country.
• Local Context: It has also been found that the local context is important. For example, dialects and wording intent can vary throughout regions, which is why GeoPoll always uses local interviewers to conduct voice calls and makes multiple checks on other survey types before sending them out. GeoPoll has also found that in countries such as Nigeria, female interviewers have higher response rates.
• Speed: The speed at which you are looking to collect data will have an impact on the mode you choose. SMS and mobile web surveys can be sent to tens of thousands of respondents at once, while the speed at which CATI interviews are conducted depends on the number of interviewers hired. GeoPoll has also found that IVR response rates can be very low, which can slow down data collection.
• Cost: Your budget for conducting research will be a factor in determining mode, sample size, and other aspects of data collection. While there is a tendency to think lower-cost equals lower-quality, this is not always the case; For many projects, SMS and mobile web surveys are a good option for gathering quality data at lower costs than CATI.
• Data Output: Data can often be delivered in multiple formats – some research organizations will provide raw data in Excel or SPSS, and others may do data analysis for you. GeoPoll has a full-service research team who do data cleanup and analysis and can provide raw data, create reports, and build custom dashboards.

Conducting Mobile Surveys during COVID-19

Although there is still much research to be done on mobile surveys, coronavirus provides researchers an opportunity to test new methodologies that will be referred back to for many years to come. While mobile may not yet reach every person on the globe, it will within the coming decades, and so we must continue to test different methodologies in order to better understand the nuances of mobile data collection.

When conducting mobile research, we recommend using a firm like GeoPoll that has years of experience in the nuances of remote mobile methodologies. To request a quote from GeoPoll for mobile-based research, please contact us here.