Testing Survey Modes to Collect Taxpayer Data in Liberia

GeoPoll regularly works with partners looking to conduct remote mobile surveys with extremely targeted populations, and our team has become experts in the survey modes and research techniques that would be most appropriate for each project based on its specific needs. One example of this was seen in DAI’s Revenue Generation for Governance and Growth (RG3) program in Liberia, which is a development program funded by the United States Agency for International Development. This project, the first of its kind, sought to understand business taxpayers’ perception and experience filing their taxes in Liberia.

The survey targeted medium and large businesses in Liberia that paid and filed relevant business taxes. The overall goal of the survey was to better understand the perceptions and experiences that Liberia’s business taxpayers had filing and paying taxes with the Liberian Revenue Authority. Before the survey was full implemented in September 2017, GeoPoll performed a series of tests via text message (SMS) to better understand the target sample.

Selecting Survey Modes

Once GeoPoll understood the complexities and sensitivities of the study as well as the limited sample size of business taxpayers, GeoPoll initially recommended to the use of Computer Assisted Telephone Interviews (CATI) as the mode for the project. However, after a number of discussions with DAI RG3 team members, the team agreed that GeoPoll would test the survey via SMS to gain insight if response rates increased after changing key factors with implementation of the survey, despite the complex and sensitive nature of the survey’s subject matter.

Beginning in July 2017, GeoPoll tested a SMS survey with 39 questions with advanced conditional logic and routing of respondents. The sample GeoPoll utilized initially consisted of 1,756 business taxpayers in Liberia, with an additional 120 eligible respondents being added after several DAI RG3 programmatic events that invited taxpayers. The majority of these sample members had been previously contacted by GeoPoll through informative one-way messages, but were not pre-notified directly regarding the survey nor had the majority of the sample previously take a SMS survey on GeoPoll’s platform. These factors were an important characteristic of the sample, since prior engagement and pre-notification of a survey are all empirically proven to increase response rates, especially if pre-notification of a survey occurs within the same mode.

For approximately a month and a half, from July until the conclusion at the end of August, GeoPoll tested:

• The frequency of reminder messages to respondents as well as sending the initial survey invitation message different times throughout the day and different days of the week.
• Increased airtime incentive amounts from five USD to 10 USD. GeoPoll offers a small airtime incentive for all of its surveys, but this amount is normally significantly less than five USD.
• Different wording of the Opt-in messages to garner more attention from potential respondents.
• Additionally, towards the end of testing, DAI RG3 staff made direct calls to a portion of the respondents in the sample to ask them to take the survey they received via SMS.

Testing Results

The SMS testing confirmed GeoPoll’s original suggestion that SMS would not be a functional mode for a study with a very limited sample and a sensitive topic. Due to the specific targeting of this survey, GeoPoll saw much lower response rates than seen with other SMS survey projects. Although nearly 60 sample members initially engaged with the opt-in message, only four completed the survey entirely. These results allowed GeoPoll and the DAI RG3 Team to make the collective decision to switch entirely to CATI for the full implementation.

The Survey

In the final stage of the project, GeoPoll utilized a sample of approximately 3,000 medium and large business tax payers in Liberia to conduct a 41 question CATI survey, achieving 506 completed surveys. CATI is a survey method by which data is collected by an interviewer through calling phone numbers to reach respondents, asking them questions from a computerized script and recording the data electronically on a computer. CATI software is customizable and allows for researchers to control the flow of a survey based on a respondent’s answers to questions, as well as information already known about the respondent. The benefits of this mode include:

• Interviewers allow for in depth probing to complex/difficult questions.
• Interviewers are able to gauge respondent understanding of questions, and probe for clarification if needed, and ensure appropriate responses are given.
• The mode reaches populations that are not as accessible using other modes.

These benefits made CATI an ideal mode for conducting a very targeted survey on a complex and sensitive topic such as filing taxes.

Impact Assessed

An initial assessment of the data displays that business taxpayers in Liberia believe that many of filing processes are burdensome, although the majority of respondents noted that it only took one or two hours to file their respective returns. However, the majority of respondents had problems filing their business taxes due to the tax s system being down or because the wait at the Liberian Revenue Authority was too long. DAI’s RG3 program and GeoPoll will continue to analyze the data and results yielded from this unique study.