Sample Frame and Sample Error

In our first blog post on sample considerations, we outlined how samples are selected using probability or non-probability sampling methods. Here, we go into where samples are selected from – the sampling frame – and common sampling frames GeoPoll uses in our own research.

What is A Sample Frame?

The sample frame is the specific source of respondents that is used to draw the sample from. This could be a map from which specific areas are outlined, a list of registered voters, a phonebook, or another source which specifically defines who will and will not be included in the sample. The sample frame should be representative of the sample universe, which is the broader definition of the sample makeup. For example, if a researcher is looking to study attitudes of students at a specific university, the definitions may look like the below:

• Sample Universe: Current students at University X
• Sample Frame: List of all 10,000 currently enrolled students provided by the admissions office
• Sample: 400 randomly selected students from the list of enrolled students who participate in the research study.

In a general population study, the sample frame may be ‘all households in Country A,’ from which a researcher can randomly select which households take part in a study.

Sampling Error or Non-Sampling Error

When speaking about a sample frame and it’s representatively of the overall population being studied, we must also consider who is not included in the sample frame. Often those who did not participate in a research study are just as important to consider as those who were represented, as without them, key items may be skewed or missed. There are a few types of sampling error, also referred to as non-sampling error:

• Coverage Error: When a sampling frame does not sufficiently cover the population required for a study there is a coverage error. For example, if a national survey is being conducted by telephone and the sample frame is taken from a phonebook, but not all households are listed in the phonebook. A telephone or internet survey will also exclude those who do not use telephones or the internet.
• Nonresponse Error: This error describes those who were contacted for a survey but were unable to or did not want to participate. This could include those who are selected for a telephone or in-person interview and do not pick up the phone or answer their door, or those who answer but refuse to participate.
• Interviewer Error: This error occurs when an interviewer incorrectly records a response for a participant of a study. This is a form of interviewer bias that can be introduced in telephone and in-person interviews. This bias could be due to voice tone or other characteristics and may influence a respondent’s likelihood to participation or their actual answers. For example, GeoPoll has found that females may be more comfortable answering questions from female interviewers.
• Processing Error: This error refers to the technical processing of a study’s data points and errors that occur as data is collected with the use of a technology platform, or during data entry as well as data coding, cleaning, and editing.
• Response Error: This error describes those who participate in a study that either intentionally or accidentally provide inaccurate responses to a study’s questions. This can occur for a variety of reasons related to the comprehension and memory of a study’s participants. Additionally, response error can occur due to social desirability bias that can be introduced into a study when a participant answers in a way they believe would be more acceptable and accurate to their conceptualization of a study’s objective or in a way that abides by social norms. Social desirability has the potential to be introduced into any study, but if often apparent in studies covering sensitive or taboo topics for a particular society.

The above errors can be mitigated through careful sample frame selection and testing of various modes to reduce non-sampling errors. For interview-administered surveys, rigorous training of interviewers is needed to help reduce the influence of biases. For self-administered surveys, understanding local context while in the design stage is important to be able to formulate questions that can be understood clearly and accepted as valid areas of inquiry by the population of interest.

GeoPoll Sample Frames

The creation of a sampling frame for GeoPoll projects depends on client needs, project specifications, and other factors including survey mode. While sampling frames are unique for each project, there are a few common sampling frames that we use which are outlined below.

• Mobile subscribers within a certain country: GeoPoll primarily conducts research through mobile-based methodologies including voice calls and SMS messages. Due to this, sample frames for our studies are often those who have access to a mobile device within each country. GeoPoll reaches mobile subscribers in two primary ways: Partnerships with mobile network operators which enable us to call or send messages to their opted-in subscribers, and Random Digit Dialing (RDD). Using an intelligent RDD process, GeoPoll is able to randomly generate valid phone numbers that match the format of those in each country.
• Census data: GeoPoll also relies on census data and census estimates both to inform nationally representative demographic breakdowns and to create sample frames when conducting in-person research. The availability of up-to-date census data varies by country and requires a researcher to understand what information from reputable sources is available. One resource that can be used to look at each country’s local bureau of statistics and at the U.S. Census Bureau’s International Data Base.
• Aid Beneficiaries: When working with international development clients, GeoPoll is able to survey aid beneficiaries if given their contact information. This requires organizations to provide GeoPoll with a list of beneficiaries’ phone numbers or other contact information.

Determining the appropriate sample frame and other sample criteria for any one project is a complex process that cannot be represented in full here, however, we hope we have given you some insight into how GeoPoll approaches sampling. To learn more about GeoPoll’s processes please contact us here.