Research Panel vs. Sample

The terms “research panel” and “sample” may seem like they are used interchangeably, but, in fact, the two are different in both what they are and how they are used.

Market Research Panels

In the most general sense, panels refer to groups of pre-recruited respondents that have consented to taking surveys; however, the word “panel” can refer to both large, general market research panels recruited by a research company to take any study, and more specific panels who take studies on a specific topic or even the same study over time. For example, a research organization may have a panel based on a specific location or interest—like a USA panel or a panel of farmers. Each panelist on this type of panel consents to be contacted by the research organization if they are selected to participate in a future research project and typically provides the organization with their demographic information. Researchers then select respondents to participate in research projects as they arise based on the demographic needs of each particular project. This type of panel keeps recruitment costs and turnaround time low for individual projects. Market research firms typically have access to these panels, and they are often used by organizations who do not have access to their own respondents to take research studies.

The other use of the word panel refers to strategically selected groups of people who have agreed to participate in a series of surveys that occur at regular intervals over a given period of time. This type of panel is often used for gathering longitudinal data on a particular topic that all participants have some sort of connection to. Sometimes, questionnaires are even identical for each survey in the series that a research panel participates in, which allows for insights to be drawn based on changes in responses to the same questions, by the same respondents, over the course of the survey series.

As an example of this type of panel, an organization focused on providing aid to people living without adequate access to food may create a research panel with the goal of tracking their organization’s success at getting food into the mouths of low-income earners in Darfur. The respondents on the panel would be recruited based on the qualifications of: income below a pre-determined threshold and must live in Darfur. Researchers typically take this one step further when creating a research panel by ensuring that the panel has representation per the demographic distribution of the population in the area—in order to have the panel be representative of the population being studied. Depending on the panel setup and size, each survey may be distributed to the entire panel, or may be distributed to a random selection of respondents in the larger panel. Once there have been two or more surveys distributed in the series, the organization can begin to compare and contrast the resulting data and draw insights on how their food security projects are performing, which is why utilizing a research panel methodology is desirable.

Research Sample

The word sample refers to the specific group who responded to a study. Every single research project has a sample, whether the sample was sourced from a larger panel, or the sample was recruited specifically for the project at hand. Even studies that are conducted over time with the same set of respondents have a sample. The word sample simply refers to the people who participated in a single study, so in survey series’ each individual survey’s participants would be called the sample—even if the samples are the same for each survey in the series.

Pros and Cons of Research Panels

A research panel methodology provides more benefits than just tracking changes over time. For example, research panels often result in high response rates and cheaper recruitment costs overall due to the fact that the respondents opt-in to participate in surveys before beginning the survey series. These benefits, however, come with some risk. Depending on the length and/or complexity of the questionnaire, as well as how frequently the respondents are expected to participate, respondents may become fatigued with the work involved in participation in the long-term. Panels often need to be refreshed so that new respondents are brought in to replace those who no longer wish to participate.

To maintain strong panel retention and high response rates, it is best to weigh the opinions of research experts on how to best balance questionnaire length, frequency of survey participation, and incentives provided to respondents based on the mode of research. Researchers that conduct panel surveys often have learned through years of trial and error how to best engage various populations, which can ensure a project makes the most of the allotted budget.

GeoPoll’s Panels and Capabilities

GeoPoll is a research company focused on utilizing mobile telecommunications technology to reach people in areas of sub-Saharan Africa, Asia, and Latin America. Our expertise in rural and remote areas of the world lend tremendous resources to humanitarian aid organizations and expanding corporate enterprises alike. We use several sampling methodologies, and can create new panels for ongoing research or recruit respondents for one-time studies from our own database of respondents who are pre-stratified by demographics. To learn more about how GeoPoll can help your team accomplish research goals, contact us today.

The post What are Research Panels? appeared first on GeoPoll.

What is A Sample Frame?

The sample frame is the specific source of respondents that is used to draw the sample from. This could be a map from which specific areas are outlined, a list of registered voters, a phonebook, or another source which specifically defines who will and will not be included in the sample. The sample frame should be representative of the sample universe, which is the broader definition of the sample makeup. For example, if a researcher is looking to study attitudes of students at a specific university, the definitions may look like the below:

• Sample Universe: Current students at University X
• Sample Frame: List of all 10,000 currently enrolled students provided by the admissions office
• Sample: 400 randomly selected students from the list of enrolled students who participate in the research study.

In a general population study, the sample frame may be ‘all households in Country A,’ from which a researcher can randomly select which households take part in a study.

Sampling Error or Non-Sampling Error

When speaking about a sample frame and it’s representatively of the overall population being studied, we must also consider who is not included in the sample frame. Often those who did not participate in a research study are just as important to consider as those who were represented, as without them, key items may be skewed or missed. There are a few types of sampling error, also referred to as non-sampling error:

• Coverage Error: When a sampling frame does not sufficiently cover the population required for a study there is a coverage error. For example, if a national survey is being conducted by telephone and the sample frame is taken from a phonebook, but not all households are listed in the phonebook. A telephone or internet survey will also exclude those who do not use telephones or the internet.
• Nonresponse Error: This error describes those who were contacted for a survey but were unable to or did not want to participate. This could include those who are selected for a telephone or in-person interview and do not pick up the phone or answer their door, or those who answer but refuse to participate.
• Interviewer Error: This error occurs when an interviewer incorrectly records a response for a participant of a study. This is a form of interviewer bias that can be introduced in telephone and in-person interviews. This bias could be due to voice tone or other characteristics and may influence a respondent’s likelihood to participation or their actual answers. For example, GeoPoll has found that females may be more comfortable answering questions from female interviewers.
• Processing Error: This error refers to the technical processing of a study’s data points and errors that occur as data is collected with the use of a technology platform, or during data entry as well as data coding, cleaning, and editing.
• Response Error: This error describes those who participate in a study that either intentionally or accidentally provide inaccurate responses to a study’s questions. This can occur for a variety of reasons related to the comprehension and memory of a study’s participants. Additionally, response error can occur due to social desirability bias that can be introduced into a study when a participant answers in a way they believe would be more acceptable and accurate to their conceptualization of a study’s objective or in a way that abides by social norms. Social desirability has the potential to be introduced into any study, but if often apparent in studies covering sensitive or taboo topics for a particular society.

The above errors can be mitigated through careful sample frame selection and testing of various modes to reduce non-sampling errors. For interview-administered surveys, rigorous training of interviewers is needed to help reduce the influence of biases. For self-administered surveys, understanding local context while in the design stage is important to be able to formulate questions that can be understood clearly and accepted as valid areas of inquiry by the population of interest.

GeoPoll Sample Frames

The creation of a sampling frame for GeoPoll projects depends on client needs, project specifications, and other factors including survey mode. While sampling frames are unique for each project, there are a few common sampling frames that we use which are outlined below.

• Mobile subscribers within a certain country: GeoPoll primarily conducts research through mobile-based methodologies including voice calls and SMS messages. Due to this, sample frames for our studies are often those who have access to a mobile device within each country. GeoPoll reaches mobile subscribers in two primary ways: Partnerships with mobile network operators which enable us to call or send messages to their opted-in subscribers, and Random Digit Dialing (RDD). Using an intelligent RDD process, GeoPoll is able to randomly generate valid phone numbers that match the format of those in each country.
• Census data: GeoPoll also relies on census data and census estimates both to inform nationally representative demographic breakdowns and to create sample frames when conducting in-person research. The availability of up-to-date census data varies by country and requires a researcher to understand what information from reputable sources is available. One resource that can be used to look at each country’s local bureau of statistics and at the U.S. Census Bureau’s International Data Base.
• Aid Beneficiaries: When working with international development clients, GeoPoll is able to survey aid beneficiaries if given their contact information. This requires organizations to provide GeoPoll with a list of beneficiaries’ phone numbers or other contact information.

Determining the appropriate sample frame and other sample criteria for any one project is a complex process that cannot be represented in full here, however, we hope we have given you some insight into how GeoPoll approaches sampling. To learn more about GeoPoll’s processes please contact us here.

The post Sample Frame and Sample Error appeared first on GeoPoll.

In this series of blog posts, GeoPoll will outline the various aspects that make up a sample and why each one is important. First, we will examine how sample is selected and the differences between a probability sample and a non-probability sample.

Probability Sample vs Non-Probability Sample

There are two main methods of sampling: Probability sampling and non-probability sampling. In probability sampling, respondents are randomly selected to take part in a survey or other mode of research. For a sample to qualify as a probability sample, each person in a population must have an equal chance of being selected for a study, and the researcher must know the probability that an individual will be selected. Probability sampling is the most common form of sampling for public opinion studies, election polling, and other studies in which results will be applied to a wider population. This is the case whether or not the wider population is very large, such as the population of an entire country, or small, such as young females living in a specific town.

Non-probability sampling is when a sample is created through a non-random process. This could include a researcher sending a survey link to their friends or stopping people on the street. This type of sampling would also include any targeted research that intentionally samples from specific lists such as aid beneficiaries, or participants in a specific training course. Non-probability samples are often used during the exploratory stage of a research project, and in qualitative research, which is more subjective than quantitative research, but are also used for research with specific target populations in mind, such as farmers that grow maize.

Generally speaking, non-probability sampling can be a more cost-effective and faster approach than probability sampling, but this depends on a number of variables including the target population being studied. Certain types of non-probability sampling can also introduce bias into the sample and results. For general population studies intended to represent the entire population of a country or state, probability sampling is usually the preferred method.

Types of Probability Sampling

There are several sampling methods that fall under probability sampling. In each method, those who are within the sample frame have some chance of being selected to participate in a study. Four of the common types of probability sampling are:

Simple Random Sample: The most basic form of probability sampling, in a simple random sample each member of a population is assigned an identifier such as a number, and those selected to be within the sample are picked at random, often using an automated software program.

Stratified Random Sample: A stratified random sample is a step up from complexity from a simple random sample. In this method, the population is divided into sub-groups, such as male and female, and within those sub-groups a simple random sample is performed. This enables a random sample that is representative of a larger population and its specific makeup, such as a country’s population. 

Cluster Sample: In cluster sampling, a population is divided into clusters which are unique, yet represent a diverse group – for example, cities are often used as clusters. From the list of clusters, a select number are randomly selected to take part in a study.

Systematic Sample: Using a systematic sample, participants are selected to be part of a sample using a fixed interval. For example, if using an interval of 5, the sample may consist of the fifth, 10^th, 15^th, and 20^th, and so forth person on a list.

Types of Non-Probability Sample

In non-probability sampling, those who participate in a research study are selected not by random, but due to some factor that gives them the chance of participating in a study that others in the population do not have. Types of non-probability sample include:

Convenience Sample: As its name implies, this method uses people who are convenient to access to complete a study. This could include friends, people walking down a street, or those enrolled in a university course. Convenience sampling is quick and easy, but will not yield results that can be applied to a broader population.

Snowball Sample: A snowball sample works by recruiting some sample members who in turn recruit people they know to join a sample. This method works well for reaching very specific populations who are likely to know others who meet the selection criteria.

Quota Sample: In quota sampling, a population is divided into subgroups by characteristics such as age or location and targets are set for the number of respondents needed from each subgroup. The main difference between quota sampling and stratified random sampling is that a random sampling technique is not used in quota sampling; For example, a researcher could conduct a convenience sample with specific quotas to ensure an equal number of males and females are included, but this technique would still not give every member of the population a chance of being selected and thus would not be a probability sample.

Purposive or Judgmental Sample: Using a purposive or judgmental sampling technique, the sample selection is left up to the researcher and their knowledge of who will fit the study criteria. For example, a purposive sample may include only PhD candidates in a specific subject matter. When studying specific characteristics this selection method may be used, however as the researcher can influence those who are selected to take place in the study, bias may be introduced.

GeoPoll Sampling Methods

GeoPoll uses all of the sampling approaches described above based on the needs and can use probability-based methods for our sample selection, including stratified random sampling, to build nationally representative samples. To learn more, please contact us.

The post Probability and Non-Probability Samples appeared first on GeoPoll.