“One barrier to the creation of probability-based online panels is that providing internet access to potential panel members who do not have it is expensive. An alternative mainly online approach is to contact potential survey respondents directly by ABS (i.e., not via a panel) and provide them with a designated URL where they can respond online. The contact letter would also include a telephone number so people who do not have internet service or prefer not to respond that way can call in and arrange to be interviewed by telephone. This mixed-mode approach retains the probability base.”
How Do Probability-Based Online Panels Work
Some polls in the Health Poll Database are “probability-based online panels.” What does that mean? First, let’s look at what pollsters mean when they talk about survey data collection mode and survey sampling procedures.
Data Collection Mode is the phrase used to describe the method by which the selected participants complete the survey. Survey modes include telephone, mail, in-person, or online.
Sampling procedure refers to the process by which researchers choose the respondents for a poll. The methods used can be probability or non-probability-based. A probability-based poll utilizes a randomized selection process where every person in the target population theoretically has an equal chance to be selected as a respondent. In a nonprobability poll, not every member of the target population could be selected to participate, which can introduce bias.
In the earliest days of polling, most polls were conducted using nonprobability quota methods. After roughly 1950, most U.S. polling organizations shifted to probability methods. Except for surveys conducted in the early years, the polls in the Health Poll Database have all been conducted using probability-based sampling procedures.
Probability-based sampling can be used with any of these modes of data collection. With telephone polling, Random Digit Dialing (RDD) of active U.S. phone exchanges provides nearly comprehensive coverage of the national population and therefore provides a practical way to build a probability-based sample by randomly selecting telephone numbers. Address-Based Sampling (ABS) relies on random selection from the U.S. Postal Service (USPS) Computerized Delivery Sequence File, which also provides nearly complete coverage of residential mailing addresses in the U.S.
But using a probability-based sample to conduct an online survey presents a challenge. There is no comprehensive database of email addresses like there is of mailing addresses or active phone exchanges from which to make a random selection. Email addresses are not tied to residency, and not all Americans have an email address, while many have more than one, making it impossible to create a probability-based sampling frame to represent the total population.
With response rates for telephone polls decreasing dramatically as costs skyrocket, many organizations have decided that polling needs to move online, where contacting people is cheaper. The easiest and cheapest way to conduct online polling (but the least scientifically rigorous) is to use a nonprobability-based online panel. Polls using this sampling approach are numerous and popular but have some inherent bias and are not generalizable to the larger target population. These relatively inexpensive polls are based on large panels of respondents who agree to answer surveys, usually in return for small rewards like points that can be exchanged for gift certificates. Panelists can be recruited through email lists, online ads, or other methods. Samples for specific polls are often built using quotas for different demographic groups, and weighting is used to try make samples more representative of the target population. But only those who are already online can be part of these panels, creating an inherent bias, and making it inappropriate to generalize the results of these polls to the full target population.
Researchers who want to retain the advantages of probability-based sampling (e.g. scientific rigor, ability to generalize beyond the sample of respondents to the target population), find a few online options. Online probability panels polls – which are newer, less common and more expensive than nonprobability online polls – use traditional probability-based samples, like ABS, to make the first contact with a respondent or household. Those people who do not have web access are provided such access. A large number of respondents are selected to be a part of the panel, and then random selections are conducted within the panel or subpopulations of the panel to be invited to answer particular surveys.
Several organizations in the Health Poll Database conduct polling via online probability panels, including KnowledgePanel (formerly KnowledgeNetworks), NORC AmeriSpeak, the Pew American Trends Panel, the RAND American Life Panel, and the SSRS Probability Panel.