A typical Pew Research Center national survey – regardless of mode – is designed and implemented with a total survey error approach in mind, aimed at minimizing coverage error (deviations of frame population from target population), sampling error (deviation of sample from sampling frame), nonresponse error (respondents’ deviation from sample), measurement error (deviations of responses from underlying measurements/attributes), and processing and adjustment error (error introduced post data collection but prior to data analysis).
Since 2014, Pew Research Center has conducted surveys online in the United States using our American Trends Panel (ATP), a randomly selected, probability-based sample of U.S. adults ages 18 and older. The panel was initially built to supplement the prevalent mode of data collection at the Center during that time: random-digit-dial (RDD) telephone surveys. However, at this point, the Center has switched almost completely to conducting its U.S. surveys online using the ATP.
Panel members are recruited offline, and survey questionnaires are taken via self-administered online surveys. Those who don’t have internet access can take our surveys on internet-enabled tablets we provide to them. Panelists typically take one to three surveys each month.
Most of our surveys are representative of the entire noninstitutionalized adult population of the United States. We do not exclude anyone from our analyses based on demographic characteristics. Some surveys are conducted among subgroups in the panel, such as Black Americans or young people, and may include a supplement sample (called “oversamples”) from another panel such as Ipsos’ KnowledgePanel. Pew Research Center also conducts international surveys that involve sampling and interviewing people in multiple countries.
On occasion, Center researchers conduct surveys with special populations, such as topic experts (e.g., technologists) or members of a certain profession (e.g., journalists). The principles are the same whether the sample is of the general population or some other group. Decisions must be made about the size of the sample and the level of precision desired so that the survey can provide accurate estimates for the population of interest and any subgroups within the population that will be analyzed. Some special challenges arise when sampling these populations. In particular, it may be difficult to find a sampling frame or list for the population of interest, and this may influence how the population is defined. In addition, information may be available for only some methods of contacting potential respondents (e.g., email addresses but not phone numbers) and may vary for people within the sample. If most members in the population of interest have internet access, and email addresses are available for contacting them, the web often provides a convenient and inexpensive way to survey experts or other industry-based populations.
Creating good measures requires asking unambiguous and unbiased questions. There are several steps involved in developing a survey questionnaire . The first is identifying what topics will be covered in the survey. For Pew Research Center surveys, this involves thinking about what is happening in our nation and the world and what will be relevant to the public, policymakers and the media. We also track opinion on a variety of issues over time, so we often ensure that we update these trends on a regular basis so we can understand whether people’s opinions are changing.
Each Pew Research Center survey report includes a “topline questionnaire” with all of the questions from that survey with the exact question wording and response options as they were read to respondents. This topline provides the results from the current survey for each question, as well as results from previous surveys in which the same questions were asked. For most studies, it is our policy to release ATP datasets within twelve months of data collection. Please visit our datasets page for further information.
