APPENDIX: SURVEY METHODOLOGY
About the General Public Survey
Results for the general public survey are based on telephone interviews conducted under the direction of Princeton Survey Research Associates International among a nationwide sample of 2,001 adults, 18 years of age or older, from April 28 to May 12, 2009 (1,500 respondents were interviewed on a landline telephone, and 501 were interviewed on a cell phone, including 198 who had no landline telephone). Both the landline and cell phone samples were provided by Survey Sampling International. Interviews were conducted in English and Spanish. For detailed information about our survey methodology, see https://legacy.pewresearch.org/politics/methodology/.
The combined landline and cell phone sample are weighted using an iterative technique that matches gender, age, education, race/ethnicity, region, and population density to parameters from the March 2008 Census Bureau’s Current Population Survey. The sample is also weighted to match current patterns of telephone status and relative usage of landline and cell phones (for those with both), based on extrapolations from the 2008 National Health Interview Survey. The weighting procedure also accounts for the fact that respondents with both landline and cell phones have a greater probability of being included in the sample.
The following table shows the error attributable to sampling that would be expected at the 95% level of confidence for different groups in the survey:
In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.
About the Science Knowledge Quiz
Results for the science knowledge quiz and the spending priority questions are based on telephone interviews conducted under the direction of Princeton Survey Research Associates International among a nationwide sample of 1,005 adults, 18 years of age or older, from June 18-21, 2009 (705 respondents were interviewed on a landline telephone, and 300 were interviewed on a cell phone, including 118 who had no landline telephone. The data were weighted using similar procedure as the general public survey. Both the landline and cell phone samples were provided by Survey Sampling International. The error attributable to sampling is 3.5 percentage points at the 95% level of confidence.
About the Scientist Survey
Results for the scientist survey are based on 2,533 online interviews conducted from May 1 to June 14, 2009 with members of the American Association for the Advancement of Science (AAAS), under the direction of Princeton Survey Research Associates International. A sample of 9,998 members was drawn from the AAAS membership list excluding those who were not based in the United States or whose membership type identified them as primary or secondary-level educators.
Founded in 1848, AAAS is the world’s largest general scientific society, and includes members representing all scientific fields. AAAS publishes Science, one of the most widely circulated peer-reviewed scientific journals in the world. Membership in AAAS is open to all.
Each person sampled was mailed a letter on stationery with logos of both the Pew Research Center for the People & the Press and AAAS. The letter was signed by Andrew Kohut, President of the Pew Research Center and Alan I. Leshner, Chief Executive Officer of AAAS. These letters were intended to introduce the survey to prospective respondents, describe the nature and purpose of the survey and encourage participation in the survey. The advance letter contained a URL and a password for a secure website where the survey could be completed. The letter also included a toll-free number for respondents to call if they had questions.
Subsequent requests to complete the survey were sent to those who had not yet responded. These requests were sent by email for those who could be contacted this way (three e-mail reminders were sent) and by postal mail for members who had told AAAS they preferred not be contacted by e-mail (a postcard and letter reminder were sent).
A total of 1,411 of the 5,816 sampled members in the e-mail group completed the interview for a response rate of 24%. In the mail group, 1,122 members of the 4,182 sampled completed the survey for a response rate of 27%. The overall response rate for the study was 25% (2,533 completes/9,998 sampled members). Nearly all respondents completed the survey online; however, a very small number requested to complete the survey in another mode; twenty interviews were completed by telephone.
Nonresponse in surveys can produce biases in survey-derived estimates because participation may vary for subgroups of a population, who may differ on questions of substantive interest. In order to correct for these biases, weighting is often employed.
To evaluate the possibility of nonresponse bias in the scientist survey, respondent characteristics from the obtained sample were compared with known characteristics of the population, based on membership and demographic information in the AAAS membership database. For most characteristics the sample was very representative of the population of all members. The most notable differences were that the sample underrepresented student members and overrepresented those with emeritus status. There also were differences in response rates between those who could be contacted by e-mail and those for whom no e-mail address existed or e-mail contact was not permitted. To correct these potential biases, the data were weighted so that the sample matched the two parameters of contact mode and member category from the AAAS membership database.
The following table shows the error attributable to sampling that would be expected at the 95% level of confidence for different groups in the scientist survey:
In addition to sampling error, one should bear in mind that question wording and practical difficulties in conducting surveys can introduce error or bias into the findings of opinion polls.
ABOUT THE CENTER
The Pew Research Center for the People & the Press is an independent opinion research group that studies attitudes toward the press, politics and public policy issues. We are sponsored by The Pew Charitable Trusts and are one of seven projects that make up the Pew Research Center, a nonpartisan “fact tank” that provides information on the issues, attitudes and trends shaping America and the world.
The Center’s purpose is to serve as a forum for ideas on the media and public policy through public opinion research. In this role it serves as an important information resource for political leaders, journalists, scholars, and public interest organizations. All of our current survey results are made available free of charge.
All of the Center’s research and reports are collaborative products based on the input and analysis of the entire Center staff consisting of:
Andrew Kohut, Director
Scott Keeter, Director of Survey Research
Carroll Doherty and Michael Dimock, Associate Directors
Michael Remez, Senior Writer
Juliana Menasce Horowitz, Robert Suls, Shawn Neidorf, Leah Christian and Jocelyn Kiley,
Research Associates
Kathleen Holzwart and Alec Tyson, Research Analysts
The research staff of the Pew Forum on Religion and Public Life, Greg Smith, Scott Clement and Neha Sahgal also contributed substantially to this project.
Cary Funk of Virginia Commonwealth University was a consultant on this project.
