by Scott Keeter, Director of Survey Research, Pew Research Center
The Pew Research Center study “Muslim Americans: Middle Class and Mostly Mainstream” attracted a great deal of attention but also raised a number of questions about the research, especially regarding how the study was conducted. Here are some of the most common questions we received together with our responses. Readers who have other questions are invited to send them to us; responses will be posted in the future.
Why did Pew conduct this study?
Muslim Americans are a small percentage of the total U.S. population but there has been a great deal of interest in their views about American foreign policy and in their experiences here since 9/11. In addition, because the U.S. Census does not include religion among its demographic measures, little is known about the demography or even the size of the Muslim population. As part of its ongoing interest in public opinion, religion, and demography, the Pew Research Center decided to undertake this study to contribute to public understanding of this population.
How did you determine who is a Muslim?
Respondents were considered to be Muslims if they told us that they were Muslim when asked their religious preference early in the interview. The religious preference question read as follows: “What is your religious preference – Protestant, Roman Catholic, Orthodox Christian, Jewish, Muslim, Hindu, Buddhist, or something else?” If respondents did not say they were Muslim but said that they were “Sunni” or some other type of Muslim, they may not have been interviewed at the time of the initial survey call, but that answer was written down and later reviewed by project staff. Respondents who said something that indicated that they were Muslim were subsequently recontacted to determine their eligibility for the study.
Are members of the Nation of Islam included in the survey as Muslim?
If a respondent told us that they were Muslim when asked their religious preference, they were included in the study. Later in the interview, respondents were asked “Are you Shi’a, Sunni, or another tradition?” In response to this question, some respondents answered that they were members of the Nation of Islam, and this was noted.
You interviewed in English and in three other languages (Farsi, Urdu, and Arabic). But aren’t there Muslims in the U.S who speak languages other than these four?
Yes. But based on our consultation with demographers and other experts regarding the immigrant population, it was determined that a majority of non-English speaking Muslim immigrants speak one of these three languages. Practical considerations limited our ability to include languages spoken by smaller numbers of potential respondents. It is very costly to translate the questionnaire, hire and train qualified bilingual interviewers, and train the rest of the interviewing staff to recognize the language when they encounter it during an initial household contact.
Many estimates of the size of the Muslim population are much higher than Pew’s. Why should we trust the Pew estimate?
It’s important to keep in mind that all estimates of the size of the Muslim population are just that: estimates. Because the U.S. Census does not measure religious affiliation, there is no official estimate of the Muslim population. The Pew number is a scientifically derived estimate from the large random survey we conducted. It is slightly higher than most others derived from surveys, probably as a result of the fact that we interviewed in three languages in addition to English. The larger estimates that have circulated over the past several years are derived from a variety of sources that, while sometimes using surveys as part of their calculation, make additional assumptions for which is there is no solid evidence.
Aren’t Muslims likely to refuse to state their religion to a stranger on the phone these days? If so, haven’t you missed a lot of this population?
We’ve heard this asserted, and it’s an understandable concern, but there is no evidence that it actually happens enough to create a bias in the survey. Our questionnaire was designed to put respondents at ease and gain their trust before asking about religion. The initial questions asked for an opinion about their community, their assessment of how their lives are going, and their participation in several activities such as taking college classes or running a small business. The religious affiliation question came next, asking respondents to choose from a list of religious traditions that included Buddhist and Hindu as well as Protestant, Catholic, Jewish, Orthodox Christian, and Muslim.
There is a good bit of evidence that Muslims are not particularly reluctant to say they are Muslim. First, the General Social Survey, which interviews respondents face-to-face in their homes, reports that only about one-third of one percent of all respondents typically decline to state a religious preference; almost none of those who do refuse to state a preference are immigrants from predominately Muslim nations or give other indications that they are Muslim. Second, we observed no significant change in the Muslim incidence in our own monthly surveys when comparing the period prior to 9/11 and the period afterward. If, as some have argued, Muslims were more reluctant to report their religious affiliation in the aftermath of the terrorist attacks, we should have seen a decline in the proportion of self-declared Muslims in our samples. It’s also important to mention that once we revealed the purpose of the survey to Muslim respondents, the vast majority agreed to continue with the survey. About eight-in-ten (79%) finished the survey (which lasted an average of 30 minutes), a completion rate nearly as high as the average for most Pew surveys (85%-90%). But even if a respondent quit the interview, they were counted as Muslim in the estimation of the size of the population.
What role, if any, did Muslims play in designing or conducting the study?
The project was planned with the assistance of an outside advisory board of eight noted authorities on Islam and Muslim Americans, most of whom are Muslims. The board was chaired by Dr. Amaney Jamal of Princeton University, who also served as senior project advisor. The board met twice in Washington to provide advice and counsel to project staff regarding sampling, questionnaire development, and survey administration. Two members of the advisory board conducted focus groups among Muslims in several U.S. cities to obtain input from the Muslim American public. In addition, all of the interviewers who conducted interviews in Arabic, Farsi, or Urdu were Muslims. The contribution of all of these Muslim Americans was invaluable. But it is important to note that Pew’s project staff had final editorial authority over the questionnaire and the study’s report, and the responsibility for the choices made in the study’s design and the interpretations presented in the report rests with them.
The report says the sample was based partly on a list of “Muslim sounding” names. Doesn’t this create a bias?
It would if we had relied only on the list. But the list was estimated to include only about 20% of all Muslim Americans, and so respondents interviewed from the list are statistically weighted to represent just 20% of the full sample
Wouldn’t many Muslim Americans be afraid to be honest about their views on many sensitive questions, especially where the U.S. government and foreign policy are concerned? Wouldn’t they hesitate to say that they don’t like President Bush or the war on terrorism?
We were concerned about this but don’t think it was a particularly serious problem. Before asking the most sensitive questions – regarding suicide bombing, Al Qaeda, the Israeli-Palestinian conflict and the like – we attempted to establish a bond of trust with the respondent by asking about a broad range of topics including their own experiences and problems in the U.S. We hoped that the effect of giving respondents an opportunity to talk about the challenges they face, their social ties, their religious beliefs, and other topics would put them at ease about the goals of the study and the uses to which it would be put.
Indeed, we saw no reluctance to express unhappiness with U.S. government policy or with the president. Only 15% of Muslim American respondents said they approve of the job President Bush is doing, and just 12% believe the U.S. made the right decision to use military force in Iraq. More than half (55%) say the U.S.-led war on terrorism is not a sincere effort to reduce international terrorism. These responses strongly suggest that there was little hesitation on the part of most respondents to say what they thought. It is noteworthy that these findings and others in the survey are very similar to those of Muslim voters’ responses to the 2004 exit poll conducted on Election Day that year. The exit poll is an anonymous survey, filled out in private by the voter and dropped into a box along with the questionnaires from other voters; the interviewer does not see the responses until later and could not associate a particular questionnaire with any given voter.
