Knowing the gender of our survey respondents is critical to a variety of analyses we do at Pew Research Center. Gender affects how a person sees and is seen by the world. It’s predictive of things like voting behavior, the wage gap and household responsibilities.
On nearly all surveys we conduct in the United States, we ask people, “Are you male or female?” (Or, in Spanish: ¿Es usted hombre o mujer?) While this wording has been the standard question we’ve used for years, we wondered if we could find a new way to ask about gender that would acknowledge changing norms around gender identity and improve data quality and accuracy, while still maintaining the neutrality that defines the Center.
We organized a research team to answer this question and — if the answer turned out to be yes — determine how we might modify the way we ask about respondents’ gender. Ultimately, we settled on a new version of the question: “Do you describe yourself as a man, a woman, or in some other way?” (Or, in Spanish: ¿Se describe a sí mismo(a) como un hombre, una mujer o de alguna otra manera?)
Here, we detail how and why we came to this decision.
We considered attitudes, laws and best practices in research
Gender is a social construct based on how people see themselves and how others see them. Sex, by contrast, is a biological construct assigned at birth. An estimated 1.4 million U.S. adults are transgender, which is defined in this post as people who identify as neither a man nor a woman (sometimes people in this group use the term “nonbinary”) or those who identify with a gender different from the sex they were assigned at birth. (It is worth noting that some people in this group might use terms other than “transgender” to describe themselves.) Roughly one-in-five U.S. adults know someone who uses a gender neutral pronoun such as “they” instead of “he” or “she,” and 17 states and the District of Columbia have adapted to this evolution in gender identity by adding a nonbinary option to driver’s licenses.
Other survey researchers have also begun adapting their gender questions. Some surveys ask a two-step question to determine both sex assigned at birth and current gender identity. Others ask whether a respondent is male or female and whether they self-identify as transgender. Meanwhile, qualitative research has looked at the best terms to use when asking about gender identity.
Given that many Americans view gender in a way that is more complex than can be captured in just the two response options of “male” and “female,” adding a third option could improve data accuracy by ensuring the data are representative and inclusive of all types of voices. Proponents of a third gender response option may also perceive the lack of such an option as exclusionary. On the other hand, the Center was concerned that adding a third option could possibly alienate the 56% of U.S. adults who said in a 2018 survey that forms or online profiles should not include an option other than “man” and “woman.” If either group stopped participating in our surveys, it would introduce bias and harm data quality. Another consideration was how a third gender response option would translate into Spanish. The Center’s surveys are almost always conducted in both English and Spanish, and little existing research has addressed how to phrase a revised gender question in Spanish.
Prior research informed our new gender identity question
In previous studies, researchers have conducted a variety of tests about the best way to ask about gender. Informed by this research, we decided to make our starting point the question wording used in the National Crime Victimization Survey: “Do you currently describe yourself as male, female or transgender?” We then made two changes based on previous research.
First, we replaced “transgender” with the phrase “in some other way.” Some people who identify as neither male nor female do not identify with the term “transgender,” instead preferring “nonbinary,” “genderfluid” or other terms. “Transgender” also isn’t necessarily mutually exclusive from “male” and “female.” For example, some people consider themselves both male and transgender. In addition, there is no Spanish equivalent of “transgender,” and around three-in-ten U.S. adults do not have a clear understanding of what “transgender” means.
Second, we replaced “male” and “female” with “a man” and “a woman.” The words “male” and “female” refer to a person’s biology or sex, while the words “man” and “woman” are cultural terms that are more consistent with the concept of gender.
We conducted an experiment to address outstanding concerns
In June and July 2020, we conducted an online opt-in survey of 5,903 internet panelists to test two alternative gender question options in English and Spanish. A total of 4,931 responded in English and 972 responded in Spanish. The survey took an average of 12 minutes to complete and included questions about the coronavirus outbreak, political approval and other demographics.
Respondents were randomly assigned to receive one of three sets of gender questions after receiving other demographic questions. One group of respondents (n=1,654) received the standard Pew Research Center question. Another group (n=2,611) received a question about sex, followed by a question about gender:
1. “Were you born male or female?” (¿Su sexo al nacer fue masculino o femenino?)
2. “Do you describe yourself as a man, a woman, or in some other way?” (¿Se describe a sí mismo(a) como un hombre, una mujer o de alguna otra manera?) The phrase “some other way” also included a text box for respondent specification.
In asking these questions, it was especially important that we minimized the number of adults who might incorrectly be classified as transgender. Since the U.S. transgender population is small, estimates of the size of this population might be artificially inflated even if a small number of respondents accidentally chose the wrong answer. To minimize this error, we asked a third follow-up question of those whose responses to these two questions did not match:
3. “Just to confirm, you were born [male/female] and now describe yourself [as a man/as a woman/in some other way]. Is that correct?” (Solo para confirmar, su sexo al nacer fue [masculino/femenino] y ahora se describe a sí mismo(a) [como hombre/como mujer/de otra manera]. ¿Es eso correcto?)
If respondents said their answers were not recorded correctly, they were given the opportunity to re-answer both questions.
A final group (n=1,638) received the same questions as the second group, but in reverse order; they received the question about gender identity before the question about sex assigned at birth. This group was also asked to confirm if their answers to the two questions didn’t match.
At the end of the survey, immediately following the gender/sex questions, we asked all respondents whether they thought our questions were biased (and, if so, why), followed by two questions about their views on transgender rights and offering a third gender option on forms and online profiles.
Unweighted estimates from this experiment are sprinkled throughout the remainder of this post. While opt-in samples are not ideal for creating point estimates, they are useful when comparing experimental wording conditions, as was the purpose here. To further improve comparability and to ensure we had reliable estimates, we also implemented a series of quotas and included an oversample of less-acculturated Hispanics (as determined by a combination of language skills, years living in the U.S., type of media consumption and self-described cultural affiliation).
Data quality improved with the new gender question
Regardless of the way in which we asked respondents their sex/gender, we observed consistent distributions among men and women, suggesting little effect of the question wording for most people. However, among English-speaking adults who received the new gender questions, 1.5% selected the third, nonbinary response option or chose a gender inconsistent with their sex, suggesting that the inclusion of the third option had the intended effect on being more inclusive and improving data accuracy.
There was some concern that people who consider themselves a man or woman may not wish to be defined by their gender and instead opt for the third option and write in a non-gender term to describe themselves, such as “human” or “mother” or “scientist.” A large number of these types of responses would have suggested that the revised question would introduce error and hurt data quality. Luckily, we only observed four of these types of responses among English speakers, affecting the estimate by 0.1 percentage point and suggesting that it was not a problem.
Providing a third gender option didn’t alienate people
Fewer English-speaking respondents to our 2020 survey said they opposed the inclusion of a third gender response option on forms or online profiles (44%) than in our 2018 survey. However, 44% is still a substantial share of respondents. We were concerned that opponents would find our new, more inclusive question off-putting. This concern did not appear in the data. Fewer than 1% opted not to answer the gender question.
The revised gender question also didn’t cause more people to stop taking the survey before completing it. No more than 0.5% of English-speaking respondents across all experimental conditions opted to stop participating at or after the sex and gender questions.
The revised questions also did not appear to be affecting the Center’s credibility as a nonpartisan research organization. When asked whether respondents thought the survey was politically neutral, about three-in-four English speakers said the survey was not at all or not very biased, regardless of which gender questions they received.
The new gender question worked in Spanish, too
A small proportion of Spanish-speaking respondents (3.6%) opted for the third gender choice or chose a gender inconsistent with their sex. While this was significantly higher than the estimate among English speakers (1.5%), both English and Spanish speakers were asked to confirm their choice if sex and gender responses did not match. Spanish speakers consistently confirmed their answers to both sex and gender. Moreover, when asked if they found any questions confusing, only six Spanish speakers mentioned the gender question wording, and none of the six were coded as transgender.
Similar to English-speaking respondents, Spanish speakers did not seem to be affected by the various question wordings. And the addition of a third option had no perceivable alienation effect. Some 40% of Spanish speakers said they disapproved of the addition of a third gender choice on forms, but the addition of a third response category to the gender question did not cause more people to stop participating in the survey prematurely, nor did it affect the proportion of Spanish-speakers who perceived the survey as somewhat or very biased.
Asking about sex in addition to gender wasn’t necessary
While our experiment used a two-step question to deduce both a respondent’s sex and gender, moving forward, we will limit most of the Center’s U.S. surveys to the single gender question referenced at the top of this post.
Based on comments collected at the end of our experiment, fewer than 1% of respondents mentioned the sex and gender questions in any way. Among those who did, 30% expressed frustration over the presence of multiple questions or concern about the use of the term “born” in the question on sex. Also, the inclusion of a question about respondent’s sex isn’t necessary for most of the Center’s research, which focuses predominantly on gender. And while we do construct weights based on population estimates by sex, the correlation between gender and sex suggests that gender can reliably be used as a proxy for sex for this purpose. Another important consideration is respondents’ time: If we don’t need to ask a question, we won’t.
In conclusion, this change in how we ask about gender was not taken lightly, as we were driven to ensure inclusivity and accuracy while maintaining the rigorousness and neutrality that characterizes the Center.
