This is one of an occasional series of posts on polling.
Donald Trump’s victory in the 2016 U.S. presidential election, as well as the UK’s decision to leave the European Union through “Brexit,” rattled public confidence in polls. Since these two major world events occurred, we have been asked the same question when giving presentations, on social media, in interviews, and from our own friends and neighbors: “Can we still trust polls?”
Our new video explains why well-designed polls can be trusted.
Those who felt led astray by surveys conducted during the 2016 U.S. presidential election may be surprised to learn that national polling was generally quite accurate.
National pre-election polls in 2016 indicated that Hillary Clinton would win the national popular vote by a 3-point margin, and in fact she won by 2 points. The major problem was with state-level polls, many of which missed a late swing to Trump among undecided voters and did not correct for the fact that their responding samples contained proportionally too many college-educated voters (who were more likely to favor Clinton). A silver lining is that both of these problems can be overcome, to some extent, by more rigorous survey weighting and heightened attention to the possibility of late shifts in voter preferences.
It’s also important to remember that election polls are just one kind of poll, and that they’re not the best barometer for the accuracy of polling in general. Why not? Because an election poll has an extra hurdle to jump: It not only has to measure public opinion, it also has to predict which of the people interviewed are going to vote and how they will vote – a notoriously difficult task.
So, if election polls aren’t a reliable measure of polling accuracy, what is? There are a number of other ways we can measure the health of polling. One is to look at how polling on an issue tracks with real-world events. For example, on the issue of same-sex marriage in the United States, polls showed growing acceptance around the same time that advocates were winning statewide referenda legalizing it. The polls, in other words, were corroborated by real-world events.
There are also a number of high-quality, government-funded surveys that provide us with quite accurate benchmark estimates for a range of characteristics of the U.S. population. Pollsters can ask the same questions these government surveys do to see how their results compare. In the case of Pew Research Center polls, our trends track very closely with those of high-quality benchmark polls on questions like religious identity and political affiliation. This gives us additional confidence that the trends we’re recording are accurate.
So, yes, we can still trust polls. But it’s important to be realistic about the precision they can provide.
Want to learn more about polling? Check out our Methods 101 videos:
How can a survey of 1,000 people tell you what the whole U.S. thinks?
How do you write survey questions that accurately measure public opinion?
