Can the Exit Polls be Fixed?

For years, the exit polls conducted by the news media were considered one of the modern marvels of opinion research. Rather than asking people what they might do, the exit polls involved more than a thousand interviewers scattered throughout the country asking people leaving their polling places what they had already done–specifically who they voted for and why.

The polls not only proved remarkably accurate. They also gave journalists, academicians and politicians a strikingly clear picture of how the American electorate felt and the reasons behind those sentiments. It was an invaluable tool for understanding the democratic process and, if they wanted, a clearer direction for those with the responsibility of governing.

In the last three election cycles, however, the once-vaunted exit poll began to prove faulty–in part because the razor thin electoral majority began to expose growing weaknesses in the poll. After being consolidated from three exit polls–one for each TV network–into one, and then suffering further cutbacks as the networks saw their audiences and profits dwindle, in the last three election cycles exit poll problems have resulted in disasters of various magnitudes that confused viewers, embarrassed the networks, and called the whole process into question.

In 2000, the combined media exit poll originally showed that Al Gore was going to win Florida and most likely, the White House. But after 37 days of deadlock and a Supreme Court ruling, both Florida and the election officially went to George W. Bush.

In 2002, the computer system handling the data from the exits crashed, and the poll never was fully completed.

And in 2004, early exit returns showed Democrat John Kerry was on the road to a big victory. The numbers were wallpapered all over the Internet. Democrats went home and turned on the TV preparing to celebrate the official returns. Again the winner was George W. Bush. The polling model, it turned out, was marred by flaws again.

What are the plans for 2006? The National Election Pool (NEP), a consortium of media groups, will be exit polling again on Tuesday, trying to get a read on how people voted in the 2006 midterm election and why. There are changes this year, and the NEP promises the problems have been fixed. Interviewers, for instance, have been trained better and will be more closely supervised.

There will also be a new attempt to fix another recent growing problem – early leaks of exit poll results, often partial or early samples that by themselves mean nothing. While consortium members don’t reveal those results—at least those reflecting who is winning—until after the polls close, a number of web sites have leaked early results in recent years, especially bloggers. One concern is that prematurely released exit polls may depress the turnout of voters who haven’t yet headed to the polls and who think the election may already be resolved.

To halt that practice, NEP promises that exit poll data for this election will be housed in a windowless and ominous-sounding “quarantine room.” The few reporters allowed to enter will be stripped of cell phones and blackberries not allowed to call anyone until after 5 pm. Will that stanch the leaks? No one can be sure.

But some of those involved in that practice on past election days told us they are less inclined to do so this time around.

For intance, Slate magazine made a point of publishing the exits in past elections. But in an email to PEJ, Slate editor Jacob Weisberg said that “given the effort to keep them secret until at least 5 PM, I think it’s likely to be a moot point this year.”

Jerome Armstrong, founder of the political blog MyDD, which posted the exits in 2004, also sounds less than enthused. “I doubt I would release them, simply because the weight of traffic from having released them first in ’04 crashed MyDD for the rest of the day repeatedly, as a million people tried to log onto the website,” Armstrong wrote in an email. “I don’t put much weight in them either, and don’t view it as a loss that they will not be reported on before the polls close.”

The bigger issue, however, probably involves the methodological problems that have plagued the exit poll in recent election cycles. Why is it that they have missed the mark so badly, and how certain is it the problems have been fixed?

For starters, exit polling has always been full of potential pratfalls. If polling is an art form, exit polling is an art form practiced in poor weather under daunting circumstances, polling professionals told us. There are, as with all polls, questions of the sampling of respondents and the wording of questions, particularly when it comes to why voters cast their ballots the way they did.

But exit polls add more potential for human error. “It’s logistically more complicated because it’s face-to-face and interviewers are spread around so they can’t really be watched closely,” said Diane Colasanto, a former president of the American Association of Public Opinion Research and a former exit pollster.

Another problem is the number of voters who will simply decline to be interviewed—specifically because they don’t want to be part of an exit poll—potentially skewing the survey sample and the results. “It’s not a question of lying to interviewers,” Colasanto told PEJ. “Every person you talk to could tell you the truth and the numbers could still be wrong,” if certain types of voters are more inclined to decline talking to exit pollsters than other groups. This isn’t as much of an issue with pre-election telephone polls. In those instances, people are usually refusing to talk before they even know what the poll is about or that it relates to politics.

An analysis by the NEP of the 2004 election found that several factors probably contributed to the poor data – everything from bad weather in some areas to distance restrictions imposed upon interviewers by election officials. The NEP analysis was delicately worded, but it also suggested some of the interviewers may have been too young and improperly trained.

Is that problem fixable? Yes, said Joe Lenski, co-founder and Executive Vice President of Edison Media Research, one of the two polling organizations that conduct the exit polls for NEP.

In a recent interview with Andy Kohut, president of the Pew Research Center, Lenski said the NEP has “put together new recruiting guidelines, new training guidelines, a new training video, a new training call, a kind of quiz to make sure that after the training the interviewers retain the most important pieces of knowledge with interviewing rates being one of those most important pieces of knowledge.”

Will all this be enough to guarantee that election day 2006—and the reporting of it—will go smoothly and accurately? It will help, says Colasanto. But nothing is foolproof.

“I think the numbers will be better and more accurate, but you’re never going to get 100% of the people to cooperate,” she told us. “And even though everyone remembers 2004, there have been problems with exit polls in the past.”