Overestimates of Sen. John Kerry’s support in presidential exit polls conducted on Election Day do not support allegations of fraud, according to an examination of the $10 million system released Wednesday.
The examination concludes that the exit-polling system worked properly, stressing that in the end, none of its clients — ABC, CBS, CNN, Fox, NBC and The Associated Press — was misled into making an inaccurate projection of the winner of any race. “However, the estimates produced by the exit poll data on November 2nd were not as accurate as we have produced with previous exit polls,” it said.
In fact, the polls overstated Kerry’s support by statistically significant margins in 26 states — more than half of them — and President Bush’s in four others. That was the worst performance since news organizations joined to begin using national exit polls together in 1988.
“We believe that the positives greatly outweigh the negatives,” said the report, issued by Edison Media Research and Mitofsky International, which created the National Election Pool, or NEP. “However, we share the members’ desire for an even more reliable operating system and more accurate data on Election Day.”
Fueled suspicions of vote rigging
The news organizations use the NEP data to help shape their coverage on Election Day. No data are supposed to be reported until all polls close in a particular state, but a series of leaked reports of preliminary survey results during the day led many supporters of Kerry to conclude that he had won the election.
When Bush turned out to be the winner, some Kerry partisans used the exit polls to insist that the Republicans had somehow stolen votes, especially in the swing states of Pennsylvania and Ohio and in districts that were using electronic voting systems for the first time.
But Edison and Mitofsky said a thorough examination of the survey data demonstrated that Bush won the election fair and square.
“Exit polls do not support the allegations of fraud due to rigging of voting equipment,” the report concluded. “Our analysis of the difference between the vote count and the exit poll at each polling location in our sample has found no systematic differences for precincts using touch screen and optical scan voting equipment.”
Top target: the leakers
In fact, the exit polling data suggest that the electronic voting systems may have been more accurate than the traditional mechanical systems they were designed to replace after the disputed vote count in Florida in 2000. In districts that used the new electronic systems, “these differences are similar to the differences for punch card voting equipment, and less than the difference for mechanical voting equipment,” the report found.
Instead, the uncertainty and contentiousness on Election Day were the fault of recipients who leaked preliminary, incomplete data from early in the day, said Sheldon Gawiser, NBC News’ director of elections coverage and former chairman of the consortium’s steering committee. Prominent publications that received such leaks and reported raw data out of context included Slate.com, which at the time was an editorial partner of MSNBC.com, which is a joint venture of Microsoft Corp. and NBC News.
“Those of us that work with the stuff know you don’t start playing with the data” until all of them are in, Gawiser said in an interview.
To crack down on such leaks next time, he said, even the NEP’s clients will not get any data until 6 p.m. ET, when polls close in many Eastern states. While the embargo will likely hamper news organizations’ coverage plans, he said, it should cut down on misuse of the data by untrained Weblog editors and writers who do not understand the system.
Why Kerry ‘won’ the exit polls
The main impetus for the report was the perception that the NEP exit polls were flawed because they consistently found more support for Kerry than was reflected in the final voting tallies.
Not true, said the report, which found that no “systematic problem in how the exit poll data were collected and processed” explained the inaccuracies.
Instead, the report’s authors said, a variety of anomalies, some of them outside the pollsters’ control, added up to create a nationwide overstatement of Democratic support, especially in Ohio and Pennsylvania. Those factors differed from precinct to precinct, combining in various permutations to create “a sizeable overstatement of the estimated percentage of the vote for John Kerry.”
Most significantly, “it appears that Democrats are just more likely to take the exit polls than Republicans,” Gawiser said. “Why does that happen? What can we do to reduce that?”
He added: “It’s an internal problem. It’s one that NBC feels very strongly that they have to look at to minimize. They seem to be looking at them.”
Preliminary handling of data from voters questioned early in the day also overstated the proportion of women who voted, who tended to favor Kerry, the report said. The error was corrected for subsequent interim reports, but by then news of Kerry’s “victory” was already rocketing around the World Wide Web.
Other factors
The report also blamed election officials in some states who barred survey-takers from approaching voters close to their polling places, as well as bad weather that led to lower completion rates at some locations.
Edison/Mitofsky conceded that some internal factors may have contributed to the skewed results, zeroing in on “interactions between respondents and interviewers that can contribute to differential non-response rates.”
Gawiser welcomed the report’s call for better training of interviewers and for paying more attention to “interviewer characteristics” — the report found that error rates rose for younger or less well-educated interviewers. “How much that will solve the problem, I don’t know,” he said, but it is a start.
In the end, there will always be some error in any poll, no matter how large or well-controlled, and that error can be magnified by the stresses of a particularly close or hard-fought race, Gawiser said.
As for 2008, he said, “I think it will be better, but until I know what the shape of the election is going to be, I can’t be sure.”