One poll last week had President Bush leading Democrat John Kerry by 13 percentage points while another showed the candidates tied in the election race. What is going on with public opinion polls?
As U.S. voters are swamped by a blizzard of rival surveys in the final weeks of the Nov. 2 presidential election campaign, pollsters themselves are facing increasing questions about their methodology and accuracy.
Polls do not only measure campaigns. They may also affect them. If one candidate starts trailing in the polls, some of his supporters may become demoralized and decide not to vote.
“Polls impact how voters feel about a candidate, especially because of the huge coverage they attract on cable TV and the Internet,” said Dean Spiliotes, director of the New Hampshire Institute of Politics at Saint Anselm College in Manchester.
“In many respects, they have become a campaign tool for mobilizing a candidate’s base and demoralizing the opposition,” he said.
Moveon.org, an Internet-based group working to defeat Bush, took out a full-page advertisement in Tuesday’s New York Times attacking the Gallup poll that put Bush ahead for employing faulty methodology that overrepresented Republicans and undercounted Democrats.
Some say the gyrations in poll results reflects a volatile and undecided electorate.
“Voter opinion has become quite unsettled in the past month. The number of swing voters has actually grown from August to September, which is not what we usually see,” said Andrew Kohut who directs polling for the Pew Research Center.
Pollsters acknowledge that there are many ways errors can slip into their surveys including sampling errors, interview bias and poor questions. Even the order in which questions are asked and slight variations in wording can produce vastly different results.
“Some of us try to hide behind the concept of a statistical margin of error but I don’t use that phrase because the errors are in fact unquantifiable. There is no fixed, finite, measurable margin of error,” said Humphrey Taylor of Harris Interactive, a New York polling company.
Part science, part art
John Zogby, who will begin polling on the presidential race for Reuters early next month, said: “For me, polling is 80 percent science and 20 percent art.”
Pollsters are each in their own way trying to come up with a cross-section of the electorate that will actually show up to vote on Election Day. But each company has a different method of deciding who should be counted as a “likely voter” in an election in which probably only around 50 percent of the electorate will vote.
For example, pollsters ask voters a number of screening questions aimed at excluding respondents unlikely to vote. These may include: did they vote in the last election, do they remember who they voted for, do they know where people in their neighborhood go to vote, are they registered with a particular party and if so which?
Some companies simply exclude anyone failing to respond appropriately to one or more of these questions. Others, including Gallup, statistically weight participants in the poll depending on their responses to these questions.
“We either screen people in or out of the poll but we don’t weight their responses. Empirically, we’ve found it produces better results,” said Taylor.
The other major issue dividing pollsters is whether they should include weighting for party identification -- that is whether they should statistically adjust their raw numbers to reflect preset assumptions about the numbers of Republicans, Democrats and independents who will take part in the election.
All pollsters weight their samples by gender, age, race and other factors such as income, geographical location and education level to produce what they believe is a true reflection of the electorate.
Zogby is a staunch advocate of including party identification in his statistical method.
“I know that to some pollsters I am a heretic but I have found that weighting for party ID is a proven way of ensuring you have a proper sample,” he said.
Other pollsters criticize this, since they say that people respond differently when asked which party they belong to depending on events in the campaign.