Oct. 30, 2012 at 11:57 PM ET
Nationwide polls may portray the presidential campaign as a neck-and-neck horse race, but less conventional data-crunching methods spit out a different picture, with President Barack Obama edging out GOP challenger Mitt Romney.
One big variable remains to be factored in: the effect of Hurricane Sandy. And one big state that's been relatively unaffected by the storm holds the key to the outcome: Ohio. "It's been that way for the entire election cycle," said David Rothschild, an expert on opinion modeling at Microsoft Research and Columbia University's Applied Statistics Center.
Rothschild, who lays out election forecasts at the Predictwise website and blogs about prediction science on The Signal at Yahoo, surveyed the state of the art this weekend at the New Horizons in Science symposium, presented as part of the ScienceWriters2012 conference.
In the final days of the campaign, the divergent spins on the election outlook have sparked a few fireworks. Statistician Nate Silver's analysis for The New York Times' Five Thirty Eight column, which has consistently favored Obama even as many others were reporting a tightening of the race, drew criticism from the National Review's Josh Jordan for including "a little bit too much hope of an Obama victory against what appears to be a surge of Romney momentum."
This week, MSNBC's Joe Scarborough virtually called for Silver's pundit license to be revoked. "Anybody that thinks that this race is anything but a tossup right now is such an ideologue they should be kept away from typewriters, computers, laptops and microphones for the next 10 days, because they're jokes," he said on "Morning Joe."
Economist Paul Krugman went to Silver's defense in his own column for the Times, decrying the "war on objectivity" and saying that "all the election modelers have been faithful to their models, letting the numbers fall where they may."
"This is really scary," Krugman wrote. "It means that if these people triumph, science — or any kind of scholarship — will become impossible. Everything must pass a political test; if it isn't what the right wants to hear, the messenger is subjected to a smear campaign."
Silver's analysis is based on a state-by-state assessment of polling data from multiple sources, translated into an electoral vote count. Political prediction markets, such as those studied by Rothschild, use a different method to come up with a surprisingly similar snapshot of the horse race.
The markets offer a glorified kind of gambling on political fortunes: The winner-take-all markets let players "invest" in the prospects of a particular candidate. If the candidate wins, the investor gets, say, $1 a share. If the candidate loses, the investor gets nothing. Leading up to Election Day, investors can buy or sell shares in candidates to match their expectations of success.
The shifting share prices reflect the perceived probability of success. For example, Intrade's market sets the probability of Obama's re-election at 63 percent. The Iowa Electronic Markets go with a little more than 63 percent, while the trading at Betfair puts the probability at 70 percent. That's in the same ballpark as Silver's 72.9 percent estimate.
Intentions vs. expectations
What the prediction markets provide is a probability figure, not a vote share figure. It reflects expectations about a given outcome, just as the Vegas odds reflected the expectation that the Giants would win the World Series, even before they swept the Tigers. There was a chance all the way up to the final out that the Tigers could roar back and take four games in a row to win the series. But in this case, at least, the Vegas marketplace predicted the outcome.
So what's the success rate of prediction markets? How do surveys that gauge expectations perform, compared with traditional surveys that gauge what voters say they intend to do? That's where Rothschild's research comes in: He and a colleague, Penn economist Justin Wolfers, looked at the predictions produced by traditional polls ("For whom do you intend to vote?"), as opposed to less traditional surveys ("Whom do you expect to win?"), in 345 political races.
Most of the time, the predictions from the two types of forecasts were in agreement. But in those cases where the predictions were different, the expectation survey was right 76 percent of the time, while the traditional intention survey was right only 24 percent of the time.
Rothschild said the strength of expectation polls may lie in the fact that investors can absorb information from other sources to come up with a consensus that reflects the wisdom of crowds. "Asking people about expectations is equivalent to as if people went out to 10 random voters and reported the binary result," he said.
Based on the prediction markets, it's as if Obama is the favored team in the seventh game of the World Series. The betting odds have been in his favor for the past year — even though there have been ups and downs, such as his slump in the first presidential debate. Now that all the debates are done, most of the uncertainty has been wrung out of the campaign.
"There's one more unexpected event: this hurricane," Rothschild said.
After the storm
Lots of prognosticators have pointed to the uncertainties raised by Hurricane Sandy. The conventional wisdom was that Romney would benefit from a long-recognized anti-incumbent effect in late pre-election polling, as well as a race-tightening effect. However, Sandy changes the calculus.
"Generally, natural disasters benefit incumbents," Rothschild said. There's a tendency to put politics aside, rally 'round the flag and let the president look presidential. (That effect can go negative if the disaster response doesn't go well, as President George W. Bush found out in the wake of Hurricane Katrina.) Even before the storm hit, Scarborough said Romney's momentum could stall in Sandy's aftermath. "It changes everything with a week to go," he said.
There's already some evidence that the rally effect has kicked in: For example, today New Jersey's Republican governor, Chris Christie, said hat Obama's response to the storm crisis was "outstanding" and that he didn't "give a damn about Election Day." Christie is due to tour devastated areas with the president on Wednesday.
The catastrophic aftermath of the storm may affect early voting as well as the Election Day turnout in places like New York and New Jersey. That could cut into the Democratic vote. Research has shown that obstacles to voting tend to hit Democrats harder than Republicans. But in Sandy's case, that statistical effect may not be critical because those states are relatively safe for Obama.
Sandy's effect may be more crucial hundreds of miles from the worst of the storm, in Ohio. For the past year, Ohio has been the "flip state" in Rothschild's calculations. Neither candidate has a clear path to victory unless he wins Ohio's electoral votes, Rothschild said. That's one reason why Romney was the headliner for a storm-relief rally in Ohio today — and why Vice President Joe Biden and former President Bill Clinton are canvassing the state while Obama tours the hurricane zone.
How will it all turn out? There are lots of statistical models floating around, and no matter which way it turns out, some will score a home run while others will strike out. In addition to the political prediction markets we've been talking about, here are a few more forecasts to watch:
We'll be talking about the scientific angles to the political campaign at 9 p.m. ET Wednesday on "Virtually Speaking Science," an hourlong talk show airing on BlogTalkRadio and in the Second Life virtual world. My guest will be Shawn Lawrence Otto, a founder of ScienceDebate.org and author of "Fool Me Twice: Fighting the Assault on Science in America."
Alan Boyle is NBCNews.com's science editor and vice president of the Council for the Advancement of Science Writing, which presented the New Horizons in Science symposium. Connect with the Cosmic Log community by "liking" the log's Facebook page, following @b0yle on Twitter and adding the Cosmic Log page to your Google+ presence. To keep up with Cosmic Log as well as NBCNews.com's other stories about science and space, sign up for the Tech & Science newsletter, delivered to your email in-box every weekday. You can also check out "The Case for Pluto," my book about the controversial dwarf planet and the search for new worlds.