How much does the pollster concept of "measurement error" inform the debate about counting the so-called popular vote in the Democratic primaries and caucuses this year? More than you may think.
Measurement error is the term survey researchers use to describe what happens when a respondent's answer (or an interviewer's record of that answer) does not accurately describe the underlying opinion or characteristic that the pollster was hoping to measure.
Suppose the respondent does not hear or understand the question, or feels too embarrassed to share the true answer, or perhaps the interviewer checks the wrong box. Any of these problems can lead to inaccurate measurements.
Although he did not use the term, UCLA history professor Theodore Porter invoked the concept of measurement error in a penned in the midst of the Florida recount battle eight years ago. Since the lead that George W. Bush held over Al Gore in Florida at the time hovered "around one one-hundredth of one percent," he argued that the race was "essentially a draw" in which "the number of votes lost due to errors in Florida is much greater than the spread between Bush and Gore."
The final report of the National Opinion Research Center's Florida Ballots Project proves Porter's point. Although Bush led on the final certified count by just 537 votes, 61,190 Floridians invalidated their ballots with "overvotes" (usually by marking more than one candidate). According to the NORC report, these ballots included 5,277 in Palm Beach, where voters punched out chads for both Al Gore and Pat Buchanan on the now infamous butterfly ballot. The report also found thousands of officially uncounted ballots where voters incorrectly circled or wrote in a candidate's name on an optical scan ballot.
The point here is not to reopen the debate about which ballots should have been legally counted or recounted, but to highlight the fact that the final result fell well within the margin of measurement error for Florida's vote-counting system. Whatever the legal consequences, we know that tens of thousands of Floridians went to the polls eight years ago intending to express a preference that was not measured accurately by the counting system.
Fast-forward to the "popular votes" cast so far in the 2008 Democratic primaries and we confront much bigger measurement issues. If we check the RealClearPolitics tally of the votes cast in the 41 "official" contests recognized by the Democratic National Committee that released popular vote counts, Barack Obama currently leads by 717,086 votes, or a margin of 2.6 percent of the more than 13 million votes cast.
That part is easy. Unfortunately, setting aside the run-of-the-mill counting errors that were on display in Florida in 2000, we confront several bigger counting issues:
- Four states (Iowa, Nevada, Maine and Washington) held caucuses but did not report a head count of the preferences of participants. They reported only the number of delegates elected to subsequent state conventions. How do we count the preferences of the voters that participated?
- RealClearPolitics publishes a count that includes popular vote estimates for the four caucus states. That count adds roughly 110,000 votes to Obama's total, presumably imputed from each candidate's share of the state delegates to the estimated turnout totals reported for each state (Obama-friendly DailyKos provides more detail on an estimate that has roughly the same bottom line). Of course, these estimates are inherently imprecise given the switching of preferences on the second round of caucus voting.
- In Washington, the turnout for the nonbinding primary was more than double that of the officially recognized caucus that chose the convention delegates. Using the primary rather than the caucus cuts Obama's lead by roughly 50,000 votes.
Which brings us back, of course, to Florida or Michigan, the states whose primaries have not been recognized as legitimate by the DNC. Setting aside for a moment the politically charged (and obviously ironic) question of whether those votes should be counted, consider the measurement problems in Michigan, where the ballot featured only Hillary Rodham Clinton and Dennis Kucinich. Jay Cost provides a concise summary:
"[We] could (a) give Obama the 'unaffiliated' vote, (b) not give Obama the 'unaffiliated' vote, or (c) reallocate the vote based upon whom voters claimed in the exit poll they would support if all candidates had been on the ballot."
For now, at least, Obama's popular vote advantage exceeds the margin of measurement error. He leads on all six counts posted by RealClearPolitics, including those that include unadjusted votes in Florida and Michigan. But before we start to ponder where these counts may end up after all of the contests are complete, we need to consider a bigger issue, which is less about accuracy than about what sort of counting is most appropriate. Or, to use the language of pollsters, the issue is not just about the accuracy of the vote count, but its validity.
I will take that question up next week.