As , let's take a look at how the growth of "cell-phone-only" households may be affecting the political polls we are all obsessing over.
To review, we know that more Americans than ever are living in households without traditional landline telephone service. According to the estimates compiled by the National Center for Health Statistics, by the last six months of 2007, 14.5 percent of all adults were living in households that could only be reached by a cell phone, a number that had more than tripled (from 4.4 percent) since 2004.
Younger Americans especially are out of reach. Mining the same NCHS study, the Pew Research Center reports that nearly a third (30.6 percent) of 18-to-24-year-olds can be reached only by cell phone.
This change worries pollsters because most political telephone surveys are conducted via landlines, not cell phones, and these missing cell-phone-only households create the potential for what pollsters call "coverage" bias.
The jargon may be unfamiliar, but the concept is simple. For a survey to provide a true random sampling of some larger population, every member of that population should have a chance of being selected to participate. When some members of the population go uncovered by the sample, a bias will creep into the results to the degree that two conditions are present: The people who are missed have to be significantly different from those not covered, and the missing segment has to be big enough to make a difference.
In 2004, exit polls told us that voters living in cell-phone-only households were different. They tended to be younger and more likely to support John Kerry. But these differences were not great, and the size of affected population was still relatively small. (Just 7 percent of voters were cell-phone only.) More important, the younger cell-only voters were not much different in their political views than younger landline-phone voters. So when pollsters weighted by age, the potential coverage error largely disappeared.
This year the share of cell-phone-only voters is obviously much larger, so the key question is whether the uncovered cell-phone-only voters are making different choices in the presidential race, especially when we look at the results by age. More specifically, are and running differently among younger voters reachable only by cell phone than among younger voters reachable by landline phones?
The good news is that, unlike in 2004, some of the pollsters doing national surveys have been gathering data on this issue. The Pew Research Center, Gallup and the CBS/New York Times partnership have been conducting some of their interviews by cell phone all year. They have been joined in this endeavor more recently by the NBC/Wall Street Journal, Time/SRBI and AP/GfK national polls.
Until this year, pollsters had been reluctant to interview voters by cell phone, due in large part to the legal and logistical hurdles. Federal regulations require that interviewers hand-dial each number, and pollsters must begin with a series of questions to try to screen out those cell-phone users who theoretically could be reached on their home phone. Costs for this process are higher, and the procedures to combine the cell phone and landline interviewers are still considered experimental. But because some of the larger national survey organizations have expended the time and money, we now have hard data on the otherwise mysterious cell-phone-only voters.
And what is that data telling us?
• The Pew Research Center looked at the three major political surveys they have conducted since June, comparing the results with cell-phone interviews included against the landline-only figures. They saw "a virtually identical pattern" on all three surveys: "In each case, including cell-phone interviews resulted in slightly more support for Obama and slightly less for McCain, a consistent difference of 2 to 3 points in the margin."
The biggest reason is a large difference they observed among registered voters under 30: Those in cell-phone-only households preferred Obama by a 35-percentage-point margin (62 percent to 27 percent), while Obama led by a much closer margin (52 percent to 39 percent) among those in households with wired phones.
• This past spring, Gallup managing editor Jeff Jones looked at six Gallup/USA Today surveys conducted during the first four months of 2008 and saw a similar pattern. Adding cell-phone interviews appeared to add roughly 2 points to Obama's total and take roughly 2 points from McCain.
I checked in with Jones this week to see if the pattern has held in the ongoing Gallup Daily tracking conducted over the course of the year. They are "still seeing the same pattern (crudely)," he wrote back, a "point or two more for McCain and a point or two less for Obama excluding cell-phone interviews." Jones hedged a bit, however, saying that they did not re-weight their wired-phone data (as Pew did) for this comparison.
• A few weeks ago, blogger Nate Silver compared the house effects of national polls that have been interviewing by cell phone with those that have not. He found that including cell phone interviews appeared to improve Obama's standing by 2 to 3 percentage points on the margin.
While each source is appropriately cautious, the pattern is consistent: Interviewing by landline phone only may cost the Obama-Biden ticket roughly 2-3 points on the margin.
This finding may not have much practical import for a campaign pondering one internal poll of 400 likely voters, where the "coverage" bias is still less than the built-in margin of error. On the other hand, those of us focused on poll averages and trend estimates need to consider that this one issue -- cell phone coverage -- may be adding a few points to McCain's numbers.
Of course, as noted here last week, the cell-phone issue is just one of three giving pollsters heartburn. We will take up the other two in the coming weeks.