IE 11 is not supported. For an optimal experience visit our site on another browser.

Cellphones challenge poll sampling

With more American households giving up their old-fashioned land lines and using cellphones for all calls, public opinion researchers are facing a challenge of how to make sure they are getting representative samples when conducting polls.
/ Source: The New York Times

With more American households giving up their old-fashioned land lines and using cellphones for all calls, public opinion researchers are facing a challenge of how to make sure they are getting representative samples when conducting polls.

Since the 1970s, pollsters have relied on sampling techniques that depend on talking with people on their home land line telephones. For the most part, the polls sample the public by randomly dialing telephone numbers in every region from a list of area codes and exchanges known to be residences. The sample is weighted to the results of the latest census.

But cellphones are not geographically based, forcing pollsters to adjust their methods. In addition, a land line often represents a household and a cellphone often represents an individual.

Pollsters say they are also concerned about low response rates among people reached on cellphones. Because wireless carriers charge customers by the minute, people may be less likely to agree to complete lengthy cellphone surveys.

The survey industry is exploring reimbursing respondents for minutes used.

Researchers using computers to dial may encounter legal complications. The Federal Communications Commission requires an interviewer to dial the number when calling a cellphone. No autodialers are allowed.

Survey researches have dealt with sampling and low response rates before. But cellphones bring up a new concern, safety. Calling someone driving or engaged in another activity that requires concentration raises ethics and liability questions.

Bloggers and media critics have been questioning pollsters for months about whether 2008 polls are truly representative without including cellphone-only households.

The issue came up in 2004, but cellphone-only households in 2003 were 3 percent of the total. They now run 16 percent, according to Mediamark Research.

The F.C.C. estimates that more than 60 percent of households have at least one mobile phone.

The demographic groups that tend to be cellphone-only households are also historically less likely to vote, reducing the effects of underrepresentation in pre-election polls.

According to data from the Centers for Disease Control and Prevention’s National Health Interview Survey, adults with cellphones and no land lines are more likely to be young — half of exclusively wireless users are younger than 30 — male, Hispanic, living in poverty, renting a residence and living in metropolitan regions.

The Pew Research Center conducted four studies last year on the differences between cellphone and land line respondents. The studies said the differences were not significant enough to influence surveys properly weighted to census data. With the increase in cellphone-only households, that may not be the case next year. Researchers, including the New York Times/CBS News poll will test that by incorporating cellphones in samples.

The estimates in the Health Interview Study suggest that cellphone-only households are steadily increasing.

“If the percentage of adults living in cell-only households continues to grow at the rate it has been growing for the past four years, I have projected that it will exceed 25 percent by the end of 2008,” Stephen J. Blumberg, a senior scientist at the National Center for Health Statistics, wrote in an e-mail message.

The American Association for Public Opinion Research has been examining the question and formed a group to study it. The association says it will issue its report early next year.

Paul J. Lavrakas, a survey methodologist and a former professor at Northwestern and Ohio State, has been a driving force behind the research at the association. Mr. Lavrakas said that he could not “imagine how anyone can feel safe in planning their election coverage without including cellphone sampling for the 2008 election.”

He added that much will be learned next year as researchers experiment with calling cellphones and measure the accuracy of their polls.

The New York Times/CBS News Poll has been addressing the question. Kathleen A. Frankovic, director of surveys at CBS News, said that without intending to dial cellphones, its interviewers “already complete as many as 2 percent of our interviews per poll on cellphones, and we are experimenting this fall with dialing cellphone samples.”

The industry generally agrees that a truly representative sample should include cellphone-only households, land-line-only households and mixed households. Many pollsters are working on assembling such samples.

“Until Internet polling gets a decent sampling frame, telephone surveys are necessary, and we can’t exclude cellphones from telephone polling,” said Martin Frankel, a professor of statistics at Baruch College.