• by Jack Loechner , Staff Writer @mp_research, July 11, 2007

Phone Surveys Skewed by Cell-Only Homeowners

According to government statistics released last month, reports a recent PEWresearch study, 12.8% of U.S. households cannot now be reached by the typical telephone survey because they have only a cell phone and no landline telephone. Twenty years ago the survey research profession worried mostly about the roughly 7% of U.S. households that could not be interviewed because they had no telephone.

If people who can only be reached by cell phone were just like those with landlines, their absence from surveys would not create a problem for polling. But cell-only adults are very different.

The National Health Interview Survey found them to be much younger, more likely to be African American or Hispanic, less likely to be married, and less likely to be a homeowner than adults with landline telephones. These demographic characteristics are correlated with a wide range of social and political behaviors.

In early 2003, just 3.2% of households were cell-only. By the fall of 2004, pollsters and journalists were openly worrying about the potential bias that cell-only households might create for political surveys. The National Election Pool's exit poll found that 7.1% of those who voted on Election Day had only a cell phone, and these cell-only voters were somewhat more Democratic and liberal than those who said they had a landline telephone.

Given the speed with which the number of cell-only households has increased, there is growing concern within the polling business about how long the landline telephone survey will remain a viable data collection tool, at least by itself. At the annual meeting of the American Association for Public Opinion Research, a government researcher told the audience that the size of the cell-only group could approach 25% by the end of 2008 if the current rate of increase is sustained.

The Pew Research Center conducted four studies that included samples of cell phone numbers as well as landline numbers. The surveys covered a very wide range of topics, including use of technology, media consumption, political and social attitudes, and electoral engagement. Comparing the cell-only respondents with those reached on landlines allowed an assessment of the degree to which our traditional surveys are biased by the absence of the cell-only respondents.

The good news, says the report, is that none of the measures would change by more than 2 percentage points when the cell-only respondents were blended into the landline sample and weighted according to U.S. Census parameters on basic demographic characteristics.

However, while the cell-only problem is currently not biasing polls based on the entire population, it may very well be damaging estimates for certain subgroups in which the use of only a cell phone is more common. According to the most recent government estimate, more than 25% of those under age 30 use only a cell phone. An analysis of young people ages 18-25 in one of the Pew polls found that the exclusion of the cell-only respondents resulted in significantly lower estimates of this age group's approval of alcohol consumption and marijuana use.

Perhaps, opines the report, excluding the cell-only respondents also yields lower estimates of technological sophistication. For example, the overall estimate for the proportion of 18-25 year olds using social networking sites is 57% when the cell-only sample is blended with the landline sample, while the estimate based only on the landline sample is 50%.

Including a cell-only sample with a traditional landline-based poll is feasible, but cell-only surveys are considerably more difficult and expensive to conduct than landline surveys.

• Federal law prohibits the use of automated dialing devices when calling cell phones, so each number in the cell phone sample must be dialed manually.
• It also is common practice to provide respondents with a small monetary incentive to offset the cost of the airtime used during the interview.
• The screening necessary to reach cell-only respondents among all of those reached on a cell phone greatly increases the effort needed to complete a given number of interviews.
• Pew estimates that interviewing a cell-only respondent costs approximately four to five times as much as a landline respondent.

Pollsters recognize that some type of accommodation for the cell-only population will have to be made eventually.

For more from PEW Research, please visit here.