• by Josh Chasin , Op-Ed Contributor, January 20, 2009

I grew up in the field of audience measurement.  I was working in the Statistical Services department at Arbitron in 1980, before I was out of college; I would take the subway uptown from Washington Square wearing a shirt and tie under my leather jacket. In fact I spent the first seven years of my professional career in that department, working with and learning from some of the all-stars and leading lights of media research. I remember making a suggestion one time, and my boss Mike Occhiogrosso smiling at me, impressed, and saying, "We'll make a sampling statistician out of you yet." While at Arbitron I even had a chance to work with Hugh Malcolm Beville, the first executive director of what is now the MRC, and the man who, literally,  wrote the book on audience measurement.

I'm indulging in this little trip down memory lane to make the point that I had a pretty good bottom-up schooling in the discipline of audience measurement; a practical education.  I want to make this point, because over the past five years I've had to unlearn everything I knew about audience measurement.  That may be overly dramatic, but I have come to view the discipline I grew up with as "analog" audience measurement, whereas what we need to deploy today is digital, or 21st century, audience measurement (I'm resisting the temptation to call it "audience measurement 2.0.")  Dr. Cheryl Harris and I presented a paper at the ARF's first Audience Measurement Symposium in 2006, titled "The Effect of Technological Innovation on Media Exposure Tracking: In Search of the New Traditional."  Our hypothesis was that changes in the technological landscape were rendering traditional measurement methods and techniques obsolete, but that solutions -- the "new traditional" -- would likewise emerge from the application of technology.

This paper pre-dated my association with comScore. But it helped convince the folks at comScore that I was the guy for them.

One of the bedrock assumptions of the analog era of audience measurement was that high quality audience measurement began with a Random Digit Dial (RDD) frame. Sure, 2% or so of American households lacked phone service, but that was a bias we were all prepared to live with.  And back when I started at Arbitron, if you were doing a telephone survey and you weren't getting a response rate of 70% or more, you were doing something wrong.  So RDD, with near-complete universe coverage in the frame, and robust response rates, was the path to projectable and representative studies.

We all know what's happened to survey response rates since then.  And we know, too, that the cell phone is rapidly making the landline, the "house phone," a thing of the past.  (One change as we become a cell phone nation is that the telephone number is increasingly associated with a person, not a household; if you want to reach me you call my cell, and if you want my wife you call hers. Using telephones to sample households as opposed to persons will become increasingly difficult.)

The CDC's National Health Interview Survey has become a highly visible source in the research community for tracking the incidence of what they call wireless-only, and what media researchers generally call cell phone-only, households.  On Dec. 17, their latest findings were released, covering the first six months of 2008. According to the NHIS, 17.5% of US households were wireless only.  To put that into perspective, just three years prior, in the same study, only 7.3% of US households were wireless-only.  

When we look at target demographics, the cell-only situation becomes even more dramatic.  Fully 21.6% of US Hispanics live in cell-only households.  And consider this: 31.4% of 18-24 year-olds live in cell-only households, as do 35.7%-- over a third!-of 25-29 year-olds. Now, this might be a function of age and life stage, but it might also be a generational phenomenon.  In other words, I don't expect all those 18-29 year-olds to wake up one day, say, "Hey, I'm 40, I have a house in the suburbs-time to get a land line!"  No.  These kids are as likely to get a landline as they are to start buying vinyl LPs.

Why is this a research issue?  Well, most RDD is done at calling centers, using auto-dial systems that automatically place hundreds of calls, handing off the call to a live interviewer when someone answers.  But it is against the law to auto-dial cell phones.  So RDD sampling systematically and by design excludes cell phone exchanges.  In order to sample cell phones, a human has to manually dial the number, rendering the process several times more costly than RDD dialing of land lines.  

In May of 2008, the Media Rating Council (MRC) issued a statement that "Accreditation will be at risk for any service affected by this issue that does not have a known solution and timeline approved by MRC to effectively measure cell phone only households."

But I think the issue is even more complex than cell-only. Because there are more than two flavors of households (those with landlines, and those with cell phones only.)  In their recent release, the NHIS reports that another 13.3% of US households have landlines, but "received all or almost all their calls" via cell phone. At comScore we've been using the term "cell phone primary" to describe this type of household.

ComScore moved away from the "analog age" recruitment technique of RDD some time ago, and we recruit our panelists exclusively online (save for our "Calibration sample," a control sample that is recruited randomly and offline, but not via RDD.). This has many benefits, but for today's purposes, it allows us to assure that we represent persons from all phone-status households.  A year ago we did a study of our U.S. panel, in order to understand both phone status composition of the panel, and how Web usage might vary by phone status.  We found that 19% of our panelists were cell-only, and that another 23% were cell-primary; in other words, if we'd relied on RDD we would have totally excluded 19% of the panel, and dramatically under-represented another 23%.   

More important, we found that Internet usage behavior varied markedly based on phone status.  Cell-only persons were heavier users of rich media entertainment content, games, and conversational media; cell-primary were heavier users of travel, business/finance, and other service-oriented content.

The macro lesson here is clear: that we can no longer expect representative samples in media research to accrue from RDD.  The bigger lesson, and perhaps the more difficult one to assimilate, is that we as an industry need to revisit all the bedrock assumptions we grew up with -- especially those of you as old as me.