• by John Spiropoulos , February 23, 2006

We keep believing it. If Nielsen reports it, the data must be right. That's the mantra sung by more than a few in the industry and press when the TV ratings company finally began measuring and reporting households with digital video recorders. Two days of measurement and "winners" and "losers" regarding DVR usage were announced. Stop. Can you really ascertain winners and losers after just two days?

The problem is simple. Nielsen has been so behind the consumer marketplace with DVR measurement it had to move forward with something. And we now have something. Let's be grateful for this small miracle--if only for the reason that it distracts from the headache of transitioning from a single data stream to multiple. With neither the DVR measurement issue nor the multiple data stream transition complete, it's shaping up to be an interesting Upfront this year.

But there's a bigger picture here than just DVR measurement. DVRs are yesterday's technology. Time shifting is so last century. The real question is, how can any company trying to measure television keep up with the changing consumer technology landscape? What about other technologies like broadband and mobile, which do not yet have enough content or consumer usage to economically justify large-scale measurement, but require measurement all the same? These questions are dark clouds closer to us than the horizon.

Each small step forward by Nielsen only reinforces concerns about the limitations of today's measurement. As an industry, we have two major hurdles to overcome:

The first hurdle is sample research versus census-data collection.

Can sample-based research really support the world of tomorrow? It will be tricky if the industry continues to focus on "average audience" statistics. How can any sample reasonably measure the difference in a video-on-demand offering watched by 0.003 percent of the population versus 0.01 (a difference of roughly 15,000 adults). There's no way any sample can offer the accountability needed. But that's the world we'll live in. Possible alternatives include: moving to census-based research, measuring cumulative audiences that counteract some sample size issues by looking at usage over time, or accepting that a rating below 0.2 is insignificant and leaving it at that. No option is a slam dunk. The acceptance option makes the most sense, but the industry endlessly wants to measure ever smaller.

The second hurdle is active versus passive measurement.

Active measurement devices, like Nielsen's People Meter technology, are not a long-term solution. Passive devices, like Arbitron's relatively new PPM, have more promise. The major difference between active and passive devices is how the respondent must interact with the measurement tool. The more active a respondent must be with the measurement device, the more implicit measurement bias that exists in the data. Bias, of any kind, negatively impacts data integrity.

The short-term view of this is simple. We're using increasingly obsolete data. To measure audience changes of even 100,000 people (or less than 0.03 percent of the population) across the U.S. is unnecessary and irrelevant considering the big picture. The industry must decide what's truly important: hyper-detailed exposure information or consumer attitudes through the many facets of viewer engagement.

As we move to a world of smaller and smaller audiences, we must embrace the mental leap from quantity to quality. We must first get a clear understanding of all consumer interaction that constitutes engagement (exposure being just one). Microscopic exposure detail is a fool's errand.