• by Frank S. Foster , March 31, 2008

While lacking perhaps the historical significance of the transition between Charles VI and Charles the VII in 15th century France, the U.S. television audience measurement industry is nonetheless bound by the same fate, I fear -- uninterrupted succession. The demise of the small sample, the 60-year-old foundation upon which television audience ratings were built, has been all but guaranteed by Comcast's and Time Warner's support of CableLabs' Canoe project.

Don't get me wrong, I am not complaining. Eras come and go and this one, like Daly's Chicago and the Long brothers' Louisiana, has run its course. My complaint is not with the Sample. No, my complaint lies with its evil progeny, Sample II.

Sample II? Ah yes, some explanation may be in order. Sample II is an utterly unimaginative name for an equally uninspired concept that combines perfectly good set-top-box data with commonly biased, error-prone sample data. In search of demographic ratings, the process often renders high quality set-top box data nearly useless. But who would do such a thing? No one -- certainly no one with an eye toward accuracy, quality or insight -- would put their name on such a plan.

Alas, I have been proven wrong. I was recently approached by several companies who asked for my opinion of a Sample II methodology. It seems that a certain television audience measurement company is "floating" the idea of using its much-maligned panels as a source for demographic attributes when dealing with set-top-box data. Should they decide to proceed there will be no vote and very little vetting if the LPM, AP meter and demise of the PPM are any indication. My expressed opinion was a question: "Have we not had enough of the tyranny already?" My goodness.

Why must the successor to the small, opt-in sampling process be chosen so quickly? Historical perspective would suggest that rash decisions are rarely viewed among the best decisions. With hundreds of billions of dollars at stake on the East and West Coasts (not to mention the rest of the country) what is the hurry? My advice, albeit unsolicited, is simple, straightforward and ,dare I say, logical. If undertaken, the following initiatives might allow the entire television industry to move forward while the media buyers and sellers grapple with new technologies and the desire for accountability:

a) The MRC should introduce a new level of accreditation that would require confidence intervals and error margins accompany all published television ratings and associated research (a long-time detractor of set-top box based research, surely Nielsen would embrace a requirement that would openly disclose the accuracy of all television audience research).
b) The ARF should sponsor a research competition focused on Inferential Statistics, where researchers try to reach conclusions that extend beyond the immediate data alone. Instead of assuming that race, age or sex is the reason people watch television, a researcher might study whether or not a characteristic drives viewing of a particular genre, for instance.
c) The broadcast television networks should jointly sponsor a study to analyze various targeting technologies for addressable advertising. Multiple research organizations could analyze the same data set and compete based on their analysis of targeting techniques.
d) Cable operators Time Warner and Comcast should sponsor a competition to develop a next-generation, local television report fueled by set-top-box data, complete with ratings and new metrics for all ad-supported networks and local stations.

The biggest problem with the current king of audience research is that everything is so damned secret. How accurate are the ratings? Go read the research supplement, if you can get one. (Spoiler Alert!Nielsen ignores all major biases and sources of error.) How well is the AP meter working? Shhh... only the MRC knows, and they are not talking. How different were the ratings for markets that ran both the LPM and set-meter-diary panels side by side? Did the ratings even point to the same overall distribution of viewing? Nielsen neglected to publish that information. Since data has been produced from set-top boxes since 2000 (e.g. TargetTV), why has no one ever contrasted the television ratings from set-top-box data with the local Nielsen ratings in any market large or small? Seems odd. Maybe someone should ask Jed Meyer at Nielsen's DigitalPlus group to publish such a comparison.

The bottom line is simple. The only party who profits from a quick and uninterrupted succession is the king himself. Up until now, the king has been unwilling to let go of the sample because not only is it his cash cow; it is protectable and proprietary. A quick transition to Sample II would allow Nielsen to keep its "gold standard" panel nearly intact and maintain its stranglehold on the market.

I know it sounds ominous, but a solution inspired by the French exists. Storm the Bastille! If we simply admit that the king is no longer working for the good of the people, we can toss the family lineage in favor of a more righteous king. Or better yet, we could conduct a search for a new research company who will work without outrageously expensive, long term, staggered contracts. After all, recent events point to the obvious; the current king is only in it for the money. And last I checked, he's not dead yet -- he just wants to go public .