• by Frank S. Foster , August 4, 2008

"Ninety percent of all statistics can be made to say anything -- fifty percent of the time," words of wisdom lifted from a television service provider's recent commercial.  I reviewed the advertisement on my DVR to insure accuracy.  If I had not seen the ad for myself, I might have thought the commercial was referring to the Nielsen Company and their television audience measurement group.  Alas, it was a DirecTV parody. 

It hasn't always been this way.  Arthur C. Nielsen was a researcher.  There was a time when Nielsen developed ground-breaking technology and innovative approaches to difficult problems.  But times have changed.  Today, the company is a marketing information juggernaut and in its mad search for profits and efficiency, the television audience measurement group has lost its way.  With nothing more than a TV Guide and a pencil at her disposal, I would argue that my Aunt Everdine could be as accurate.  This is where I will lose a number of readers.  But if you bear with me for a moment, I am leading you somewhere.

 I can hear the Nielsen luddites cry, "The MRC has certified Nielsen's services!" To which the appropriate response should be, "Yep, but haven't they certified diaries as well?"  Query George Ivie.  Certification has very little to do with accuracy.  Case in point, the local people meter.  After a somewhat rocky start in Boston, the local people meter technology has been rolled out with great fanfare to most of the top 15 markets.  Despite the fact that transitional data obtained from simultaneous set meter/diary and LPM panels often provide radically different ratings, there appears to be little resistance to the technology's march.  Current plans have LPM slated for deployment down to Fort Myers, market 63 for those who are wondering. 

But the lack of accuracy is an enormous problem.  Using last night's fast national numbers as a guide, FOX was number one in A18-49 with a 1.8 rating (http://www.thefutoncritic.com/ratings.aspx?id=sunday).  According to the CAB Web site, Nielsen has deployed an 800-household LPM panel in New York, the country's largest television market.  In that panel you would expect somewhere in the neighborhood of 925 - 930 people in the A18-49 category.  If New York's panel were comparable to the national people meter panel, a 1.8 rating would translate into roughly 16 panel members aged A18-49 watching the FOX affiliate, WNYW.  Using sample size error calculations alone, the error is 25%.  So the rating is no better than 1.8 +/- .45. 

Which numbers inspire more confidence:

a)      Nielsen reports in New York last night that WNYW secured a 1.8 rating in A18-49 for a total of 154,000 young adult viewers. 

b)      In New York last night, Nielsen estimates that somewhere between 115,000 and 193,000 adults aged 18 to 49 tuned into WNYW during prime time,  making it perhaps the most watched broadcast station.

There are more than 60 ad-supported television stations and locally insertable cable networks in the New York market.  Last night, the ratings for A18-49 for all those networks were subject to sample size errors larger than 25%, and the error associated with the vast majority were much larger.  For markets with panels smaller than New York's, the situation is worse still.  Keep in mind that we have ignored every other source of error and bias.  Were we to take the exercise further, accuracy could only drop.

 Can you blame Nielsen for ignoring the error when it reports ratings?  Not really.  Nielsen is a privately held business with aspirations of going public.  Nielsen is intent on driving LPM technology into the top 60 markets because it is a more profitable business model.  LPM data can be used nationally as well as locally.  Nielsen can double-dip and charge more to both local and national customers. 

 So, for those who have followed me to this juncture, here is my point.  The blame for the current situation does not lie solely with Nielsen, but also with the governing body created to keep Nielsen in check -- the MRC.  If the MRC required that certified services openly disclose the accuracy of ratings, does anyone think it would make fiscal sense to switch to LPM technology?  An MRC "certification" that a researcher is following his own processes means little if the ratings are inaccurate.

 With new forms of distribution, (telco TV, terrestrial digital, broadband video) and new technologies (addressable advertising, ad-supported video on demand, interactive television) the landscape for audience measurement is about to be radically transformed.  What is the MRC doing to prevent a total collapse of the local television business model?  The future of television may be at stake.