• by Frank S. Foster , September 2, 2008

A friend and I were having lunch the other day in New York when a mutual acquaintance stopped by our table.  She grabbed my companion by the arm and dropped the Council for Research Excellence's set-top box data RFP on the table.  

"Can I get your answer today?" she asked politely.
"I think you already have it," he answered.
"But it is such a great opportunity and I so want you and I to do it together," she said.
"Thank you for thinking of me, but I'm not really interested." he replied.
"The industry is embracing set-top box data -- I thought you would be thrilled," she lamented, outwardly disappointed by his reaction.  He shrugged his shoulders and she moved on.
When she was out of earshot, my friend picked up the abandoned request for proposals document and said to me, "Why would anyone who knows anything about set-top box data participate in this sham?"

Given an opportunity to showcase his knowledge and expertise in this area, one would expect this particular gentleman to be excited by the prospect -- yet he has no interest in participating.  More importantly, one of the brightest people working with set-top box data today will not be presenting a proposal which makes this an industry problem.  

Why is he not embracing the opportunity?  There are several reasons, but two in particular come to mind.  First, the proposal appears to be a poorly conceived competitive analysis project for Nielsen Media Research -- which is just plain "icky," as my son would say.  Is there a more devious way to get competitive information than by hiring an outsider to research an "industry initiative"?  Nielsen has had people working on set-top box data since the late 1990s, so why do they need to be the sole owners of work commissioned by the industry?

Secondly, if you were to ask my friend, TNS, Rentrak, Google and anyone else hoping to crack the market would be foolish to work with anyone representing the CRE on this initiative.  But there are those who have built relationships who could get such information, if unofficially.  I suppose my friend could answer some of the questions based on what he has learned over the years.  He might choose to further leverage his contacts to obtain insight and information from Nielsen's competitors as well.  For the project he might produce an analysis of TNS' approach to problems regarding error analysis, projection to non-cable households or channel-surfing thresholds. 

All great stuff for the industry, but what happens when the report is turned over to Nielsen?  There are some who would suggest that Nielsen would not be above using the information to discredit TNS' business practices.  I would suggest that is precisely why the company commissioned the study.  Delay, obfuscate and deny: Nielsen's go-to market strategy.  What happens to my friend, then?  Could he go work for TNS?  How about Google or Rentrak?  I would propose he might obtain pariah status: not a good thing for him, or the industry.

Which brings us around to the core problem: if the Council for Research Excellence has it wrong, what is the right way to get smart people to work on television audience research?  If the industry truly desires input from the best minds in the industry so that they may identify the potential research value of set-top box data, it should go after the problem in a different way.

1.    Create a level playing field at the national level. If the industry wants smart, entrepreneurial people to jump aboard the cause, it must at least create the illusion of a level playing field.  There is no disputing that Nielsen owns more than 99% of the television audience measurement business.  If the industry wants better research, it should slice off 10% of the market for anyone other than Nielsen.  Let outside companies compete for that very small slice of the pie.  If the industry cannot create a budget for the project, it  could at least deduct the money from Nielsen invoices, citing "shoddy research practices."  How could Nielsen argue?  Create opportunities for VOD, DVR and broadband video data analysis.  Make companies feel they can compete.

2.    Showcase research, not marketing. Force those companies who participate to disclose margins of error and confidence intervals.  While they are at it, a thorough bias analysis should be included as well.  Quality research does not hide those figures in supplemental publications that come out once a year.

3.    Embrace demographics, psychographics and data mashups. Assigning small panel demographics to set-top box data is one way to approach the problem of demographic attribution, but it introduces both error and bias into the analysis.  There are other solutions, and they should be explored.  Ask participants to provide alternative approaches to demographic and psychographic ascription.

4.    Focus on local market problems and set a date for a showdown. At the end of the day, to get investment for research in the space, the market has to be open.  If the industry were to create a local market showdown between the current state-of-the-art technology (LPM) and competitors, investment would not be an issue.  An LPM market should be selected where the infrastructure for set-top box data collection is in place.  The date should be within a reasonable timeframe.  My choice would be Tampa, Fla. at the start of the 2010/2011 television season.  Advertisers would be interested in the results and it would give the industry two years to narrow the competitors.  At the end of the test period, the industry would have a side-by-side comparison of set-top box based ratings and small panel based ratings.  Let's see how Nielsen stacks up.    

At the end of the day, I think everyone wants better audience estimates.  With the deployment of digital video recorders rising daily, the availability of on-demand content growing every week and digital broadcast on the horizon, something must be done.  To leave it all in the hands of Nielsen is more than a little scary.