Second Thoughts On Set-Top-Box Measurement

• Comment (1)
• Recommend (4)
• Tweet
• 

For more than half a decade, many in the TV industry have looked forward to set-top-box measurement as an elixir that would deliver better TV ratings. The theory is that if Nielsen or another ratings company could extract viewing information from the millions of set-top boxes already installed in viewers’ homes, the larger sample size would deliver more accurate -- or at least more stable -- ratings.

To that end, in 2007 Nielsen created a separate business unit (then called Nielsen Digital Plus) to build a set-top-box business.  Rentrak developed its own set-top box enterprise, which has attracted many TV station clients.  And set-top-box measurement was given as one of the main rationales for creating The Coalition for Innovative Media Measurement (CIMM).

I don’t want to rain on anyone’s parade, but I don’t think we are going to get true “currency” measurement from set-top boxes for a long time -- if ever. Set-top boxes can be a valuable analytical tool, but most of the barriers to set-top-box currency ratings that existed five years ago still remain and don’t seem to be coming down very fast. 

The methodological objections to set-top box measurement are well-known:

1) There are no set-top boxes in homes without cable, which effectively disenfranchises many low-income and minority homes; 2) in many homes, the set-top box stays on after the TV has been turned off, so viewing would be counted when no one is watching; 3) there is no way to tell WHO in the house is watching even when the television is turned on (i.e., no demographics); 4) many homes have multiple TVs but only one set-top box, which means viewing in kids’ rooms and master bedrooms would not be captured; 5) most set-top boxes don’t have the capability to report data back to a measurement company (this  problem is especially true in satellite homes, which would require a connection to a phone line to report back data).

There’s also the practical fact that set-top boxes are not designed to measure TV viewing.  That’s not their job, nor is it the main focus of the cable and satellite companies who own them.  With only a limited amount of information that can flow through cables, many cable companies have made delivering more channels a priority over capturing data. So to obtain viewing information, a measurement company has to jury-rig a reporting system that works with some boxes and not with others. 

What this boils down to is that a set-top box sample may be larger, but wouldn’t be more accurate, because it’s not representative of the entire population. By definition, it would only include homes with cable -- and would furthermore only include boxes capable of reporting back to a measurement company.

Having said all that, I am confident Nielsen can develop a hybrid measurement that combines numbers from both its People Meters and a set-top-box sample.  Of course, that would require some fancy modeling to cover the gaps and address the issues discussed above.  (As an aside, I pity the client services people at Nielsen if the company ever adopts a currency that is based on a model that only stat geniuses can understand.  Nielsen is already accused of using a “secret sauce” to massage the People Meter numbers. If Nielsen adopted modeling, it really would be using a “black box” into which raw numbers are poured, and from which a completely new rating emerges.  This model would probably be blamed every time a network had a ratings decline.)

The real question is whether this hybrid measurement would be better.  Theoretically, a bigger sample might provide more stability for the smaller national networks whose ratings tend to bounce around a lot (if your network only averages a rating of 0.2, this number can fluctuate significantly on any given day because the number of People Meter homes that tune in that network is pretty small.) Then too, set-top boxes could provide some kind of measurement for so-called “long-tail” networks audiences so small they don’t even generate a 0.1 rating.  

But unmeasured networks only constitute about 6% of total viewing, and you have to wonder if the other national networks would be better off with the significant change that a hybrid measurement would entail.  Maybe I drank the Kool-Aid when I worked at Nielsen, but I think ratings for national networks are more accurate when they are drawn from a good, well-maintained panel than from an unrepresentative, albeit larger, sample.

For all these reasons, Nielsen and its largest clients generally agree that set-top data can be useful for analytical purposes, but that the People Meter sample will remain the primary tool for collecting ratings on a national basis.  As a result, little, if any effort is being put into integrating set-top boxes into a national ratings system.

The situation is different at the local level, where paper diaries are still being used to measure local TV stations.  In my next column, I will look at efforts to use set-top-box data in local markets.

• Comment (1)
• Recommend (4)
• Tweet
• 

1 comment on "Second Thoughts On Set-Top-Box Measurement ".

1. Bruce Goerlich from Rentrak
   

   commented on: December 7, 2011 at 6:58 p.m.
   The marketplace has said yes to the use of return path data. Today, 39 TV nets, 7 of the top 13 media agencies, and over 100 TV stations are Rentrak clients. We have 3rd party financial processing feeds with Strata, One Domain and Donovan which enable buys. We have data from more than 19 million TVs in 8 million HHs (coming from cable, telephone [telco] and satellite providers). (Not only cable provides return path data; satellite providers have both phone and Internet feedsand AT&T and Verizon are not inconsiderable TV businesses.) What we also provide: Stability. Millions of HHs and return path data give ratings that are stable, not only for smaller networks (which we see at 14% of hours) and programs but also at the level of granularity that advertisers want. We produce national TV Exact Commercial RatingsTM that allow advertisers to seefor the first timestable and accurate numbers, enabling excellent stewardship, value pricing and minimal ADUs benefits for all sides of the industry. Our stability also goes into local markets, where quarter-hour posting is the norm. Our stability extends to household demographics, where one can buy a highly targeted network like CNBC and expect to achieve the post on HHI $125K+. Projectability. Rentrak collects viewing from 3 of the 4 TV strata, satellite, cable and telco. Yes, we do model for Over the Air households, but our process is not a black box. We weight at the zip code level for Hispanic, African-American, presence-of-children and geographic distribution within the DMA. We have 10,080 (and soon to be more) probability curves to account for the TV set being off. We have horizontal weights to count for those of our operators that do not have a separate return path for each TV.(Some operators do have return path data for every TV.) Our processes are undergoing MRC accreditation. And, reluctantly, we must ask the writer of the blog to think about the trade-offs the sample currency has to make. Where are they with MRC status? Can a sample with heavily stratified selection truly be called “random”? Can samples with low response rates truly be called “representative”? And may changes in a complicated Hispanic weighting schema be causing a reported drop in children’s viewing? Advanced Targeting. Because we have millions of data points, we can integrate data sources such as Polk, our box office data, and direct marketing info to provide stable and projectable ratings that go beyond age and sex. It’s true that we cannot say who is in front of a TV set. However, we can say, “This home has a $250,000+ income and you can expect to see a similar rating for that show tomorrow.” We can say, “This HH contains strong liberals.” These are only a couple examples, but the real question is, “What would an advertiser rather know? What the relevant buying behavior of the HHs watching are for her product category, or just how many A18-49 eyeballs are watching?” The TV marketplace now has multiple currencies.

Leave a Comment