• by Mitch Oscar , Op-Ed Contributor, May 27, 2008

At the last Chicago Carat Exchange (April 11) one panel discussion focused on set-top-box and viewing data. Participants included (alphabetically by company): EVAD Consulting's Frank Foster, Nielsen's Jeff Boehme, Rentrak's Cathy Hetzel, and TNS Media's Bud Breheney. As a follow-up to the roundtable, we fielded five questions to the participants. The following are their uncensored responses in the order of receivership:

 

QUESTION 1: VIEWERSHIP VS. TUNING/SET-TOP-BOX DATA

Is there a significant difference between tuning/set-top-box data (non-people-centric) and traditional viewership data, which offers demographic insights?

 

Cathy Hetzel (Rentrak):

Top 15-20 programs (mostly broadcast) are very similar, but the difference in viewership is significant for cable networks. Rentrak believes that due to the fact that we are collecting data from millions of set-top boxes (vs. a small sample of "people") that using a combination of geo-demographics, segmentation data and usage information results in much more business intelligence about the audience that can be used to target advertising. We have been using these tactics in our other applications for many years.

 

Bud Breheney (TNS Media):

The main advantages of Return Path Data are the robust sample size, full channel lineup reporting, second by second data and the related benefits these features provide.  In all of TNS work in the US, we are dealing with a census approach or a sample of at least 100,000 HH.  We report on the entire channel lineup and provide our customers the ability to perform second by second analysis.  Another key advantage our customers see with our product is the integration of commercial occurrence data from TNS Media Intelligence.  This allows customers to mine the data whether it is the Charter LA sample or the soon to be released national panel with DirecTV called DIRECTView.

In addition, TNS has announced plans to offer household demographics in DIRECTView through an opt-in panel capability in Phase 2.    Demographics are not solely a panel based metric.

 

Frank Foster (EVAD Consulting):

In a traditional (non-peoplemeter) local panel, tuning data and set top box data are similar.  A tuner meter generally "samples" or captures what channel a television is tuned to at a given time.  This is generally done several times during a 30 second period and the channel that is most often captured "wins" the period.  The winner of the 30 second period is then noted by the meter and that information is delivered to the researcher.  In this way, tuner meter data provides an estimate of a specific television's tuning behavior.  In a modern set-top box, every state change is captured by the software in the set-top box.  The operator has the ability to ignore channel changes where the tuning was of insufficient length to warrant attention, but technically most data collection services could be configured to provide a perfect representation of viewing behavior including channel surfing habits. When the television is left on the same channel for an extended period of time, the data sets will tell the same story.  However, if I client were interested in how audiences reacted to particular commercials or even events within a commercial, the tuner meter data is too imprecise to shed much light on the issue.  Set top box data often proves more valuable in this scenario.

 

The remaining differences between the two data sets relate to timing and sample size.  Tuner meter data can be based on an internal clock or on a code embedded in the broadcast signal as is the case with the A/P meter.  The internal clock may be a source of error due to drift issues, but as long as the code can be accurately identified, the A/P meter should alleviate many of those problems.  Set top box data provides the researcher with the opportunity for very precise tuning data regarding live and delayed tuning as well as recording events.  Latency can be an issue, but generally speaking the error associated with latency is static in nature. 

 

Jeff Boehme (Nielsen):

Yes.

Nielsen is in both the digital STB data and panel measurement businesses, and we have been carefully analyzing both datasets for some time. Very clearly, there are significant differences between STB tuning and traditional viewing data. We know this because we have the unique capability of developing 'common home' metrics, comparing results from TV sets and STBs from the same home.

 

We have documented clear and substantial tuning differences between the STB and the TV set, and two examples are device "on/off" and non-STB tuning. One of the most significant differences can be attributed to "on/off," when the STB is not turned off in conjunction with the TV. On average, our studies have shown that more than 30% of the STBs are not turned off and about 10% are never turned off.

 

In addition, many TV sets may not be connected to a STB or a backchannel (data retrieval system). In those cases, no records are available so any tuning could not be counted. Most apparent, however, is the absence of demographics from STBs, as the device captures no information about who is tuning for any given time period.

The comparison of actual STB tuning and panel-based-TV set viewing usage gives us a comprehensive source of data to create robust solutions for these issues. Based on our analytics, we are not only able to normalize these data, but integrate it across multiple consumer datasets for greater insights and value.

 

 

QUESTION 2: LATENCY

If advertisers or agencies are truly interested in commercial ratings is it technically feasible for research companies to provide the accuracy necessary to deliver second by second analysis given the complexity of cable and satellite networks and consumer electronics appliances.

 

Cathy Hetzel (Rentrak):

With the current network latencies, 10 seconds commercial average audience results are identical (+- 1%) to sec-by-sec

 

Bud Breheney (TNS Media):

The skill to manage the complexities of Return Path Data is new to most research providers.  TNS has been working with Return Path Data for over 5 years and operates panels on four continents including our announced work in the US with DirecTV, Charter, Time Warner and Comcast. The latency question and other data challenges such as set on/set off maybe new to persons who are just beginning to get involved with RPD but they are not new to TNS.  In our 5+ years of experience, we have developed data processing and reporting techniques that are unique to Return Path Data panels. 

 

Frank Foster (EVAD Consulting):

Cable and satellite networks as a general rule do not buffer television signals.  Propagation delay is therefore consistent which means that when you and your brother are watching Monday Night Football two hundred miles apart, he sees the touchdown pass a second before you do-every time.  The delay is constant and does not change. 

Although some equipment deployed in the field such as DVRs do have a five to seven second buffer, the delay is also consistent.  Because the timing disruptions are predictable, the data associated with tuning can be adjusted automatically to improveaccuracy.  Therefore commercial ratings based on set-top box data are technically feasible.   

 

Jeff Boehme (Nielsen):

Feasible, yes, depending on the specific application, technical capabilities of the devices and the associated business rules.

 

We are closely examining the various latency issues across the multitude of devices and platforms to determine current thresholds and technical capabilities to discern the lowest level of precision possible.

 

 

QUESTION 3:RELEVANT DATA

How long must a viewer be tuned to a program before the act of staying on the channel or surfing away from it is meaningful information.

 

Cathy Hetzel (Rentrak):

20 Seconds -Rentrak has calculated this number as a result of receiving data from multiple operators and platforms across multiple markets.

 

Bud Breheney (TNS Media):

This is dependent upon the research objective of the advertiser or analyst.  A benefit of Return Path Data is the granularity of drilling down to viewership on a second by second basis.  Our work with addressable advertising called for a different approach then our efforts with syndication and other advanced analytics    

 

Frank Foster (EVAD Consulting):

Generally speaking, modern set top boxes do not change channels as quickly as their analog counterparts.  Typical boxes take 0.2 to 0.8 seconds to tune.  From a research perspective, most would agree an active viewer must invest between three and fifteen seconds to determine if content is interesting.  I would argue that the minimum dwell time for a tuning event to be relevant is five seconds.  Researchers should avoid discarding data with dwell times exceeding fifteen seconds as valuable information may be lost. 

 

Jeff Boehme (Nielsen):

It seems that 5-second duration appears to be a minimum amount of time for any relevant tuning record. This would obviously depend on the accuracy of each of the device capabilities.

 

 

QUESTION 4: SIMULTANEOUS VIEWING/RECORDING

How does your company handle measuring multiple viewing environments such as:

 

Cathy Hetzel (Rentrak):

¨ Picture in Picture  

We are currently receiving PIP data and working to verify the quality and to better understand the audience behavior before including it in our reporting system.

 

¨ DVR Recording of content

We are implementing a report to show Live, Live +30m, Live +1Hr, 2Hrs, 4Hrs, 8Hrs, 24Hrs, 72hrs, 7 Days

 

¨ Playback of recorded content

Included in the above report but can also be viewed separately Not all data providers are able to provide information beyond "record" at this time.

 

¨ Broadband like viewing on TV when multiple windows are open simultaneously

Same as PIP if multiple windows are TV windows, otherwise full impression.

 

Bud Breheney (TNS Media):

There are more viewing options today then ever before.  Reporting on these alternative data sources require a skilled research and technical team.  Another challenge is the capture of the viewing data.  DIRECTView will offer reporting on time-shifted viewing as we currently do with BSkyB in the UK.  If data provider is able to provide PIP and Broadband data, we are ready and eager to work with them on reporting this activity

 

Frank Foster (EVAD Consulting):

¨ Picture in Picture

Traditionally picture in picture is not accompanied by an audio feed and the resulting data is not captured.  While in a set top box environment tuning data from the second tuner may be available, it is usually ignored as well.

 

¨ DVR Recording of content

DVR Recording on a set top box can be captured as a state change like linear viewing or via a DVR log similar to how VOD data is obtained.

 

¨ Playback of recorded content

Playback of DVR content is gathered as state change data just as linear viewing is obtained.

 

¨ Broadband like viewing on TV when multiple windows are open simultaneously

Multiple screen environments on a single television are not widely available other than as a single broadcast feed, e.g. viewers cannot easily change which channels are available on the screen.  As this technology becomes more readily available, the measurement solution will be addressed.

 

Jeff Boehme (Nielsen):

¨ Picture in Picture

Dominant content audio currently receives credit. However, we are examining the usefulness of PIP data in future STB applications.

 

¨ DVR Recording of content

We don't include DVR record in audience because a significant percentage of recorded material is not played back.

 

¨ Playback of recorded content

Recorded content (for both program and commercial content) contributes to the overall HHs using TV (HUT) credited across four data streams: live, live-same day, live +3, and live+7.

 

¨ Broadband like viewing on TV when multiple windows are open simultaneously

Currently, dominant content audio receives credit, which is consistent with our current methodologies.

 

 

QUESTION 5: PANELS VS. POPULATION

What is the best way for advertisers, agencies and researchers to ensure that panels are representative i.e., age, sex, race, income, new technology propositions (DVR, gaming consoles), and distribution methodologies (cable, over the air, digital terrestrial, telco, satellite). 

 

Cathy Hetzel (Rentrak):

With 400 Networks ,multiple distribution platforms , hundreds of programming packages and a growing number of people watching TV on internet, Rentrak believes that the only solution is vast amounts of geographically and demographically balanced STB data.  

 

Bud Breheney (TNS Media):

To work with an experienced research organization that is familiar with panel construction and management.  At TNS, we have a panel management group that has gained experience working with Return Path Data as well establishing panels for a number of other research objectives.  We look forward to detailing this process with any data provider that would be interested in establishing an opt- in panel.

 

Frank Foster (EVAD Consulting):

One of the best ways for any entity to determine the value of research would be through careful examination of standard and relative error rates as well as error margins and confidence intervals.  Panels should be carefully constructed and information about the

recruiting, data collection and tenure of households in the panel should be closely followed.  In today's television environment, an argument could be made that the panel should be stratified along technology variables, but small panels are often of insufficient size for this approach to be taken.

 

Jeff Boehme (Nielsen):

Nielsen employs the highest standards in the research industry to provide the most accurate and representative panels for media measurement. We ensure that the panels are high quality, with transparent methodology. To do so, we compare panel profiles with reliable external sources and weight to these external sources where appropriate.

We have the most stringent policies and procedures in place, from engineering, field, recruitment, membership representatives, and statistical research. We consistently examine and innovate new techniques to capture all relevant information from all types of devices, across all types of platforms, for all types of consumers and households. Importantly, Nielsen funds various external industry audits and participation (MRC, Council for Research Excellence, etc.) to ensure the highest quality possible.