CIMM Launches Best Practices Study On Combining Smart TV, STB Data

The fragmentation created by multiple viewing devices, data sources and measurement systems continues to be an obstacle to fully employing the sophisticated audience-targeting capabilities possible through advanced TV platforms. 

Case in point: Smart TV and set-top-box (STB) data each have advantages and disadvantages for audience targeting and attribution purposes. As a result, the practice of combining the two types of privacy-compliant data to try to build large-scale, granular datasets capable of measuring all forms of TV viewing has been on the rise. 

Now, the Coalition for Innovative Media Measurement (CIMM), in partnership with Pre-Meditated Media and Janus Strategy & Insights, is launching a study to develop best practices for this emerging capability. 

“Our goal is to provide the industry with an understanding of how to leverage smart TV and STB data together, not only helping to improve data quality within a household, but create a blueprint for more nationally representative, deterministic TV viewing datasets that can be used for cross-platform planning, activation and measurement,” sums up Howard Shimmel, president, Janus Strategy & Insights. 



Smart TV data can provide a broad footprint of data spread geographically across the country, and help refine the edit rules used for STB data, while STB data provides a fuller picture of TV usage for most TV sets in a household.  

“STB data are currently being made available to some media measurement vendors, and also to audience-based planning and buying platforms,” but “none of the vendors analyzing such data have a nationally representative footprint,” points out Jane Clarke, CIMM’s managing director and CEO. “This study aims to examine how smart TV data can complement STB data to increase the value of the combined datasets.” 

“Smart TVs can report viewing data in near-real time, but the ACR (automated content recognition) data collected don’t represent all TV sets in the average home,” adds Gerard Broussard, principal, Pre-Meditated Media. 

“STB data are sourced from a much larger household footprint than smart TVs,” but “calibrating the STB signals to viewing metrics and matching program names adds more time to the reporting process,” he says. “We hope to find how, perhaps by combining the data, there might be a better solution for releasing more accurate tuning data on a timely basis.” 

Even in combination, the two datasets lack data on broadcast network viewing in over-the-air (OTA) households — those without broadband access or pay-TV subscriptions. But the combination comes closer to providing data on most U.S. households, which can then be further calibrated to provide the OTA viewing via more traditional panel methodology, according to the study partners. Panels (i.e. Nielsen methodology) are also still required to understand who and how many people are watching a TV set.

Reflecting this, the study also aims to add transparency and increase industry confidence in using new data-plus-panel TV measurement approaches. 

The study will be conducted in two phases. 

Phase 1 is a review of ACR and STB data providers, through interviews with providers and aggregators, to identify next steps. Factors examined will include sample size, data captured and reported, data processing rules, and availability. 

Phase 2 will consist of interviewing companies integrating ACR and STB datasets to review existing methods, learn about best practices and potential obstacles, and identify specific design recommendations for creating an integrated reporting system. 

A report on the results will be released later this year.

10 comments about "CIMM Launches Best Practices Study On Combining Smart TV, STB Data".
Check to receive email when comments are posted.
  1. Ed Papazian from Media Dynamics Inc, May 1, 2020 at 7:32 p.m.

    Can smart TV sets really report on "viewing behavior"---namely who is watching, not just that the set is tuned in to a particular channel or program? Just out of curiosity, how is this data made available? Is there a hidden "camera" in the smart set that secretly captures eyes-on-screen activity---second by second for each resident in the household and/or visitors? Is this being done without the permission of the set owners?Color me curious.

  2. John Grono from GAP Research, May 4, 2020 at 5:30 p.m.

    "Smart TVs can report viewing data in near-real time.

    Not quite right.   They can detect tuning, not viewing.   I'm all for more accurtate tuning data in that TV usage sub-set, but call it what it is ... tuning.

  3. Tony Jarvis from Olympic Media Consultancy, May 5, 2020 at 2:39 p.m.

    Once again Ed and John have to remind the industry (CIMM and its associates should know better, Mr. Schimmel!) of the difference between tuning and "viewing" - per Ed, quite correctly, "eyes-on-screen".  Sadly Industry reporters, even those with extensive experience from Media Post, are happy to consistently go along with the sham use of the term "viewing" in their  reporting of research that does not meet the requirements of that term.  In the political arena I think this would be called fake news and would be castigated! 
    Karlene, Wayne, Joe: What is this triumvirate of regular commentators, recognised world wide, missing? 

  4. Ed Papazian from Media Dynamics Inc, May 5, 2020 at 6:30 p.m.

    I think that many people who use the words "viewing" and set usage" interchangeably  think it is only a small , techincal distinction and you will probably get the same answer anyway--so what's the fuss all about? They are wrong. For example, consider this. A typical TV show gets only 1.2-1.3 viewers per set tuned in, yet the same home contains 2.6 residents. Which means that the average resident watches only half the time when a household set is tuned in. Moreover, if you use set usage by age of household head as a surrogate for viewing by age, you will find---perhaps to your surprise---that TV's most frequent viewers---in terms of total time spent---- are young adult by a wide margin, while old adults are the least frequent viewers. Do the same thing by household income and---surprise, surprise---TV's heaviest viewers are affluent adults while low income folks are the least frequent viewers. Does that sound right? Of course not. Indeed  the exact reverse is true when you tally Nielsen's viewing data using viewing claims made by its panel members.

    Why is there such a discrepancy? It's simple. A typical home headed by an adult aged under 45 has over three residents any of whom can turn on a TV set. But a typical home headed by an adult aged 70+ has , perhaps 1.5-1.7 residents---half as many as its younger counterpart. Which means that much of the set usage in younger homes does not involve either of the two household heads---and especially the male head---someone else is viewing. But in older homes the odds that a given resident is watching when the set is tuned in are much, much greater. The same distinction---though to a lesser degree----applies between affluent and low brow homes.

    If you think that overall stats such as these are sobering, wait until you try to make sense of things on a "granular" basis---minute by minute, including ads--- and show by show to boot. LOL.

  5. Debbie Reichig from In-Focus Media Consulting, May 6, 2020 at 10:31 a.m.

    Yes, there is an important distinction between tuning and viewing, as I am confident the key players involved in this study are all aware.  This is likely why the initial phases of this project are based on interviews to obtain a more granular understanding of what types of data are available and what types are needed.   It is a worthy and complex issue that will likely require at least a few additional phases.  

  6. Tony Jarvis from Olympic Media Consultancy replied, May 6, 2020 at 4:38 p.m.

    Your confidence that the "key players" are aware of the critcal distinction is not supported by the mountain of evidence reflecting the misuse of the term "Viewing" over many many years.  So, either they are aware and are inappropriately motivated to perpetuate the sham (most likely?) or they actually are not aware (unlikely?) or perhaps they are not really "key players"? 
    As to the the latter, can one be a key media research player if one knowingly misleads the industry?  It's a rhetorical question!

  7. Ed Papazian from Media Dynamics Inc, May 6, 2020 at 6:10 p.m.

    To Tony's point, Debbie, if there is so much awareness that viewing is not necessarily defined by set usage, why is the word viewing used so often in cunjunction with "smart sets" and set-top-boxes and all of the "insights" about  "viewing" ,that they supposedly offer? I understand that it is deemed necessary to promote ventures of this sort---and over claiming is a common practice---but much more often than one may think, hordes of people in our business who know very little about how TV "viewing" is and is not measured will eagerly await astounding findings  that simply aren't going to be forthcoming.

    If, as you say, this project will begin by asking advertisers and agency people what they want in cross platform audience measurements and then examine whether smart sets or set-top-box simply data might posssibly deliver these goods, you can rest assured that just about everyone involved will use the term "viewing" to describe any findings. It reminds me of the constant cautions about sample size and "standard error" calculations as they apply to surveys. Just about everyone now thinks that you can determine the real world accuracy of any survey by knowing its sample size---hence a survey using a million homes is infinitely more accurate than one using a mere 40,000 homes. This, too, is nonsense, yet we hear it constantly. 

    I have a lot of respect for Jane and CIMM---they have done much good work and I hope that they continue to do so wih this interesting new project. But please, let's not get carried away about set top boxes ---with their huge samples---and "smart sets". All that they record is whether the set is on and what channel was tuned to.  Neither tells you nearly as much as you need to know about "eyes-on-screen viewing" behavior---as the folks at TVision will, I assume, be happy to confirm.

  8. John Grono from GAP Research, May 6, 2020 at 6:24 p.m.

    A nice debate.

    Ed, the other strong correlate is 'devices per household'.   Back in the 'one-set' days age of the HOH was key.   These days devices are prevalant.  In fact it is not uncommon for devices to exceed household members.   (My guess - I don't have the data at hand - is that it would a rarity that devices<people.).

    This is akin to the Zoom 'participants' versus 'participations' issue I prevbiously commented about.   Both are important, but largely misunderstood, and regularly mis-used.

    I do, however. tend to draw the line at 'eyes-on' and not beyond.   By this I am referring to ads rather than programmes.

    The media owner's key responsibility is is to get people watching programmes (which we measure by average minute audience, rather than peaks or cumulatives).   The advertiser's key responsibility is to make ads that people want to watch/accept (currently the average audience of the minute that the ad is shown).

    If the audience in the ad breaks is lower than that of the programme - whose fault is that?   Why would that require a discount or rebate?   I think it was Dave Poltrack back in the '80s or '90s who said (my rough interpretation) ... why would I give you a credit for your crap ad because people didn't want to watch it when I just spent tens of millions of dollars to get them!?!?   Maybe we should charge a premium!

    And don't even start me on device screens with multiple-tabs being served different ads when the ad-server has no measure of whether that tab is in focus, yet they are all counted equally.   I note that 'screen-in-screen' TV went nowhere ... I wonder why.

    The need for standards in measurement, and their correct usage, has never been greater.

  9. Ed Papazian from Media Dynamics Inc, May 6, 2020 at 6:50 p.m.

    John, you make an interesting point about the number of sets and, yes, in the states the average home does have more sets than residents. You also noted the crucial distinction between program content and commercials. A while ago some of the large agencies spent a lot of money to obtain home by home and person by person data from Nielsen---the stated reasons being to examine viewing behavior on a "granular" basis as well as investigating alternative daypart mixes using "real" data as opposed to formulas. These investigations often focused on specific commercials but the agencies were very disappointed by the findings, which didn't reveal very much sensitivity. All  Nielsen is really reporting is that a person claims to be a program viewer when the channel is first tuned in. Beyond this the system assumes that all content ---including ads---is "viewed" for every second the channel remains tuned to the show---unless the panel member who claimed to be watching the program at the outset indicates that he/she is no longer watching---which almost never happens----even though absence from the room and inattention are common avoidance practices. In other words they thought they were going to analyze "viewing" but what they mostly got was set usage data.

    Regarding the medium's responsibility to the advertiser, while I agree that it's not the TV networks' responsibility to make viewers stay in the roon and pay attention to ad messages, the networks---and stations and cable channels---do cause many viewers to avoid commercials when they offer highly cluttered ad and promotional breaks. That is their responsibility---even though too many advertisers, chasing the great CPM god let the sellers get away with burdensome breaks that drive viewers away.

  10. Tony Jarvis from Olympic Media Consultancy, May 7, 2020 at 10:51 a.m.

    Ed and John: We have offered a crystal clear reminder that the Nielsen meter is a relatively sophisticated set tuning meter, no more and no less.  Even at that I beleive it credits/awards any whole minute to a station or network based on a minumum of 5 seconds of the minute tuned (?) and to Ed's point with nothing to do with whether anyone is in the room or not.  Yet another issue related to so called "viewing". 

    CIMM has and continues to drive some terrific, primarily video, measurement intitiaitves.  Now, as part of ARF, it should surely be even more precise and painstaking in its use of media research terms and their defintions especially when it comes to the basis for media currencies?  (MRC please please take note!)  Perhaps "we" shoud be invited to vet the CIMM report from this latest project to ensure it meets the accuracy and accountability that advertisers and their agencies deserve?  

    Regarding the onus for "viewing" (and subsequent impact and ultimatley ROI) it is a two edged sword.  The broadcasters have willingly allowed increased ad clutter however their primary responsibility is to generate "viewing" to the associated programs and often their very special content and context which will "feed" into the ad audience.  The "impact" of an ad, which requires exposure or "eyes-on-screen" not merely some kind of nebulous gross impression, is driven by a wide array of elements out of the media platform's control, notably the creative execution.  So I would side with John, David Poltrack and most other major media platforms that the buck stops with delivering Eyes, or Ears, On to a brand's target audience.   

Next story loading loading..