A Convenient Truth

The impetus for the media industry’s new JIC (joint industry committee) is at least partly due to Nielsen’s inability to accurately measure TV viewing during the pandemic. For reasons that seem obvious, Nielsen couldn’t keep sample homes in-tab and the decline in sample caused a reduction in TV viewing estimates.

The problem was related to sample quality. At the same time, the JIC wants to open its certification to companies that are using less rigorous sample methodologies.

The data that comes in from TV distributors and manufacturers that comprise the basis for these companies’ measurement are neither a census nor a random sample. Moreover, the companies have yet to even apply for accreditation to the Media Rating Council (MRC), whose  requirements which are tough even for Nielsen.

When you think about it, these facts don’t align.

If the “alt” companies’ -- which is now what they’re starting to be called -- samples aren’t random or a census, then what are they? If they’re not random or a census, they’re not representative of a population.



Their data samples are what some researchers might refer to as “convenience samples.” You get the name comes from the fact that their data is conveniently available.

The researchers don’t need to create a sample frame, call or visit lots of homes to ask people to participate or send out a lot of surveys. They simply collect it from TV set-top boxes or smart TVs.

There is no creation of a new sample frame with the data from each new census, no random choosing of census block groups or of homes on a block group. There are no rules that must be followed to use another home on the same block if a home drops out. You simply get what you get.

This is not to say convenience samples have no value. The sample sizes are enormous. They are fantastic for certain types of research, just not for traditional TV measurement. At least not as standalone services unsupported by panels.

Back in the day when ErinMedia and Nielsen were attempting to make TV ratings out of newly available TV set-top box data, there was another startup that didn’t believe the data was suitable for TV measurement. It instead believed that the best use of the data was for TV ad targeting and media return on investment research. The name of that company was TRA, which was acquired by TV and is now called TiVo Research.

If you’re not familiar with the company, TRA was started by the legendary Bill Harvey and his co-founder, Mark Lieberman.

Bill was a long-time Arbitron executive. He is often credited with having invented the area of dominant influence or ADI. He was the first winner of the Advertising Research Foundation’s (ARF’s) Erwin Ephron Demystification Award. The ARF’s highest honor. He beat out the legendary David Poltrack, CBS’ then Chief Research Officer, who won it the next year. It wouldn’t be a leap to say that he is the father of TV set-top box ad targeting. There is probably not a more respected researcher in all of media research.

This isn’t to bash the “alt” research companies, or for that matter, the use of convenience samples. Convenience samples are phenomenal for ad targeting. For this use, the data doesn’t need to be perfect. What you need is lots of highly granular data that enables the advertiser to send ads to homes with people who are most likely to respond to the advertising. You’re not measuring anything.

They’re also amazing for gauging return-on-media-investment. As TRA and TiVo Research taught us with their product when it first landed on the scene, you could double blind match large TV set-top box databases with purchase data from loyalty card programs and know exactly which homes were exposed to ads and which homes bought the product. This was and is done with no modeling. In research, it’s the Holy Grail – it’s as good as it gets.

The issue is that the JIC isn’t looking to use the “alt” research companies’ data in this way. It’s positioning it as a Nielsen replacement for currency measurement.

Unless the industry is planning to change the basis for how it transacts business from cost-per-thousand (CPM) to some other metric based on conversion, convenience samples by themselves are not a good replacement for a proper, randomly chosen, probability sample.

3 comments about "A Convenient Truth".
Check to receive email when comments are posted.
  1. Ed Papazian from Media Dynamics Inc, May 4, 2023 at 8:35 a.m.

    Exactly, Ed. It's very simple. The "JIC" is not really intending its "so-called" alt currencies to be currencies. And that is why there is no push for them to be accredited by the MRC--assuming that most could even pass its tests---which I doubt. There will be only one standard "currency"---a big data, digital media- style system which estimates "impressions"---based not on attentiveness or presence in the room, but merely that the content played out on a screen. That will give the ruling sellers what they want---big numbers to "monetize'---- while advertisers who just can't be bothered to fund a better system will simply have to pay up. Meanwhile they can delude themselves that their TV buys are reaching large numbers of people---actually, screens, not necesarily people---- and worry about excess frequency when in reality, there isn't a lot of frequency for their ad messages.

  2. Jason Rocco from Media, May 6, 2023 at 12:02 p.m.

    These arguments would be far more compelling if they didn't include wrongful assertions positioned as fact. A simple google search would reveal Inscape the source of ACR data everyone is using has a product being used by currency alternatives that is balanced to census.

    iSpot, VideoAmp and others have done their own major investments to take big data and align it -- as if networks and brands funding the $70B TV ads economy  wouldn't care about that as they push for new currencies. Good take here on that

    And a simple visit to the MRC website would show you that iSpot is getting accredited for ad occurrence data -- which will be the backbone for ad first measurement at scale and the new cross platform currencies and that Innovid has done si as well.

    It's one thing to have an opinion but let's atleast get the facts straight in an argument - if the intention is to help the industry get more informed --MediaPost has a responsibility to fact check this stuff more adequately and the author himself should probably talk to more people are the companies making these changes and or fire his ghost writer! 


  3. Ed Papazian from Media Dynamics Inc, May 7, 2023 at 2:15 p.m.

    Jason,  what your reference seems to be describing is what is often called "sample balancing" whereby a sample's individual members ---in this case ACR sets----are reweighted  for "audience" projections so that when looked at nationally the total sample resembles the total population demographically. In other words if you have too many ACR sets from upper income homes and too few from lower income homes the projection weights of the sample's upper and lower income homes are adjusted so when tallied in total, the profile of your panel matches that of the country. Is that correct?  

Next story loading loading..