TV network advertising trade group the VAB is demanding that Nielsen immediately stop releasing its new more granular “Big Data” TV monthly reports because of numerous methodology issues.
This comes just weeks before the upfront TV advertising market is set to begin.
One major point made by the VAB is that Nielsen is offering two different measures, in effect -- its new Big Data effort and its current, long-time data set derived from its legacy 40,000-home Nielsen Television Panel -- and this could cause confusion.
Nielsen's “Big Data” effort includes more granular measurement of TV audiences -- including set-top-box data and smart TV Automated Content Recognition (ACR) data -- along with its traditional measures of total linear viewership by age and gender.
“The VAB had high hopes for Big Data being a big leap forward in what Nielsen’s measurement and currency can bring to marketers, but after in-depth analysis it’s clear to us that this first data set is rife with serious problems,” says Sean Cunningham, chief executive officer of VAB, in a statement.
In a letter to David Kenny, chief executive officer of Nielsen, Cunningham writes that Nielsen has failed to “provide meaningful level of disclosures, verifications or explanations as to how ‘Big Data’ was/is calculated, including the National Panel/Big Data calculations.”
Cunningham says immediate disclosures need to be revealed, including how the new data was created, and says there are many comparative examples that are troubling.
For one, he says, there were “illogical demographic audience” results when comparing Big Data versus current Nielsen panel-based data “within the same dayparts.”
Cunningham says: “Big Data claims significant audience gains on persons 2+ and P18-49...but also significant simultaneous declines on P25-54.”
He adds that “there are wild swings in comparative gender results in many dayparts / genres /programs -- in which there are double-digit plus/minus swings in male and female viewership that defies any logical pattern or explanation.”
With regard to sports TV programming, Cunningham says that in comparing the two sets of data, live-same day ratings are down, while “persons using TV sets for those same programs are up double digits.”
The VAB also says that while Nielsen touted the benefit of Big Data -- which was supposed to result in larger audiences for smaller networks -- much of this has yet to reveal itself with those shifts. “Comparisons (Big Data vs. current Panel) reveal that nearly 30% of the smallest 100 networks had lower overall audiences per Big Data."
The VAB cites other challenges, including that methodology has not been disclosed for the set-top-box data that comes from pay TV viewers, or for Automated Content Recognition data, which comes from smart TV sets. Another issue is that TV homes from smart TV manufacturer Vizio and smart TV platform and set-top-box app Roku have not been disclosed.
In a statement, Nielsen says, in response to the VAB: “Up until this letter was issued, we have not received questions from the VAB. In addition, a trade group associated with traditional TV channels is an incomplete and biased subset of the video marketplace. We prefer to work openly with the entire industry to get to the best measurement solution.”
With regard to the upfront ad market, it adds: “Based on feedback across buyers and sellers we made the decision to allow either data set to be used for trading in the fall.... Our approach will enable buyers and sellers to trade against big data plus panel metrics if they so choose, while giving our clients runway to adapt to this launch.”
The VAB has had major complaints about Nielsen over the last year-and-a-half, including undercounting of TV viewing for most of 2020 through early 2021 due to the COVID-19 pandemic, which prevented Nielsen field engineers from entering Nielsen-panel TV homes and conducting maintenance.
More recently, the VAB expressed concerns after Nielsen revealed that it had understated out-of-home TV viewing.