Set-top Boxes May Underrepresent Minority Populations
- by Wayne Friedman , March 13, 2019
Nielsen says set-top-box/return-path data coming from pay TV providers and other sources can underrepresent certain viewer groups -- in particular Hispanic and African-American homes, compared to other household types.
Compared with official U.S. Census estimates and Nielsen’s representative national panel, these homes -- many coming from cable, satellite and telco platforms -- underrepresent Hispanics by 33%, Spanish-language dominant Hispanics by 49% and African Americans by 34%.
Nielsen adds that when you compare set-top-box/return-path data homes with over-the-air/broadband only, the situation is even worse.
advertisement
advertisement
Set-top-box data measurement under-represents Hispanics by 50%, Spanish-language dominant Hispanics by 68% and African Americans by 38%.
Over-the-air TV and broadband-only homes have grown to 28 million in 2018 from 15 million in 2014, where it says “41% of the consumers in those 28 million homes are multicultural (either Hispanic, African American or Asian) and 10% are a younger demographic (18-24).”
Some of this has to do with “weighting” specific groups against other measures. Nielsen says, overall, “A large biased sample is still biased.”
Even though this is a bit self-serving Nielsen has a point. It has never been established that "big data" panels drawn from cable system subscribers are a valid representation of the population as a whole. So far the main selling point of the big data set-top-box panel purveyers is their large sample bases however even if sample balancing is used to supposedly bring these panels in line with known total population demos, that's just a statistical bandaid which assumes that whatever younger, older or affluent homes you do have in your panel are just like all younger, older and/or affluent homes---which may not be the case. To be fair, however, it has not been established that Nielsen's peoplemeter panel which is now cobbled together by mixing local market homes with those selected nationally is representative either. Perhaps this might be the subject of an objective industry analysis to clear the air. Or is getteing data all that matters ---even if we don't have a fix on how accurate that data is?
Ed, you are correct and raise some very valid points.
Weighting a unrepresentative sample can make problems worse unless you know all the cohorts that best represent viewing and then weight them back to their universe count. The thing is that if you knew all the cohorts you wouldn't have the problem!
Therefore the weighting matrix has to use 'known proxies' that best represent/fit the viewing universe. Further the number of cohorts generally need to be limited so as to efficiently re-weight the raw data. Using 'composite' cohorts (e.g. two-person households over 55 with 2+ TVs as an example) helps substantially.
Irrespective of any and all weighting systems there are certain problems that simply can't be fully accounted for. A core issue in all markets that I am aware of is recruting or gaining the co-operation of people who do not speak the native language of the market.
Here in Australia people of a Non-English Speaking Background (NESB) are hard to recruit and gain permission from. It is easier in larger NESB households as the school-age children can translate for their parents. We use multi-lingual recruitment but it is hard to people on hand who can handle (say) 15 different languages. Despite a massive effort and great expense NESBs are always 'on the low side' in the ratings. But as a buyer of media we focus more on the targeting of the ad into the right content rather than the tonnage of the audience.
We are fortunate here in Australia that OzTAM (who contract Nielsen) do report in depth on the panel composition back to a committee who oversee the ongoing service, While the reports are not public 'the insiders' know the panel performance and are not scared to rattle the cage when needed, which fortunately is very infrequent. This means that the energy of the committee can be channeled into important current and future measurement issues.
Ed & John, I am in fundamental agreement.
I applaud your analyses and sentiments.
Given some recent experiences, permit to share my reactions to these recent hyped
pronouncements by CEO David Kenny and Nielsen:
“Spying” on a consumer’s mobile digital usage is not representative sampling,
nor does it create truth or trust in clients or societies.
Moreover, STB Panels are not per se random! Why does Nielsen have to research
the self-evident truth?
—Was it because Nielsen had considered using the “Big Data” of Cable, Satellite and Fiber Optic Systems
instead of its own data derived from the “Small Data” of random, representative samples of TV Households
just to reduce operating expenses or to impress the world with large, self-selected samples?
—And had Nielsen really considered giving incentives to all STB TV Households ... not to mention the component of the People Meter through which “people viewing” is registered actively, as opposed to “set tuning” passively?
And why doesn’t Nielsen find matters of client and consumer trust to be self-evident too?
Just who is Nielsen kidding? It should be held accountable for what it does and what it fails to do!
#Truth #Trust #RandomSampling #Representativeness #Accountability #FactsStillMatter