As viewing options and devices grow, samples become less capable of measuring them. The whole purpose of Nielsen’s national TV sample is that you and your cohorts watch the same thing, based on a dozen or so categories that Nielsen has determined impact TV viewing.
This worked quite well through the 1990s, when the average household could only receive about 30 or so channels and could only watch TV content on a television set. If I watched Seinfeld, and came into the office talking about being the master of my domain, virtually everyone (at least everyone Nielsen said was just like me) knew exactly what I was talking about.
Today, I recently noted to some of my friends that the woman in the new Terminator movie was one of the stars of Game of Thrones. None of them watch that show and didn’t know who she was. These days, simply because of the number of viewing options available, examples of you and your “cohorts” watching different things (or the same things at different times) is commonplace.
The problem with modeling is that it's not good research for measuring viewing behavior, no matter how good the modelers. And even though Nielsen does have some of the very best researchers, calling them “scientists” doesn’t make them better.
Should we accept that modeling viewing behavior from its national 25.000 people-meter sample to 13,000 local TV set meters will effectively double the national sample because Nielsen tells us the math works out? It’s not quite just making up numbers, but it’s close.
You can model overall reach of a network or group of networks, but not individual average program ratings. In other words, overall viewing is much more predictable, specific viewing much less so.
When I was head of research at ION, we would often ask Nielsen how it was possible that an hourlong show like “Criminal Minds” can possibly lose half of its adult 25-54 audience in the second half-hour and gain 40% among adults 18-24? The next day it might be the exact opposite.
They would diligently analyze the data and then tell us that 10 people in the sample switched to something else, and that resulted in a reported rating decline of 48%. This is something not unique to ION. If you really scrutinize the daily cable ratings of virtually any cable network, you’ll see many “illogical” fluctuations. If I were at a cable network now, I would favor almost anything that would significantly increase sample sizes.
I was one of the founding members of the Council for Research Excellence, which should be advising Nielsen on how to proceed here. This is the main purpose of the CRE. To quote from its website:
“Methodological research is concerned with the accuracy of audience measurements and the effects of possible changes in methods. It provides the foundation for valid, reliable and credible audience measurement. The Council for Research Excellence is intended to give Nielsen's client base greater voice in the design and execution of methodological research.”I haven’t heard anything from the CRE on this yet, but a POV distributed to the industry would be a great idea. CRE members are from all sides of the industry, and its seal of approval would give Nielsen’s plan credibility, which, no matter how much “impact data” it releases, will otherwise be lacking.