January 25, 2023
Commentary
Garbage In, Garbage Out
- by Joe Mandese @mp_joemandese, April 14, 2022
It’s one of the most fundamental principles of computer programming and data science, but based on some descriptions for the “test and learn” marketplace onboarding multiple, “alternative” currencies that advertising deals will be based on in the 2022-23 upfront marketplace, I think it is probably a good principle to keep …
Joe Mandese is the Editor in Chief of MediaPost. You can reach Joe at joe@mediapost.com.
Interesting perspective, Joe. As you know I came from the era when the major ad agencies were right on top of methodological questions and were pushing ,first for viewing not set usage data and later for demos and product usage information for national TV shows. In fact, I was one of the pioneers in that movement along with some fine colleagues. All of us were primarily "we" oriented, meaning that we wanted the best measures for our industry, not just one agency trying to look good at the expsnse of its rivals.
While we made lots of progress on the magazine, radio and local TV fronts, we were stymied when it came to network TV buys as the buyers and sellers rejected virtually all efforts to employ more brand-relevant metrics and as the upfront process became entrenched, the buyers more or less went their separate way, agreeing with the sellers on a single, broad based "demo" as the currency for each upfront corporate negotiation. So that's where adults 18-49 and women 25-54 came from with many clients thinking that the use of such "curriencies" was a big improvement over household ratings---which it was---but why stop there?Yet this is exactly what happened and even now, in 2022, we are still using the same currencies as if they meant something.
To be fair to the "demand side" we should note that it's thinking is focused primarily on how the ad dollars are split between various sellers---and at what CPM--- not whether the audience surveys are right or wrong. In other words, even if we don't really know who is watching, so long as the research we use isn't greatly misleading us as regards share of audience, then we are making valid buys. If and when it can be demonstrated that national TV's rating currency is greatly distorted regarding how the aidience pie ---whatever it may be ---is split up differently than we have been led to believe, that's quite a different matter.
So far, none of the various competitors to Nielsen's national TV rating service over the years ----Arbitron, Burke, Audits & Surveys, R.D. Percy, AGB, SMART, etc.--- have been able to show that its share of auidience split is far off the mark. In fact, none of them even tried to make this critical point---so they all failed to win support.
Bravo Joe, and plus one Ed.
I've worked on multiple JICs in AU and I agree that "we" is the stongest requirement and most needed component.
I've had to push the media owners to invest in new methodologies, new data sources, new technology etc. (which all comes at a cost) because if they don't the media agencies would probably drop the research expensecompletely.
I've also had to reel in the media agencies/marketers expectations to be more realistic about ther outcomes they want .. unless they want to pay a ship-load of money towards the research via the advertising rates.
It feels like we're heading to a "tell me the answer you want, and I can get the research to show that". (Yes, that was the first research thing I learned at university on how you could produce a pre-ordained answer. We had to statistically prove that every husband is, or has been, a wife basher - which clearly is not true - but yes we could get a 100% wife-basher resukt by the way we wrote the questionnaire.).
We call that the Hot Chocolate approach when there is no currency ... Every One's A Winner, Baby ... but unfortunately it's not the truth.
Once again, Joe has the best 20-20 vision and insight. I was wondering when someone was going to remember when the presence of even two measurement companies drove Poltrack to observe that "one of them is right, or the other one is right, but they can't both be right!" It will indeed be an interesting time.
Jack, the only real comparisone between rival TV rating services occured at the local market level between Arbitron and Nielsen. Here we had very small samples and a mixture of methodologies---diaries only in most markets plus meters and diaries in some of the larger cities. While individual stations lived or died based on misicule rating differences between the two services, most of these---especially for independent stations---- were noted in markets where one service used meters plus diaries---the latter for viewer-per-set projections---- while the other used diaries entirely. So they weren't employing the same methodology and that plus the small sample sizes explained the differences.
However, in most cases where both services measured audiences in an identical manner the results were very similar---a few exceptions notwithstanding. So Dave was right---- under comparable circumstances they were both "right". Which is why one of them eventually lost out. There was no need for two services supplying essentially the same information. If they had produced really significant differences---that would have been a different matter---but it never got to that point.
Having been on the sales side, the differences even where methods were quite similar, would be troublesome enough to be vexing. Plus, agencies and advertisers declared themselves to be Arbitron or Nielsen "shops" so sellers had to keep two sets of books. It was hard enough when two suppliers had similar but not the same numbers. This time it's going to be more challenging. Way more.