Commentary
What Would Ad Industry Do If It Heard 22% Of Nielsen's TV Panel Were Bots?
- by Dave Morgan , Featured Contributor, October 17, 2019
Folks would be up in arms. We’d probably see stories in the ad trades calling for Congressional investigations, and networks and agencies would suspend their payments for the ratings.
Don’t worry. It’s a rhetorical question. Nielsen does NOT have a panel bot problem.
Juxtapose the anticipated rabid reaction to that hypothetical problem with the tepid industry reaction to research from Pixalate telling us 22% of the programmatic ad delivery on OTT (over-the-top) and CTV (connected TV) today is fraudulent.
What gives?
How come most folks yawn (or avert their eyes) when you talk about the clear, unmistakable problems of fraud and opacity in digital?
No one is yelling to the rooftops about the impenetrability of determining provenance, value and accuracy in data used in digital ad targeting, a multibillion-dollar market — especially when the data sometimes represents 80+% of the raw cost of a targeted digital ad campaigns.
advertisement
advertisement
Why? Because many in the ad industry hold new, buzzy digital channels to different standards than they do legacy media.
That needs to change. Digital advertising in the U.S. now receives more spend than any other channel, including TV, and there is no question that building trust and transparency in digital ad data and in OTT/CTV needs to be front and center.
What can be done? Here are some of my thoughts on where we can start:
Simplify the targeting data story. We may not
understand meteorology and the science behind it, but we all know how to talk about sun, wind, rain and snow.
Meanwhile, much of what happens around data and targeting is purposefully complex.
Just look at how much of it is communicated in acronyms, not words.
Ask hard questions about how shareholders’ money is being spent. This is not just about relationship management. This is about spending shareholders’ money.
Folks buying targeted ad campaigns need to ask (and have answered) key questions like: How do I know the data is correct? Where it came from? How it was manufactured? Did the data/targeting add value? Was the price fair, at market? Who made money on the impression? Who made money on the data? And, how many “daisy chains” did each go through? Uncomfortable questions to ask partners and suppliers for sure -- but essential.
Do it now, before that targeting data “black hole” invades TV ad budgets through OTT and CTV. There's no question digital ad approaches are going to remake and revolutionize TV. That's a great thing. Advertising on OTT and CTV will be big parts of that future. However, let’s take the good and leave the bad behind. Let’s have a future of fewer, more relevant ads, with no fraud and no opacity. That will lead to happy viewers, profitable publishers and profitable, growing advertisers.
Get involved in the ANA Trust Consortium. The Association of National Advertisers is taking the lead here. These are the kinds of issues that they’re trying to tackle. Help them and help yourselves and be part of the process.
What do you think? Do we hold digital ad platforms like targeting data, OTT and CTV to different standards than we hold Nielsen and TV?
Dave, of course "we" need to hold digital platforms accountable and the process starts when we stop relying on the sellers to provide "audience" data and switch to objective, third party measurements that are audited and adhere to the same standards for every platform. How this will come about is anyone's guess, however.
Dave, when you state 22% of OTT, CTV ad delivery is fraudulent, who is being defrauded? Are viewers seeing ads for non-existent products? Or is it ad inventory being purchased by brands for non-existent audience segments or even shows? The word delvery has me confused here.
Henry, aplogies for the confusion. The 22% fraud is or non-existant audience. The promised audience "delivery" includes a substantial portion of views of the ads by robots who are not people.
Thanks.
Dave-
I agree with the sentiment here - there needs to be a better way - but let's clarify what's happening. According to the report, 22% of traffic is invalid, as defined by Pixalate. They define invalid as "fraud and ad serving and measurement errors caused by technical malfunctions". So what's the breakout here? Is fraud a majority of the 22% or is it mostly issues related to the serving and measurement of SSAI which has a 20% higher rate of invalid traffic? Do those impressions actually get delivered to real people but not reported as such? We can't figure out a solution without knowing the real culprit and root cause.
Dave-
I agree with the sentiment here - there needs to be a better way - but let's clarify what's happening. According to the report, 22% of traffic is invalid, as defined by Pixalate. They define invalid as "fraud and ad serving and measurement errors caused by technical malfunctions". So what's the breakout here? Is fraud a majority of the 22% or is it mostly issues related to the serving and measurement of SSAI which has a 20% higher rate of invalid traffic? Do those impressions actually get delivered to real people but not reported as such? We can't figure out a solution without knowing the real culprit and root cause.
The article doesn't discuss solutions or causes apart from apathy and acceptance of fraud and slop. The article is about how fraud laden digital measures are accepted for conducting business even though they fall well below any acceptable standard. Even below the minimum standard applied to traditional media measured by third parties. How grading your own homework is how it's done and accepted. When media companies complain that third party providers don't match internal figures, they pretend these problems don't exist. New media could meet or exceed standards of measurement used used for old. Wasn't that the promise?
Considering that almost all of Nielsen's Audience Ratings are an extreme extrapolation with no way to validate that someone was watching TV at the same time as a panel member that they are said to be matched with demographically, this is a poor comparison. Digital is served 1:1 only when someone is actually there - yes, there might be some bots and fraud, but it is never served out into the ether like TV where the advertiser is charged for people that might never have seen their ad. And Digital has verification services that can show fraud and I don't know about you all, but my clients have make-good requirements for fraud above a certain % and it is not anywhere near 20%.
This sort of nonsense is exactly the problem. Unless the laws of statistics, probability, and the law of large numbers have vanished, sampling is an efficient and trustworthy approach to measurement. Sampling is extrapolation. The doubters dispute the laws of statistics. How many spoons of your soup do you need to extrapolate that it has too much salt? Where would scientific progress be without accepting the laws of probability. A vote to accept fraud and grading ones own homework is a vote for the scammers. A bot has a zero chance of being a person. I go with the statistics I learned from Hays, accepted by reputable scientists across the globe.
Garbiel, I disagree. While Nielsen might not be ad accurate at the person/imprssion level as wek all would like (leaves lots and lots to be desired), it is a balanced panel, all viewing and channel changes, etc. is directly monitored and it is carefully monitored to insure that there are no attempts to create false viewing.
As Jack pointed out above, a big issue with ads on OTT & CTV today is lack of concern and vigilance. Many of the ad deliveries can't be verified, and it's very hard, as you know, to verify that some/many of them aren't deliverd to proxy servers. That can't happen on linear TV. I don't harken for the time of yesteryear - OTT is a big part of the future. I just want us all to care, to be vigilant, and to push the tech and practices harder and harder to stamp out all fruad.
Sorry Gabriel for my tardy reply.
I loaded up the MediaPost articles about an hour ago, but my wife just got back from overseas so I made her a cuppa and we had a chat. I then hung the washing out, did some work outside then came back inside to continue with reading my MediaPost articles.
So that is about an hour with about 40 articles open, all serving (or the potential to serve) ads to me when I wasn't there.
Lesson: an ad served is not necessarily (and indedd very often) seen.
And being an old fogey I remember when one ad in one of my first regional Pizza Hut campaigns didn't run as booked. The drama was high, the error was rapidly corrected. Yes, just one ad. Now, no-one seems to give a hoot about such errors.
Gabriel, are you aware of the camera -style studies that show that a significant part of the digital ad "audience"---page views, that is---is not present ---just like TV?While I think that the figures might be a bit inflated a recent study byTVision found that about 30% of the time, nobody was present when the ads appeared. Also, there is evidence that this applies not only to desktop devices but also to smartphones. Here, a recent study noted that about 10% of the time there were indications that the ads were on-screen but nobody was there.
Thanks for the debate, gents. I will say to @Jack re: my non-sence going against statistics and probability 'laws' - that I don't know any multi-billion dollar industry whose currency is based off of a panel that represents 1-2% of the US households. Soup costs $.99, not hundreds of millions that the top brands spend on TV.
Let's remember the rules of statistics require statistical relevancy - we have accepted 1-2% of the population extrapolated to the other 98-99% for too long and called it relevant and transacted billions of dollars by this 'source of truth' - let's get real here when saying 20% of digital isn't seen by humans and think logically, not with our feelings, about how much of TV is not seen by the audience paid for. That study won't happen, because major TV companies with money would never let it. Also, again to my point, no decent advertisers in the US are paying for 20% fraud - most might tolerate 1-2% before require makegoods.
Would love to have a good chat with anyone offline that likes to talk through these things!
Gabriel, you ask about any multi-billion dollar industry who currency is based off of (sic) a panel thet represents 1-2% of the US households.
In the media industry let's start with newspapers, magazines and radio plus all consumer research.
But I can think of another one. A larger one. A far more important one.
Medicine.
You go the doctor as you aren't feeling well and the doctor gets a blood test done. 5 ml of blood goes into a test tube and off to the pathology lab. You go back when the doctor gets the results.
The average adult body has 5 litres of blood. 5ml is one one-thousandth of your blood. Yes, 0.1%. Yes, TV ratings sampling ration is 10x to 20x times bigger than that very important blood test.
So, I suppose to be sure, you should ask for all your blood to be taken and you'll come back in a week for the results. Just so you are sure.
And if you think that is an extreme example., consider when you are coooking. You put the green peas into the pot and boil them until they are the right tenderness. But how do you check that they are done? You get a spoon, and take one out and test it. That would probably be <1% sample size. Or do you take all the peas out just to be certain.
When you cook a cake you pierce it in the centre with a wooden skewer and if it comes out clean then the cake is done. You don't have to skewer it all over, or even eat it just to be sure.
I wish you good health Gabriel and happy cooking.
Gabriel, "that study" you referred to as never happening has happened many times in the past---the first one I recall was in 1957 with a camera study of TV audiences by a university and the latest is an ongoing panel of 7,000 persons by TVision which uses a modern day "eye-camera" to determine whether people are in the room and also whether their eyes are fixed on the TV screen. In between these there have been many other studies which we chronicle for our MDI Direct subscribers in a publication called "TV Dimensions". Not only that, advertisers have conducted thousands of ad recall studies ---probably a million of them---maybe more---and they are perfectably aware that only a portion of the "audience" of a given TV show can recall seeing an average commercial--why?--because the others left the room, or weren't paying attention.
As regards sample size, it has been proven time after time that a well designed study does not need a gigantic---like millions of respondents----sample to produce reasonably accurate information. I first learned this many years ago when I compared Nielsen's national TV home ratings ---based on a panel of only 1,000 homes at the time--with local market ratings done by ARB---later called Arbitron----in 150+ cities. The latter's combined sample, when all of the cities were totalled up, was about 150 times what Nielsen was using for its national ratings, but produced nearly identical national ratings on a show by show basis.
It is a myth that sample size equaltes with survey accuracy. The three primary determinants are how you get the data---meters, diaries, telephone calls, online recall studies, etc., how you ask the question and whether you have a proper sample that is as representative as possible. Sample size becomes vitally important only when you try to break down the findings on a "granular" basis---by age, income, ethnicity, etc. In such cases you need a large enough sample for each break or you will get "unstable" results. Nielsen's national people meter rating service, which is now based on a panel of about 40,000 homes, while it has issues like all surveys, is considered to be an extremely accurate barometer of who watches what TV shows by almost every company that uses its data. And rest assured that the networks watch like hawks for any sign of odd or suspect findings as the ratings provide a vital "currency" for their ad sales as well as programming operations.
Sorry Gabriel. Sampling is widely used in many industries and goes well beyond media. Manifacturing? Aren't samples pulled off the line and tested? Have you gone to Kellogg's of Baytle Creek to watch them make and pack corn flakes? I have. They pull samples and test them. Same is true for autos, appliances, canned goods, alcohol, wine. Samples are drawn from those stills and barrels to know when they are ready, no? Seems to me nonsense once again to assert that sampling is not acceptable outside of electronic media. As to your 1-2% threshold, more nonsense. It applies if and when you detect fraud. Nobody believes fraud is even close to being properly measured and identified. Any advertiser that believes otherwise should also be in the market for a certain bridge. This article states what is well known, argues that it should not be acceptable and offers some ways to address it. To do otherwise allows the scammers, crooks, and cheats to continue to steal our money.
Thank you John, Ed and Jack for providing much-needed context - and a big dose of reality - for folks who argue against panels and sampling and don't recognize how powerful those techniques are - particularly when so much of digital relies on directly measuared data from "requests" from non-normlized software clients on the open Internet, where proxies and bots operate freely, with only limited capacity to know that those "requests" come from real people.
Thank you, gentlemen! I feel much more informed now - I hope you all have a great week and best of luck to you all! Let's keep fighting the fight against fraud in any form.
On this we heartily agree.
I'll be in NY around May next year.
Maybe we catch up and have a chinwag and try to solve the problems of the world, and finally get to meet!
Cheers.
That would be great John. I'd like that very much.