The Most Important Media-Mix Study, So Far

Bill Harvey is one of the smartest guys in the business. He is also a great writer. So when he sent me an email to an article he wrote last week, I definitely took notice and read it all the way through.
I mean, how could you resist the headline?: "The Most Important Media Mix ROI Study Of All Time."

The article, which was published by Jack Myers' Media Village, opens with a powerful anecdote about Harvey feeling chagrined after teaching a college class last year, "because it became obvious to me that the students felt that they could "do it all" for a brand using only social media.

Harvey then goes on to cite the most important media-mix study of all time, and it became obvious to me that its authors felt they could "do it all" by adding increments of TV to any campaign's media mix.

The study certainly was an impressive one, an Advertising Research Foundation (ARF) project analyzing more than 4,000 different global case studies of the ROI of advertising: "from a blue chip list of research companies," Harvey noted, citing what likely actually is the most important finding of the study, though one that anyone in the ad biz should have already baked into their thinking: "The study shows conclusively that the more media you use the greater your ROI will be, all other things (like creative) being equal."

He then goes on to illustrate a number of incredibly TV-centric findings showing how adding the medium into any mix boosts results.


The reason incremental media generate incremental ROI to a campaign's mix is because in our hyper-fragmented media world, audiences are dispersed and incremental media usually delivers incremental reach. In other words, more people are exposed to it. Pure and simple.

Don't get me wrong, I'm not knocking the ARF's findings, or Harvey's case that it's the best ROI study ever done. I would just add a little modifier: "so far."

The reason is I think the actual media universe is much greater than the one the ad industry typically looks at, even when it aggregates 4,000 case studies.

And the reason for that is they tend to measure the effects only of the media that they know exist and can actually measure.

And as I've written before, there is a vast dimension of media exposure the industry doesn't accurately measure, and may not know actually exists.

Years ago, I labeled that "dark" media and drew analogies between the physical universe and the media universe. And just like the physical universe, most of it is stuff we cannot see and know very little about (dark matter/dark energy).

For years, marketing-mix ROI models have tried to account for some of these unseeable and unmeasurable media, using convenient open-ended terms like "word-of-mouth," "friends and family," or a litany of ambient media we know exists, but don't have the faculties as an industry to accurately measure and account for.

The funny thing is that most of those early studies also found adding TV to the mix typically boosted returns.

Years later, experts agreed that at least part of that phenomenon was due to the fact that there generally was better data available for factoring TV into the models and that results were biased in favor of TV.

Then digital began going down its own infinite rabbit holes of attribution models, leveraging real- or near-time data on actual digital advertising exposures to real product sales.

That evolved into multimedia attribution studies and lots of heads hurting during presentations at industry events like the ARF's conferences.

I don't mean to poo-poo all this hard work, because much of has certainly advanced our understanding of media-mix effects. I'm just pointing out that they don't account for everything -- and maybe don't account for the most important things.

So, sure -- if you're looking at a narrow sliver of a universe, you will obviously draw some simple, repeatable conclusions. That's how science works. But it's all cause and effect until you learn about the parts you haven't measured up to then.

Don't get me wrong -- I don't know what the right solution is, because the media universe is so ridiculously infinite and fragmented that I don't think human beings will ever have the ability to see it completely.

But we can start trying.

And the best way to do that is via good, scalable observational studies analyzing what people actually do with media.

And by that, I don't mean the infinite array of computer vision, eye-tracking and other biometric technologies, though they definitely should be incorporated as part of one.

Some years back, when she was head of research at Time Inc., Betsy Frank commissioned some pretty good research executed by then neuromarketing startup Innerscope, which utilized a combination of eye-tracking and other biometrics, as well as a camera test subjects wore around their necks so Innerscope's analysts could sync what the respondents were being exposed to with how they responded to it, consciously and unconsciously.

The study was expensive and conducted with a relatively small sample, but the insights -- as anecdotal as they may have been -- were fascinating. Among other things, it showed marked differences in the way younger (Millennials) and older (Boomers) subjects physically engaged with hand-perceived media.

Observational studies are hard and costly, but the insights they can glean will help inform the total media universe -- not just the ones the industry has syndicated and custom research providers typically focus on, because, well, that's where the money is.

In recent months, I have been surprised -- and written frequently -- about a variety of initiatives by suppliers of big TV-centric companies -- mainly NBCUniversal -- that are in the process of remaking themselves into big digital ones. And in the process, redefining digital ad "video," whether it is the CTV, OTT, streaming, or whatever new forms of TV they are peddling.

What has shocked me most is how much and how quickly the demand-side -- mainly the ad-agency executives I have been trying to get reactions from -- have capitulated and gone all in on these initiatives as their saving grace.

Never mind the obvious implications for "media neutrality" and the inherent bias big agencies have when it comes to working with big media suppliers that make their lives simpler and easier -- even if they tend to reinforce a bias for them in their media mix.

One of the reasons that capitulation has happened so quickly is that many of the people calling the shots in big agency strategy, planning and execution  are people who came of age with digital media platforms -- you know, "walled gardens" -- that have historically done that for them. And they seem to want NBCU to be one of their new walled gardens.

Let me give props to NBCU for getting out in front of this problem and understanding the opportunity it presented. And based on what I observed at last week's NBCU developer's conference, I think congratulations are in order. You won NBCU and you will dominate the conversation for the new TV-centric mix ROI going forward.

Call it "cross-media," or "cross-platform," and say you are using census-level consumer identity data to do it -- the bottom line is that it will tilt the industry more in that way of thinking about mixes and outcomes, not a more expansive view of what the media universe actually is and how it works.

It has been nearly 100 years since Ball State University conducted one of the best-known observation studies about how people actually use media -- the so-called "Middletown Studies."

They used grad students to follow real people in Muncie, Indiana around as they really used media and reported the results.

It was arguably the first, best ROI study -- and the best one for its time.

Earlier in this century, Ball State's Center for Media Design replicated some observational media studies, and ultimately spun off and influenced some commercial research endeavors, some of which were commissioned by the Nielsen-funded Council for Research Excellence.

The CRE has suspended active research projects, but some of its studies can still be read on its site.

If you ask me, the most important media-mix ROI study the ad industry could commission for these times is a really good, scalable observational study to understand all the media that people are actually exposed to, and how it influences what they think and feel and how they behave.

7 comments about "The Most Important Media-Mix Study, So Far".
Check to receive email when comments are posted.
  1. Kevin Killion from Stone House Systems, Inc., March 28, 2022 at 11:51 a.m.

    My co-worker friend for many years Beth Uyenco sometimes used to ask new media hires to estimate the prevalance of various groups within the U.S., such as urban/rural, college grads and so on.  The result was great consistence in how these young people overestimated the proportion of all the groups that they themselves were members of. That's consistent with Harvey's observation of college kids inflating the role of social media.

    On the larger question of media mixing, Harvey is spot on again.  While balancing media choices is crucial, it's always the case that mixing produces benefits.  There were spectacular store studies decades ago documentating the sales bump from adding radio to a retail client's ad schedule.

  2. Mike Bloxham from Magid, March 28, 2022 at 12:47 p.m.

    Hi Joe.
    It's not often I see the Middletown Media Studies cited these days!
    Your comments about unmeasured media are spot on. By way of illustration, I recall when we conducted the first of the studies in 2003, many people questioned why we chose to include use of the cell phone in what we measured - they didn't consider it a medium.
    That all changed in less than a year of course, but it shows how you don't get to understand the whole picture unless you measure it.

  3. Jack Wakshlag from Media Strategy, Research & Analytics replied, March 28, 2022 at 4:09 p.m.

    Middletown and the follow up CRE studies, led by Mike and his Ball State group were incredibly useful in their time. I'm sure another would prove equally useful every ten years or so. Such collaborative efforts are sadly not likely anymore without finding from measurement suppliers. 

  4. John Grono from GAP Research, March 28, 2022 at 6:31 p.m.

    Great post Joe.

    You are spot on with your comments such as "I think the actual media universe is much greater than the one the ad industry typically looks at", and "they tend to measure the effects only of the media that they know exist and can actually measure", and "there is a vast dimension of media exposure the industry doesn't accurately measure, and may not know actually exists."

    But they made me think of another pretty smart bloke, albeit not in the media sector. who has been attributed to have written on a blackboard "'Not everything that can be counted counts, and not everything that counts can be counted."   Whether Eisntein did or didn't say that. it is still very sage advice.   As a parallel, we only know a scintilla about the universe, but what we do know is mind-blowing and enough for us to put a man on the moon just over 50 years ago.

    And then there is the delicious irony of the reference to the "walled garden" of the digital age.   We may know a lot about what is within a walled garden (albeit self-generated) but it is still less than what we know about the much larger "media market" with its omissions, incompleteness and varying accuracy.

  5. Tony Jarvis from Olympic Media Consultancy, March 28, 2022 at 6:47 p.m.

    As Erwin Ephron espoused years ago regarding the value of increasing the mix of media, it's about target audience reach, preferably exposed i.e., Eyes or Ears-On.  Frequency, as he opined in one of his famous newsletters, is "crab grass".  CPM, which is driven by frequency, stands for "Completely Positively Mad", of course!
    In addition, I would posit that ROI studies are campaign rather than media studies (ANA please take note regarding hyour Cross Media Measurment initiative) due to the overwhelming influnce of the creative to drive attention and therefore impact plus, the interest in the brand category and the equity of the brand being advertised going in.  However, all things being equal, incremental target group Eyes/Ears-On reach will typically improve campaign outcomes based on the "right" message. 
    Please remember John Grono's warning in all this.  Brilliant creative can overcome a bad media plan but not vice versa!!

  6. Ed Papazian from Media Dynamics Inc, March 28, 2022 at 7:22 p.m.

    While reach is obviously important it matters a lot whether we are considering the very beginning of a branding ad campaign---new product or new positioning for an old product---or whether we are in the second year of the campaign ---or the third year---by which time just about everyone who is going to respond to your message will have seen your ads many times and gotten your message..

    Erwin was promoting the maximize one -week reach aspect of his "recency" theory. This called for a 70% reach week after week for the entire budget year. If executed---very difficult now---this would automatically have become a high frequency campaign by week two or three.  It would have taken about 200-220 TV GRPs in 1995 to attain a one- week reach of 70%----- by the time you finished your second week you would have amasssed 400-440 GRPs with a reach of around 85-90%---remember , those were the good old days for TV. This meant that your average frequency in two weeks was about 5. Take the entire year---all 52 weeks--- and you had about 11,000 GRPs which meant That your yearly frequency was roughly 115. That's a lot of "crabgrass".

    My point is that you can't avoid the "crabgrass" as it's part of the reach and frequency equation. Moreover, frequency is usually required to make a sale and later, to reinforce the consumer'spositive feelings  aroused by  earlier exposures. It's not something to be avoided, rather, it's something to try to orchestrate so you don't overdo it in any short period---like a day or a week---or bunch up your ads in the same episode of a program.

    As for which medium comes first, which second, which third, etc. and how much to spend in each medium there is no way to develop  a magic formula that gives you an answer which can be applied to all ad campaigns.

  7. Gordon Borrell from Borrell Associates, March 29, 2022 at 9:06 a.m.

    Thanks for putting into words what I think whenever I read about "the right media mix" including a larger investment in the type of media favored by whoever funded the research.

Next story loading loading..