Harvey then goes on to cite the most important media-mix study of all time, and it became obvious to me that its authors felt they could "do it all" by adding increments of TV to any campaign's media mix.

The study certainly was an impressive one, an Advertising Research Foundation (ARF) project analyzing more than 4,000 different global case studies of the ROI of advertising: "from a blue chip list of research companies," Harvey noted, citing what likely actually is the most important finding of the study, though one that anyone in the ad biz should have already baked into their thinking: "The study shows conclusively that the more media you use the greater your ROI will be, all other things (like creative) being equal."

He then goes on to illustrate a number of incredibly TV-centric findings showing how adding the medium into any mix boosts results.

Duh.

The reason incremental media generate incremental ROI to a campaign's mix is because in our hyper-fragmented media world, audiences are dispersed and incremental media usually delivers incremental reach. In other words, more people are exposed to it. Pure and simple.

Don't get me wrong, I'm not knocking the ARF's findings, or Harvey's case that it's the best ROI study ever done. I would just add a little modifier: "so far."

The reason is I think the actual media universe is much greater than the one the ad industry typically looks at, even when it aggregates 4,000 case studies.

And the reason for that is they tend to measure the effects only of the media that they know exist and can actually measure.

And as I've written before, there is a vast dimension of media exposure the industry doesn't accurately measure, and may not know actually exists.

Years ago, I labeled that "dark" media and drew analogies between the physical universe and the media universe. And just like the physical universe, most of it is stuff we cannot see and know very little about (dark matter/dark energy).

For years, marketing-mix ROI models have tried to account for some of these unseeable and unmeasurable media, using convenient open-ended terms like "word-of-mouth," "friends and family," or a litany of ambient media we know exists, but don't have the faculties as an industry to accurately measure and account for.

The funny thing is that most of those early studies also found adding TV to the mix typically boosted returns.

Years later, experts agreed that at least part of that phenomenon was due to the fact that there generally was better data available for factoring TV into the models and that results were biased in favor of TV.

Then digital began going down its own infinite rabbit holes of attribution models, leveraging real- or near-time data on actual digital advertising exposures to real product sales.

That evolved into multimedia attribution studies and lots of heads hurting during presentations at industry events like the ARF's conferences.

I don't mean to poo-poo all this hard work, because much of has certainly advanced our understanding of media-mix effects. I'm just pointing out that they don't account for everything -- and maybe don't account for the most important things.

So, sure -- if you're looking at a narrow sliver of a universe, you will obviously draw some simple, repeatable conclusions. That's how science works. But it's all cause and effect until you learn about the parts you haven't measured up to then.

Don't get me wrong -- I don't know what the right solution is, because the media universe is so ridiculously infinite and fragmented that I don't think human beings will ever have the ability to see it completely.

But we can start trying.

And the best way to do that is via good, scalable observational studies analyzing what people actually do with media.

And by that, I don't mean the infinite array of computer vision, eye-tracking and other biometric technologies, though they definitely should be incorporated as part of one.

Some years back, when she was head of research at Time Inc., Betsy Frank commissioned some pretty good research executed by then neuromarketing startup Innerscope, which utilized a combination of eye-tracking and other biometrics, as well as a camera test subjects wore around their necks so Innerscope's analysts could sync what the respondents were being exposed to with how they responded to it, consciously and unconsciously.

The study was expensive and conducted with a relatively small sample, but the insights -- as anecdotal as they may have been -- were fascinating. Among other things, it showed marked differences in the way younger (Millennials) and older (Boomers) subjects physically engaged with hand-perceived media.

Observational studies are hard and costly, but the insights they can glean will help inform the total media universe -- not just the ones the industry has syndicated and custom research providers typically focus on, because, well, that's where the money is.

In recent months, I have been surprised -- and written frequently -- about a variety of initiatives by suppliers of big TV-centric companies -- mainly NBCUniversal -- that are in the process of remaking themselves into big digital ones. And in the process, redefining digital ad "video," whether it is the CTV, OTT, streaming, or whatever new forms of TV they are peddling.

What has shocked me most is how much and how quickly the demand-side -- mainly the ad-agency executives I have been trying to get reactions from -- have capitulated and gone all in on these initiatives as their saving grace.

Never mind the obvious implications for "media neutrality" and the inherent bias big agencies have when it comes to working with big media suppliers that make their lives simpler and easier -- even if they tend to reinforce a bias for them in their media mix.

One of the reasons that capitulation has happened so quickly is that many of the people calling the shots in big agency strategy, planning and execution  are people who came of age with digital media platforms -- you know, "walled gardens" -- that have historically done that for them. And they seem to want NBCU to be one of their new walled gardens.

Let me give props to NBCU for getting out in front of this problem and understanding the opportunity it presented. And based on what I observed at last week's NBCU developer's conference, I think congratulations are in order. You won NBCU and you will dominate the conversation for the new TV-centric mix ROI going forward.

Call it "cross-media," or "cross-platform," and say you are using census-level consumer identity data to do it -- the bottom line is that it will tilt the industry more in that way of thinking about mixes and outcomes, not a more expansive view of what the media universe actually is and how it works.

It has been nearly 100 years since Ball State University conducted one of the best-known observation studies about how people actually use media -- the so-called "Middletown Studies."

They used grad students to follow real people in Muncie, Indiana around as they really used media and reported the results.

It was arguably the first, best ROI study -- and the best one for its time.

Earlier in this century, Ball State's Center for Media Design replicated some observational media studies, and ultimately spun off and influenced some commercial research endeavors, some of which were commissioned by the Nielsen-funded Council for Research Excellence.

The CRE has suspended active research projects, but some of its studies can still be read on its site.

If you ask me, the most important media-mix ROI study the ad industry could commission for these times is a really good, scalable observational study to understand all the media that people are actually exposed to, and how it influences what they think and feel and how they behave.