Commentary

A Tale Of Two AI Measurement Futures: Dystopian, More Utopian

The Advertising Research Foundation’s CIMM (Coalition for Innovative Media Measurement) summit got off on a potentially dystopian note in New York City Tuesday morning, kicking things off with a keynote from Omnicom Media’s Ben Hovaness warning that the industry’s embrace of AI could exacerbate distortions already well underway in media measurement.

“If you think that this is an overly pessimistic view, I want to assure you that it isn’t, because in some corners of the industry, this is how we’ve been operating for years,” asserted Hovaness, Omnicom Media’s executive vice president of specialized capabilities, noting that it’s already been five years since “modeled conversions” became “the standard practice in certain quarters of the industry,” citing Google’s hybrid approach to comingling empirically observed conversions with modeled ones, albeit with little or no transparency on which is which.

advertisement

advertisement

“We have the opportunity to use AI to usher in a Golden Age of measurement, but only if we choose it,” Hovaness offered, adding that unless industry action is taken now, “AI will actually take us to a darker, more obfuscated place.”

I tried to compress two of Hovaness’ slides illustrating the pre- and post- into the one depicted below. In both cases, Hovaness described the scenarios as self-perpetuating “loops.” The not-so-utopian loop above is the scenario Hovaness describes as exacerbating the role of modeled conversion data that ends up in various industry ad effectiveness models – MMMs (marketing mix models), “agile MMMs,” incrementality testing, etc.

Without improvements in the transparency between modeled and empirically observed results, Hovaness implied the dystopian scenario will only get worse vis a vis agentic systems extrapolating synthetic data and making it worse.

“Vendor reports performance. And then the buyer’s AI agent – agents working on behalf of buyers will be ubiquitous in 2031 – they’re ealready increasingly wide-spread today, so that doesn’t seem far-fetched,” he noted citing the following potential process:

  • Vendor reports performance.

  • It gets fed into the buyer’s agent.

  • The buyer’s agent says, “Oh wow, this vendor is reporting a lot of conversions.”

  • Now this buyer’s agent does a budget reallocation based on that very favorable assessment by the vendor.

  • It leads to more media being purchased.

  • Which generates conversions – both real and more modeled conversions on top – and the loop just perpetuates itself.

Hovaness noted that such “bad measurement” practices has “two sets of victims” – the buy-side, as well as the sell-side – because it would likely lead to “overinvesting and overpaying for advertising” among media sellers using bogus data at the expense of media suppliers using more empirical or representative data.

In the more utopian loop scenario, Hovaness made the case that AI can be used to “collapse the time requirements and the labor requirements to do good studies, analyze the results and integrated them into models,” effectively accelerating the turnaround time and reducing the cost of good quality studies to produce them more frequently and update models to make them more representative of the most current marketplace conditions.

“That means we could be headed towards a world with a type of measurement that’s quite routine and no longer such a pain,” Hovaness envisioned.

Next story loading loading..