
While attending the ANA's recent Measurement
& Analytics Conference I couldn’t help but appreciate a deep irony running through the marketing mix modeling (MMM) supply chain as it evolves from a cottage industry to industrial-grade
analytics. The tension is simple: are we a media tool, or a tool to defend marketing investment in the boardroom?
As an industry, we want to move from “interesting
analysis” to a disciplined process company boards and CFOs can use to explain results, shape guidance, and move budgets. As marketers, it’s an exciting moment to define a toolkit that can
defend and win budgets in increasingly competitive corporate environments.
But it’s also fraught with regulatory risk. And regulatory knowledge is not something marketers
have always been savvy about.
To my mind, one of the greatest risks globally is the rapid rollout of open source MMM across the market, with little to no consideration for the
implications of unreliable ROI measurement.
The stakes are getting higher as some of the biggest digital media platforms -- Meta's
Robyn and Google's Meridian -- are broadly accelerating adoption of mix modeling tools that once were relegated to the most sophisticated users
in the inner sanctum of marketing.
The moment MMM influences numbers or narratives that boardrooms rely on to talk to investors about decisions, it triggers some risk for
publicly-traded companies in the U.S.: Sarbanes-Oxley.
For those who don't recall, Sarbanes-Oxley is a law requiring CEOs and CFOs of publicly-traded companies to certify the fairness of
reporting. It means management must:
- Design and assess effective controls over financial reporting.
- Auditors must be able to test the evidence.
- Audit-relevant records must be preserved, not overwritten or “lost.”
For MMM to be used at the boardroom level, that means it must be a regulated project, not just a marketing project.
Let's drill into it a little deeper and consider how Sarbanes-Oxley works.
Section 404 requires management to assess the effectiveness of internal control over financial reporting and, for larger issuers, for auditors to attest to that assessment.
Section 302 puts the CEO/CFO signature on the line. Audit record-retention rules require keeping the work behind conclusions for seven years, including materials that contradict the final
view. If MMM informs what you say publicly about marketing effectiveness the model, the inputs and the governance frameworks become audit-relevant material.
Why MMM falls into this scope is straightforward. When you use a model to explain “what moved
revenue,” “how marketing drove margin,” or similar MD&A claims, auditors evaluate the data as they do for any accounting estimate.
The Security and Exchange Commission's
(SEC) KPI guidance also demands clear definitions, consistent methods over time (for example: you shouldn’t be building and tweaking a random, bespoke model every 3 months made up by an
analyst), and disclosure of changes that would alter how a metric is read. If MMM underpins a KPI at the C-Suite, expect auditors to want productionized pipelines and governance programs
alongside them.
Most MMM programs were not built with this standard in
mind. That has to change. Auditors need to see what changed, when, and why. Record-retention rules make the archive non-negotiable.
Independence is the next pillar. Two decades ago, regulators forced hard separation between
investment banking and equity research because conflicted “measurement” misled investors. Advertising has a similar conflict structure. If you own, sell or trade media, you should not be
the authority measuring its financial impact when that output may inform investor communications.
Similarly, there has been a divorce between consulting firms and audit
functions for the same reasons. You can’t muck around giving business advice and also certify the result of that advice. The outcome is clear:
In measurement, we have
similar patterns emerging.
For example, today many technology companies who sell advertising are rapidly rolling out large scale MMM programs. Those technology companies are also
building analytics platforms to control the reporting and scorecard of the inventory that they sell. It’s an open question as to whether this meets the threshold for internal controls.
We’ve seen this movie before. The Global Research Analyst Settlement makes it exceedingly clear that research and trading must
be separated. It’s pretty obvious when you think through the principles that the same might need to apply here.
In-house teams can face similar problems. For example, if
someone directly reports to a manager with a KPI to improve marketing ROI (as most marketing effectiveness teams would) and builds the MMM, this would be seen as a breach of internal controls. If
priors are placed in as "business context," which really reflect what the business wants to believe, this could be seen as a breach of internal controls.
And if you don’t
validate your models with robust, reproducible standards then you would be caught not governing them properly. It’s critically important, then, that we level up governance.
How should we navigate this challenge?
Firstly, we need clear governance standards for Enterprises running large MMM programs.
Secondly, we need to recognize
that conflict of interest in measurement is a standard that major advertisers cannot accept if they want their measurement programs to inform board-level numbers. There’s a stark choice: work
with conflicted programs may invite regulatory scrutiny. It’s time for marketers to make the choice to grow up from the drug of ‘free’ and invest in robust measurement programs that
exist independently from potential conflict .
Thirdly, we need better audit trails as third-party MMM providers.
That means leveling up our data ingestion practices and
logs, having robust governance that is radically transparent and ensuring methodologies and model versions are logged over time. That’s a big shift from the consulting mindset that pervades the
MMM industry. But it’s one that is necessary if we want to be credible partners in boardroom.
Marketing measurement aims to deliver in the boardroom, defending and
explaining why marketing is an investment, not a cost. If we’re to get there, it also means acknowledging that it’s time to grow up as an industry on governance and managing conflict as
well.