Advertisers are demanding greater accountability. They are looking for data points that help to reduce risk and maximize the sales return on their marketing investments.

For many, the data points they look toward are contained in marketing mix modeling results. Specifically, what is the sales contribution of each investment (e.g. spending in TV, radio, couponing, etc.) and what is the corresponding sales return on investment?

Building marketing models requires assembling a historical database of key performance indicators (KPIs like sales, gross margin, etc.) and a complete set of factors believed to influence the selected KPIs.

Under-Measurement of Impacts

There is a history of marketing mix models understating the contribution of media channels that have been incorrectly expressed within the models. The result has been a distorted view of sales contributions, ROIs and optimal cross channel media budget allocations. These allocations have tended to favor television.

Analysts charged with quantifying impact for each sales driver have been frustrated by the difficulty of isolating display and online video impacts. Modeling these media may benefit from recent investigations into how best to express media activities within the model development process.

Industry Investigation

In recent years, industry stakeholders for the magazine and radio fields have separately initiated investigations of the under-measurement problem for their respective media.

The results of these investigations have been both extensive and beneficial. In both cases, improved ad exposure measures (inputs to the models) were developed and implemented. Measured sales contributions and ROI estimates increased. Contributions were found to shift toward the historically under-measured channels.

The Magazine Industry Inquiry

The magazine industry initiative was spearheaded by the Magazine Publishers Association with the involvement of GfK MRI and several well-established modeling consultancies.

Examples of findings include:

1. More specific media inputs yielded a better match to marketing outcomes suggesting the use of weekly data based on issue-specific measurement (to capture audience accumulation) at a market-level.

2. GRPs (rather than spending data) are a truer measure of ad exposure. Target audience GRPs should be preferred over more broadly defined GRPs, e.g. based on all adults or households.

These suggested practices, which were tested by the modelers, resulted in increases in magazine sales contribution. The contributions and ROIs of other media channels decreased proportionately with the improvement in magazine performance. Performance improved at above threshold exposure levels.

The Radio Industry Inquiry

The radio industry initiative was spearheaded by Arbitron, the Westwood One and Premiere Radio Networks.

Up to this point, modelers reported using a variety of data types, including expenditures, and planned rather than actualized weight levels. Target rating points were not the standard. Arbitron radio diary data (reported as a 3-month moving average) was being used to feed the models.

Arbitron had recently introduced the PPM (Portable People Meter). It provided a continuous (passively measured) stream of radio listenership. Integrating PPM data with commercial occurrence data from Media Monitors produced a more exact measure of ad exposure.

Examples of findings include:

1. GRP delivery data for each commercial occurrence should be built up to weekly, rather than a multi-week rolling average. This data is more variable and was found to align more closely with the KPIs considered within the radio industry study.

2. Local market measurement should be used to capture variation in local market delivery, e.g. to account for varying radio network clearances.

3. Not all plans are strong enough to produce a measureable effect in the models.

Like the magazine case above, these practices tended to increase radio sales contribution. Performance improved at above threshold exposure levels.

The outcome of utilizing more consistent and precise measures of ad exposure benefited the previously under-measured channels through improved statistical significance, increased sales contributions and larger ROIs.

A bigger picture benefit was a redefinition of the “optimal” cross-media allocation.

The Implications for Display and Online Video

Using the above cases as examples of approaches to resolving the under-measurement problem, there may be opportunities for improving display and online video ROIs.

Some steps that may be taken to improve the estimated return on investment for display and online video include:

1. Utilizing “as ran” as opposed to “planned” impression delivery.

2. Utilizing “target” rather than gross measures of ad exposure.

3. Utilizing “viewable” impressions (for online video – completed views)

4. Seeking opportunities to align impression delivery to geography.

5. Spending at “above threshold” levels to assure the effects are measurable.

6. Varying impression levels – independent of other channels – to statistically isolate display and

   OLV impacts.

The statistical jargon for the problem that this paper discusses is “measurement error.” Factors that are incorrectly and inconsistently stated tend to be under-measured. This paper reviews methods that have been used in recent years to resolve the under-measurement problem in media.

A consistent theme throughout the investigations completed for the magazine and radio industries has been the work-intense nature of developing newer, better estimates of ad exposure. In both instances, syndicated solutions have been developed for creating the more detailed modeling inputs.

Improving the quantification of display and online video contributions (and, hence, ROI) is going to require systematic efforts to build databases of accurate, consistent and complete historical KPI and exposure data. This needs to begin as soon as possible to lay a foundation for future improvements in the estimation of display and OLV sales contributions and corresponding return-on-investment.

The benefits of more accurate estimates of display/OLV contribution are obvious.