Commentary

Enter The Digital Media Performance Score

by Jason Heller , January 31, 2012

Last week we addressed the rampant problem of misaligned measurement and the key questions we seek to answer from a branding campaign.

The key takeaways are straightforward enough:

1) Establish clear objectives; and

2) Align a measurement approach according to these objectives.

Simple, right? Rule #1: If your objective is “branding” or “awareness,” then measure branding effectiveness. I support this statement 100% and this is my primary recommendation to clients.

However, things are not always that simple and straightforward.

An increasing number of marketers are beginning to question or even dismiss the often-unimpressive single-digit lifts in branding effectiveness measures. Some marketers, including brands with smaller or fragmented budgets that support multiple initiatives, are not budgeting for branding effectiveness measurement on a regular basis, if at all.

Regardless of the reason, there will be a time when every digital media strategist or planner must develop a creative (and analytically sound) measurement approach that is generally aligned with your objectives.

I’ve been working on perfecting a calculation for a normalized index of multiple objectives and priorities that allows for the comparative analysis of campaigns and elements within campaigns. The resulting digital media performance score is not skewed by campaign scale and should be well formulated such that the calculation does not change over time.

Of course, I can’t give away the entire punchline here, but I will walk you through the general framework for creating your own model. This should provide some serious fodder for conversations with your agency/client analytics teams on how to develop your own calculation and model. I welcome feedback, challenges, debates, and ideas on how to improve this approach.

The model incorporates three broad measurement buckets: Reach & Frequency, Engagement and Commerce.

Reach & Frequency: Although there are plenty of issues around the GRP/TRP approach, there is no contesting that reach and frequency must be included in any calculation of a performance score. All else being equal, the efficiency of reaching your target should positively affect your score. Establishing an optimal frequency target can be calculated separately (via branding effectiveness studies) and can also be applied to the model.

Engagement: One of the unique attributes of digital media is the ability to measure engagement -- interactions with rich media, on landing pages and beyond. Ordinarily there are several potential elements of engagement within a campaign. These elements can be weighted based on the relative value of each behavior to the brand. For example, inputting a zip code in a store locator is a higher expression of intentions than watching a video and would be weighted accordingly (maybe in reality it is not, but we have to stick the stake in the sand somewhere). A weighted cost per engagement would be included as a component of the score calculation.

Commerce: While not a primary expectation for a non “performance” campaign, measurable (multichannel) revenue or lead generation should be included in the score as well. The normalized way to do so is simply with ROAS or CPL. These numbers will not look good from a return perspective, but again, this is not the objective of the campaign. We are merely including all of the pertinent measurable elements, whereby any significant fluctuation in one metric will visibly affect the score and trendline over time.

Digital Media Performance Score

The performance score combines disparate metrics that will need to be prioritized and weighted. If running a branding effectiveness study, brand lifts can also be incorporated into the score. Arbitrary coefficients should be used to enable each of the measurement buckets to yield ranges of measures that can contribute to the score in a visual manner. This will prevent any needles in the haystack, so to speak.

How to Use The Digital Media Performance Score

Remember, this score is not an absolute value of anything, but rather a tangible comparative index that is influenced by the efficiency of reaching your target, the level of brand engagement, and the measureable revenue or leads that result. It’s an augment to existing measurement, and a creative approach to the gray area between hardcore direct response and branding effectiveness measurement. This may not be the be-all and end-all of digital media measurement, but it just may be your salvation if you’re still struggling with measuring digital media in general or if you’re somehow still using click rate (gasp, shame).

I’d love to hear your feedback, challenges, or additions to the approach. Leave your comments below or hit me on Twitter @jasonheller

metrics

5 comments about "Enter The Digital Media Performance Score".

Check to receive email when comments are posted.

Caroline Bloomer from Laurinus CCC , February 1, 2012 at 4:16 a.m.
Thanks, good blog, but 'easier blogged than achieved' ... how much do, particularly small, cash-strapped, businesses have to invest on top of their promotional/digital activities for adequate measurement - with tracking on the way to being banned. Do they cust their budget in half: half for action and half for measurement?
Reply

Nick D from ___, February 1, 2012 at 9:12 a.m.

Interesting article, with well-made points. I'd suggest that the industry is slowly heading in the direction of 'RFE' - comScore continues to push the R&F angle with its vGRP launch this year; and MediaMind's sterling work on engagement/ dwell highlights that we need a qualitative metric in addition to the quantitative figures of R&F. There is still a way to go before it's accepted industry practice, and before the industry moves beyond vendor-specific solutions (eg the vGRP from comScore) to one standard, but this would still seem to be the best route.

Can you explain the 'Commerce' metric in more detail though; and how it would scale for big and small campaigns?

And what 'E' value would we give to relatively static, conventional online ads? Video/ rich media is straightforward, but for those without interactive elements, or tracking of time? Does it become an arbitrary percentage of the time spent on the page (1/4 page ad, 1 minute spent on the page - 15s arbitrary 'engagement' time)?

Jason Heller from AGILITi, February 1, 2012 at 3:51 p.m.

@Nick -The commerce metric, actually, all metrics in this model, are normalized so that scale of the campaign won't skew it per se. By using ROAS, CPL, and/or some type of measurable offline metric that comes back to a 'cost per', the score for that bucket will not be affected by scale unless scale itself improves performance of the bucket.

The 'E' value for static ads would be engagement with landing pages/website activities - it will be a smaller number, and thus would impact the 'E' portion of the score accordingly.

This model changes a bit for online video, when the objective can be "passive exposure" (sometimes online video budgets are augments of TV budgets) and not brand engagement in the same interactive way that rich media provides. In this instance the 'E' can be a metric tied to duration of exposure, but this is a very different form of engagement and not an apples to apples comparison. That said, the difference in metrics and measurement of display/rich media and video has been a tough point generally.

The performance score as outlined in this article is really for display (or interactive video).

Annie F from Analytics, February 6, 2012 at 7:42 p.m.

Thanks for this article - it was exactly what I was looking for in terms of setting up a metric for a branding campaign for a typically DR focused client. Your combination of Reach and Frequency along with engagement makes a lot of sense and aligned with the direction we were moving in. However, I was surprised to see your inclusion of measurable revenue. In my mind, ROI is similar to CPA which I believe you stated in your previous post as not being a valid proxy for branding effectiveness (which I totally agree with). Can you explain why you feel that ROI, but not CPA is a valid measure of branding?

Jason Heller from AGILITi, February 7, 2012 at 3:13 p.m.

@Annie - neither ROAS nor CPA is a valid measure of branding, of course. The reason why "commerce" has a bucket in the performance score is to add the dimension of measurable sales as part of the score to whatever weighted degree you see fit. It should be weighted less than the other metrics so as to not overpower the score. ROAS can certainly be swapped out for CPA. They both do not take scale into account, which is the point. Thanks for the reply.