'Any Idiot Can Optimize One Variable'
That was how one former colleague expressed his frustration when a new employer framed all the company's goals in terms of new customer orders. A predictable consequence followed: managers dug out every kind of discount packaging ever hatched by the JV sales team in pursuit of scoring orders; and lo and behold, new customers came and ordered -- almost as quickly as profit margins fell. It's easy for most of us in online advertising to chuckle at a story of one-metric management, given that as a group we have more performance data than any other group outside of the Federal Reserve.
In fact, thanks to the measurement systems of publishers, agencies, advertisers and Web measurement companies, we can analyze the success of our ads and campaigns seven ways to Sunday and still have new options left for Monday. Reach. Reach in target. Increase in brand awareness. Clickthrough. Viewthrough. Conversion. eCPM. Cost per action. Estimates for lifetime customer value. Ongoing measurement of customer value. The list goes on, and all these metrics can be delivered fresh to your inbox, daily.
Even with the plethora of data available, how many planners and buyers focus only on one metric when setting goals, judging success, and optimizing? Surprisingly, the majority of them. The metric of choice varies, especially across industries -- retailers set their compass by cost-per-transaction, while a consumer electronics manufacturer might dwell exclusively on unaided awareness -- but devotion to a single metric is very common and very dangerous nonetheless. Sure, narrowing your measurement focus to one variable reduces opportunities for confusion, delayed action, and for decision-making hinged on too much complexity. However, taking this mono-metric approach also shuns the potential power of the data at our fingertips, leaving marketers blinded to new opportunities and ill-equipped to achieve our objectives.
Two metrics are better than one
If online advertisers entirely lose ourselves to one metric, then we aren't as smart as we could be. This is most obviously true when devotion to one metric leaves us ill-equipped to act on a secondary objective. Consider the media buying group of one advertiser where the metric of choice is cost of acquisition, and media planners are also directed to make "strategic" buys on key sites. Once that word "strategic" is invoked, planners not only get to set aside devotion to the single metric - they get to set aside reference to any metric whatsoever. The result is.... well, nobody can tell what the result is.
There's growing awareness in the industry that all campaigns inevitably yield both direct response and branding value. Given all the data that's available, it's critical that media teams evaluate success in at least two dimensions. A baseball team has to rate its shortstop both as a batter and as a fielder. Likewise, the direct-response advertiser should still have a measurement plan and process that evaluates branding value, even if only by a gross measure like exposure.
One metric also becomes a barrier to success when it leaves us out of touch with the other variables that can be directly influenced or accurately predicted. We don't always get to choose cost-per-action, after all -- instead, we make changes that can favorably impact rates at different points along the path to action. We need to maintain a view one level deeper -- e.g., of both open rates and take rates on e-mail campaigns, since those are variables we can truly impact by changing subject lines and calls to action. We all know this, perhaps, but do we always do it? Do we do it routinely enough to know whether open rates or take rates are easier to influence given the audience and offer? It becomes all too easy to simply repeat what's produced the highest "score," and to lose our grip on the levers that might enable even greater success.
Leaving opportunities on the table
Afflicted by a case of mono-number-osis, online advertisers also lose the ability to make new discoveries -- both about underlying bellwethers that might be open to influence and prediction, and about the shortcomings of our current charting. We all know that simply taking a longer-term view of the "one metric" will often necessitate a revised evaluation of campaign choices. A longer-term view of several metrics might reveal that increases near the top end of the funnel (reach, brand awareness) do materially contribute to success on the bottom line.
By going even deeper and by looking at more discrete metrics, it's possible to make truly tactical, actionable discoveries - perhaps the key indicator of long-term "direct-response" impact for a "branding" ad is the amount of research on review sites and queries on search engines stimulated by the ad. In the research and measurement business, we uncover surprises like this all the time.
Baseball players don't improve by deciding to hit more, they improve by discovering a better way to swing. That is to say that we can't help but manage to one metric sometimes; and we shouldn't stop trying to look for a single metric that could serve us better.
Maybe one company moves from short-term, cost-per-action to a metric that takes lifetime value into fuller account. That kind of innovation, the development of a new, better shorthand for success wouldn't be idiotic. It would be efficient, effective, and smart.
But standing pat on that one new metric, forgetting the elements that feed into the new metric and facilitate action and prediction limits the success of campaign -- that old colleague might say it's not a very wise idea. We have so much data, and so many numbers, and they contain so much value. Let's not forget that we have to keep more of them on the table and triangulate -- not just calculate -- in order to outfit ourselves for greater success.