The white paper analyzes the results of mailer and ESP surveys fielded by the eec's deliverability roundtable. "The survey results make it unequivocally clear that demystifying our metrics is a critical step to moving our industry forward," notes Deirdre Baird, CEO of Pivotal Veracity and chair of the roundtable,
The head scratching "go figure" begins when contemplating the impact of the wide variance in how our industry calculates key metrics -- delivery, open and click rates. You'll be scratching your head harder when you consider the different interpretations of basic deliverability terms. And you won't have any hair left when you put yourself in the position of those who lack the data or visibility into their results to do much about them.
The implications of the findings are profound. Aside from being unable to properly assess campaign performance, the confusion around metric and delivery definitions makes it impossible to produce reliable industry benchmarks and make valid comparisons across service or solution providers. Such confusion even threatens email as a channel when its contribution can't be measured and compared to others'.
Now comes the "do the math" part. If you doubt the significance of the findings, take a look at how the various methods play out on a hypothetical campaign:
Which click rate is right -- 8%, 10%, 12%, 15%, 33%, 50% or something entirely different? They're all right, depending on who you ask. And similar variation would apply to the other metrics.
Having done the math, you'll at least know the potential pitfalls and where to ask probing questions the next time you hear click rates or other results quoted. Now apply that critical thinking to what's reported by your own service provider or in-house solution. Ask yourself whether you're getting the data to properly assess performance and manage your business, such as the reasons for hard bounces. If your answer is "no," or you've got gaping holes in the data, take solace in knowing that you're not alone. Or better, you can seek out solutions that don't leave you so handicapped.
This is direct marketing, so the "go figure" imperative to do the math will always be with us. But I'll be exceedingly disappointed if we're still in the "what's the meaning of this?" mode a year from now. It's a mode we must outgrow to realize the full potential of email and gain acceptance as a true direct-marketing discipline. Whether we do so or not depends in large measure on you as an email marketer. It's up to you to demand full disclosure on how your metrics are calculated -- and become a vocal advocate for change.