• by Dave Lewis , March 8, 2007

When people say "go figure," they're probably not imploring you to do a mathematical calculation. More likely, the phrase is being used in the colloquial sense of "what's the meaning of this?" and would be followed by ample head scratching. Regardless, you'll want to apply both meanings after reading "The State of Email Metrics & Bounce Management," a white paper just issued by the Email Experience Council.

The white paper analyzes the results of mailer and ESP surveys fielded by the eec's deliverability roundtable. "The survey results make it unequivocally clear that demystifying our metrics is a critical step to moving our industry forward," notes Deirdre Baird, CEO of Pivotal Veracity and chair of the roundtable,

The head scratching "go figure" begins when contemplating the impact of the wide variance in how our industry calculates key metrics -- delivery, open and click rates. You'll be scratching your head harder when you consider the different interpretations of basic deliverability terms. And you won't have any hair left when you put yourself in the position of those who lack the data or visibility into their results to do much about them.

The implications of the findings are profound. Aside from being unable to properly assess campaign performance, the confusion around metric and delivery definitions makes it impossible to produce reliable industry benchmarks and make valid comparisons across service or solution providers. Such confusion even threatens email as a channel when its contribution can't be measured and compared to others'.

Now comes the "do the math" part. If you doubt the significance of the findings, take a look at how the various methods play out on a hypothetical campaign:

1,000M mailed (more if retries included)

800M delivered

100M "permanent" bounce; 50M bad address

240M opened

120M total clicked; 80M unique clicked

Which click rate is right -- 8%, 10%, 12%, 15%, 33%, 50% or something entirely different? They're all right, depending on who you ask. And similar variation would apply to the other metrics.

Having done the math, you'll at least know the potential pitfalls and where to ask probing questions the next time you hear click rates or other results quoted. Now apply that critical thinking to what's reported by your own service provider or in-house solution. Ask yourself whether you're getting the data to properly assess performance and manage your business, such as the reasons for hard bounces. If your answer is "no," or you've got gaping holes in the data, take solace in knowing that you're not alone. Or better, you can seek out solutions that don't leave you so handicapped.

This is direct marketing, so the "go figure" imperative to do the math will always be with us. But I'll be exceedingly disappointed if we're still in the "what's the meaning of this?" mode a year from now. It's a mode we must outgrow to realize the full potential of email and gain acceptance as a true direct-marketing discipline. Whether we do so or not depends in large measure on you as an email marketer. It's up to you to demand full disclosure on how your metrics are calculated -- and become a vocal advocate for change.