• by George Bilbrey , Op-Ed Contributor, July 26, 2017

I’ve found that companies in many different fields — finance, dating, travel, e-commerce, etc. —  all have one thing in common: they really dislike the reporting they get from their email service provider (ESP). It's a dislike that borders on loathing.

Let me give you a couple of scenarios where ESP reporting falls short.

Scenario 1: The company had falling open rates. Strategists were sure that they had a delivery problem, reasoning that because nothing else had changed, delivery had to be the culprit.

A review of their deliverability data indicated that they had a modest delivery problem at Gmail, but the delivery problem was centered around subscribers that rarely opened messages from the company.

But the data indicated that actually something else had changed. The company was sending more mail to subscribers. This was driving lower open rates (but more overall opens per subscriber). Sending more mail did drive increased unsubscribes and complaints,  but the increase in open rates overwhelmed the increased terminations. So, in the end, there was nothing to worry about. Unfortunately, its ESP reporting only showed declining open rates.

Scenario 2: The company saw dramatically increasing open and click rates. Good news! It had been spending some time on improving its subject lines, so this must be the cause, right? Maybe its  inbox placement had improved?

In fact, its deliverability had not improved much. Reviewing sending behavior, it seemed the company was sending to fewer but more active subscribers. Looking at historical open rates for that group, normalizing for the kinds of messages being sent, results were about average. So the work on the subject lines wasn’t really paying off.

There was some good news, however. Because the company was sending mail to its most engaged subscribers, unsubscribes and complaints (terminations) per subscriber fell pretty dramatically. A little “back of the envelope” math showed that the expected 12 month revenue from sending only to more engaged customers was actually higher.

Based on these examples, how should ESP analytics change? Or, if marketers have the time, what kind of analytics should they build themselves?

Look at key metrics on both a per-message basis (e.g., open rate) and on a per-subscriber/week (or month) basis. Per-message rates are misleading. They vary a lot by whom you are sending to (e.g., suppression) and how much mail you send. At the end of the week, what matters is how many opens, clicks, and conversions you have. Looking at both rates per message and per subscriber, you can get a better feel of how you are doing for these metrics that matter.

Include deliverability metrics measuring true inbox placement with open, click, and conversion data. How else can you know if a change in a key metric like open rate is based on a delivery problem or something else?

Deliverability and open/click/conversion data should be “segmented” by historical activity level. A delivery problem at Gmail that is clustered among clients who have rarely opened or clicked isn't really a problem. If your open rates are the same among the most active subscribers but your overall opens are increasing,  you are probably just mailing more frequently to you best subscribers.

Opens and clicks should be looked at on a per-receiving mailbox provider (ISP) basis. This is another good way to understand if you have a delivery issue.

Reporting should look for what has changed, particularly in sending behavior. Are you sending to new subscribers? Have you changed the suppression policy and are now only sending to the most active subscribers? Are you sending more frequently? Did you send a new kind of content?

What do you think? What other changes would you like to see in ESP reporting?