• by Marissa , Ken Takahashi , June 29, 2005

I spend a great deal of my time visiting clients and other organizations in the e-mail industry and in any discussions that I've had recently, deliverability is a red-hot topic. But there is a lot of confusion around what this term means, and some of this confusion is attributable to e-mail service providers. For example, ask any e-mail service provider to describe their delivery rates and you will get as many definitions as the number of vendors that you ask.

The truth about e-mail deliverability is that no vendor can truly measure it. That is, no vendor can provide complete accuracy in calculating precisely how many intended recipients actually received an e-mail. This is not a failure on the part of any individual e-mail service provider, but a limitation of SMTP (simple mail transfer protocol) technology.

Fundamentally, however, e-mail marketers are concerned with actual inbox delivery and any reason their mail isn't delivered. Methods to measure these vary widely and the actions a company should take to combat these challenges differ as well.

In general terms, there are direct ways and indirect ways of determining if a message goes undelivered.

In the direct way, the marketer receives a bounce-back message from the receiving server for the address. While this is commonly referred to as a delivery rate, it is really a 'non-bounced' rate. A bounce-back message generally has one of the following root causes:

1) The e-mail address is invalid, either because it was entered incorrectly at point-of-capture or because the recipient has changed their address.
2) The address is temporarily not receiving mail, either because of a full mailbox or because of receiving server issues.
3) Mail has been flagged as spam by the receiving server, either because of customer complaints, incorrectly configured mail servers, or content that is confused with spam.

E-mail service providers attempt to classify bounces into "hard" and "soft" categories, but this isn't an exact science, given the lack of consistency in classification across ISPs and their receiving inbound e-mail servers. The e-mail service provider and sender must also decide how aggressively to treat bounces. Essentially, they must walk the line between not continuing to mail known bounces, and not giving up on a customer relationship too early. This also makes comparison of bounce rates across companies somewhat meaningless because it is primarily dependent on the accuracy of addresses captured as well as business decisions such as how often to retry mailing to a bounced address.

Often, however, when an ISP is blocking or filtering messages, there is not a bounce back provided, and hence, there is no direct indication of the problem. To understand this issue (in fact this is where most companies have problems), you need to take indirect measurements of the issues. This is best accomplished through seed-list testing, which involves the creation and monitoring of sample e-mail accounts using the same ISPs as those on a marketer's customer file. By monitoring the success for these sample accounts, it is possible to estimate the level at which your e-mail may be being blocked by ISPs, which not only provides a more accurate delivery rate, but is also an opportunity to work with the ISP to learn about why your e-mail is being blocked and hopefully have that decision overturned.

Importantly, seed-list testing not only allows senders to estimate messages that were not delivered at all, but also messages that were filtered into a bulk or spam folder. In many respects, a seeded delivery rate provides the best measure of what is truly happening with a marketer's mail from a blocking or filtering perspective, whereas bounce rates are much more of a measure of list hygiene.

So, given these issues, what should marketers do? First, they should understand that a lot of confusion exists around what the term delivery rate means, and be careful in making comparisons. Secondly, they should work with a provider that can provide insights in to these different aspects of delivery. And, finally, they should dig into their data and identify whether any delivery problems that they have are driven by list hygiene or by ISP blocking and filtering. By digging down to the root causes, marketers can take corrective action and improve any delivery issues that they have - whatever way you define it!