• by George Bilbrey , Op-Ed Contributor, August 4, 2010

In my last article for this column, I provided a daily checklist for deliverability.    By following this checklist, you may find that you've run into a deliverability problem.  In today's column I'm going to help you determine why you have a delivery problem.  Next month I'll show you what to do about it.

Let's put on our lab coats and stethoscopes and dive in.

Step One:  Gather Data

Much like Dr. House, you'll need to gather a lot of data to get a good diagnosis for you problem.

SMTP Log Data - SMTP log file information is a record of the conversation between your mail server and the receiving mail server.  Whoever is in charge of sending your email -- whether in house or outsourced -- can get you this data. The log file is frequently the most important bit of information in diagnosing a delivery problem.  Many of the larger ISPs provide good diagnostic codes in their bounce messages.  Usually it's a URL that links to a postmaster page much like this page [http://postmaster.aol.com/Postmaster.Errors.html] at AOL. 

Reputation Data - Over 80% of delivery problems are due to poor reputations.  Start by creating a list of IPs and domains that are having delivery problems and which ISPs they are failing at. Then, for each IP and domain, gather these metrics: 

o    Complaint Rates - You can sign up for feedback loops at ISPs like AOL (http://postmaster.info.aol.com/Postmaster.FeedbackLoop.html ), Microsoft (https://support.msn.com/eform.aspx?productKey=edfsjmrpp&page=support_home_options_form_byemail&ct=eform ts&st=1&wfxredirect=1), and Yahoo (http://feedbackloop.yahoo.net/ ).  High complaint rates often drive delivery problems. Reputation monitoring tools like those used by Return Path also include complaint rate data for select ISPs.

o    Spam traps  -  Spam traps are email addresses that should not be receiving commercial email - usually because they were created by the ISP/blacklist operator but never signed up for any messages.  Best sources for finding out if you have spam traps on your list are Microsoft's SNDS service (see below) and Sender Score (www.senderscore.org).

o    Blacklist review - There are a variety of open source blacklist checkers out there.  Please note that some blacklists put nearly every commercial mailer on their lists and aren't widely used.  Knowing which ones matter and which ones don't is important.

o    Unknown users -  Your mailing system or ESP should be able to provide you with the percent of addresses on your list that are "dead" addresses.

o    Unsubscribes - Trends in unsubscribe rates are interesting as well.  It is helpful to look at unsubscribe rates by campaign, type of email (e.g., newsletters vs. offers) and other dimensions.

o    Open/Click Trends - Look at open and click rates by campaign, by type of email and by customer group (e.g., subscribers that were acquired from a particular source) over time to see how each aligns with reputation data and inbox placement rates (IPR).

o    Microsoft's Smart Network Data Services -  SNDS [https://postmaster.live.com/snds/index.aspx] is an incredibly rich source of data for your reputation at Hotmail. 

o    A free, single place to go to collect a lot of this information is the Sender Score site (www.senderscore.org).

Step Two:  Rule out Content Problems

Less than 10% of the time delivery problems are caused by content issues. That is, it's not IP or domain reputation that is causing the problem, but the content of the message looks "spammy."

The quickest way to determine if you have a content problem is to send the message to seed addresses from a clean IP address.  "Clean" means that the mail server (IP address) sending the mail isn't having delivery problems at the ISP in question.  If you send out the same creative, and it has a delivery problem from the "clean" IP (the reputation is good), then it's likely the content.  Many mailers maintain a clean IP, either with another ESP or in-house, to perform this content test.  Deliverability vendors will frequently have tools to perform this kind of test.

Step Three: Rule Out Mail Server Infrastructure Problems

There is a very long list of checks followed by mailbox providers when determining whether a mail server looks like it's one that sends spam.  We don't have time to list here.  If you are using an ESP, it's likely that your infrastructure is pretty sound.  I'll address what a well configured infrastructure looks like in a future article.

Bottom line: The Information You Need is Within Your Reach

We find we can diagnose more than 90% of deliverability challenges by reviewing the log files, assessing the reputation metrics and ruling out content or infrastructure problems.   In tandem with digging into this data we ask the marketer what changes they've made to their programs.  Significant changes in data sources, mailing frequency (both increases and decreases) and creative strategy can all lead to changes in reputation that will, in turn, lead to lowered inbox placement rates (IPR).  In some cases we find that the deliverability failure is due to a change at the ISP.  What's interesting is that even when the change is on the ISP side, the data we gather on the marketer's side inevitably leads to a quicker resolution.  I'll expand more on why that is so in a future column.