• by daniel.c , Melinda Krueger , February 21, 2006

Many thanks to everyone who responded to my informal survey about what you'd like to see in the Email Insider. Your feedback was very helpful and will be the basis for these columns for a long time to come.

One common theme among the responses was a desire to see case studies and results. The reason you don't see more of them in this and other columns is that clients are not eager to share their hard-won knowledge with the outside world, particularly their competitors. But fear not, there are ways to get around this obstacle.

Assuming your goal is to see how your program stacks up against others, and get ideas to improve your program quality and results, I recommend three things:

1. Look at industry stats. If you are interested in being able to tell your client/manager how your e-mail efforts compare to others, there are two good sources of response rate data: Marketing Sherpa and DoubleClick E-mail Solutions. There are always two questions you must ask as you look at industry stats: are they applicable to your particular program, and did they calculate their numbers the same way you do? If you can't answer these questions, it may be much more useful to track your own data and see how your program compares to itself over time. For every e-mail sent, I do a report with a results summary like this:

The elements of your snapshot will vary depending on your program, e.g., you should include purchase or conversion data if applicable. It's a simple thing, but by comparing a single effort to the program overall, it causes you to ask why this e-mail was better or worse than the average. This, in turn, should inspire you to develop theories to test in the future. As you look at your overall results for the year (this data for all e-mails in one table), you can draw additional insight from the star performers and the dogs.

2. Look at what your competitors are doing. It's easy to sign up for your competitors' e-mail programs. If your competitors' programs are not up to your standards, look at a comparable industry with bigger budgets. Although we're not supposed to plug products in these columns, I've found that a truly exceptional resource is the E-mail Analyst. Similar to the direct response offering Who's Mailing What, the E-mail Analyst tool allows you to retrieve and review e-mail programs by several search criteria.

3. Grow your own. The best insight into what resonates with your customers will come from tests that you conduct. Develop a theory, randomly split your list into two groups, hold all factors constant except for the test variable and see what happens. I have found great reluctance from my clients to expend the time and effort necessary for testing. The attitude seems to be: "It's hard enough to get one e-mail out on time, and now you want me to do two versions?" But this is the best way to learn, and a small investment compared with the overall cost of an e-mail program.

Some test ideas will come up in the creative process: the client/manager doesn't like your approach and suggests another; you believe your approach is best. The knee-jerk reaction of direct marketers in this situation is, "Let's test it." It should become your mantra as well. Another way to ensure that you are learning from your efforts is to develop a list of things you'd like to test and work your way down the list. Remember to test big, and you'll be on your way to major program improvement.