| ||||||||||||
PRINT
COMMENT
SUBSCRIBE
RSSUsing the open rate to measure email marketing performance is akin to the music industry measuring sales based on the number of CDs sold. It just doesn't reflect how people buy music today.
How did we get here? The email open rate became a useful metric once HMTL messages became the primary email-message format, because it counted each time a tracking image loaded when the recipient opened an email.
This allowed marketers and publishers to measure how many people were "reading" their emails, distinct from those who also clicked on the email. Still, this metric was not a great measure because it didn't distinguish between someone who reads every word in a newsletter and someone who opens and then deletes immediately.
As soon as the major ISPs and email clients began disabling images by default, though, the open rate's accuracy and value plummeted. When combined with growing use of the preview pane, emailers got the double whammy of metrics mush.
Various studies (including my own from a few years ago) support that both consumers and business users of email view from a quarter to a half of their emails in a preview pane and 50% to 60% percent block images by default.
Now, add text emails to the mix. Someone, please, tell me how you can view open rates as an accurate and meaningful measure of anything other than the percentage of your HTML emails where images rendered?
Here are some real-world examples of the inaccuracies and inconsistencies of email opens:
Meet Loren McDonald at Email Insider Summit Utah!
Loren McDonald will be there speaking during "Looking Past Email Measurement" on December 09 at 9:45 AM. Top executives will be there. Will you?
Register today and save.
Register today and save.
· The email is "opened" (launched), but images are blocked: not counted as an open
· The email is not opened (launched), but images are enabled and is read in the preview pane: counted as an open
· The text version of a multi-part message is read on a BlackBerry. The HTML version (with images blocked) is later opened in Gmail (or other email service/client). The email has been opened and read twice -- but zero opens are recorded.
· A text version is opened and read but not clicked: not counted as an open
· A text version is opened and read, but the user clicks a link: not counted as an open with some email software. Others assign an open because the email was clicked on, which assumes an open.
I think you get my point. With marketers increasingly being held accountable for their marketing spends and actions, do they really want to base performance reports and marketing decisions on such a flawed and inconsistent metric?
Further, the open rate is a process metric that does not measure return on investment or how well the campaign helped you achieve a strategic initiative for your company. Showing how much email contributes to the bottom line, not how many people opened your email, will help you secure a bigger share of the marketing budget.
So, what good is the open rate? Some of my peers line up behind it, agreeing that it's flawed but still provides some diagnostic value (e.g., click-to-opens). Maybe I'm getting crotchety in middle age, but, come on, folks, we can do better than "well, it sort of works."
In a time when email continues to fight for its rightful place in the marketing mix and budget, our industry must communicate the channel's success with accurate and meaningful success metrics.
Over the years, I've tried to get marketers to think beyond the open rate to measure email performance, from replacing it with the more useful click-to-open rate, to creating "balanced scorecards." I'm now acting as a champion of both change and standardization as co-chair of the Email Experience Council's Measurement Accuracy Roundtable with David Daniels, vice president of JupiterResearch.
If you are passionate about helping the industry speak with a common metrics language, please join David and me in the Measurement Accuracy Roundtable.
Whether you decide to join the Roundtable or not, I'd love to hear your thoughts on the email open rate in the comments box below: Is it a tired old metric that is ready for retirement, or a slightly flawed but still valuable tool in the email metrics toolbox?
2 people recommend this article.
RSS FEEDS
One solution is that "open rate" should be renamed to something more realistic.
Loren has taken the initiative and I think we have to try and find another way to track the real "open rate"
If all can come forward to a common platform of measurement which is effective as well as simple, this problem can be solved.
No need to rehash what has already been said, right?
I look forward to continuing this discussion...
dj at bronto
I think most marketers can improve by focusing on measurements that matter and click rate is one of them. For those people that only want the email read, than measure other things like (noted above) phone calls.
Too many beginning email marketers focus on open rates and it distracts them from focusing on creating content that works - drives people to the action they want them to take - which is what email marketing is mostly about.
I wrote something about email open rates on my blog last month. http://emailmarketing101.blogspot.com/2008/03/open-rates-have-lost-their-usefulness.html
I guess what I would advocate is let the advanced marketers use open rates for subject line testing and list fatigue reporting, but as a group let's work more at focusing on engaagement - which results in phone calls, clicks, donations, sales, etc.
Not opens.
Use the open rate - just understand that it has limitations and doesn't measure your success at reaching your actual business goal - calls to the toll-free number.
As far as a replacement, I'll talk more about that in my next column. But what I'm really saying is that the "open" in open rate is a misnomer. Keeping and using it are fine, let's just rename it and redefine it to mean what it actually measures - when images are rendered in an HTML email. So perhaps we call it the "Render Rate" or something - which tells us when recipients are viewing the email as it was theoretically intended - with the images rendered.
Love the discussion....stay tuned.
Shawn Fassett Director of Marketing My Choice Medical Holdings Inc. A division of Vertrue Inc.
I am beyond sick and tired of trying to explain to clients that this metric really doesn't matter. Inevitably it gets their underwear all tied in knots if they see an alarmingly low rate. We have to sit them down, remind them to breathe, and go through this whole explanation (as Loren did quite well above) as to why this is highly flawed -- but, in most cases, the technology just doesn't offer anything better, at least not yet.
In one case a client, just incredulous about the sad state of this open rate, stepped back to question my credentials. She flat-out didn't believe my explanation and wanted a second opinion on this. Life's too short. Let's move on.
I have to disagree that clicks are a better open indicator.
Subject lines cause a message to be opened and read. Value proposition and call to action cause a link to be clicked. What happens after the click causes conversion.
Therefore, a weak subject line with a good value proposition and strong call to action may get more clicks even though fewer people have "read" the message than a good subject line where more people have "read" the message with a weak value proposition and call to action. From that, which is the better subject line?
The people who buy from you or read your newsletter and really want what you have to offer will turn images on. They will show open rates along with click rates. These people are a good indicator of what people like them are interested in, and if that's your target market I'd say that there's some pretty good intelligence to be had there.
Let's not go from navel gazing to toe gazing.... :)
Although mail clients block images by default, those interested in receiving a message will turn images on for email that they want to receive – just like myself and others turned on the images of this message (so we could see a big header and banners). Does that measure overall readership? Of course not. In the end it's all about conversion (click-to-conversion anyone?) as it always has been - even when it was vogue to write article after article after article about open rates not that many years ago.
I think that open rates are an indicator of other things, such as subject line testing as mentioned by Tim, so I don't think that discounting open rates wholesale is any more beneficial than myopically focusing on them.
The open rate shouldn't die, just put into perspective as it should have been from the start.
But is also is an example of what is wrong with the open rate - to some it is based on total opens; others if someone clicks on a text message, etc. But this is more a standardization issue - than just killing it.
Tim, Morgan, Nick, Mark - You are corrent in that I didn't offer up any great alternatives. My purpose for this article was to stir up some debate and hopefully get people involved in the Roundtable and helping standardize the metric and come up with better measures.
That being said, for most companies the purpose of sending an email is not to see how many people open it. It is about getting the recipient to take an action. So for testing subject lines, I've always preferred to use click-through rates - and ultimately conversions to determine the best subject line. I've done tests with retailers in the past where emails with lower open rates generate higher CTRs, orders and revenue. In these and most cases the purpose of the subject line is to motivate people to buy - not just open.
In the case of publishers (such as MediaPost) or confirmation emails, open rate is certainly one measure of is anybody reading the Email Insider columns. But for them, actually the key metrics are more likely around ad rates, list size, demand by publishers, click rates on the sponsors ads, etc. These are what drive their business - not whether my column got a 40% open rate one week and 36% another. And in my own case, I tend to use the number of comments posted on a column - and in comparison to the other columnists - to see if I'm hitting the mark or not.
But perhaps you've set me up for the follow-on column to discuss alternatives to the open rate...
Our team all acknowledges that our Open Rate is low. We know people personally who open our e-mails but never show up in the open list in our ESP's metrics.
There needs to be another measure, but one that works for the entire industry, not just those driving traffic to a site, but those who are publishing information or driving people off line.
Perhaps the answer is technological...a better way to measure e-mails opened, read, responded to...
The only 2 metrics I care about are: 1) How many emails did we schedule? 2) How much money did we make?
==> $ earned/emails scheduled is my bottom line.
Notes: -If you're concerned about ROI, then you also should factor in the cost of sending those emails. Some ESP's charge per email sent and others by list size.
I prefer the list size pricing because it let's us send relationship (= more $ later) content loaded emails to our members for "free". I tend to recommend http://productinfo.aweber.com as an ESP that fits this model.
-To tie the $ earned back to your email you can use any number of tracking systems including Google Analytics and affiliate subid's if making 3rd party offers.
-If you're not selling but trying to get leads, then figure out your average $value per lead (lifetime customer value) and multiply it times leads generated to get the $ "earned" number.
Yes you can get into split testing subject headings and bodies etc. but keep the focus on which of A/B made more money per email scheduled as the metric to judge the winner, and not open rate or click through rate.
Using this system we're saving over $5k per month for our clients on the cost side and in March we hit record revenue per email scheduled.
Works for me.
I read through the first 1/3 of this post trying to figure out what you were talking about because 'open rate' in the non-email advertising world (where I grew up) means the standard media rate card price before negotiation. I finally figured out that in the email world it means the rate at which emails are opened and I just laughed.
Clearly, email, like a lot of other disciplines has developed a language of its own. And like everyone else, emailers fall into talking to themselves, using their own jargon, and confusing outsiders.
The industry might have greater credibility if your customers understood what you were talking about.
BUT...as a comparative measure (as others mention) it still has value.
Open rate issues are symptomatic of a wider problem, namely that there's a lack of critical thinking around when examining email marketing numbers.
If we are to move beyond using open rates, what can we replace it with? You don't seem to offer any suggestions other than to suggest we can do better.
Click through rates are nice, and can be used – but not all emails have the goal of driving a click. Some just want to be read.
As "one of the peers that lines up behind it" I find it interesting that Loren has yet to offer an alternative for the evaluation of subject line tests. In order for the campaign to kill Open Rates to have any merit there needs to be an alternate means of evaluating subject lines that is statistically valid. I have not seen a viable alternative proposed.
If I am A/B testing two subject lines, I assume that, with a random sample, any open issues would be evenly distributed, and therefore, the open rates do provide some insight as the the effectiveness of the different subject lines.
In the end, the conversions are the most important metric but I do think that open rates provide some useful information.