Click-Through Metrics Come Under Fire
- by Gavin O'Malley @mp_gavin, April 25, 2011
While marketers continue to rely on click-through rates as a primary measure of campaign effectiveness, new research suggests that the metric should be taken with a grain of salt -- or completely taken out of the equation.
Indeed, 99% of stable cookies examined by ad network and technology provider Collective show no evidence of clicks. The finding is even more pronounced than an often-referenced comScore study, Natural Born Clickers, which revealed that only a small minority of Web users make up the majority of click activity online.
To conduct its research, Collective said it examined the ad interactions of more than 100 million anonymous user profiles, and over 1 billion ad impressions served in the first months of 2011. In addition, it reviewed the action lift of 100 campaigns, and brand lift reported in 400 campaigns.
"We integrated an enormous volume of impression, user and performance data in our 'super-computer' from early 2011," said Jeremy Stanley, vice president of analytics at Collective. "We then drilled down into this data to understand what fraction of users were clicking, what characterized those users, in what circumstances they tended to click."
What it found was that users who have clicked in the past are twice as likely to click again in the future, while nearly 20% of ads that received any click activity received multiple clicks within the same impression, suggesting that these clicks were unintentional.
This effect was seen especially among online gamers, who clicked 43% more often than non-gamers, and on mobile devices where users clicked 123% more often, according to Collective.
In addition, an examination of who usually clicked painted a picture of an audience that may not be attractive to most advertisers. Clickers tended to be lower-income, older and late technical neophytes.
Users with incomes under $40k click 30% more often than users with incomes over $200k, Collective found. Users with only 'fair' credit scores click 20% more often than users with 'excellent' credit scores.
What's more, users who are late adopters of new technologies clicked 50% more often than early adopters, while users who are economizing clicked 65% more often than users who purchase frequently online.
Many clicked ads did not materialize in terms of action or brand lift; Collective could not even find a discernible correlation between CTR and post-impression action.
Furthermore, the highest-performing CTR campaigns examined (top 20%) had a 150% higher CTR but an 8% lower post-impression action rate.
Similarly, a preliminary study of 100 campaigns showed no correlation between CTR and brand lift and purchase intent as measured by VIZU post-impression surveys. Collective suggests that optimization of campaigns to achieve higher CTR may, in fact, reduce brand ROI.
Stanley believes that marketers should stop using CTR to measure performance of their campaigns. "In fact, they should stop tracking CTR altogether," he stressed. "We believe that the CTR is not only a meaningless performance measure, but it can actually destroy value for brands if they optimize their campaigns to perform on CTR."
Wait, we are STILL having this conversation??
I would suggest that CTR varies in importance on a category by category basis. Even within verticals the value of CTR data should not be tossed aside.
SMBs, where conversion metrics are not easily measured (typically for the smaller retailer) take some comfort in CTR data.
Paul Benjou
www.MyOpenKimono.com
It is easy to say don't use the click as a tool. Much harder to say what it should be replaced with. So, what should we use instead? It is easy for ad networks to say well we got you a lot of valuable impressions. That's an unquantifiable statement. Consumer surveys are not always an option and, indeed, they too tend to be biased towards lower incomes. So, given the needs to track online activity, what and how do we track?
Some alternative metrics include engagement (time spent with the ad, interaction rate), post impression action lift (time spent on brand website, branded search), post impression surveys to study message recall and/or purchase intent (though, I suspect Andrew is right regarding a bias here), and in-store sales lift.
I would encourage folks to read the whitepaper Lotame released last year which was the first ever cross-metric study of brand equity measures and their relationships to clicks, interactivity, and time spent. Nice to see the conclusions we made are further validated nearly a year later.
http://is.gd/9dLMDk
Sincerely,
Eric Porres, Lotame
@Jennifer: Nice!
@Andrew: In addition to Christine's list I'd like to add view through. Also, whether a consumer clicked or not matters less than the fact of whether or not they did something. Did they do what the advertiser intended them to do, be they site visitation/revisitation brand engagement or conversions being measured post impression.
A presentation detailing the research covered in this article is available at our website at:
http://tinyurl.com/6zxa79o
Best,
Peter Weingard
collective.com
While I agree that CTR is not the best metric, what does the author suggest as an alternate?
In my opinion, the best measures are the following:
1) For direct advertisers, measure online actions and attribute them to impressions (not clicks) using controlled experiments (actions will *not* be fully allocated)
2) For brand advertisers, measure brand lift (post impression increase in awareness, opinion, intent) using controlled experiments (we frequently use Vizu for this)
Using anything less than these options (e.g., CTR, last click / impression action) can provide a highly misleading measure of performance, and when combined with aggressive optimization may in fact lead a marketer to destroy value (poor quality inventory, spamming existing customers).
Instead, if 1) or 2) are not possible, I would recommend the following strategy:
* Choose an audience segment that your advertisement will resonate with
* Run the advertisement on high quality inventory
* Target contextually if appropriate
* Measure view time and interaction rate to ensure the ad is being seen
Now while I agree that more should be looked at beyond CTR, I don't believe it should be tossed out all together. While all of the other measurement tools are great and would be ideal for all clients to use as measurement. They are only used by the most sophisticated brands in the digital space who have matching budgets. These tools aren't as easy to utilize for medium to small brands or niche industry brands. For no other reason than they can't afford to spend money on these tools or should I say they can't convince themselves to spend the money on them. All in all I'm in agreement with Jeremy in that the first thing to do is put your ads in front of the right audience because it's been proven that an interested/engaged audience is more responsive.
To say that CTR should be dispensed with speaks not to Stanley's sophistication but to the lack thereof. While campaigns should ultimately be optimized around more ROI-relevant metrics (responses/conversions/downloads/purchases/etc.), demonstrable strong correlations can be drawn, in certain cases, between CTR and rates of those deeper metrics. Those correlations can be exploited to accelerate optimization cycles, and a truly sharp marketer can take advantage of that. Rarely are things so "black and white" as he's painting them to be.
CTR is like "hits" to your site: Interesting, but mostly meaningless. It does provide information about ad position and creative effectiveness. But most of us know getting a high CTR is easy, but getting a low Cost-per-conversion can be hard. The conversions and ROI are the bottom line. CTR is used to impress the clueless or to CYA. True, a high CTR will help reduce your CPC, but without conversions you only waste less per click.
"...while nearly 20% of ads that received any click activity received multiple clicks within the same impression, suggesting that these clicks were unintentional. "
I don't think the clicks were unintentional, just the extra clicks. Many users are still confused on when to click once and when to click twice. Of course some ads are positioned next to a scroll bar and get clicked on my mistake, but multiple clicks from the same user should not be a topic of discussion since they should never be charged to an advertiser.
"...online gamers, who clicked 43% more often than non-gamers, and on mobile devices where users clicked 123% more often..."
In our option, gamers fit the profile of those that engage in click fraud, and the new platform of choice for click fraud is mobile. We feel this way due to the reduced availability of tracking, and that some platforms register a "click" after an initial action without actually clicking through to a site in some cases like some click-to-call offerings.
"... Collective could not even find a discernible correlation between CTR and post-impression action."
"Furthermore, the highest-performing CTR campaigns examined (top 20%) had a 150% higher CTR but an 8% lower post-impression action rate."
"Collective suggests that optimization of campaigns to achieve higher CTR may, in fact, reduce brand ROI. "
This confirms what we have been seeing in some campaigns. And this is huge. If you don't think so, you need to talk to your marketing people. If you are in marketing... never mind, there's no point explaining.
The factor that is missing from this article is the 800 lb gorilla in the corner: click fraud. What I was hoping would be mentioned since the study seemed to be so comprehensive, was the evidence of click fraud that should have been detected. Patterns of abuse, IPs with excessive clicks, geo locations (down to the city level) that exhibit high CTR with few or no conversions. If you have a Google Adwords account, you may be able to see this information under the "Dimensions" tab. There is nothing that identifies possible fraud, you will have to put that label on it yourself when the patterns are observed.
Given the results of the study, would it not make more sense that advertisers be provided the option for "flat rate" advertising? They would pay for their ads to appear for a certain period of time, and pay only for that exposure, NOT how many clicks they received. Then they can look at the expense and the return and decide if they should continue paying at the current rate or not. Click fraud would be effectively declawed.
What about view-through (non-click) traffic? Is that regarded as a post-impression action?
Jeremy -
You article brings up an excellent point: the 'click' should not be valued as the end game and other metrics need to be utilized ... but you missed the boat by not recommending Attribution as a 'best measure' for both direct and brand advertisers.
Full funnel attribution can properly measures all engagements - impressions + clicks for direct advertisers and brand lift + purchase intent for brand advertisers.
As we move forward towards more RTB, Real Time Attribution (RTA) is going to be necessary or as you said ... 'aggressive optimization may in fact lead a marketer to destroy (brand) value'.
If you're interested in learning more, our White Paper (at http://www.c3metrics.com ) has all of the details.
Any intelligent search marketer knows that measuring performance based on one metric is unwise. First of all, measuring CTR% should always be done with segmentation. It's all in the context of what you're trying to accomplish. It all depends on what keywords you're bidding on, head term vs. long tail. Of course CTR% is important. CTR is fundamental in improving your quality score, minimize your CPCs and get better ROI. At the end of the day you're looking to drive qualified traffic to your site. If you believe that there's a drop off point then optimize your CTR to that level. The article needs to go more in depth, showing the countless nuances a paid search marketer has to consider. Also, as other commenters have pointed out, if one is to make an assertion like this, one should be able to provide an alternative. Otherwise, the article sort of falls flat.
@ Jennifer... amen.
It's interesting data, but any company still using CTR as an actionable KPI for display probably deserves to get subpar traffic.
The case in favor of display is pretty simple for retailers, isn't it... do you or do you not want millions of eyeballs on your brand in an environment where consumers spend the majority of their day?
I’ve seen us evolve far beyond click through rates as the most accurate measure of campaign effectiveness. Perhaps it’s time to look at the market through a new lens. Consider share of click. In any product category, there are a finite number of clicks, and a true measure of effectiveness is seeing if you can positively affect your share, increasing the percentage of total clicks on your product. More interesting is to analyze this data against the backdrop of a competitive set to see from whom you’ve captured clicks. Premium position, rich media, featured zones, they all offer solutions to increase share of click. Perhaps it’s the metric that matters most.
The REAL 800-pound gorilla in the room? The platform.
It is much, much easier for agencies and advertisers to rely on their central serving and tracking platforms tools to do small-scale research and evaluation for them, than use the raw data (if they can even get it) and play mad scientist. Let's look at Doubleclick, which owns the pipeline, the methodology, and increasingly the media placements; why not just stick with reports and numbers that the platform takes care of for you, because it takes care of everything else for you? Clicks are deeply built into the infrastructure of Doubleclick's reporting offering, tools, and methodologies (by extension, the market), so what's the average media planner supposed to do? Try to export "raw" data, hope excel doesn't crash, and go (back) to the drawing board? Some enterprising agencies do and we work with a number of them. But the crisis here is in the platform: in providing limited or no access to raw data alongside plenty of access to "granular" pre-packaged data that feels raw.
We strive to work with our agency partners towards new KPIs, but it's a difficult battle because the capital-P-Platform (I'm still looking at you Doubleclick) shapes our understanding of what's possible and limits what can be investigated by way of its toolset. The market is built on The Platform and we are really feeling the strain of that paradigm as new formats and new tracking technologies begin to take hold and struggle to make sense of themselves relative to the larger market.
Analysis aside, I'm happy to see continued attacks on clicks. In our own limited research into ad exposure and site behavior (a dozen or so campaigns across several advertiser categories), we've seen similar trends. More than 99% of folks who click on video ads abandon the advertiser site without any additional interaction: farmed, accidental, misintentional ("I thought clicking might skip through the ad so I could get back to the Dancing-with-the-Stars") clicks. So, most of the site's "landing-page" traffic comes from clicks on ads, yes, but nearly no meaningful conversion is related to clickthrough; instead, for video ads specifically, users are exposed to several ads across sites and placements and then navigate to the site themselves (search) later. We are unwaveringly confident that clickthrough is a video metric with vanishing significance in any real sense; now it's about busting it out of our sales (CPC) and operations.
@Jennifer We MUST still have this conversation because we still bend over backwards to try to fit clicks into our clients' reporting and research in a meaningful way, when everything increasingly points to meaninglessness, or worse: misdirection and fraud.
This would've been news in 2003....
@Jeremy Stanley: You suggest above that "For direct advertisers, measure online actions and attribute them to impressions (not clicks) using controlled experiments (actions will *not* be fully allocated)". While I agree fully with your suggestion this is often hard to operationalize, especially if your ad placements change frequently or do not generate enough impression volume for a conclusive controlled experiment.
Advertisers generally would like a default answer (e.g. we can attribute 30% of post-impression actions to our display campaigns) but this is hardly ever accurate. I still believe - and here I agree strongly with Michael Meier's comment above - that post-impression attribution should be built into The Platform for it to be properly adopted and used in the industry.
The hard reality, and part of the reason why controlled experiments are not run more frequently by agencies, is that impressions very often have little to NO impact on conversions. Ad visibility (the fact that many DR impressions are served below the fold) is a major factor here, but poor contextual relevance, ad clutter, ad blocking and just plain bad creative are all contributors.
It is sobering to realise 15 years into this project that there is still so much work to be done to prove the true value of display advertising.
Click through rates are simply the latest case of advertisers choosing to count what's easy to count instead of what's meaningful. The fiction of GRPs and CPMS set based on pumped up sweeps-week programs has been replaced by the fiction of clicks. Evolution on this issue will be characterized by our species growing spines.