Brands Waste Budgets In Misdirected Attribution, Media Buys
- by Laurie Sullivan @lauriesullivan, June 13, 2011
Between 36% and 60% of companies' digital advertising revenue could be at risk for misdirected media buys. That's according to a ClearSaleing study released Monday. The company looked at the attribution model of 13 advertisers across retail, education and financial verticals, analyzing more than 1 billion advertising impressions, 81.5 million advertising clicks, and 3 million conversions.
The findings, which were intended to quantify the amount of revenue misallocated by the last-ad attribution model, were compared with the company's algorithms built into an application Morris Martin, director of product marketing and strategy, refers to as the "glass-box" -- the ability to see contributions from all channels.
Morris, a former senior analyst at the Microsoft Atlas Institute, has been working with attribution models for 6.5 years. He says the study attempts to identify the amount of revenue at risk for the 13 advertisers and give them the tools to move beyond the last ad method.
The platform, ClearSaleing Altitude, aims to do just that. If campaigns only get optimized under the last ad, or click, brands would ignore all the other steps contributing to the bottom line. The last ad can undervalue other steps in the sales funnel and mislead marketers to misallocate budgets, according to the study.
Brands will begin moving beyond the last ad when it comes to attribution, Martin said, suggesting that many will begin to see the value in display advertising as a contributing factor to finding consumers that might not know about the brand's product or service. He points to eMarketer predictions for better-than-expected ad spending this year for display ads. Brands will increase the amount spent this year by 20% to $31.3 billion in 2011, up from a previous forecast of $28.5 billion, eMarketer estimates.
ClearSaleing's statistical model examines the relationship between the exposure to media channels and consumer conversions and supports CS Altitude, which helps to determine allocation. The platform looks at all the impressions and clicks of a brand's campaign and analyzes the causal relationship between a person being exposed to a variety of media, such as affiliate, display, comparison shopping engine, natural search, social and their propensity to convert.
Paid-search promotes navigation. A retailer's most profitable keywords will likely include the brand's name. It becomes the last click before the conversion. Display, for example, does well when it's one of the first steps. The model not only measures conversions, but also the relationship of how a consumer interacts with a specific channel.
What should advertisers do? Martin suggests identifying the amount of revenue at risk for the last ad method. "In one case, an advertiser participating in the study had 60% of its revenue tied into multiple steps to conversion," Martin said. "So it didn't give this advertiser the full picture. In fact, most medium-to-large advertisers have customers that take multistep paths. So they need to move past the last ad."
It will lead brands to reopen their media plans and display ad channels. Advertisers do a great job creating marketing briefs, but when they optimize it against the last ad attribution model, the larger publishers are cut from the campaign because the premium display units underperform, based on key performance indicators.
Many consumers discover products and services in a display ad, then turn to search before making the purchase. Display ads can help brands reach consumers well before they're in the market to make a purchase. The consumer base doesn't grow by buying more branded keywords, Martin said.
With more specificity: 44% of all transactions are Originated by Display for advertisers using C3 Metrics attribution.
More here from C3 Metrics:
http://c3metrics.com/labs/
So, Mark, absent display ads 44% of revenue goes away? Gimme a break. I understand the need for marketing patter, but this "study" is tremendously, and intentionally misleading. We're big believers that Display advertising can be done cost effectively for many (not all) businesses with proper attention to the details. We're also firm believers in the importance of shrewd attribution. We're also firm believers in integrity, and while blasting this "study" to pieces point by point would be fun, I suppose it would be impolite. Advertisers need to carefully assess the incremental value of media buys, and stats modeling is a piece of the puzzle -- hold out tests for email, display, and direct mail are also essential pieces.
As a rule of thumb, we at Visual IQ also see that at least 40% of converters have more than one touch in their experience prior to conversion. Whether it is 36% or 44% or 40%, we certainly agree that many converting consumers experience more than one marketing touch prior to conversion. So all of us in the attribution management business agree that that last click attribution is a flawed methodology in this context. But I would NOT agree with George’s comments that the conclusions in ClearSaleing’s documents are intentionally misleading, though I do agree that they are unsubstantiated.
The document merely points out that when display ad impressions took place during the multi-touch path to a conversion that the value of the resulting conversions was greater than when only one touchpoint was observed. The document does NOT provide a methodology for basing any conclusions WHY this was the case, nor does it provide any proof that were those display impressions to have not occurred, that the resulting conversions would not have taken place.
An attribution methodology that arrives at the right attribution weights for each touchpoint in the converting user’s experience by executing virtual A/B tests between paths that include that touchpoint and paths that do not include that touchpoint using 100% of touchpoint data from both converters and non-converters gets to the WHY behind the findings and provides a solid basis for future performance predictability. This accomplishes what George suggests without the pain and cost of executing hold-out tests for each placement and creative. It’s our belief that until such a methodology is utilized that the “cause-tied-to-effect-tied-to-outcome-predictability” conclusions in this document cannot be justified.
Gentlemen, thank you for taking the time to read the published case study and comment on the Media Post article. We have reached a significant milestone in evolving beyond the last-ad attribution methodology and I thank you for echoing the need for more robust, advanced analytics.
Bill, your comment "An attribution methodology that arrives at the right attribution weights for each touchpoint in the converting user’s experience by executing virtual A/B tests between paths…" shows that we are in complete agreement in that the methodology must include the determination of a causal versus coincidental relationship between advertising and revenue.
As stated on page 2 of the study in the methodology section, CS Altitude leverages level of influence at its foundation. "A statistical standard, Bayesian Regression, was used to analyze 100% of impressions and clicks from both converters and non-converters to identify the causal relationship between marketing channels and consumer conversion." This was across more than 1.1 billion impressions and more than 81.5 million clicks across converting (3 million) and non-converting paths over a 4-month period. Furthermore hold out analysis is standard procedure in our coefficient generation validation process; coefficients are also refreshed on a regular basis to insure stability. We are enthused that you agree with our approach.
Additionally, it appears that you may have misunderstood the actual study. When a consumer has multiple steps within their Purchase Path to conversion, revenue associated with the conversion is 42% higher than single paths only. Multiple step paths are not limited to the display channel but rather are inclusive of channels such as Paid Search, Affiliates, Email, Comparison Shopping Engines, etc. Display, having been one of the most undervalued channels in digital advertising, is one of the key benefactors of moving to the robust CS Altitude methodology, however, channels such as Email and Paid Search also benefit.
Certainly, we enjoy this discussion on the topic and we invite you to join the webinar on June 21st at 2EST.
We enjoy the discussion as well Morris. The data driven nature and scientific rooting of this discussion should help marketers and their agencies feel more confident about making investments in attribution. It also speaks to how the space has come a long way. The marketplace is increasingly recognizing the superior nature of algorithmic attribution process – a process that Visual IQ has pioneered for several years now.
I’d like to offer a final couple words of clarification. If I understand George’s comments correctly, the hold outs he is referring to are campaign-level hold outs in the traditional marketing sense, meaning you run a test for email by holding back email for a week and then introduce email back the following week to measure the difference in response rates. These obviously differ from statistical hold outs which involve holding back some sample data points during the modeling process for cross-validation later. Naturally, both are meaningful in their context. A statistical modeling algorithm will only be able to tease out the impact of a marketing stimuli when there are sample observations in the data that speak to response states with and without a particular stimuli, or at least varying degrees of it. This is why George’s point is a precursor to using a model to estimate/quantify the impact of a marketing program.
Let's keep sight of the end goal here. Advertisers NEED to have proper attribution reporting as part of their campaign measurement tool set so that they can properly spend their online budget in the right places. Even at its most basic level using equal attribution, an advertiser can gain the benefit of knowing that display campaigns can be a useful branding mechanism for helping to drive conversions for other campaigns, such as search.
I'm absolutely loving the competitive spirit of these attribution conversations because at Yahoo, you can go back all the way to 2006 when we did studies with ComScore talking about how using a combination of display + search can help to drive a higher ROI for the advertiser. And our critics use to say, "Well of course Yahoo is going to report this, display and search is their bread and butter."
Well, the evidence is out there people....and it's solid. And it's not coming out so much anymore from companies who are in the business of search and display (Yahoo, Microsoft (Atlas), Google), it's coming from very talented companies such as Clear Saleing, Visual IQ, and C3 who are focused on the topic of attribution. They are proving out the theories with testing and providing you the data in reports and case studies.
For those advertisers who run both display and search campaigns and are not focused on attribution reporting or who have their campaigns being managed by a 3rd party agency who does not pay close attention to attribution when managing your campaigns, you better start paying closer attention.
Attribution helps you gain tighter control of your online spend. It will weed out the underperforming campaigns and will help you drive more more conversions by focusing more spend on the most efficient campaigns.
It's a no brainer people. You need to get attribution into you measurement mix.
Bill, Morris, for clarity, the 'hold out' tests of which I speak are what direct marketers have been doing for decades, and no, it doesn't mean off-on-off-on. It means random split tests with a control group. This is the gold standard.
RKG's attribution system also takes advantage of high-end statistical modeling to estimate lifts associated with different types of ads, it also recognizes critical distinctions between brand and non-brand search, between display impressions and clicks, etc. That said, we also have enough knowledge of stats and integrity as advocates of our clients to acknowledge that the mathematical assumptions underlying these techniques are often wrong, and that sanity checks with hold out testing is critically important.
We wouldn't have built the service offering if we didn't think it an important piece of the puzzle. My complaint is that those who have an interest in selling more advertising (agencies, publishers, and attribution service providers) have a huge incentive to over-inflate the importance of these interactions. This is a disservice to the advertisers we are supposed to serve.
Morris, one other comment. You mentioned using Bayesian Regression modeling for the analysis. That seems an odd choice to us. We found dynamic Bayesian models produced lousy attribution modeling because these models don't understand sequential ordering. See this piece: http://www.rimmkaufman.com/rkgblog/2010/06/21/advanced-statistics-and-other-meaningless-drivel/
Matt, thank you for continuing to beat the drum that is the case for attribution as it is imperative for advertisers to adopt a methodology that is a more robust, intelligent representation of their holistic marketing mix. Evolving beyond the last ad via the channel-agnostic CS Altitude is a win-win situation for advertisers and publishers alike. Advertisers are empowered with improved optimization of 100% of their marketing data while publishers are rewarded for reaching an advertiser’s target audience rather than simply being the last ad.
George, you’ve made several astute points and I thank you for sharing your team’s efforts in producing an attribution methodology. You’ve highlighted the need for an advanced attribution model to be transparent and identify and account for sequence of ad exposures and clicks. As noted on page 2 of the case study, CS Altitude is a glass-box algorithm, revealing all inputs in addition to outputs to advertisers. Furthermore, in addition to quantifying the relationship between channel exposure and conversion based on 100% of converter and non-converter data, “Purchase Path Position, the relationship between when a consumer is exposed to a channel, e.g. Introducer (first engagement), Influencer (middle) or Closer (last) and onsite activity was examined.” Finally, we differ in approach of model specification and data collection technology to build a best in class attribution model but your core tenants of a channel agnostic evaluation of media, transparency and marketing acumen are precisely the foundation of CS Altitude.
We look forward to hosting you on tomorrow’s webinar. Regards.
Thanks for the discussion, Morris. I have no doubt your team has built a fine product.
Like George Michie at R-K, we at AdStrategist are believers in integrity. However, I don't believe blasting a study to pieces would be impolite. Publishing a study full of drec, is.
This makes all of the attribution vendors look bad. Bayesian Regression Modeling!!!??
Give me a break guys..