Marketing Attribution: The Answer To The Validation Question

• Comment (2)
• Recommend (8)
• Tweet
• 

I authored a MediaPost article  last June that asked the question “Can the Accuracy of Attribution Be Validated?”  It examined a number of the most common marketing attribution methodologies of the day, and challenged readers to think about if and how the accuracy of each could be mathematically corroborated.  The topic sparked a lot of guesses and a lot of dialogue, both online and in conversations with readers.  Options that were suggested included:

• Develop a statistical model and statistically validate the model
• Identify the lift between two groups
• Identify the lift between two bias-corrected groups
• Identify lift between two bias-corrected groups that also includes ad verification
• Develop a model that compensates for cookie deletion

Given the diversity of opinions, I thought I’d revisit the topic and provide the answer that I withheld in that original piece.

Only One Real Answer

The fact is, there is only one attribution methodology discussed in that article whose accuracy can be scientifically and mathematically validated, and then subsequently adjusted to become increasingly accurate over time: the algorithmic methodology.  Forrester Research demonstrated its endorsement of algorithmic attribution in its recently published Wave Interactive Attribution Vendors and Wave Cross-Channel Attribution Vendors reports, which ranked attribution solutions using objective algorithmic attribution as having superior methodology to those using more subjective rules-based methodologies.

By its very definition, “validation” is being able to prove that “A” is equal to “B” -- and in the case of validating algorithmic attribution, “A” is the media spend recommendations and result predictions being provided by the attribution vendor, and “B” are the independently derived, actual results obtained by putting those spend recommendations/predictions into market.  Using just the lift won’t validate the accuracy of the model because in that case both “A” and “B” are provided by the attribution vendor, with neither being independently produced.

How Does It Work?

When your marketing performance data is processed through an algorithmic attribution solution, a number of mathematical techniques (its algorithm) are used to calculate a “weight” for every attribute associated with every marketing touch that takes place with consumers exposed to your marketing.  So a weight is assigned to every channel, ad size, placement, keyword, search engine, publisher,  creative, etc., with that weight corresponding to the relative impact that each particular attribute has on producing your desired business outcome (such as a conversion).

Using these weights, the most sophisticated algorithmic solutions will formulate predictions of future performance. They will also present you with extremely detailed, placement-level recommendations on how to invest your future marketing budget in a more effective way, with those recommendations mapping to the marketing tactics containing the attributes with the highest weights.

The Validation Process – Coming Full Circle

When those predictions/recommendations are accepted by you, and put into action within your marketing spend, they produce a new set of marketing performance data, which is in turn brought into your attribution solution and processed.  When this happens, the algorithm behind your solution compares your actual results against the predicted results and validates its own accuracy by looking at the delta between the two, and identifying a percent margin of error. 

The best algorithmic solutions in the market today can regularly validate their accuracy between a 0.5% to 2% margin of error. Almost all rule-based systems have margins of error ranging from 40% to 90%, which can at times be less accurate than even last click attribution

Enter Machine Learning – The Next Level

The highest level of model accuracy and validation comes from the use of the self-correcting machine learning systems used on different types of brands within multiple industry verticals over several years. When an attribution solution is machine-learning enabled, it will look at the margin of error between your predicted results and your actual results and then make automatic adjustments to fine-tune the algorithm.  This produces more accurate predictions during the next measure-predict-optimize cycle.  And of course, rarely is your marketing mix going to be totally consistent and static between cycles – probably adding or subtracting channels, publishers, keywords, creatives, etc.  As a result, every cycle produces data that the algorithm is processing, weighing and predicting for the first time. 

But with every cycle, the machine learning process is getting smarter at predicting your future performance, and during the process the recommendations you are putting into action produce ever-improving results.  The more cycles and the more “new” information the algorithm gets to churn through and subsequently fine-tune, the more the accuracy of the algorithm is validated, the more accurate its predictions become, and the more the recommendations you put into market succeed.

• Comment (2)
• Recommend (8)
• Tweet
• 

2 comments on "Marketing Attribution: The Answer To The Validation Question".

1. John Grono from GAP Research
   

   commented on: June 5, 2012 at 5:37 p.m.
   "The highest level of model accuracy and validation comes from the use of the self-correcting machine learning systems..." as long as all marketing contacts and marketing results (sales?) are machine-based. Otherwise, best to focus on a "total marketing" model to understand what is really happening.
2. Anto Chittilappilly from Visual IQ
   

   commented on: June 6, 2012 at 4:24 p.m.
   Thank you for your comment John. I was referring to “attribution models” that are having the self-correcting and machine-learning capabilities. Such attribution models use channel data, sales data, econometric data, competitive data and other several relevant data sets available to marketers. Such input data may not come from automated sources or may not have any particular level of sophistication. That’s the ground reality. The reason for the higher accuracy of the attribution model is because these self-learning algorithms have often been in place, learning and refining their accuracy over several years, within different client types, conversions, verticals, and levels of media spend. I hope that clarifies.

Leave a Comment