Commentary

Can The Accuracy Of Attribution Be Validated?

by Anto Chittilappilly , Op-Ed Contributor, June 30, 2011

Just as not all scales use the same methodology to calculate the weight of objects, not all forms of attribution management use the same methodology for calculating how credit for conversions should be attributed. We can easily discover if a weighing scale is faulty, or the level of error that can be expected from it by using a particular methodology to validate its accuracy. But can we validate the accuracy of a given attribution management methodology? Before we answer these questions, we need to understand how the different attribution methodologies being used in the marketplace today assign credit to the various marketing touchpoints that you create with your prospective customers.

Touchpoint Selection

Every attribution methodology works by analyzing some or all of the marketing touchpoints (search, email or display ad click or impression, direct mail receipt, TV ad view, etc.) experienced by individuals who are exposed to your marketing efforts. So the first component that differentiates the various attribution methodologies is the choice of which touchpoints each uses in its analysis. Below are those touchpoints most commonly used today:

1. First and last touchpoint. Regardless of how many touchpoints a consumer experiences while being exposed to various marketing efforts prior to a conversion, this methodology uses two - the first and last - and assigns 50% of the credit for the conversion to each. This methodology assumes there are at least two touchpoints on the path to conversion, and if there is not, then 100% of credit is assigned to the one and only touchpoint.

2. Up to last 3 touchpoints. In the instance where there are three or more touchpoints, this methodology uses the last three prior to conversion, assigning roughly 25%, 25% and 50%, respectively to those touchpoints. If there are less than three touchpoints, this methodology will default to using the methodology above.

3. Even distribution with exclusions. This methodology allows the marketer to select a subset of all touchpoints and assigns an equal percentage of the credit across those touchpoints. So if the software uses five touchpoints, it assigns 20% credit for the conversion to each one.

4. Even distribution across all touchpoints. This works the same as described in #3 above, except no touchpoints are excluded from the even distribution. So if there are 20 total touchpoints recorded, each one gets 5% credit for the conversion.

5. Assigned Weight. This methodology allows marketers to select any of the four methodologies above, but to pre-define how much credit each touchpoint should receive. So the first and last touchpoints could be used with 25% and 75%, respectively. Or the last three could be used, with 10%, 20% and 70%.

6. Real-time. This methodology records and stores several digital touchpoints in a shared cookie on the user's computer. Once the user converts on a web page, the software assigns a weight that's pre-defined by marketer to the touchpoints stored in the user's cookie.

7. Algorithmic. This methodology uses machine learning systems to calculate the weight for all of the touchpoints and re-calibrate such weight over time using the learnings from the marketer's data.

Touchpoint Dimension/Trait Selection

The second component that differentiates the various attribution methodologies - that works in concert with the touchpoint selection component described above - is the choice of which touchpoint dimensions (or traits) each methodology uses in its analysis. These dimensions include characteristics associated with each touchpoint like channel, publisher, creative, keyword, timing, etc. Below are the dimension methodologies used by the most common solutions available today:

· Single dimensional. This methodology analyzes just a single dimension - such as a publisher - that can be associated with the touchpoints used by methodologies #1-6 above.

· Fixed multi-dimensional. This methodology analyzes a fixed number of dimensions - that are hard-wired into the attribution software - that can be associated with touchpoints used by methodologies #1-6 above.

· Unlimited multi-dimensional. This methodology analyzes an unlimited number of dimensions that can be supplied to the attribution software and typically employs a self-filtering system that identifies the dimensions that are impactful and those that are not. These unlimited dimensions can be associated with the touchpoints used in all of the methodologies above.

Response-only vs. All Touchpoints

Associated with all the touchpoint selection methodologies above is another factor that varies within the marketplace, which is that of response-only touchpoints vs. response and impression touchpoints:

· Response-only touchpoints. This methodology analyzes only the clicks, website visits or responses, but NOT impressions.

· Response & impression touchpoints. This methodology analyzes impressions, clicks, responses and website visits.

Converter-only vs. Converter & Non-Converter Touchpoints

Also associated with the touchpoint selection methodologies above is yet another factor that varies within the marketplace, which is that of converter-only touchpoints vs. converter and non-converter touchpoints:

· Converter-only. This methodology analyzes the the importance and weights of the touchpoints by only looking at users who converted.

· Convertor & non-convertor. This methodology determines the weight of the touchpoints by looking at the difference in touchpoint exposure between users who converted and users who did not convert.

Data Universe

Finally, all the touchpoint selection methodologies can utilize either of the following data universes to analyze:

· Sampling. This methodology selects a subset of the individuals who were exposed to your marketing touchpoints to analyze.

· All users. This uses 100% of the individuals who were exposed to your marketing touchpoints.

By piecing together various combinations of touchpoint selection, touchpoint dimension, response-only vs. response and impression, converter options, and data universe methodologies, you can imagine how each methodology would have its own degree of efficiency and accuracy in delivering insights to marketers. But probably the most important question to ask is, "how can the accuracy of each combination of methodologies are validated?" I'd be interested in your comments below to see the answers your dialogue produces. Here's a clue - think about how future data could be used to validate the accuracy of insight.

metrics

5 comments about "Can The Accuracy Of Attribution Be Validated?".

Check to receive email when comments are posted.

Timothy Brooke` from Media Performance Group, July 1, 2011 at 11:46 a.m.
Since every marketer's end goal is profitability on their marketing investments, why not use sales (actually profit) as the common benchmark and work back from there.
All media vehicles can be defined by the set geography they cover (e.g. DMA, Cable Zone, ZIP, Carrier Route, HH) and some can be targeted to specific subsets of their total coverage (i.e. Newspapers inserting to specific ZIP Codes within their coverage area). If we can purchase media by each geography (call it a Medi Buying Unit or MBU for short) and can measure sales by the same geography, then we can directly measure media attribution to sales.
Since each media's geography is different (e.g. cable zone vs Zip code) we need a way to standardize across MBUs. This is done by generating a composite score for each MBU based on the number of targeted HHs covered and then divided by the cost of purchasing each MBU.
By using the Scientific Method (test v. control) and measuring sales over time for each MBU, we can then directly attribute incremental sales to each individual media tactic (or more likely - mix of integrated media) for each MBU.
Read more about this approach at www.mediaperformancegroup.com
Reply

Anto Chittilappilly from Visual IQ, July 5, 2011 at 1:53 p.m.

Timothy has made some very good points about establishing common benchmarks, finding correlations between sales and media spend on specific channels/segments, as well as the use of A/B testing. Lots of marketers have employed these methods for decades to measure their general marketing effectiveness.

The question being asked in the article is how a marketer can validate the accuracy of their attribution methodology. The accuracy of assigning credit to each touchpoint in consumers' paths to conversion needs to be validated before the marketer can feel confident making any changes to their marketing tactics based on the learnings derived from those credit assignments.

Matt Curcio from Aggregate Knowledge, July 7, 2011 at 3:19 p.m.

The accuracy of attribution is the wrong question to be asking. The important question is is how well the attribution models output enables you to improve your ad spend and ROI. Attribution shouldn't be thought of as an oracle, rather it is one of the many tools a marketer has in their arsenal to improve the ROI of their campaigns.

We, at AK, think of attribution just like any financial (or portfolio) model. When you are buying stocks you want many different models to be telling you the same thing, you would never rely on the output of one particular model. Nor are there arguments about which financial alpha is more accurate. There are ones that are better under certain market conditions or models that have contributed more to individual success but they all have their place. Attribution should be thought of in the same light. Advertisers and agencies should use all the tools available to them to find insights. This should include different views of attribution, including last touch, first touch, and multi-touch.

michael Kaushansky from Havas Helia, July 10, 2011 at 10:22 a.m.

Attribution modeling should only be considered once you havbe solved the entire media mix question. Then use attribution to refine your daily optimization strategies to ensure daily metrics are optimized. Also, when we speak about optimization we never mention a control group; which is essential, else you end up overfitting the data and run into adverse selection effects.

Anto Chittilappilly from Visual IQ, July 12, 2011 at 10:02 a.m.

@Michael Kaushansky
Michael, you bring up a good point about the entire media mix. Cross-channel attribution should consider the following aspects:

1. The entire marketing mix

2. The Brand Equity that the marketer has built over the years or even decades.

3. Econometrics that affect the business of the marketer.

Cross channel attribution a.k.a marketing mix modeling is used to optimize the marketing spend across channels. But digital attribution or intra-channel attribution is used to optimize the marketing spend within a channel or across a subset of channels (e.g. digital channels).

@Matt Curcio
While we all agree that one of the main objectives of the marketing function is to improve the ROI of marketing spend, it is quite important to ensure the ROI measurement method is validated to be accurate. If your attribution is flawed, then your ROI measurement is flawed and you don’t know whether you are actually improving your ROI not.