• by Omar Tawakol , June 25, 2004

I have been entertained by the fact that, while plenty of vendors promise targeting capabilities, few actually provide a simple way of assessing whether or not their targeting works. An advertiser's question to a publisher is, "If I pay more for a targeted campaign from you how much of the campaign was delivered to an in-target audience?" The good news is that the answer to this question is an extremely simple metric that is already part of the industry vernacular.

Before I give you the answer, I'd like to suggest that there are three key criteria for any good quality metric, specifically, Comparability, Ease, and Consistency.

Value Comparability: Advertisers should be able to use the metric to easily compare two audiences. The comparison has to correlate directly to economic value.

Ease of Use: A workable metric must be well-accepted and understood, as well as simple and inexpensive to deploy and validate.

Consistency: A metric must mean the same thing every time it is used. If the metric fails this test, it no longer allows comparison.

The prospect of having a clear-cut quality metric makes some publishers nervous. They think that by staying away from a metric that compares them with competitors, they can get more money from the market. The opposite has been proven to be true. Economists have fondly named this phenomenon Grenshaw's Law. The Law suggests that when there is no metric of quality, low quality drives out higher quality and price becomes the only deciding factor. This is bad news for quality media properties that want to leverage their high-value audience and bad news for advertisers that want to reach the highest-quality target.

So what should a behavioral targeting metric measure? Behavioral targeting promises that it will get you the right people, so we need a metric that tells you if that promise is fulfilled. Fortunately, we don't have to create something out of whole cloth and then educate everyone about it because there's a metric that's been used successfully for a long time: Audience Composition. In the offline print world ad dollars are allocated using audience composition numbers. In the online world, agencies use audience composition when allocating ad dollars at the site level. That's why Nielsen exists.

Knowing composition allows advertisers to make qualitative comparisons that show them that while Wall Street Journal, Slashdot and USA Today are all news publications, technology geeks read Slashdot, travelers read USA Today and C-level executives read the Wall Street Journal. Since marketers already ask about the composition of the site, why don't they ask about the composition of an audience within the site?

So let's check and see if Audience Composition meets our three criteria.

Does it allow value comparability? Yes, because it takes the context of the site's audience into consideration. In fact, it directly correlates to economic value since you can calculate cost per thousand targets by dividing CPM by composition.

Is it easy to use? It's being used everyday in both offline and online media and can be measured by a variety of vendors.

Is it consistent? Audience Composition always measures the same characteristic of the audience being purchased. A 35 percent audience composition for a 5+ business traveler is conclusive, regardless of how the audience was constructed or which site was used. That's the point of a metric.

So if it's so simple and clear, why is there a debate? Some in the industry are proposing a naming standard for the inputs of an audience, as opposed to a quality metric that measures the output. The idea being proposed is that if we use standard nomenclature we could all agree that an "active traveler" is someone who clicks on the travel section 10 times in the past month. In other words, rather than apply a quality metric to the resulting audience, they suggest we should agree on how to name the pieces that go in to defining the audience. Let's apply the criteria from above to see how well this works.

First, can you compare the value of two audiences using the naming method? The answer is "no," and here is why. Site A and Site B both define "active travelers" as visiting their travel page 10 times a month so advertisers know they are getting the same audience, right? Wrong, because if the Site A's Composition of "active travelers is just 2 percent and Site B's is 50 percent, the in-target audience from site B is 25 times larger than site A's.

Which would you buy? Number of visits doesn't correlate to quality across sites.

On to the next criteria: Is the naming standard easy to use? Again the answer is "no." When you use algorithms that look at dozens of attributes in the design of an audience, describing that set of inputs can be very complex. Finally, does the application of a naming standard create consistent quality? Again the answer is "no." Just because someone calls something they sell you "good" does not necessarily make it so. If Slashdot and USAToday both define an audience of 10 time business travelers, the only thing in common between those groups would be their name.

Applying the simple criteria of comparability, ease, and consistency shows that Audience Composition provides an excellent metric for behavioral targeting quality. There are precedents for this approach in other industries such as the financial industry where risk grades judge the quality of a loan population and allow you to compare two pools of loans. Rather than agreeing on how to name the hundreds of attributes that define a given loan group, the industry came up with a simple grade that describes the risk of the pool. Standards are made to help buyers make value decisions and hence they must describe the quality of the output not the complexity of the inputs!

Let me be clear. Audience Composition is not meant to replace post-campaign evaluation such as conversions or brand metrics. There will still be questions about whether the campaign was compelling, whether people clicked on it, and whether it resulted in dollars that are influenced by the ad creative and other issues. But that is not the task at hand. As Randy Kilgore, the Vice President of Advertising at the Wall Street Journal Online says, "Behavioral targeting allows advertisers to precisely target the audience they are trying to reach with the message they are trying to convey and Audience Composition is the simple and clear way to show how precise that targeting is."