The Numbers Just Don't Add Up

A funny thing happened on the way to the CMO's office.

Between the realization of an eye-opening, game-changing insight gleaned from advertising test results and Web behavior data, the report you're gleefully ferrying to the C-Suite wilted, turns brown at the edges and starts to dribble a slimy substance with a conspicuous stench.

The CMO immediately develops a nose-squint. The VP of Corporate Communications has that "Oooo, you're in for it!" look in her eye and the VP of Advertising nudges the Director of Direct Marketing and says sotto-voce, "The golden boy is about to find out his day in the sun has turned him to toast."

The CMO points to (but does not touch)

        a traffic report from comScore

        a traffic report from Hitwise

        a chart from



        an ad banner report from Atlas

        a traffic report from Omniture and

        another from Google Analytics

"It's like the old joke," she said with no humor at all. "If you take all the economists in the world and line them up end-to-end, they all point different directions. What the hell is going on with these numbers? Are we getting thirty two and a half million people on our Web site or forty-four million?"

The first time you ran into this nest of nettles, you hopped over to the white board and cheerfully explained all about

        cookie deletion

        cookie blocking

        multiple machine browsing

        multiple browser browsing

        multiple people on the same cookie

        non-human traffic

        dynamic IP addressing

        page caching

        javascript loading

        called pixel placement

You didn't even get to the good stuff about comparing miles to gallons and how

        different tools using

        different date cut-off routines and

        different methods to capture

        different types of data to store in

        different kinds of databases with a

        different method of data cleansing and

        different slicing and dicing segmentation to produce

        different kinds of reports that ended up in

        different feed for integration into

        different datawarehouses

...before you were thanked for your help and shown the door -- permanently.

You don't fall for it this time.

This time you explain that the world of online marketing has been suffering from an delusion of precision and an expectation of exactitude.

You tell them that we live in a world of statistics and probabilities. We can't count all the stars in the sky, so we don't try. We don't try to get an actual count of

        television watchers

        radio listeners

        magazine readers

        billboard readers

        bus poster readers

        floor sticker readers

        airline ticket jacket readers

        sandwich board readers

Instead, we count some and estimate the rest.

You share the good news that we can do this better than any of the above -- and we've got some astonishing tools and techniques for dynamically targeting the audience and optimizing each one's experience.

You say, "We get 36.3 million people coming to our Web site."

The CMO lowers her half-glasses and gives you the look you last saw when caught using the office copy machine for party invitations. So you add, "With a 4% margin of error and it's a benchmark we can compare month over month from now on."

"So somewhere between 34 and a half and 38 million," she says.

"Pretty much right between them, in fact."

Disparagingly, she asks, "You really can't give me a more accurate number of how many people saw this digital marketing masterpiece that costs me tens of millions a year?"

"I can tell you whether our digital visitors are more engaged with our brand, come back more often, buy from us and discuss our products with their friends. How many people buy our products who saw our ads on CNN and 'Oprah' that cost you hundreds of millions a year?"

The VP of Advertising makes himself visibly smaller.

"I came here to show you a way that could save four million dollars of search marketing while boosting online sales by 6 to 8%," you say.

The scowl leaves the CMO's face. The odor of dubious data dissipates. Her eyes narrow as she leans forward and says, "Show me."

The numbers don't have to be precise -- just compelling.

20 comments about "The Numbers Just Don't Add Up ".
Check to receive email when comments are posted.
  1. Wendy Hidenrick from AwesomenessTV, October 2, 2009 at 1:32 p.m.

    Although I am not on the client side, I LOVED this article! It gave me some insight on the client side pain points, but also helped me think more strategically when positioning my website. Thank you.

  2. Michael Meier from VINDICO Group, October 2, 2009 at 1:43 p.m.

    Hear, hear!

  3. Holly Hamann from TapInfluence, October 2, 2009 at 1:44 p.m.

    Brilliant! I have been a marketing exec in the web space for years and a long-time subscriber to MediaPost newsletters. This is a brilliant post and probably one of the best articles on this subject I have ever read. Nice job, Jim!

  4. Carol Lewis from Riverton Media, October 2, 2009 at 1:49 p.m.

    Thank you (!!!!!).

  5. Bart Foreman from Endgame Marketing Solutions, October 2, 2009 at 2:09 p.m.

    I love this. Wonderful insight and comparisons.

  6. Elise Richardson, October 2, 2009 at 2:15 p.m.

    great post! thank you!

  7. Jared stivers, October 2, 2009 at 2:37 p.m.

    "we live in a world of statistics and probabilities. We can't count all the stars in the sky, so we don't try"


  8. Howie Goldfarb from Blue Star Strategic Marketing, October 2, 2009 at 2:39 p.m.

    Great post! I hate the dubious numbers game that Marketers play. But in reality it is impossible to get exact measurements, which sadly lead to much inflating of numbers by biased interests. I leave a few websites up in my browser tabs all day long, but yet spend maybe 20 minutes on each a day...does that Website claim 8 hours or 20 minutes? But when you deal with the CFO all they care about is did they get sales! That is the goal anyway.

  9. Ken Nicholas from VideoAmp, October 2, 2009 at 3:04 p.m.

    Sometimes the best ideas/lessons/thought-provoking points to be made, come from stories & analogies. Great, great story here!

    Diagramming Cookie Deletion and Dyn IP Addressing for the CMO??? Too funny...!

  10. David Hawthorne from HCI LearningWorks, October 2, 2009 at 3:06 p.m.

    The CMO and the VP of Advertising return to their offices and immediating begin to dissemble. The nearest competitor hires away the R&D team, and a new technology is introduced that cuts the cost of doing what your client does by 20%.

    You return to yours and try to figure out precisely 'how' you are going to deliver 6 - 8% when the GDP plunges 10%, and choose between Oprah and CNN when she is boycotted by whackos and CNN discovers that it's become an entertainment/talk channel instead of news.

    As the old showbiz saying goes, "Nobody knows nothin'." Probablistic stats are better than nothing but not worth much without good analytics and informatics that can tell you "which" of your decisions are working.

  11. Erin Ulicki from Centro, October 2, 2009 at 4:18 p.m.

    Wonderful column - it sums up nicely the double-edged sword online created for itself years ago that still plagues us. Often when clients get stuck in the metric details we have to make them step back and say "Tell me, how are you measuring your other media?"

    Then we get into the discussion of determining how we helped them sell product and get butts in seats - because really, that is what matters.

    Thank you!

  12. John Jainschigg from World2Worlds, Inc., October 2, 2009 at 5:53 p.m.

    What a great article. I agree with Holly - it's not everybody who can deliver a barrel-full of laughs AND profound wisdom in under 1000 words. Nice job!

  13. John Grono from GAP Research, October 2, 2009 at 5:53 p.m.

    Stands and applauds. Bravo Jim!

    That is simply the best piece I have read on online metrics in ages, if not ever (apologies to Josh Chasin who also "gets it" and writes brilliantly as well).

    May I have permission to use:

    * "the world of online marketing has been suffering from an delusion of precision and an expectation of exactitude"; and

    * "we can't count all the stars in the sky, so we don't try"

    down here in Australia when called on to explain the self-same situation.

  14. monica levy, October 2, 2009 at 6:04 p.m.

    great post.

    i think the giant problem with reconciling today's metrics, whether they come from the web or advertising or pr or other functional discipline, is that we still (apart from direct-response print or tv that makes no use of any other medium) don't know how each of these tactics -- individually or collectively with others -- affects a customer's perceptions or purchase behavior.

    i may see a print or tv ad, follow it up by going to a website, and not do anything about it for weeks or months until i overhear a conversation between two people on a plane who're talking about the product or the brand .... and i may STILL not do anything until much, much later, when i actually DO buy whatever it is, but have completely forgotten about how i came to know about the thing. so even my response on a survey asking me "how did you hear about us" is suspect, because i will probably lie to avoid looking like an idiot.

    this is why i personally favor net promoter score as the key metric that should be tracked above all others ... we do know, statistically, that people are more often likely to trust the recommendations of family/friends/colleagues, and we can go about getting NPS data in a fairly straightforward way. not as fun as explaining cookie deletion and dyn ip to the cmo ... but probably much more worthwhile, in the long run.

  15. Robert Rose from Big Blue Moose, October 2, 2009 at 8:36 p.m.

    Wow was this ever good. It's just so refreshing to see this kind of thinking. Thank for a great post. I've been making this point to clients for so many years - and it's just great to have the story so well told.

  16. Mark Hughes from C3 Metrics, October 2, 2009 at 9:24 p.m.

    Yes! Now read the White Paper on C3 Metrics which solves this.

  17. Eric Melchor from Smart Digital Spending, October 3, 2009 at 3:29 p.m.

    So typical of a CMO, wanting to track and measure every possible metric across a variety of tools. It should be established in the beginning that one source will be used as the benchmark.

    Great post Jim!


  18. Jim Sterne from Target Marketing, October 4, 2009 at 8:09 p.m.

    I am absolutely delighted and humbled by all of your comments. Thank you so much. I'm speechless... until the next <a href="">eMetrics Marketing Optimization Summit</a> ;-)

  19. Katie Paine from Paine Publishing, October 5, 2009 at 5:55 a.m.

    Great stuff, Jim. The problem is that the Ad and PR Agency reps are selling the CMO on "eyeballs" and the web metrics guys are delivering revenue. Which is the CEO going to buy?

  20. Brewster Barclay from Clickstream Technologies PLC, October 5, 2009 at 7:44 a.m.

    Jim, Great post which has universal approval from all those that are actually working at the coal face of on-line marketing. Don't remain speechless for too long.

Next story loading loading..