Welcome | View My Profile | Sign Out
MediaPost Home About MediaPost Privacy/Terms Media Kit Sitemap
Publications Home News
Online Media Daily Media Daily News Marketing Daily Mobile Marketing Daily Search Marketing Daily
Daily Feed> Email Daily Feed> Video Daily Feed> Social
Online Blogs
Online Spin Email Insider Search Insider Behavioral Insider Online Publishing Insider Mobile Insider Video Insider Gaming Insider Performance Insider Metrics Insider Social Media Insider Just An Online Minute Daily Online Examiner Raw Blog
Media Blogs
Research Brief Diane Mermigas:On Media TV Watch TV Board Magazine Rack Media Creativity Notes From the Digital Frontier Digital Outsider Mad Blog Red White and Blog
Marketing Blogs
Engage:Hispanics Engage:Kids 6-11 Engage:Moms Engage:Boomers Engage:Gen Y Engage:Teens Marketing:Green Marketing:Sports
Magazines
OMMA Magazine Media Magazine
Subscribe
Feedback Loop RSS Feeds Archives Subscribe
Feb 24 OMMA Metrics Measurement (NYC) Feb 25 OMMA Behavioral (NYC) Mar 17 OMMA Global (San Francisco) Apr 14 Search Insider Summit (FL) Apr 18 Email Insider Summit (FL) Apr 27 Outfront Conference (NYC) May 12 OMMA Mobile (NYC) May 13 Digital Out-of-Home Awards (NYC) Jun 15 OMMA Video Jun 16 OMMA Publish (NYC) Jun 17 OMMA Social (NYC)
Recently Concluded Events
Jan 26 OMMA Social (San Francisco) Jan 25 OMMA Performance (SF) Jan 12 MEDIA Agency of the Year 2009 (NYC) Jan 11 OMMA Agency of the Year 2009 (NYC) Dec 6 Email Insider Summit (Utah) Dec 2 Search Insider Summit (Utah) Nov 3 OMMA Adnets (NYC) Oct 30 OMMA Video (LA) Oct 29 OMMA Mobile (LA) Oct 29 OMMA Mobile & Video (LA)
All MediaPost/OMMA Events Event Blogging Past Event Videos
Industry Events Calendar
2010 Digital Out-of-Home Awards
2010 MEDIA Agency of the Year 2009 2010 OMMA Agency of the Year 2009 2009 Creative Media Awards 2009 OMMA Awards 2009 Digital Out-of-Home Awards 2009 Media Agency of the Year
All Awards
Employment Situations Wanted Services Offered Post a Job
Briefs Reports Online
MediaPost Directories
Mobile Insiders Group
People Finder Edit My Profile View My Profile My Contacts My Calendar
HOME • MANAGE SUBSCRIPTIONS • MEDIA KIT
Why Don't The Numbers Match?!?
by Judah Phillips, Tuesday, June 10, 2008, 10:45 AM

SHARE

TOOLS

RELATED ARTICLES
TAGS:  Metrics

MOST READ

A question any practitioner of Internet-based analytics will be asked by many different stakeholders is, "Why don't the numbers match?"  Counts of the identically named metrics from ad servers don't match the Web analytics tool, which don't match the for-pay third-party audience measurement tools, which don't match the free audience measurement tools, which never match any of the homegrown internal measurement tools.  And none of them ever match each other.

So it's a good and certainly valid question to ask.  The answers are even fairly easy to understand, but the root causes are often difficult to pinpoint and even harder -- if at all possible --  to remedy.  The fact of the matter is that data discrepancies in analytics occur for a multitude of reasons, such as:

 

  • Different data collection methods.  We have a bunch of tools and services that collect Web data using various, non-standardized, proprietary data collection methods.  Ad servers use javascript page tags.  Many Web analytics tools use page tags too, but it's not uncommon in Web analytics to use additional methods, such as log files or packet sniffers.  Or perhaps a combination of these methods, called hybrid data collection.  And all the tools have different algorithms for processing the data collected.

On the audience measurement side, data is collected from self-selecting panels who install proprietary software (i.e. toolbars and so on) on their computers, perhaps at work or at their university, but most likely at home.  Then, the collected data from different panels is rolled-up and combined, and the limited subset of the Internet population that chooses to be monitored, in exchange for some incentive, is inflated and projected to the entire Internet audience using proprietary statistical methods.  We also have data collected from a limited set of geographically specific ISPs.  And regardless of whether we're talking about audience measurement or Web analytics, the different data collection methods often, but not always, involve cookies and all their inherent issues of cookie deletion. 

 

  • Unique data models.  Ad servers aren't focused on counting page views and the other dimension of Web analytics (visits, time, and so on).  Rather, ad servers focus on serving and counting impressions served (and loads of related derivative calculations, like CTR, CPC, and view-through).  Metrics are based on an ad request and an ad code.  Ads may or may not be targeted to a page, and instead to various constructs, like a "zone" or "keyword."  What that means is that the "page" dimension may not even exist in your ad server's data model.  In other words, you aren't looking at impressions measured on a page, but rather at the number of impressions served in a different conceptual construct.  That's one of the reasons that people say metrics and ad-serving systems "don't measure the same thing."

 

  • Untagged pages.  Specific to technologies that collect data or serve ads using javascript page tags, there are challenges to ensuring and verifying complete coverage of page tags across every page on a site.  When the pages aren't all tagged with the different tags for the assorted technologies, guess what?  The numbers won't come close to falling within tolerable variances.  And questions and skepticism will ensue.

 

  • Non-JS-executing clients and ad blocking software.  Let's imagine, for the moment, your site is perfectly tagged for all technologies, so the numbers between your ad server will be close to your Web analytics system, right?  Nope, regardless of data model issues, not all browsers execute javascript -- and many Firefox users have installed Ad Block Plus. 

 

  • Cookie issues.  When you're counting based on cookies, third-party cookies get blocked (often by privacy software).  Many ad servers and Web analytics tools still serve third-party cookies, and many corporations have not tricked out their DNS to accommodate this issue.  And we all know how cookie deletion affects unique visitor counts, even if you use first-party cookies.

 

  • Many other issues.  Latency from visitors moving off the page prior to the tag executing to latency in the call to pick up an ad from a third party while your ad server counts the traffic (so your ad count differs from the agency's count), to refresh rates making it hard to correlate page views and impressions, to no rich media installed and no fallback, to robotic traffic not being filtered from logs or tags, to certain types of user agents (such as mobile devices) not executing javascript... there's a whole host of other factors that cause data discrepancies.

And of course, there's always the nebulous issue around the complete lack of consensus-based, enforceable standards for online measurement.  No industry organization can say what vendors or companies "must" do, only what they "should" do. And no industry body is going to get successful companies to change their secret sauce just because they said so...

So what's a practitioner to do?  Understand the potential sources of discrepancies.  Work with your team (from IT to vendors) to prevent and minimize the root causes when possible.  Educate your team when discrepancies are not remediable.  Ensure you use the different sources of metrics judiciously in the context of your business goals.  Finally, realize that none of the tools are more "correct" than any other.  All of our analytics tools serve different, and sometimes overlapping, business purposes -- from counting ads, to influencing media buying, to sizing audiences, to measuring business performance, and to optimizing the site.

1 person recommends this article. 

Leave a Comment

You must be signed in to comment. Sign In

Do you have strong opinions and inside knowledge about the topic of this article -- and do you want to share your insights, observations and points of view regularly with the readers of MediaPost? To be considered as a MediaPost contributing writer, please send pertinent info about your credentials, plus several column ideas and one example of your writing on the topic, to pfine@mediapost.com. Please see our editorial guidelines here first.

JUDAH PHILLIPS


AUTHORS

ARCHIVES

Recent Metrics Insider Articles
How To Avoid A 'Holey War' Of Spending   
My friend and business partner, Dave Reibstein of Wharton School, writes an advice column called Ask...
Audience Insights, Not Audience Measurement    
I find myself quoting Randall Rothenberg, Chairman of the IAB, more and more these days. Earlier...
The Cost Of NOT Branding   
It's a simple formula: Recession requires more tactical spending. This year's budget = + online spend...
Online Metrics: Trends For 2010   
When I look back at my 2009 predictions, I feel pretty good about my hit rate....
The Marketing Kwan   
"Love, respect, community and money" tied into one word. That is what Rod Tidwell was describing...
Forecast For 2010: Better Forecasting   
Yogi Berra said, "It's tough to make predictions, especially about the future." Yet the turn of...
Measuring What Matters Most   
There are so many things to measure, more and more marketers are getting wrapped around the...
Setting Realistic Expectations For Targeted Campaign Delivery   
We have a tendency on the Internet to believe that -- because we are, after all,...
The Future Of The Marketing Dashboard   
If conversations with clients are a good indication of the future, then 2010 will be the...
Making The 'Best' Business Case For Marketing Investments   
More than ever before, the approval of any significant marketing initiative is dependent upon a compelling...
>> Metrics Insider Archives 
ABOUT MEDIAPOST • MASTHEAD • MEDIA KIT • RSS FEEDS • PRIVACY/TERMS & CONDITIONS
©2010 MediaPost Communications. All rights reserved.
1140 Broadway, 4th Floor, New York, NY 10001
tel. 212-204-2000, fax 212-204-2038, feedback@mediapost.com