• by Judah Phillips , September 21, 2007

Over the last two weeks, my fellow Metrics Insider columnists have correctly pointed out that online metrics are neither standardized nor easily integrated across systems. Vocabulary is muddled. Numbers do not match. Data exists in silos and is isolated from related data. Systems do not adequately or easily talk to each other. Research services, ad servers, and Web analytics tools report similarly named, overlapping and often conflicting metrics. Unfortunately, these problems will not disappear anytime soon, even with emerging "standards" and continued attention paid by the industry to these important issues.

Current industry standards for Web metrics are limited, basic, and come from independent entities. Most recently, the Web Analytics Association released a set of "standards." The WAA's standards are elementary definitions of concepts from various periods of Internet measurement. Web 2.0 concepts like "events" are mingled with dated measurements like "hits." Regardless, these definitions provide a very useful starting point for framing a discussion about metrics. Recently, I've learned that the IAB and MRC are developing a set of IAB Reach Measurement Guidelines. Let's hope the IAB and WAA align their work efforts.

The IAB and MRC are also currently auditing "audience measurement" firms, like Comscore and Nielsen. It's rather unclear to practitioners what standards the IAB/MRC are applying to the audit. But the hope is that auditing will expose issues of coverage error and selection bias in the black box methodologies used to create the panels and generate the audience measurement data.

It is important to note that the IAB's audit has two parts. The first is certification, which indicates the company being audited is applying the "standards," and the second is accreditation, which demonstrates adherence to the IAB standards.

Only time will tell if companies like Hitwise, Compete, and Quantcast will be asked to submit to auditing. It's worth mentioning that legacy metrics "standards" (and audits) from historic organizations like ABCe still occur and carry weight with publishers and advertisers (especially outside of the United States). It's entirely possible that newly formed organizations, like the Association for Downloadable Media will offer their perspective on "standards" for online metrics.

The idea of "standards for the standards"--however absurd it sounds on the surface -- starts to seem like a good idea when considering that all these parallel efforts aren't intersecting. Honestly though, I question whether "standards" that are purely "definitional," even if agreed upon, will solve many of the measurement challenges companies have when trying to understand Web data and take action from it.

Standard definitions are helpful for promoting understanding and creating a controlled vocabulary for discussing online metrics, but they don't help with what I see as a huge challenge in today 's metrics technologies. The problem is this: currently available online metrics systems do not adequately separate data from presentation . That's a huge limitation preventing Web data from being easily integrated with other systems.

Detailed-level Web data (the raw data) is often costly to extract, if available at all. It is nearly impossible to deliver detailed data in real time from Web analytics, ad serving, and research-based technologies in order to feed other systems. The majority of hosted (ASP) metrics systems are closed and do not allow access to key interfaces using open software standards. For the most part, today's metrics technologies are black boxes where data goes in, but can only be extracted in various file formats after creating a report. Common export formats include csv, pdf, and doc. While XML exports are often available from many vendors, there is no standard XML schema for describing the same type of Web data across different sources!

The industry must begin collaborating and creating a standard XML schema for describing Web data. Creating a widely used, consensus-based, published, and maintained XML standard for online metrics would make it possible to more easily share, transform, and use Web data in other systems.

I firmly believe that current metrics standards must go beyond simple definitions and tackle issues pertaining to data portability and system interoperability. Then we'll all be in a better position to reuse Web data across the enterprise value chain. Once we all agree on "standard" definitions, I encourage us to start working together to develop a standard Online Metrics Markup Language.

**************

Insiders Wanted

Are you an Insider? Do you have inside knowledge about metrics -- and do you want to share your insights, observations and points of view with the readers of MediaPost? To be considered for the Insider team of online media influencers,please send pertinent info about your credentials, plus one column idea, to Insider@mediapost.com.