• by Josh Chasin , Op-Ed Contributor, February 26, 2008

Well, OK, maybe not that deep.

Arguably the biggest story in online metrics in 2007 was the fact that both NetRatings (now part of Nielsen Online) and comScore's Media Metrix commenced the MRC audit process.  In fact we at comScore began the process in 2006, with the pre-audit; but we entered the formal audit phase in September 2007.  I think that there were three major headlines associated with this news; let's look at each one. 

Audit and accreditation of the panel companies means Internet measurement, and thus Internet advertising, is finally coming of age.  I will argue that the robustness of Internet ads pend in a largely soft ad market demonstrates convincingly that measurement accreditation has not in fact slowed down the growth of the medium. That is not, however, to say that the MRC audit is not a necessary and significant step for this industry; indeed, I believe that accreditation might well make it easier for the major general market (which means, TV) agencies to justify shifting some offline dollars online.

What I do want to stress here, though, is that it is probably unwise for measurement services to submit to the audit process prematurely; the process is rigorous, the bar set high.  And because the audit and accreditation process to a large extent formalizes methodology, it is best to wait for audit until the methodology has stabilized.  In the early years of any measurement service, by definition the research company should remain flexible enough to respond to market and measurement conditions that arise.  (I've actually given this advice to some of the newer measurement companies.)  ComScore was founded in 1999, and began the MRC process in 2006; in a historical context, this is not unusual.

Audit and accreditation will assure transparency and disclosure. True, and a significant positive for the industry.  Check out the Media Rating Council's Minimum Standards for Operation and Disclosure to understand precisely what this will entail.

Now that the panel companies and some publishers are audited, we'll be able to reconcile the differences between panel-based and site-centric counts of Unique Visitors. Not so much.

It is important to note that MRC accreditation is not a reconciliation process.  I've written in the past that audience measurement data and Web analytic data are two different animals.  It is entirely possible that comScore can do what we do with excellence, and Omniture or Webtrends can do what they do with excellence, and estimates from each of us are accredited, but they will not come any closer to matching.  If readers are expecting the divergence between panel-based estimates and site-centric estimates to be ameliorated by audit and accreditation, I must urge that this will almost certainly not be the case.  I expect the IAB's document on Audience Measurement Guidelines and Definitions, which I hope will be out in the second quarter, to help the industry sort out the differences between panel-based, site-centric, and ad server-based metrics, so that the differences may be understood.  George Ivie of the MRC has been one of the primary architects of this document.

Some publishers have undergone an MRC audit of their site-centric data.  In my opinion, the process for a publisher -- using an outside Web analytics vendor, and reporting on their own audience only, and limiting that reporting to cookie-based  machine-level (as opposed to persons-level) measures -- is fundamentally different from the process a syndicated measurement company undergoes.  In fact, I've been thinking that it is probably misleading to call the results of the publisher audits "accreditation," because to do so implies an effort of the same magnitude as that involved in a research company audit.  Rather, I might suggest that publisher audits result in MRC "certification."  But I have not mentioned this to George Ivie, so please, let's keep this between us for now.

Let me know if you want to hear more about the MRC process, and I'll cover it in a future column.