• by Judah Phillips , December 28, 2007

As 2007 ends, I thought it worth looking back, from the practitioner perspective, at just a few of the issues that have shaped Internet measurement and thus online metrics over the last year:

· The Page View is Dead, Long Live the Page View. During 2007, technologies like AJAX and Flash continued to erode the construct of the page view. These technologies render content in a browser but do not always make requests to the server to do so. If the technology you are using to measure behavior requires the page request and you do not have a page request, what do you measure? The major vendors of online metrics tried to answer that question.

Various audience measurement companies claimed "total minutes" and other time-based derivatives were better alternatives to measuring the page view. Web Analytics companies rolled out features for measuring "events" subordinate or equal to the page view (and highlighted already existing time-based metrics). Ad serving companies made inroads in reconciling view-through to assist advertisers in understanding the latent effect of ad exposure on the purchasing lifecycle. Yet all these technologies still count and report page views.

· Engagement, Engagement, Engagement. One of the hot topics in 2007 was a carryover from 2006. Definitions for "engagement" emerged from the worlds of advertising, social media, online metrics, and more. Engagement has been described as "turning on a prospect to a brand idea enhanced by the surrounding context" to "repeated, satisfied interactions that strengthen the emotional connection a customer has with the brand" to "apparent interest" to the more metrical "estimate of the degree and depth of visitor interaction against a clearly defined set of goals."

"Engagement" is very specific to the site being measured and full of nuance. This fact has led agencies, consultants, and various companies to create complex engagement indices consisting of measures of key behaviors. Behaviors are tallied and segmented in order to calculate an engagement metric, which is then used as the basis for site optimization. These indices go far beyond often-cited simple time-based measures of engagement. For a well-thought-of example, see Eric Peterson's Engagement Metric

Cookie Deletion, Again! In 2005 Jupiter Research first uncovered and quantified how cookie deletion can affect unique visitor numbers in Web analytics systems. The effect of cookie deletion is not quantifiable in the basic way audience measurement companies want to quantify it in 2007 -- by only examining cookie deletion rates from self-selecting panelists who visited one portal site and an ad server.

Cookie deletion behavior varies greatly by audience segment and by site. It may be as much of an accuracy problem in Web analytics as selection bias and coverage errors are in panel measurement. It is worth noting that some audience measurement firms use cookies to collect panel data.

Black Box Audience Measurement. Many questions were asked about whether audience measurement companies adequately measure "unique visitors" or "unique users" or just the frame of self-selecting "unique panelists." In audience measurement, counts of "unique visitors" are generated using complex, black-box mathematics that project observed metrics to the entire online universe. The projections are always unequal to the same data provided by other audience measurement companies or Web analytics tools. Panel inconsistencies (across at-home, at-work, at-university, or specific to the geography being measured) may cause some level of bias and error. Accounting for the difference between actual, observed panel metrics and projected metrics is perhaps even more challenging to clarify and resolve than the measurable effect of cookie deletion.

The Continuing Need for Standards Enforcement. 2007 was the year two significant industry bodies continued working on standards related to online metrics: the Internet Advertising Bureau and the Web Analytics Association. While each organization serves the needs of different constituencies, they both share the inability to enforce standards. Both bodies can say what you should do, but not what you "must" do.

Throughout 2007, these issues (and others) brought increased attention and scrutiny to online metrics. Corporations are inextricably linking online metrics to site and channel strategy and performance, and thus to overall corporate profitability. The "numbers" are now more important than ever for managing an online business and maximizing online revenue. Nevertheless, questions are still being asked about accuracy, precision, usage, and sources of online metrics. We have a ton of collaborative work to do in 2008 to provide the best answers and numbers we can.

Happy New Year!