• by Jack Loechner , Staff Writer @mp_research, April 24, 2007

Web Audience Measurement by Cookie Counting Considerably Overestimated

comScore recently released a study that analyzes the validity of using cookie-based data to measure unique visitors and to guage the number of unique users that were served an ad by an ad server. The results indicate that Web site server logs that count unique cookies to measure unique visitors are likely to be exaggerating the size of the site's audience by a factor as high as 2.5, or an overstatement of 150 percent

A "cookie", a small text file inserted on a user's computer by a Web server, is often used by web servers to identify and authenticate, track, and maintain specific personal information. First-party cookies are those left on a computer by a Web site that has been visited, while third-party cookies are those left by a domain different than the site being visited.

The Study observed that:

• 31 percent of U.S. Internet users cleared their first-party cookies during the month.
• Within this user segment, the study found an average of 4.7 different cookies for the site.
• Among the 7 percent of computers with at least 4 cookie resets, the study found an average of 12.5 distinct first-party cookies per computer, accounting for 35 percent of all cookies observed in the analysis.
• An average of 2.5 distinct first-party cookies were observed per computer for the site being examined. 

Cookie Deletion Analysis(1st Party Cookies December 2006)
Number of Cookie Deletions/Resets	Percent of Computers	Avg. No. of  Cookies per Computer	Percent of Cookies
Total Sample	100%	2.5	100%
1 or more	31%	4.7	58%
4 or more	7%	12.5	35%
Source: comScore, Inc., April 2007

Dr. Magid Abraham, President and CEO of comScore, concludes that "...with just 7 percent of computers accounting for 35 percent of all cookies, it's clear that a certain segment of Internet users clears its cookies very frequently.  These ‘serial resetters' have the potential to wildly inflate a site's internal unique visitor tally, because just one set of ‘eyeballs' at the site may be counted as 10 or more unique visitors over the course of a month. The result is a highly inflated estimate of unique visitors for sites that rely on cookies to count their audience."

The analysis of third-party cookies revealed an average of 2.6 distinct cookies per computer in December, indicating a similar rate of overstatement as the first-party cookies.

Cookie Deletion Analysis(3rd Party Cookies December 2006)
Number of Cookie  Deletions/Resets	Percent of Computers	Avg. No. of Cookies per Computer	Percent of Cookies
Total Sample	100%	2.6	100%
1 or more	27%	5.5	57%
4 or more	7%	14.2	38%
Source: comScore, Inc., April, 2007

Dr. Abraham also noted that "There is a common perception that third-party cookie deletion rates should be significantly higher than first-party cookie deletion rates. Because many PC users reset or delete their cookies using security protection programs, conventional wisdom dictates that people are more likely to selectively expunge third-party cookies, which are generally deemed more invasive, while maintaining their first-party cookies. But these findings suggest that selective cookie management is not prevalent... confirmed via a survey, with only 4 percent of Internet users indicating that they delete third-party but not first-party cookies."

Mohanbir Sawhney, McCormick Tribune Professor of Technology at the Kellogg School of Management, Northwestern University, commented "To measure audiences more accurately, it is important to link visits to unique individuals, not unique cookies. As privacy programs become more entrenched, cookie-based audience counts will get even more unreliable."

And Bill Cook, PhD, Senior Vice President for Research and Standards at the Advertising Research Foundation, adds   "Reach and frequency metrics are the cornerstone of any media plan, and given the size of the discrepancies that can occur when counting cookies instead of people, the study underscores the importance of panel-based measurement.  For the advertising community, an accurate understanding of reach and frequency within a given target audience is vital."

For more information, please visit comScore here.