Researcher Beware

  • by March 8, 2001
Researcher Beware

While preparing a study on comparative viewer statistics by rating firms, The Center for Media Research has viewed a variety of reports, each valid from its own perspective. Our conclusion will certainly be that the data user must be aware of the methodology used, and the appropriateness of the sample selected to make projections. But before we get there, here’s an example of the material uncovered as reported by Ariana Eunjung Cha in the Washington Post.

The internal logs for one dot-com logged nearly 900,000 visitors to its site in December while one service was reporting roughly 660,000 for home users only, another said it was 530,000 for home and work users and a third counted 456,000 home and work users.

Advertising rates often are tied to the number of people viewing a particular Web site. So far, no one company has emerged as the absolute authority. The leading firms cannot even agree on a methodology. Some record users as young as 2 and some do not. One firm only counts PC users, ignoring those on Macs. No company has a clear idea of the surfing habits of those at work, where Web use is heavy. And no one monitors use at schools, libraries or other institutions

The vastness and diversity of the online world, however, adds an extra dimension of complexity, media analysts say. Traditional media auditors may have to track several hundred or possibly a few thousand broadcasts or publications. Web site researchers follow more than 300,000 sites.

Some of the biggest variations come because of the way the ratings' companies go about their jobs. Several base their ratings on the Web viewing habits of a sample of computer users. But the companies differ in what samples of people they keep tabs on. Some, track users age 2 and older. Another limits its panel to people 13 and older. Some select their participants randomly. Another uses ads on cable television stations and Web sites to recruit volunteers

The need for standardization remains strong say Wall Street analysts and investors. At least half a dozen studies by researchers at Stanford University, the University of Washington at Seattle, the University of Rochester and other places, found that Web measurement numbers are often linked to the market value of companies.

"I think I'd lean toward revenue over a particular rating system, but more often than not those two things go hand in hand," said Paul Noglows, an analyst who covers AOL, Yahoo, Terra Lycos, and other dot-coms for J.P. Morgan H&Q.

Read the complete article and links here.

Next story loading loading..