A transition that would likely have sparked a firestorm of debate in Nielsen's core medium of television - a complete overhaul of the system it uses to produce its audience estimates - has gone
virtually unnoticed in the online industry. That was until a few weeks ago, when a routine investigation revealed a strange and unexpected glitch in the methods Nielsen currently uses that grossly
exaggerated the online audiences of three of the Web's best-known publishers: Fox News, The New York Times, and USA Today (see related story). Nielsen today will release corrected, albeit lower online
audience estimates for the month of May for those publishers, and it says the new system will prevent problems like that from ever happening again, but the gaff is finally getting people in the online
business to focus on the impact a new Nielsen audience ratings system will have for Web publishers, advertisers and agencies alike.
While the new system will clearly be an improvement by many
standards - including a new sample that will be 10 times larger than the one it currently uses to measure online audiences, enabling Nielsen to dramatically increase the number of Web sites it reports
data for - a change of that magnitude also is likely to alter the relative shares of online publishers that depend on Nielsen's data.
It's a methodological change, one long-time Nielsen client
says, "could rival the people meter change of the late 1980s." The client was referring to the TV industry's transition to people meters in 1987 from the older, arguably less accurate meter and paper
diary system Nielsen used before. While a methodological improvement, that change sent ratings for the major broadcast networks plummeting, and cost them millions of dollars in lost advertising
revenues. Even in recently, years after years of vetting, the transition to people meters in local TV markets has caused TV industry eruptions, including a federal antitrust suit filed against Nielsen
by Sunbeam Television Corp.
Nielsen executives say it is routine for clients to "shoot the messenger," or blame Nielsen when it changes its methodology, even when it's an improvement, if the
changes reduce their audience estimates and decrease their share of advertising dollars. And while Nielsen's Web audience estimates don't hold the same level of "currency" value in the online industry
that they do in television, some big Web publishers have been conducting their own investigations into Nielsen's methods - though they have been oddly silent about the imminent change.
The New
York Times has been most vocal in recent months, insisting that the downward trend in its Web audience numbers are a result of Nielsen's methods and do not reflect a genuine shift in online user
behavior.
"We believe their data is in error," New York Times spokeswoman Diane McNulty said. "We're reviewing the methodology now as the numbers are inconsistent with three other independent
sources and our own internal data."
New York Times executives declined to elaborate, but McNulty's comment refers to the fact that unlike television, the online industry has a multitude of
sources for estimating, and presumably validating, online audience counts. But even in the online world, the Nielsen name carries a lot of water, especially as Nielsen moves to integrate its online,
mobile, TV, and "other screen" measurement systems in a way that other research companies may not be able to.
In fact, Nielsen Online's new system, dubbed NetView RDD//Online, will further
integrate its online and TV audience measurement systems. Three thousand of the households in the new online measurement system's 200,000 households sample come directly from Nielsen's TV audience
measurement system - a soc-called "convergence panel" that is a subset of its national TV ratings service that measures both TV and online usage behavior.
On the plus side, the new online
measurement system will be ten times the size of its old NetView system, which represents the last remnants of the original NetRatings measurement system based on the so-called "PC meter" technology
that Nielsen entered the online industry with nearly a decade ago. The expansion of the NetView sample may have another unintended consequence for big Web publishers that currently dominate the online
display advertising marketplace. By moving from a sample of 20,000 to a sample of 200,000, NetView will be able to expand the number of sites it measures and reports data for from about 3,000
currently to 25,000 with the new system. While that is clearly good for Nielsen's business, the expansion will effectively fragment and dilute the online advertising marketplace, enabling many more,
and much smaller Web publishers to compete with the industry's biggest, and to tout the same Nielsen-branded audience estimates that companies such as the New York Times, News Corp., Gannett and
others use.
All this occurs, of course, as the Web publishing industry grapples with an economic downturn, as well as what some see as a "secular" shift in the online display advertising business
(see related story) that may be commoditizing online advertising prices, even as companies such as the ones mentioned above are trying to wean themselves off higher-valued analog publishing dollars to
what are still lower priced digital advertising rates.
But the biggest problem in the shift occurring this summer, is that online publishers, as well as advertisers and agencies, simply will not
have enough data to plan, project, and prepare for the shifts that are likely to occur by changing the measurement system. Typically, when Nielsen makes such a wholesale change in its TV measurement
systems, it gives the TV industry a year's worth of "parallel" data, running the old and new systems currently for 12-months to allow its clients to prepare for the impact of the change. That happened
in 1986-87 when Nielsen converted to people meters, and it happened more recently when it switched from average TV program ratings to time-shifted commercial ratings. Even then, the TV industry has a
fair amount of uncertainty and anxiety with the shifts.
But Nielsen is only providing five-months worth of parallel data for its online measurement system shift. Ironically, it had only planned
to issue three-months worth of evaluations data before turning the new system on, but opted to extend that another two months when its audience measurement glitch was brought to light. But even that
may not be enough time or data to allow the online industry to plan for its impact.
"The biggest fear that everyone has is that they have no way of forecasting what will happen," says one Nielsen
client, adding, "While I don't expect the number to be unrecognizable, Nielsen is making wholesale changes - a new meter, a new sample, a new method, and new projection techniques for weighting the
results on a month-to-month basis."
Nielsen executives acknowledge that such changes are vexing for an industry that depends on its data to conduct commerce, but they say the trade-off is
necessary because of the improvements that will come along with the disruption. In addition to the increase in the sample size, and the ability to measure and report many more Web sites, Nielsen
executives say the sample will be far more "projectable" and more representative of the population because it will utilize a better "random probability" method to recruit panelists. The core of that
method is called "random digit dialing," a phone recruiting method that has been used successfully in television for years. In addition, Nielsen said it would augment that with "area probability"
methods, and would also include "cell phone-only" homes not normally accessed via random digital dialing, and also has means for recruiting "more Hispanic homes," and measuring "secondary computers"
within its sample households. In short, the system will be better, more accurate, and more representative than its current system. But it will also be very different, and presumably will yield
dramatically different numbers for the online industry.
"As the size of the Internet has grown, our clients wanted a measurement with more breadth and depth, without sacrificing the quality of
the measurement," a Nielsen spokesman explained.
One other issue Nielsen will need to deal with is its industry accreditation process. Nielsen, which like its closest rival comScore, has been
years into a multi-faceted accreditation process being conducted with media industry ratings watchdog the Media Rating Council, will now have to zero-base that process, and start from scratch.
"They are in the process of accrediting the whole new methodology," the Nielsen spokesman said. "We've had a number of pre-audits that have gone well."
