• by Michael Kubin , April 11, 2001

True story: A number of years ago I worked for NBC, selling time for the television stations division. The head of sales was a man named Bud Hirsch. Bud was a tough, knowledgeable, no-nonsense, take-no-prisoners guy. Bud was built like a fireplug, and he was a genuine hardass. He tolerated no crap from anyone.

In those days there were two television rating services: Nielsen and Arbitron. And while Nielsen was by far the most heavily used, Arbitron did have a respectable share of subscribers among advertising agencies. The ratings reported by each service seldom agreed exactly, but on the other hand they were seldom far apart. So no one really paid much attention to the difference between the two.

Until Chicago. The NBC affiliate, WMAQ-TV, aired The Tonight Show beginning at 10:30 pm. The show was extremely important to NBC as its ratings were always high and thus commanded lofty prices from advertisers. But there was a problem. Both Nielsen and Arbitron had consistently been giving The Tonight Show an 18 rating, month in and month out, for years. Then, for some reason, the Nielsen numbers began to slip. First down to the mid teens, then all way down to 10s and 11s. A disaster as far as WMAQ was concerned, since it meant that ad rates would have to be cut by almost half to those agencies who subscribed to Nielsen. And that was most of them.

After a few months of this, Bud could stand it no longer. He did something never done before: in protest, he canceled WMAQ's subscription to Nielsen. And while this put WMAQ in the precarious position of not having direct access to Nielsen ratings in Chicago, it also put pressure on Nielsen since its profitability in Chicago would be badly hurt.

Several months went by and the standoff continued. But then something odd happened: the Nielsen ratings for The Tonight Show began to rise. And rise. And rise. All the way back to where they had been before the skirmish began. So Bud Hirsch agreed to subscribe to Nielsen again, and the Chicago television community lived happily ever after.

While many skeptics did, I ascribe no cause-and-effect relationship to Bud's cancellation of Nielsen and the concomitant increase in The Tonight Show's ratings. But I do find it interesting.

Credibility and consistency are the hallmarks of media measurement; without them sellers find it difficult to sell since buyers don't know what to believe. And media buyers are paid to be certain; to negotiate on good, solid numbers.

Today's Nielsen Web audience sample of 25,000 and Media Metrix's sample of 52,573 are much larger numbers than those used by Nielsen to measure national television audiences. But -- and this is a VERY important 'but' -- the average household can choose from less than 200 TV channels while the average digital media user can select from thousands.

So, while you don't have to be a weatherman to know which way the wind is blowing, you also don't have to be a statistician to know that small-sample measurement is an inherently unstable way to measure Web audiences. And until marketers can feel comfortable that the numbers are stable, they will look at Web advertising with a very jaundiced eye.

- Michael Kubin is Co-CEO of Leading Web Advertisers - http://www.web-advertisers.com - one of the web's most powerful sources for online ad data. He may be reached at mkubin@web-advertisers.com