• by Douglas Bentz , June 4, 2001

"Kobe and Shaq, buddies again, roll through the playoffs." These days most front page sports sections carry stories with headlines like this one. Since the playoffs have heated up the hype around the drastic NBA rule changes has since died. The same thing happened in the online ad industry. Just a month ago you couldn't go a day without reading about the IAB's new ad units. Everyone had an opinion, some thought the new units would save the industry and other believed the intrusiveness would drive users further away from responding to the advertising. At the same time, sports columnists across the country waged battle over the outcome of the new NBA rules. So who's right on both of these fronts? And how will we know?

That's where the problem lies, how to determine if these changes actually work. It's a problem because there are different perspectives in analyzing the results. Take the NBA rule changes, there are two major perspectives: the league and the consumer. The league installed new rules in order to boost sagging viewership with increased revenues being the motivating factor. The consumer watches or attends NBA basketball games for the entertainment value. Assuming these motives are true, from 30,000 feet the NBA's logic is solid: alter a few rules that will ideally make the game more exciting which will increase the entertainment value of an NBA game which will put more butts in the seats. Is that a little too simple though? Does that logic take other factors into consideration?

It doesn't seem like it. Personally, I have been priced out of most NBA arenas and I don't think I'm unique in that respect. Also, during the NBA season, there is a "Game of the Week" on everyday. The Warriors vs. Hawks game on a Wednesday night is not particularly enticing to most basketball fans. And what if Jordan comes back next season? If he does, revenues will most likely skyrocket throughout the league. So will those increases be attributed to the rule changes?

Questions like these arise when I read about the new IAB ad units. "Study shows SkyScraper ad units yield 15% CTR!" Wow. But wait, what kind of advertiser was it? Was it the placement? Creative? Call to action? I need more info! I realize that most of this information is coming from publishers who are trying to move their wares and that in a short time frame CTR is the easiest stat to publish. Given that, these studies are a good preliminary measure of the potential of the ad units, but not enough to deem them successful from the media perspective. A 15% CTR could be for an ad that guaranteed free money or one that is uniquely targeted.

If we are to truly understand the relative value of the new ad units, we need studies that take multiple factors, such as placement, creative execution, size, price, etc., into consideration. These studies must also report out back-end factors applicable to client success (CPW) such as cost per sale, cost per action, cost per registration, cost per pageview or whatever the client needs to succeed. Not to mention metrics for brand impact pre and post-campaign. And these studies must be undertaken over time not just a single occasion. They also should not cater to only one perspective, whether it be publisher, media buyer, or user. A possible way to proceed is to create an independent organization made up of representatives from the media, agency and advertising constituency. It may mean releasing some individual data for the good of the industry. A data bank that would allow further study and analysis is a good thing. At least it would provide a starting point and may be a benchmark for increasing the value of the Internet for all interested parties. It may also mean we have to see some bad news first (but aren't we used to that by now). It may be difficult but it would lead to better online advertising decisions, which could benefit both buyers and sellers and it may even translate into more respect for the medium by advertisers.

I don't mean to oversimplify the issues at hand, because neither the results of the NBA rule changes nor analyzing the success of online advertising units can be explained in a article like this. But that is the point. Judging the success of these changes is a complex process and it needs a more thorough analysis by all parties involved.

- Doug Bentz is a Media Strategist at Mediasmith, Inc. an integrated Interactive Media planning and buying company.