Taking Measure of Measurement

Measurement is a blessing and a curse on the Internet. It's wonderful to have so many facts at our fingertips, but vexing when we don't know which ones are the most accurate.

The Internet is perhaps the most accountable ad medium there is. But the downside is there are often too many numbers to wade through - total spend estimate, return on investment, number of online views, unique visitors, ad growth and so on. Separating the wheat from the chaff can be tough. How does a media buyer, advertising executive or venture capitalist decide which reports to use? More specifically, how do they choose which projections to lean on for the online video ad market?

Because that's the segment that's causing the most confusion when it comes to numbers. We know that Web video is probably the fastest growing ad medium, but the estimates on the size of the market vary widely. Lehman Brothers said advertisers would spend $1.1 billion on online video ads this year (and look where that got them), Forrester Research pegs the year-end total at $989 million, while eMarketer predicts the dollars will come in at $505 million.

But that eMarketer figure is a recently revised estimate, down from an original prediction of $1.4 billion; it cites a change in methodology. David Hallerman, senior analyst, explains that eMarketer sliced its estimate by two-thirds after the Internet Advertising Bureau said this summer that online video ads had pulled in only $324 million in 2007. The IAB measures "digital video commercials" - those that appear before, during or after a variety of content - but not brand integration. "We believe we are capturing the biggest part of a growing market," says David Doty, senior vice president for thought leadership and marketing at the IAB.

That's also the issue with the eMarketer figure. Neither its initial nor its revised projection accounts for money flowing into brand integration, product placement and host mentions. Those ad formats drive a significant number of buys in the medium.

"The share of the money going to product placement is a bit harder to gauge," Hallerman says. He points out that most TV ad spending numbers don't take product placement into account, either. eMarketer measures pre-rolls and other ads that look like TV ads online. Hallerman says product placement should be counted, perhaps as a separate category. "The medium may be more measurable, but ad spending isn't necessarily," he adds.

The uncertainty surrounding market estimates touches nearly all parties with a stake in the Web video business, including media agencies that use macroeconomic figures when evangelizing the medium and venture capitalists who search out growing markets.

Omitting a robust and growing portion of video ad spending suggests the size of the Web video economy is being underestimated, described only in terms of the ad dollars flowing into high-profile Web shows such as nbc-backed "Gemini Division," EQAL-owned "LG15: The Resistance" and Revision3's "Diggnation." That's a problem, because they generate most of their ad revenue from product placement and host shout-outs, as do many other Web studios, including NextNewNetworks, ManiaTV and ForYourImagination.

"The vast majority of revenue we derive for our shows are from brand integration," says Greg Goodfried, one of the executive producers of "LonelyGirl15" and its spinoffs; he's inked deals with MSN, Disney, Paramount and Procter & Gamble.

Forrester Research analyst James McQuivey says his company's survey methods leave room for interpretation by agency respondents. To estimate the online video ad economy, Forrester asks advertisers if online video ad spending is trending up or down. That can be a complicated question to answer, because a large chunk of online video ad spending comes from multiplatform buys in the broadcast upfront.

"We have found it difficult to disentangle what should be the online video ad spend when a Nissan sponsors Heroes, for instance," he says. "There is a lot of room for error."

Still, McQuivey expects his original $989 million estimate to be close to the final tally at year's end. While YouTube has faced difficulties in monetizing its ads, entities such as Hulu have grown faster than expected, balancing things out, he says.

Research, by its very nature, is imprecise. Bill Tancer, the global head of research for online audience measurement firm Hitwise and author of the book Click, urges everyone to be skeptical when evaluating studies. "Open up the chart, look at the source, ask what the methodology is."

He adds, "Most research companies will ask agencies, advertisers and publishers, and all of them are incentivized to make those numbers look larger, because the larger they are, the more people will spend. That's the problem with any type of study that pegs ad dollars. It's fortune-telling."

The situation is further complicated. What is known about product placement deals is that Web TV networks such as Revision3 and ForYourImagination say they charge $60 to $80 on a cost-per-thousand basis for such buys.

So where does that leave advertisers and programmers who rely on market research? If estimates don't include product placement, are advertisers getting the full story?

Michael Hayes, executive vice president of interactive at Initiative North America, says he would like to see market estimates that include all formats, including brand integration. "We would love for it to be broken out by category," he says. Forrester's McQuivey says he's likely to break out spending by format next year. But eMarketer's Hallerman says he does not have plans to do so at the moment.

But the truth is Hayes and others say they aren't terribly troubled by the fluctuating estimates. That big picture is simply that online video ad spending is increasing this year, whether it's by 56 percent, according to eMarketer, or by more than 100 percent, per Forrester.

Besides, Web-content creators and distributors also say they don't rely on market estimates to sell their inventory to marketers. Instead, Revision3 says 100 percent of its viewers can recall at least one advertiser; NextNewNetworks talks up its 20 million monthly views; and online video health information network HealthiNation sells ads based on engagement with its content.

So does market research matter anymore?