• by Karl Greenberg , April 1, 2009

How better to launch the third day of the Advertising Research Foundation annual conference in New York on innovation than with a digital scientist who lost the biggest innovation war on the Web -- that of search.

Gary Flake, a Microsoft Technical Fellow who founded and now runs Microsoft's Live Labs, was brought into Overture in 2002 to head up its paid search product. In 2003, the former GoTo.com actually had 55% of paid search, more than Google's share. Now Google has 80% of a category that has quadrupled, and Overture was swallowed by Yahoo, which only has 15% of search.

Said Flake, "WTF? How did I lose so badly?" He lost, he said, because Overture ignored the "long tail" -- focusing solely on premium partners, exact word-search, and hands-on, personal treatment of advertisers, and editing processes. Flake said the lesson -- one that could apply to auto manufacturers as well as media companies -- is that "the first companies in an industry must be willing to destroy their existing business to create something new; they must destroy their own business before someone else does."

Google destroyed Overture's business by becoming the engine of choice for the much-larger long tail of the Web -- not just the premium portals like then-leaders AOL and Yahoo -- and automating the search-results vetting process. Google, he said, benefited by the auto-feedback nature of networks: When your network grows, the growth creates more growth through economies of scale.

"The more participants in a network, the greater the aggregate and participant value." Basically, he explained, Google beat out Overture by understanding that the future of the Web, and therefore online marketing and content, would be driven by the vast "bio-mass" of the crowd, not the few elephants.

He says that going forward, the long tail will only get longer. "As tools -- from desktop publishing and Web publishing, products like Garage Band, and Song Smith, blogging and podcasting -- get more ubiquitous, the pool of participants is only going to get larger and the barriers to entry will drop to zero."

Flake says marketers and media alike should "first embrace the change -- build ecosystems for long-tail participants, for instance. Or use the wisdom of crowd capabilities to tease out collective intelligence in this pool of content." Or, he says, one can do the opposite -- focus on the "head" instead of the tail, but raise the price of entry.

"Hollywood faces similar pressures with independent filmmakers, and the logical reaction for them is to embrace a 'blockbuster' mentality: make big, big movies so barriers to entry are quite steep."

He says the iTunes music store is a Petri dish of where the Web could go in this regard. "ITunes shows both the head and tail: from mainstream artists down to podcasting. But within a year or two, Apple will also be signing unsigned artists," he says, explaining that the "head" sets the product-quality bar for long-tail content, while the long tail puts pricing pressure on the head.

"The analytics on this are remarkable because you can use head content to make predictions about tail content and you can go vertical to vertical," he adds -- saying that marketers can predict how a person's music tastes will relate to his taste in other media.

"This is also the evolution of Amazon, eBay, and YouTube," he says, arguing that those companies have gone from merely being, say, booksellers, to selling their integration of head and tail networks. YouTube, for instance, will soon follow the Hulu model and bring in mainstream content from the likes of Disney World, "in addition to the stupid YouTube videos," he says.