• by Mark Simon , July 30, 2007

Fellow Search Insider Gord Hotchkiss tossed out a fantastic challenge for the SEM industry last week. As Gord sees it, the search engines are innovating rapidly, while the SEMs are failing to innovate. It's time, Gord argued, for SEM innovation to catch up.

Although I found the piece wonderfully thought-provoking, I have to disagree with it for two reasons. First, SEMs are innovating, and they're innovating at lightning speed -- a phenomenon I've had the pleasure to witness firsthand. Second, it's impossible to compare engine innovation with SEM innovation, because both sides are too secretive about their innovations for us to honestly compare them.

I'll elaborate on both those points in what follows.

The engines. To frame this conversation, we first need to differentiate between the two separate sides of the engines' business: search results, and advertising. When we talk about search innovation, most of what we're talking about deals with search results. We talk about advertising innovation much less.

Gord's article is a perfect example. Gord rattles off a list of innovations including "personalization, universal results, Web 2.0 functionality [and] mobile," and concludes that "search experience is about to change drastically." Gord doesn't discuss innovations on the advertising front. Instead, he focuses on the ways the engines are innovating to give users a better search experience.

The focus on search experience is not a coincidence. The engines work hard to tout benefits to users, because a better user experience draws more searchers. When it comes to advertising systems, the strategy is nearly the opposite: the engines shroud advertising innovations in a black box of mystery as dark as Google's Quality Score. When we talk about search, we tend to focus on search experience because the engines frame the discussion that way.

Take Google. At the time of this writing, Google's Labs page features roughly 30 new and developing Google products, the majority of which are search products. Meanwhile, Google's "Inside AdWords" blog features only 10 un-archived entries, only three of which are search-specific, and one of which is a 4th of July message. It's clear which type of innovation Google is looking to emphasize.

To be sure, we do hear of high-profile developments on the advertiser side of the engines. But much of that news is about advertising ventures beyond search -- like news of Google's ever-expanding ad empire -- or catch-ups, like Yahoo's Panama launch. Much of the engines' true innovation in search advertising stays under-the-radar.

The engines will say that obscurity keeps advertisers and Web sites searcher-friendly: the less marketers know about the engines' systems, the less time they'll spend gaming those systems -- and the more energy they'll spend keeping their ads and Web sites relevant. Cynics will say that the obscurity simply hides the engines' charging practices. Either way, it's clear that while the engines continually change ranking and bid algorithms, advertiser tools, and even new targeting and payment models, the engines don't place public understanding of these changes as their top priority.

Of course, the largest search players -- and especially the major SEMs -- are highly knowledgeable of the engines' advertising innovations, because those innovations are often designed with the help of those same larger players. But when we think of search engines as a whole, we tend to think of their public persona -- which is their user side, and not their advertiser side. And the user side isn't a fair point from which to compare engines to SEMs -- because the SEMs don't work on the user side of the search business. SEMS work on the advertiser side of search.

What the SEMs won't say. As I said above, I've witnessed firsthand the SEM industry's developments in new technology, click fraud prevention, strategy, and analytics -- plus the tremendous leaps that SEMs have provided in targeting, letting clients pinpoint the right customer, at the right moment, in ways that the engines can't.

But at the same time, we SEMs go to incredible lengths to keep innovations under tight wraps. It's easy to explain why: whatever we share publicly will be overheard by our competitors.

I think that Gord (and the analyst he cited) may have looked at this corporate secrecy, plus the second-rate firms that reflect poorly on what is otherwise the smartest sector in advertising, and concluded that SEM is at a slowdown.

But in reality, SEMs are innovating at an incredible rate -- if only because we have no other choice. Industry experts forecast search marketing to grow 29% this year alone --and SEM's astounding growth rate will bring severe competition and higher keyword prices with it. The only agencies left standing in the coming years will be those whose efficiencies can move clients past the competition. Creating those efficiencies will only come from rapid innovation, which is why the top SEMs see rapid innovation as our lifeblood.

And so, while Gord's piece was thoughtful, thought-provoking, and welcome, I'd have to say that he was out of his gord on this one.

Of course, none of us in the industry would want it any other way.