• by Mark Simon , February 12, 2007

If you're following the blogosphere, you'll know that there's a war of words afoot between my company's president, Dave Pasternack, and the SEO community. Dave's written a series of articles arguing that SEO should be managed in-house -- and that, rather than serving on retainer, SEO firms should serve primarily educational/consultative positions from now on.

That argument has ruffled a lot of feathers. Among other things, it's led to a sea of blog responses, and a $1000 contest for the No. 1 Google organic spot on Pasternack's name. (For a great sum-up of the controversy, see Aaron Goldman's Search Insider piece from last week.)

I'm not going to jump into that argument directly. But I would like to use the new discussion around SEO as an opportunity to introduce an observation. My observation is as follows: As search engines get smarter, SEO firms have a harder time providing real value.

To explain my point, I'd like to start with the business of relevance.

Searchers want relevant results. They'll reward or punish engines according to the relevance they provide. Advertisers, meanwhile, go where the searchers are. And so in order to keep the advertisers, who make the engines money, the engines need to make sure their search results are as relevant as possible.

That's why engines hire armies of Ph.Ds to work exclusively on creating more relevant results. Tbe sole task of those Ph.Ds is figuring out how to make the engines smarter at recognizing previously unrecognized content -- and at ignoring content that looks relevant but really isn't.

And the Ph.Ds are having huge success. Thanks to them, spiders have made leaps forward in reading content on dynamic pages, and even in understanding images. They've also learned to recognize spamming tactics like cloaking and excessive keyword stuffing.

And where the engines can't create relevance through technology, they make up for it through education in SEO best practice. After all, if your site is relevant, they want it to appear in front of the searchers who are looking to find you. And so they offer pages like Google Webmaster Central or Yahoo Search Resources for Webmasters.

The endgame for all of this is a world in which SEO doesn't matter. The engines won't need you to tell them how relevant your page actually is, because they'll understand on their own. For the same reason, they won't listen if you lie to them about a page's true value. Search results may never be unmanipulatable, but they'll be nearly so, to the point that it doesn't make business sense to try.

That doesn't mean that it's curtains for SEO firms. They'll still be in high demand as experts on information architecture and site usability. They'll help clients build sites that their searchers are really looking for, and that their searchers can use. They'll cease to have value in reverse-engineering the engine algorithms, but they'll still have value in helping clients create truly relevant Web sites.

Of course, that role might not be a retainer position. It might just involve a short consultation of a few weeks or months, in which the SEO firm explains to the client how to express itself to the customer in the best way. Meaning that while SEO might not come to an end, the practice of keeping an SEO firm on retainer just might.

Obviously, that era of perfect, unmanipulated search is far away. But as I've already highlighted, the engines have smartened up to a wide array of spamming tactics. And they're getting better all the time at recognizing content that they couldn't see before. Plus, what the engines can't do through better technology, they're working to achieve through better Webmaster education.

In other words, the SEOs certainly haven't lost value, because search still isn't perfect. But we're in an environment in which the margins that SEO firms can win by are getting slimmer all the time, because the engines get smarter and smarter every day -- and clients can learn more on their own than they ever could. The engines haven't killed off SEO, but they're making it a lot less valuable all the time.

How small are those SEO margins now? The answer to that question depends on too many factors to answer with a blanket statement -- but it's a question that every SEO client should consider today.