How many search engines do we really need?

According to the metasearch engine GoshMe, there are more than 500,000 search engines. That’s more than one for every resident of Albuquerque, New Mexico. I dare you to search them all. If anyone will accomplish the task, it’s Charles Knight, a search engine optimizer who has made a name for himself publishing monthly lists of the Top 100 Alternative Search Engines, such as this lengthy piece on Read/WriteWeb.

I’ve attempted a number of grueling feats in my day. In college, I won a challenge to see who could eat the most Deadly Chocolate Sins, a rich, fudgy, warm brownie served at Applebee’s, and I subsequently learned that along with a sugar high, there’s also such thing as a sugar hangover. I am also one of few men who will admit to having endured watching nearly every episode of “The Real Housewives of the O.C.” (the things men do for love). The weekend I spent sorting through all of the Top 100 search engines wasn’t quite so demanding as brownie-eating or “Housewives”-watching, but it was up there.

With all these search engines, and I have no doubt that the 100 Mr. Knight compiled were truly among the best, I was mining them to explore where the real innovation lies. What aspects of all these engines will improve the search experience for users over the years ahead? Even if none of these are the next Google, Yahoo, or Windows Live Search, are there diamonds in the rough that can be polished and adapted into the major engines’ algorithms and results pages?

For the most part, the answer is no.

The engines on the Top 100 list can be segmented into a handful of categories, and those categories can be further divided based on which ones will have a low impact on innovation, and which ones will matter most the rest of the decade. This week, we’ll look at the low-impact categories, and then next week we’ll see which categories are more promising.

Low-Impact Engines

• Clustering/graphic display: These engines organize search results in some sort of visual field. Quintura’s among the best of these, and it’s potentially useful for academics and brand managers, but I don’t get the benefit for general consumers. Gnod clusters results based on specific subjects such as authors, as in this example for a search on Philip Roth, yet Amazon’s recommendations are usually more than sufficient (as an aside, my regrets to the good Dr. Oliver Sacks for appearing on the literary map of Danielle Steele).
• Filtering based on categories/recommended keywords: This is one feature especially common in vertical search, but it’s also being used by other engines such as Factbites. If that’s the predominant feature, it’s not going to be incredibly useful, as it’s already being used by other engines, notably Ask.com and Windows Live Search.
• Metasearch/aggregated search: These engines search multiple sites at once or individually. Dogpile, Mamma, and Goshme all are variations on the metasearch theme, while engines like FindForward allow more features for searching select sites one by one. Even if these engines are useful at times, Dogpile and its ilk are icons of the Web’s past, not its future.
• User-ratings/voting: VMGO lets users rank search results. I’m skeptical of the longevity of this approach, as it’s too easily gamed and too biased toward early adopters. If an algorithm’s that good for natural rankings, voting won’t matter, though the whole idea of a Digg-based search engine might gain some fleeting buzz.
• Q&A: These engines, like Lexxe, aim to give you direct answers to your questions. For the post part, the innovation here has already happened, as Yahoo Answers emerged as one of the company’s biggest success stories in recent years while Google Answers folded. One of my favorite entrants in the Top 100, Ask Vox, falls into the Q&A category. Built on the Yahoo Answers API, Vox is a talking avatar who answers your questions, and you can add in your own answers when she falls short (see this vanity search as an example). For added fun, Vox says on her MySpace page that she’s going out with the retired Ask.com butler Jeeves. If you ask her directly if she’s in a relationship, she’ll confirm the tryst, though the two-timer also says she’s single if you press her.

Even though these categories are low-impact, some of these engines are innovative in their own way. Quintura keeps evolving and grows more useful with each iteration, Goshme is awe-inspiring with its breadth, and Vox was so much fun, I shared her with every visitor to my office last week.

But enough playing around. Next week, we’ll look to the engines and categories that will fuel the future of search innovation.