• by Kaila Colbin , Featured Contributor, May 26, 2009

"It is a slightly arresting notion that if you were to pick yourself apart with tweezers, one atom at a time, you would produce a mound of fine atomic dust, none of which had ever been alive but all of which had once been you." -- Bill Bryson

For those of us with only mortal brainpower, Bill Bryson's vivid imagery renders the concept of complex systems accessible. Complex systems are those that are greater than the sum of their parts; they have properties that cannot be explained through reductionism; they are economies and hurricanes and you and me.

Search is a complex system.

Search is a system that incorporates our behavior, our feedback, and our propensity to swarm. We are awash in a sea of practically infinite data, co-created through our news pages and our blogs and our Web sites and our Flickr accounts, and the infinitude of this data forces us to rely on every tool in our arsenal to get the good stuff to reveal itself. And, like cream rising to the top, a small fraction of the global information set makes its way into our consciousness.

We expect that Google and Yahoo and Kumo will do their job and find that cream for us. We hope that the bits that make it past our filtration are worthy of our attention. And we keep our fingers crossed that the engines are winning the good fight against the spammers and scammers and black hats.

Those engines have ridiculously complicated and absolutely necessary algorithms to determine relevance. When it comes down to it, though, search is a function of four immutable principles:

1. Show up. As Twitter is rapidly learning, and as Jimmy Wales already learned, to be effective in search you can't just sit back and let the crowd do the work. You have to bring something to the party.

   Yes, Twitter holds the largest database of real-time conversations in history (I assume). But Evan and Biz are still going to have to work out how to separate the wheat from the chaff. TechCrunch demonstrated this with its "Real-Time Search-Off," in which the number one result for the query "space shuttle" was "Just watched the space shuttle go overhead -- so cool."

   Find me the links, Biz. Find me the relevance. Find me the cream.

2. Pay attention. Why is real-time so exciting, and why do results from a day ago seem so last year? For the same reason that when someone asks how you are you don't generally tell them how you were yesterday. My question is now, and, if you're paying attention, you'll give me a result from now. While you're at it, you'll also disambiguate my query and solve my quadratic equations.

   Sure, Google's still raking in the billions and Twitter's still essentially penniless. But that doesn't mean the big G ain't anxious.

3. Tell the truth. If there is one absolutely fundamental requirement for search, it's that it must be trustworthy. We may not know all the ingredients in Google's secret sauce, but we trust it to be applied fairly and democratically. We trust our search engines to work hard and honestly so that our own honest hard work is rewarded above that of the losers and cheats trying to game the system.

4. Don't be attached to outcomes. Of course search engines want you to click on the paid result, every time. But if they didn't give you an honest choice, they wouldn't have any clicks at all. At the end of the day, search has to be about the searcher. Show me your evidence search results, and let me decide what I want.

We are each of us a complex system, and we live in a complex world. But even complex systems have their underlying principles. Do you agree with these?


Editor's Note: Unfortunately, Aaron Goldman's Search Insider column was emailed on Wednesday, May 20, with its ending cut off. To get the full impact of his theory -- "Is Twitter To Facebook As Google Is To Yahoo?" -- check out the complete version here.