How much control do consumers need, anyway?

That's the most profound question that strikes me the more I hear about Yahoo's latest "open search" initiative, described in some detail on the Yahoo Search Blog. So far, it's just the teaser release, the equivalent of those "Cloverfield" trailers where you see everyone running for their lives but the monster's totally out of sight. The story's gradually coming together, though some versions of it differ.

From all versions, the end result is clear, thanks to screenshots on Yahoo's blog and elsewhere. Yahoo writes, "Instead of a simple title, abstract and URL... users will see rich results that incorporate the massive amount of data buried in websites -- ratings and reviews, images, deep links, and all kinds of other useful data -- directly on the Yahoo! Search results page." The search results should in turn become more useful, so consumers may spend a little more time on the search engine results page to scan the additional content, but they'll spend less time clicking on search results that don't match what they're looking for.

The end result, while looking impressive, is a tacit defeatist declaration from the search engines, as it implies they can't figure out how to provide more enhanced search listings on their own. They're basically crowdsourcing search result descriptions; the technology, or the engines' internal staff, can't do it alone. And this isn't just about Yahoo. Google's so invested in using manpower to improve its search results that many of its emerging products use humans in some way, including Google Base, Co-Op, and Knol; it even created a game out of people tagging images to improve its results.

Consider Google Co-Op's Subscribed Links component, where users subscribe to their favorite publishers, and when those publishers have content relevant to a search from a subscribed user, a featured listing will appear on the page. For example, I subscribed to OpenTable.com's link. When I ran a search on the restaurant "Le Bernardin," the subscribed link to OpenTable appears as the fourth natural search listing, following two results for the restaurant's official site and one for New York magazine.

Fairly successful Subscribed Link publishers like Open Table and Search Engine Watch have a few thousand subscribers. Digg, a runaway success, has over 20,000. I'm amazed that many people can figure out how to subscribe, as it's practically hidden in Google. The directory has all of 48 links, hardly a vast menu of subscription options.

There are several obstacles preventing Co-Op from working. First, it requires users to take concerted actions to improve their search results, and people generally don't like working that hard, especially for something so functional. Second, users will most likely subscribe to publishers they already visit, providing the publisher with only a marginal benefit; less popular sites that could benefit more will have a hard time attracting subscribers in the first place. Third, people expect the search results to be as relevant as possible, so if a site they like is relevant to a search, it's Google's job to make it rank well.

How does this relate to Yahoo's plans for opening its search results? Google's Subscribed Links offer one way for users to take control over their search results thanks to publishers' contributions, which is conceptually similar to Yahoo's open search. Some of the best coverage of Yahoo's news came from Search Engine Watch Managing Editor Kevin Newcomb. He wrote how users will see the improved search listings when they "enable a Yahoo Search plug-in created by a publisher," though such a plug-in was barely mentioned in other outlets' coverage.

He later told me what else he heard from Amit Kumar, director of product management for Yahoo Search. Newcomb said, "From what Amit told me, publishers will have to create a plug-in that users won't have to install as much as enable. He compared it to Facebook apps, where you turn them on or off... Yahoo will be collecting plug-ins in a 'publishers gallery' to make it easier for users to find and enable them (similar to Firefox or Wordpress plug-ins), and publishers can offer them on their own sites
too. Yahoo will be automatically enabling some of the more popular plug-ins for all Yahoo visitors, but he didn't say which ones or how that would be decided."

My main concern centers on how much control users have over their search experience. Like every other pundit, I love spouting off on how consumers are taking more control over their media experience, but consumers don't need to control every part of it. Consumers can let others decide when the new Indiana Jones movie hits the theaters, whether "Quarterlife" airs on MySpace or NBC, or how natural search results rank. Media and technology companies can exert their control where it helps.

The question over Yahoo's open search is whether consumers will get too much control, and if that will be more than they need. Just look at the before and after pictures on Yahoo's blog with the Yelp example. Who wouldn't want the enhanced edition? Presumably, some publishers could abuse this platform, but there are sites like spam blogs that currently try to abuse natural rankings, and in either situation it's up to the engines to ensure these sites don't rank well, if at all.

It's Yahoo's job to improve its users search experience. If this helps users, and it clearly should, then it should be part of the natural search results. If it doesn't generally help users, Yahoo should be the arbiter. If Yahoo's too selective with enabling some publishers but not others, then it may improve some search results for some users while at the same time calling into question how fairly it ranks all results for everyone. Advertisers could also get enraged if Yahoo automatically displays open search results for their competitors but not for them.

That puts Yahoo in an unenviable position. Exert too much control and various constituents raise hell. Leave all the control to consumers and they probably won't exercise it. Yahoo can still strike the right balance and run circles around Google Co-Op in the process. For the chance to outmaneuver Google, the reward's worth the risk.