Search Engine Learns From People Interaction

No, I'm not referring to Google's personalized search technology. Rather,  trying to better understand what people seek on search engines has two Cornell University professors working on an alternative.

Research from Thorsten Joachims and Robert Kleinberg, computer science associate and assistant professors, respectively,  aim to create search engine software that considers what links users click, and how they reformulate queries when results don't pay off.

The work, funded by a four-year, $1 million grant from the National Science Foundation under federal stimulus funding, should lead to methods that improve search quality without human guidance. That's one of the goals, anyway.

Search engines serve up query data based on rankings. But what if you could change the ranking function based on the person running the query? The goal is to run small interactive experiments while people search, collecting information on clicks and queries, and change the ranking of results.

The technology will allow search engines to automatically tune and tweak themselves by learning how people interact with them. Engines will learn that certain tags and keywords hold different weights.

If you just observe how people search for information you get bias data. Running interactive experiments gives search engines more reliable data -- as, for example, having a search engine periodically presents a query result at the top of the page that would typically run at the bottom.

The research focuses on intranet and site search, but could adapt to broader Web searches on sites like Google, Microsoft Bing and Yahoo. The researchers plan to create a search engine for the physics Web site at Cornell that contains thousands of papers in physics, mathematics and computer science as one test bed.

There's no denying that people can get better query results from personalized search, but what if you could change the rankings based on the domain that the person originated from on the Web? For example, you could have a slightly different function for people who originated from Cornell.edu vs. AOL.com. Google couldn't do the tuning manually. There are just too many domains across the Internet.

Google might use some techniques for personalized search that the two researchers have been testing, but Joachims says the Mountain View, Calif., search engine has declined to disclose which ones.

Privacy concerns were one reason the two researchers chose not to focus on Web personalization. "You would need methods that guard against privacy violations, so we are looking at intranet searches where there are less privacy issues to develop the technology and later augment them with privacy-preserving techniques," Joachims says.

The portion of the research that excites Joachims the most actively explores what people are really interested in, rather than what they do. Search engines typically present the best guess, the information it thinks the person wants to see. But what if you present a slightly different ranking, he says. It may mean taking a ranking at No. 11 and putting it in position No. 4.