What Can People Teach A Web Crawler?

Search engine Web crawlers can learn from people perusing sites, according to Bill Slawski, who points to Yahoo patent application No. 20090019354 to demonstrate and detail his point. The patent abstract reads: "A user's browsing activities on a Web site are captured. The user's browsing activities includes affixing labels to Web pages and filling out forms. The captured activities are analyzed for patterns."

Slawski provides insight on how user/site interaction could assist webmasters, manual reviewers and content subscribers. He also defines two types of Web crawling: free and focused. He describes approaches for each, pointing to problems with focused crawling, and why and when a search engine might use one approach over the other.

Read the whole story at SEO by the Sea »
Tags: search
Recommend (2)