Around the Net

Google Gets Polite

A Google patent granted Tuesday describes how the search engine might schedule the crawling of Web sites "so it doesn't bring servers to their knees," according to Bill Slawski. He tells us that Google might use crawlers focused on finding different content like images, news and sports. These different bots might all decide to visit the same Web site simultaneously, which would take all the resources. He explains that the concept of scheduling visits by crawlers so it doesn't starve the load capacity of a server is known as a politeness protocol.

Read the whole story at SEO by the Sea »

Next story loading loading..