Around the Net

Herding Botnets Like Lamb

Have you found that Googlebot has mistakenly moved valuable pages into Google supplemental index or has passed PageRank to pages that do not need to rank? John Britsios provides insight on how to guide bots to avoid these problems.

Britsios discusses when to use and not to use noindex HTML meta tags, nofollow meta tags and noindex robots.txt directives. He explains that Google (unofficially) supports the noindex robots.txt directives, but at the moment it is not supported by other search engines. "Using this directive, you can block or advise Googlebot not to index a page (or de-index a page if previously indexed)," he writes.

Read the whole story at Search Engine Journal »

Next story loading loading..