What Search Engines Crawl & Index
Rand Fishkin explains that there's a "large contingent of folks doing SEO who are a little confused" about how search engines crawl and index content. It's not just a simple in and out process, he explains.
Search engines crawl the Web and choose pages to index based on a variety of metrics. An illustration provides insight into the different levels. Fishkin explains that the information can help educate "SEO outsiders" on why pages may not draw traffic, though metrics appearing in XML sitemaps show in Google Blogsearch queries.