Every wonder how or why Google would drop out content from search results for a few hours, after it had been there for at least a day? I have. I saw it happen. It made me think I was going crazy.
Thank you, Ben Hendrickson, for providing some sort of explanation in this post.
He said Google likely works with two indexes. One hypothesis is that Google removes a page from the small index only after it is in the big index. Then both indices would have the content for a while until the small index was rebuilt. This overlap means the small index is larger than necessary. Perhaps Google tries to time it perfectly to eliminate gap or overlap. Hendrickson believes that problems occur as indexes crawl faster, grow their indices, add complexity to indexing, or let the intern check in his summer project. You'll have to read the rest for the details.