Caffeine indexing system (2010): what changed and what it means now

What Google changed#

On 8 June 2010, Google announced “the completion of a new web indexing system called Caffeine” on its official blog. Caffeine changed how pages get into Google’s index, the store of pages that searches actually run against, rather than how results are ranked.

Google explained the difference from the old system. The old index had several layers, and “the main layer would update every couple of weeks.” With Caffeine, Google wrote, “we analyze the web in small portions and update our search index on a continuous basis, globally.” New pages and changes to existing pages could be added to the index as they were found.

When it rolled out#

Google opened a web developer preview of what it called a next-generation architecture for web search on 10 August 2009, asking for feedback on differences in results. It announced the system complete on 8 June 2010, which is the date used here for both start and completion. Google did not publish a separate date for when the live rollout began.

What is known about its impact#

Google’s own figure: Caffeine “provides 50 percent fresher results for web searches than our last index.” Google also said it was the largest collection of web content it had offered. It published no figures on ranking changes, because Caffeine was not designed to change rankings.

Caffeine mattered later. When Google launched its freshness improvement in November 2011, it wrote that it was “building upon the momentum from Caffeine.”

What site owners should do#

Caffeine is not in Google’s ranking systems guide, because it was never a ranking system. Its lasting effect is that new and updated pages can reach the index quickly. Google’s current guide to how Search works says the vast majority of pages in results are found automatically by its crawlers, so the practical steps are about being easy to crawl:

• Link new pages from existing pages so crawlers can find them.
• Keep a sitemap current so Google learns about new and changed URLs.
• Update pages when the facts change, rather than only changing their dates.

How to check whether it affected you#

This is historical. Search Console keeps 16 months of data, so 2010 can only be checked in archived analytics or server logs, for example by looking at how quickly new pages started receiving search visits. The Google Algorithm Updates timeline lists every update with its dates.

Related updates#

• The Mayday update was a separate ranking change weeks earlier, which Google said was unrelated.
• The Freshness update built on Caffeine in November 2011.