Google Algorithm Testing: How Big G Knows What Works

For anyone who keeps tabs on the latest developments in search engine optimization, it is common knowledge that Google Algorithm updates happen on a regular basis. When I say regularly, I’m not talking about once per month. They sometimes will roll out multiple updates in a day – for example, take 2012, when they launched a total of 665 updates in one year!

If you click through the link in the previous paragraph, you will also see that Google’s team rolled out over 7,000 live tests that same year. In other words, they test more than 10X the number of actual updates to determine which work best. And all of this is algorithmic, meaning it excludes manual reviews and penalties.

Google Algorithm Testing: What Goes Into It

Given this affinity for testing and updating, Google is obviously hard at work trying to improve the SERPs. When you add in the range of different platforms (i.e. mobile, tablets, etc.), locations, types of results (organic vs. local vs. knowledge graph), images, languages, freshness, and all the other variables they must parse to deliver quality results, it’s amazing that they have time to sleep .

Matt Cutts recently posted the below video to the Google Webmasters channel on YouTube on this very topic. It was created in response to the following question:

What are some of the metrics that Google uses to evaluate whether one iteration of the ranking algorithm is delivering better quality results to users than another?   – from James Foster, Sydney, Australia

This is a great question, and one that gives us a chance to get more insight into how Google lands on the changes they push out. Matt’s overview of how the Google algorithm is tested both in a staging environment and live is very enlightening.

For example, did you know that Google has a team of search quality raters who are heavily involved in this process? They serve a key role in evaluating whether the live Google algorithm is generating quality results on an ongoing basis. Additionally, and even more importantly, they are monitoring and rating the potential impact of unreleased tests to determine which should be rolled out for a live test with real searches.

On the live testing side, they sometimes weave in the results from the update with results from the previously existing algorithm, and track where the clicks go. But there is both a scientific and a qualitative piece to evaluating success, as sometimes spam results naturally draw clicks due to the aggressive language used in the page title and meta description.

Watch the video yourself to get an idea of a “day in the life” of a search algorithm update. For additional information, here are some related links from Google:

Search evaluation at Google (series of blog posts they released a few years ago – still relevant according to Google Webmasters on Google+)

How Search Works: Algorithms (from Google’s “Inside Search” training section)