Yesterday at South by Southwest, Danny Sullivan held a Q&A session with the search engines, with Matt Cutts representing Google and Duane Forrester representing Bing. While I followed the session on Twitter, one comment especially caught my eye: there seemed to be some general consensus that rankings are no longer a valuable metric in SEO reporting. So is it time to abandon rankings as an SEO success metric?
The Case Against Including Rankings as a Marketing Metric
The main reason that members of the panel gave for discouraging using rankings as a measurement of success is that rankings shift so greatly from searcher to searcher because of a variety of personalization factors, including location, social circle, and other factors. Forrester even compared ranking measurement to a "sugar pill," emphasizing that a far better measurement of success is conversion.
The Problem with Measuring Conversion in SEO
While I agree with Forrester that conversion should be the main metric of really any type of marketing effort, measuring conversion from SEO isn't always a measurement of whether or not your efforts drove targeted traffic to your website. As with pay-per-click advertising, you can optimize for the most targeted keyword terms and drive traffic, but if the page the searcher lands on doesn't convey your message effectively, searchers will abandon the site. That's why SEO practitioners may not want to be held accountable for conversion, as SEOs are often not themselves in control of the messaging or design of the website.
Enter Conversion Rate Optimization
Conversion rate optimization (CRO) should be applied to more than just landing pages for pay-per-click campaigns. Why not use A|B testing tools like Google Website Optimizer (GWO) on your website's pages to find out what changes might improve organic conversion? Unfortunately, not many marketers are embracing this philosophy yet. In fact, last week on a panel at Pubcon South, I was asked if GWO could be used on website pages as well as landing pages. I commented that I'm still surprised by how many website designers and agencies don't use tools like GWO or Adobe Test and Target to measure success before scrapping an entire website, essentially "throwing the baby out with the bath water."
So should you focus on conversion measurement? Absolutely. But conversions are the measurement of many factors, not least of which are the methods that drive targeted traffic to the website -- including, but not limited to, SEO.
The Case In Favor of Including Rankings as a Marketing Metric
Or should you measure rankings, too? Rankings serve as a benchmark, as do other valuable metrics, like search traffic. And while you can't always account for all of the personalization variables, you can get a good idea at least of how your rankings are improving by using benchmarks.
But how can you get the cleanest, least personalized rankings measurements? There are two ways I recommend: Advanced Web Ranking and Google Webmaster Tools' Search Queries.
Advanced Web Ranking and Proxies
Advanced Web Ranking is a rankings measurement tool that allows you to collect "clean" rankings using proxy servers -- spreading the ranking requests over several servers at once, which speeds up reports and provides more accurate ranking results.
Google Webmaster Tools' Search Queries
A few weeks ago, I covered some of the benefits of using Google Webmaster Tools (GWT) -- one of my favorites. The Search Queries report shows the top keyword queries in Google that your website ranked for in the past 30 days. The report also shows valuable information such as impressions, average ranking, and click-through rate. However, like AWR, which is essentially a snapshot in time of the rankings at the moment you run the report, the Search Queries report in GWT shows the information spanning the last 30 days and does not archive data prior to that time.
Which Way to Go
So, in the end, should you include rankings or not? I do. But rankings can't be your only measurement; they are one of many metrics search marketers need to consider when determining search success. While I agree with Forrester and others about the shortcomings of relying solely on rankings for SEO success, rankings are a baseline to work from, and if you use the right tools, you can get great insight about your movement.
What should you measure? I recommend using a combination of rankings and analytics data (including ecommerce and/or goal data). But whatever you do, make sure your measurements clearly gauge the area you're trying to improve.