Internet marketers like to pride themselves on working on "Internet time," where the gap (often traditionally a chasm) between idea and execution is radically closed. Though that pride is well earned in many areas, one realm of Internet marketing which has lagged behind Internet time is multivariate testing, as Eric Hansen, president of SiteSpect, notes below.
Behavioral Insider: How and why did SiteSpect choose to focus in such a singular manner on multivariate testing ?
Eric Hansen: SiteSpect developed in response to the desire of publishers, and particularly e-commerce sites, to run controlled experiments to guide them in optimizing their sites. From that perspective we created what we called non-intrusive optimization.
BI: Could you explain the term non-intrusive in this context?
Hansen: By non-intrusive I mean that publishers can run tests without needing to change their sites. Traditionally, running a test has been a slow, arduous process that took a lot of elbow grease and weeks, even months, to execute. You had to literally go to every page on a site and put in tags. That severely limits the flexibility of being able to get a test of an idea into the field immediately. What marketers wanted to do was to get a test up and running in a matter of hours.
BI: How does your approach differ from what others in the analytics space are doing as far as testing?
Hansen: What we do instead of tagging pages is provide a box, a device that resides on the network and works like a traffic router. As a browser requests a page, SiteSpect routes that request. The device not only routes the request, however, it also discerns who is making the request. So what happens is, as the Web server gets the request and serves up the page, the page is served back through the router. If a publisher wants to test different content for certain types of consumers, for instance, SiteSpect will, in the process of serving up that page, change its content on the fly based on who the requester is. It will shoot the altered page content back to the end user all in less than a hundredth of a second. SiteSpect will track user behavior and, based on that behavior, manipulate the content that ultimately gets served on a consumer-to-consumer basis.
BI: How does your testing platform leverage and integrate behavioral data?
Hansen: Given that flexibility, marketers can then test content offerings, messages, page creative, what have you, for very discrete segments of people defined by behavior. A publisher can say, ‘let's set up a series of test campaigns to optimize content around users of specific search terms.'
Say you're a retailer. You can optimize for people who've come to your site after using the search term ‘jewelry.' Beyond that, you can target people who've searched for jewelry but who have never visited the site before, and see how a given content offering tests with them, against people who've come to the site from a ‘jewelry' keyword search and have shopped at the site before. The test can be set up and run literally while marketers are holding a meeting to discuss strategy and tactics.
BI: Determining scale and figuring out how to interpret reporting are perceived as hurdles by many online marketers when they think about doing testing. How are you addressing that issue?
Hansen: The short, one-word answer is math. Although we've built SiteSpect to be a self-service tool, we also offer a full range of professional services to ensure successful, ongoing optimization for our clients. In using the tool, the assumption is that you're not dealing with a marketer who understands the mathematics of statistics in depth.
Instead, a test wizard walks the marketer through the process. The wizard prompts users to say how many pages they want to test and how many different offers, and then calculates an estimate of how large a sample is required to get results. As the test is running, the system can tell you how results are trending. It can also predict if the test you're running is unlikely to yield any conclusive results.
BI: What are the primary benefits of doing extensive multivariate testing?
Hansen: There are many ways to move the needle. One advantage of being able to get multi-criteria tests into the field is that you can work on optimizing the entire funnel, from product navigation to shopping cart to check-out to conversion, simultaneously. That creates the possibility of real synergy. If you can increase the number of checkouts by 5% by eliminating shopping cart abandonments and at the same time add 16% to the order value because of better product mix, you've boosted ROI from multiple dimensions.