• by Phil Leggiere , August 15, 2007

Variety is the spice of life. One of the most daunting challenges of ecommerce sites, however, is to shape the seemingly limitless variety of potential page elements into just the right mix for just the right customer. To do so, marketers need to learn to integrate statistical analysis with behavioral segmentation, as Jim Wehman, vice president of global strategic marketing at Digital River, explains below,

Behavioral Insider: Digital River works with many of the largest ecommerce sites in the world, particularly in consumer electronics and software. What are you doing related to behavioral methodologies, and what roles can behavioral targeting play in ecommerce Web site optimization and marketing?

Jim Wehman: The application of behavioral principles to e-commerce site optimization involves the marketer moving through three phases. The first is based on judgment and experience.  The next is data-driven testing. The highest level of optimization is full behavioral targeting, which we refer to as mass dynamic personalization. I would say that at this point our clients are at different stages of optimization, and that our goal is to help them achieve the highest level of optimization as quickly as possible.

BI: Can you break those phases down a little more, starting with what you mean by judgment and experience?

Wehman: As one example, we have developed an up-sell strategy that online merchants use to entice consumers to choose a premium version of a product while checking out, rather than, say, just the basic version. If we consistently find that 60% to 70% of customers opt for the higher-priced premium version, we can then introduce this strategy to a wider range of clients, with the confidence that it is generating successful results.

 
BI: And how does that migrate into data-driven testing?

Wehman: From there we do three layers of data-driven testing. The simplest level is A/B testing, which is used for testing large format changes on a Web page. For example, you could take half of a set of Web site visitors and offer them the basic shopping cart, while offering the other half another alternative - perhaps a basic shopping cart with a cross-sell option. By splitting the traffic, we can measure the difference in the performance of the two user experiences in a statistically valid way.

Then finally we have multivariate testing, where you can measure discrete items and elements on a page. This can encompass everything from headlines to calls-to-action, product offers, product placement mix, colors, promotions, pricing -- basically any of the important elements on the page.

Multivariate analysis can expand into incredibly powerful complexity. Let's say you want to test four elements and four versions of each element. That yields a potential universe of 256 different combinations of individual user experiences on one Web page. Imagine you were using A/B testing. You'd need to divide traffic 256 different ways and wait a very long time to get enough traffic to each user experience to get measurable results. With multivariate testing, the marketer can test just 16 different combinations and, based on predictive analysis, it will identify which of the 256 combinations will perform best.

BI: How does the behavioral piece relate to these more aggregated statistical testing?

Wehman: In getting to the highest level of optimization, all of the data we've accumulated from previous A/B and multivariate testing can become a starting point for an enhanced kind of behaviorally based targeting or the mass dynamic personalization of the Web site. It's based on identifying particular user patterns. We can do it by any number of variables, including channel, referral domain, operating system or browser version, correlating these variables with the kinds of response results we found in A/B and multivariate testing.

Another important behavioral variable site optimizers can use is keywords. For instance, if someone runs a query for ‘best rated OCR (optical character recognition) software,' then we lead with a quality offer. If the query is for ‘low price security software,' you can lead with a price offer. Other click-stream data that can be correlated with performance includes cart abandonment, page views, purchase history and demographic registration information.

To do more complex behavioral targeting takes longer. For example, it's easy to segment by IP address, where available, but location alone might not be particularly predictive of behavior.  The task is to find the differences that ultimately matter.

BI: What are the biggest challenges in moving large ecommerce providers toward this behavioral targeting mode?

Wehman:  You need large enough samples to be statistically valid. One of the big challenges is that people not steeped in statistics get distracted by the early noise in a trial. Sometimes you get very early gangbuster numbers and there's a lot of excitement, but those numbers may erode as the sample size increases. Likewise, some clients are depressed by disappointing early results based on samples too small to infer anything. The true winners emerge as the sample size goes up.

The gains at each phase are cumulative. For most clients it's a short ramp-up to move from judgment and best practices to statistical testing, maybe six to 12 weeks before they get results. But getting over 95% optimized with multivariate testing can be a six- to nine-month process and it can take over a year to get to 99%. But by this time, you not only have those test results, but more importantly, rich behaviorally identified segments. Over the next several years, we expect to see marketers across the Web and around the world moving into the highest level of full-fledged personalized segmentation.