• by Eric J. Hansen , February 21, 2008

Online publishers today are under tremendous pressures to ensure their sites are as efficient as possible to not only engage their readers so that they keep coming back, but also to make their sites valuable and attractive advertising properties for media buyers, advertisers and the top online lead generation providers to create revenue and remain competitive.  

The most effective way for online publishers to create an experience that keeps their readers engaged, is to run regular, targeted multivariate tests.  This allows online newspapers to give their readers what they really want, and also allows them to help customize their readers’ experience.

While the thought of creating and running multivariate tests can seem a daunting task at first, the following is a list of best practices which will help publishers run multivariate tests that will keep their readers engaged and achieve the “stickiness” which is imperative to stay competitive in today’s rapidly growing online publishing world.

Define Your Goals
o    Before you start formulating a test hypothesis, or begin running your tests, the first and most important step is to ensure your marketing goals are clearly defined.  Examine your marketing goals to define the appropriate success factors that all stakeholders can agree upon.
o    Sift through potential key performance indicators (KPIs) to determine those that will accurately measure progress towards specific marketing goals and benchmarks.
o    Composite goals: when two or more KPIs should be formulaically combined to yield a "visit score" for measuring engagement and other advanced metrics.

Define Site Factors
o    Think like an end-user by evaluating some of the common pitfalls on your site starting at your landing page and moving inward.  Is it hard to navigate back to the landing page from another page?  Little annoyances when navigating through a site can ultimately cost readers.
o    Identify which factors you should test (i.e. copy, layout, navigation, and calls-to-action), in which combinations to test them, and in what order to test to yield the results you need to meet your marketing goals.

Target Reader Segments
o    Avoid one common mistake Web marketers make while running multivariate tests, which is optimizing their site for anyone and everyone, when their audience actually is comprised of distinct, identifiable user segments.
o    Methodically choose which groups of site visitors (i.e first time site visitors vs. repeat visitors) to target for specific tests or offers in order to meet your marketing goals
o    Define user-segments based on meaningful characteristics (i.e., those who arrived on your site through internet search engines vs. those who arrived through links on other sites).

Run Your Test
o    Decide if you need to use a testing tool or if you can do it on your own, there are pros and cons of each.
o    Identify the key aspects of executing a test, from test design (full vs. fractional factorial and Taguchi arrays) to sample size estimation (rules of thumb).

Analyze Your Results
o    Determine the types of analysis that can be used to make the raw data useful and actionable.
o    Read the data in the reports from your testing tool.
o    Determine if the change you’ve observed is statistically significant or not.
o    Decide which factors were influential vs. those that were not (i.e. determining “factor effects”).
o    Begin the iterative test-learn-repeat cycle.

By running regular, targeted multivariate tests, not only will online publishers achieve the “stickiness” they need in order to remain competitive in the marketplace, but will build solid relationships with their readers.  There are several tools out there today which help you run effective multivariate tests with little effort; it is just a matter of finding the tool that fits best with your overall site goals and objectives.

advertisement

advertisement