• by Paolo Gaudiano , Op-Ed Contributor, October 24, 2014

Online publishers have begun to rely increasingly on quantitative methods to help them run their businesses. On one hand, statistical measures such as time on site, click-through rate (CTR) and bounce rate can help publishers assess the health of their sites. On the other hand, some publishers become overly focused and dependent on metrics, and they forget that behind those metrics are individual readers.

The advent of online publishing has led to an unprecedented ability to measure virtually every aspect of reader behavior. This in turn has spawned an entire industry of analytical platforms and solutions, launched to save publishers from drowning in the resulting deluge of data. The growing adoption of statistical measures is clearly beneficial, as long as their purpose and limitations are understood.

There are two common ways in which statistics are misunderstood or misused. First, it is easy to forget that, by design, statistics hide details about the observations on which they are built. For example, suppose you find out that average time on site for your online publication is two minutes. That could reflect a situation in which all your visitors stay between 1:50 and 2:10, or a situation in which half of your visitors stay one minute and the other half stays three minutes. Clearly, the high-level metric alone is hiding details that could be critical to the success of your business.

The second and more pernicious problem is that when we use high-level, statistical measures about a population, we tend to forget that these high-level measures emerge from the willful behavior of individuals.  This oversight often leads publishers to look for solutions that fix the measurement, rather than fixing the problem.

As a simple analogy, consider managing a building during the winter. Suppose you notice that the average temperature, as measured from thermostats placed in every room, is lower than it should be. Turning up the heat may seem like a reasonable action, until you realize that there are several vacant offices where the windows are wide open -- so your action is wasting energy and making the rest of the tenants uncomfortably warm.

In the online publishing domain, the reality is much more complex than the simple example above, but the parallels should be clear. For example, increasing the CTR rate of advertising is good for revenues, but if so doing angers some readers and drives others away from your site, you end up eroding trust, reducing loyalty and losing out in the long term. A more subtle problem is that increased targeting or per-user site optimization can reduce discovery -- a major driver of reader interest -- and it might actually increase bounce rates.

Overwhelmed by these complexities, it is tempting to turn to technologies driven entirely by quantitative measures, thereby abdicating your decision-making powers. But this is a mistake. You are the expert; you can understand your reader much better than they can; you care about the longevity of your publication. Back in your building, if you hire an engineer to control the overall temperature, don’t expect him to walk around the building to see how the tenants are doing: That’s your job.

Embrace technologies that leverage quantitative approaches, but think carefully about what they will do: not just in terms of the metrics they promise to fix, but in terms of how they impact the very readers that represent the lifeblood of your business. That responsibility is yours and yours alone.