• by Jack Loechner , Staff Writer @mp_research, September 13, 2018

We live in an age that inundates us with more data than we have ever had before, says the report. but more data does not necessarily mean better decisions.

How six costly data mistakes can lead you astray, by William Gadea for Marketing Profs. 

In 2012, management consultancy CEB, now a subsidiary of Gartner, studied how more than 5,000 employees in 22 global companies used data in their decision-making.The researchers concluded that there were three categories of workers:

• The visceral decision-makers, who followed their instincts rather than the numbers
• The unquestioning empiricists, who followed the data slavishly
• The informed skeptics, who considered the data, but used it with caution

The third category, those who used data with a measure of skepticism, balancing the figures with their own judgement, ultimately made the best decisions, but how can we take advantage of evidence without being led astray by numerical mirages? asks the report. Gathering data and learning from it can be such an effort that sometimes we cling too tightly, says the report, and too long to the truths we learn. Things change, and so what was true two or three years ago might not be true today.

What if in the last three years or so the returns of paid listings vs. organic listings on search has doubled or tripled? and organic click-through rate has fallen, asks the report. It's reasonable to look at your past-year performance to use as a baseline.

However, to do that you need to make sure that there's no variable that is skewing the result during the base period you're examining. 

Likewise, if you are looking at data retroactively and you are trying to see whether there is a correlation between an action you took and a result, check to see whether there are any lurking variables that might have also influenced the result.

Statistics allow you to come to a conclusion about a set of data from a random sample of its items. A mathematical formula that lets us calculate how likely it is that a sample will be an accurate reflection of the larger set; the output is called sampling error. There are various common misconceptions about sampling error:

• Saying the sampling error is +/-3% does not mean that is as large as the error can be. Indeed, 5% of the time, the sampling error could be larger than 3%.
• Saying that the sampling error is +/-3% does not mean that it is as likely to be 3% off than it is to be spot on. The probability curve peaks in the middle.
• Saying that the sampling error is 3% does not mean that is the only way the data can be wrong. This article calls out some of the other ways!

Bad data might also be technical failure, human error, or bad protocols. When you look at numbers, think of how reliable the entire process of collecting them might be. And, sadly, numbers are often used to hoodwink us, says the report. If you're basing a decision on some data, and the source of that data is an interested party, you might want to take a closer look at, if not discount altogether, the representations being made.

Not measuring the right thing is the trickiest and most costly mistake to make.

Sometimes it's made because we want to fool ourselves or make ourselves feel better. Consider the so-called vanity metric. Is website traffic booming but conversions are flat? It's easy to emphasize the former rather than the latter. It might take a rationalization, but one can manage that, but sometimes we are not measuring the right thing because measuring the right thing is too hard... or maybe not even possible.

Customer satisfaction, for example, is really hard to measure accurately, even with established metrics, concludes the report. Data can be an enormous boon, but it’s important to know where it’s coming from  Measures like engagement are far easier to see and record, but which is going to be more important to the long-term health of the business, questions the report.

For additional information from Marketing Profs, please visit here.