• by Bill Colbourne , August 31, 2007

"Lies, damned lies, and statistics," is a popular phrase attributed to Benjamin Disraeli by Mark Twain. My degree is in history, a subject I've had a passion for all my life. This remark by this most colorful of Victorian British prime ministers highlights an essential aspect of history -- those that drive the agenda control the agenda. Thus history is, for the majority, whatever those capable of doing so want it to be.

Last week, the Kremlin admitted to a policy that seeks to reappraise Russian history in a more "patriotic" manner. This will be done by recently completed historical guidelines to be sent out to schools, which reevaluate Stalin, for example, as having had no choice but to commit certain "monstrous acts" in the national interest. Apparently the team creating the guidelines included not one single historian. And so with market research: If you drive the agenda, you seize the initiative. The problem is that, increasingly, researchers are evolving into marketers to add value to their raw propositions. A great deal of research published is created specifically for certain corporate interests and for marketers that have already determined the outcomes.

In relation to my own business, I recently read a piece of research in Ad Age by a company espousing email-marketing techniques, which revealed that email marketing is preferable to contacting business leads directly on the phone. The research stated that only about 10% of decision-makers take telephone calls from companies wishing to win business from them. The evidence of my own firm's databases over the last decade -- and in the same marketplace -- shows it's possible to speak on the phone with just over 60% of this group on average and at the first point of contact. Yes, there can be instances where email marketing is preferable, but how can there be such enormous disparity in the facts themselves?

It's clear to me that four things cause distortions in research findings: intention, methodology, analysis and presentation. Each of these factors, in a multitude of different ways, can bend realities to the purpose of the research publisher or user. Governments, corporations, interest groups etc., all use "research," and make bold pronouncements based upon it to help achieve their aims. Whereas for scientists, in the sterile conditions of the lab. it may be possible (at some expense) to ratify important advancements (e.g. -- is it possible to repeat the experiment? If not, the results are invalid), in the business arena I believe it's really very hard to do so across ALL the criteria that might influence outcomes.

I do think, as advocated by another Metric Insider columnist, that a series of universally accepted research guidelines is a good thing and that this should involve, at the very least, sample size and type, date, methodology and the availability of raw data for independent auditing.

But ultimately it has to come down to the individual to trust the source. I think consumers of research are way too trusting -- transferring credibility from a media brand to the content within. Do I trust the source that said the team at the Kremlin included not one historian? I think so. But then what is a "historian"? Knowing which source to trust and why is never easy. I think this also highlights a fifth area beyond the reach of any guideline: the prejudices of the viewer.