• by Jack Loechner , Staff Writer, July 13, 2003

Put Research Data to the Test

Geoffrey Ramsey, CEO and co-founder of eMarketer, in a recent release writes, "chances are, whether you're involved in media, marketing, sales, research or all four disciplines, you probably rely on market research data to help you make strategic decisions. Or just as likely, you rely on this data to prove your case to someone else -- whether that be a client, prospect, your boss or your boss's boss. No matter what your use of research information, the integrity of the data itself is of vital importance."

eMarketer, as an aggregator of research data from over 1,400 sources, has developed a four-step process for evaluating research information. It boils down to asking four key questions every time a new piece of research data comes out.

- "Who is the source, and how reliable has its data been in the past?" Some sources are better at measuring and projecting activity in some areas than in others, and bias sometimes occurs in studies that are sponsored by interested third parties.

- "What specific definitions were used in the research?" Often, the greatest disparities seen in estimates between research firms are a result of different definitions being used. For example, while 10 leading research firms estimate the number of people online in the US, the range is between 113 million and 178 million. However, by holding the definition to include: a) all age groups; b) access through all locations (i.e., work, home and school) and c) only active, weekly users, there is remarkable convergence in the numbers.

- "What methodology was used to produce the results?" An online poll of 32 executives may not yield results that are as definitive and actionable as a survey among 4,000 respondents in a random-digit-dialed survey. Additional questions might be, "What was the margin of error?" and "How were the respondents recruited and what incentives were used?"

- "How do the results of this new research data compare with the existing body of evidence?" Don't go to market with just one piece of research. Check and see if there's other data out there -- from different sources -- that either supports, or contradicts, the information you're reviewing. If there is a high degree of convergence between the new data and all the historical data from other sources, you are minimizing your risk of making a wrong decision.

For instance, the eMarketer data show that nine out of eleven researchers agree there are roughly 16 million to 17 million households with broadband connections as of 2002. What's more, all researchers are predicting at least 33% growth for 2003. That high degree of convergence should give marketers and other industry watchers great confidence that broadband has indeed reached critical mass and that it will continue to grow rapidly.

Broadband Household Growth in the US, 2003 (% increase over prior year) Comparative Estimates by Various Sources

• EMarketer, March 2003 - 41%
• Strategy Analytics, January 2003 - 41%
• Gartner Dataquest, October 2001 - 40%
• Parks Associates, August 2000 - 38%
• Jupiter Research, October 2002 - 36%
• NetProfit, August 2001 - 36%
• PricewaterhouseCoopers, May 2002 - 34%
• Strategis Group, January 2001 - 34%
• Yankee Group, April 2002 - 33%

Source: eMarketer, March 2003 & Others as noted When it comes to future projections, research data is only as good as the assumptions made to produce that data. And, the bottom line, according to Ramsey, is a careful evaluation and weighting of multiple sources usually leads to a more accurate picture than any single source could possibly provide.

You can find out more here.