• by Karlene Lukovitz  @KLmarketdaily, November 23, 2009

Nearly 90% of blue-chip companies aren't fully leveraging their market research functions, according to a new report from The Boston Consulting Group (BCG).

BCG surveyed more than 800 executives at 40 global, consumer-facing companies (serving industries such as packaged goods, financial services, retail, apparel, travel and technology) with sales of $1.5 billion or more. Half of those surveyed were in the consumer insight function and half in line management. BCG also conducted nearly 200 interviews with executives and gathered benchmarking data on the companies' structures, staffing and spending.

Just 35% of survey respondents reported having good time-series data on how consumer needs and behavior change over time, and just 47% said that they know what drives market share, who is buying, and why. Less than 45% believe that their market research/consumer insight function provides a competitive advantage or high ROI.

Companies are also failing to use research data fully. While nearly three-quarters (72%) reported using this research to help with new product and marketing message development, only 30% use it to help make channel and distribution strategy decisions, only 33% use it in creating financial forecasts, and only 38% use it to drive pricing. Moreover, just 31% said they feel that their companies are above average at having widespread institutional knowledge of their consumers (who buys and who doesn't, and why).

According to BCG, low ROI on consumer insight in part results from many companies continuing to run the function in outmoded fashion: They ask market researchers to take orders rather than to act as strategic partners generating breakthrough insights. For instance, just over half (54%) report that senior market research staff have a seat at the table for the company's most critical decision-making.

Poor communication and lack of empathy contribute to the consumer insight integration problem. For example, 73% of researchers said that they always answer the question "so what?" about the data they provide, but just 34% of their counterparts in the business units agreed. Fewer than half of researchers said that they believe that senior management would pass a pop quiz on basic facts about the consumer, and only about half (56%) of total respondents reported that their business leaders attend qualitative research sessions.

Neither of the groups surveyed had "put on the other's hat in a meaningful way," observes Mary Egan, a New York-based BCG partner and leader of the study. "Research teams feel that they're not included or empowered, and that they could do much more if they were, but don't recognize that if they have not been producing high-quality research, they have not earned these privileges. Line managers feel that the consumer insight function lacks the capabilities and talent needed. They don't recognize that, in many companies, the talent is there, but the research team is unable to perform to full potential because they are being sidelined."

The study found that companies that use market research most effectively actually spend less on it. "Throwing a lot of money at a small research team leads to hopping from study to study and inability for the organization to absorb the data," says Egan. "It's like drinking water from a fire hose." To make the most of spending, companies need to adjust the mix of tactical versus strategic work and invest in synthesizing the results, the report concludes.

The best companies also give the consumer insight team a seat at the table and address the hiring and talent, career path and training issues needed to do this successfully, points out Kate Manfred, a Chicago-based BCG principal and a co-author of the report.

The best companies demonstrated a higher quality of engagement between insight teams and other executives, driven by four principles on the part of management: Ensuring an appropriate interface with senior executives; moving beyond a narrow marketing scope; prioritizing strategic work and saying 'no' to some projects; and strongly embedding the insight team into the business units.

Egan notes that the name change from market research to consumer insight at many companies is evidence that a perception shift has taken place, but that the accompanying changes in practice are still in process.

Manfred adds that consumer insight is far from alone in struggling to integrate into the larger company at present. "Finance, HR and other roles traditionally considered support roles are evolving to bring significant contributions to the business," she says. "But we believe that when consumer insight is working very well, it can bring competitive advantage in a unique way."