• by J. Walker Smith , Op-Ed Contributor, May 15, 2024

The old joke about survey research is that the best thing about it is also the worst: people answer the questions.

It’s good because we get answers that help us understand people’s preferences and behaviors. It’s bad because, all too often, we also get answers that reflect nothing but the fact that people had to tick one box instead of another.

Nothing new. But it’s subject worth reviewing again because the stakes of misleading ourselves with survey data are higher than ever.

Most of the conversation these days is about data fraud from bots and bad actors. This is a pressing priority, yet certainly not the only priority and perhaps not even the biggest one when it comes to getting public opinion right.

There is an inherent bit of mushiness built into every survey result. Some portion of total agreement with any question is mushy, which is to say not a reliable part of measuring opinion about an issue or topic. Such mushiness can be a big problem if most respondents know or care little. Answers will be unstable and contradictory. But even a small number of respondents like that can be more than enough to misguide us.

Dan Yankelovich, founder of the firm I worked for and ran in years past, tackled this problem in the early 1980s with something he called the Mushiness Index. An extensive, two-year research-on-research project underwritten by Time magazine -- we conducted their public opinion poll --created a four-item battery that was able to determine if public opinion on an issue was firm or mushy.

The original plan was for Time to visually signal the degree of mushiness in its own polling results, thereby setting an example for other polling firms that Dan hoped would be followed. For various reasons, this did not happen.

Dan was philosophical about the failure of mushiness to catch on. Years later in retrospect, he said he had come to realize that mushiness was at cross-purposes with journalists seeking facts and verity. But I think reporters and researchers missed a moment. Mushiness tells us a lot about the facts.

Climate. In my last column, I wrote about the so-called value-action gap between sustainable attitudes and green behaviors. This gap is easy to understand once you realize that most environmental attitudes are mushy. Broadly speaking, these are not firmly held values, mostly because they are rarely grounded in any sort of personal relevance. Which is the first of the four things that figures into the measure of mushiness.

Fake news. Today’s fractious politics are often explained as a consequence of disinformation campaigns by shadowy figures trying to inflame political passions. The lament is ‘fake news,’ for which the antidote is said to be ‘the facts.’ The mushiness work found that higher knowledgeability predicts firm opinions. This is the second thing that figures into mushiness. People who ‘know more,’ quote/unquote, are less mushy in their attitudes. It doesn’t matter if they know ‘correct facts’ or not. Just that they feel knowledgeable. Meaning that fighting ‘fake facts’ with counter-facts will always be a losing battle. There is nothing mushy to affect or firm up. It’s true of politics and it’s true of brands, too.

Social media. Dan’s view of the general public coming to judgment, as he put it, is rooted in a belief that as people deliberate the issues of the day with one another, opinions will settle out pro or con. Talking about things with others is the third element of mushiness. This view now seems quaint to me, no disrespect intended, in our era of social media. Echo chambers lead to more extreme views not to views more accommodating of democratic compromise. Many people have abandoned news for memes. Attention spans seem to have shrunk. It is correct, as the mushiness work found, that discussing issues with others leads to firmer opinions, but it turns out that these opinions are more radical and less conciliatory as well. Which is not how fixing mushiness is supposed to work.

Boycotts. Beliefs that are held less strongly and attitudes that people admit they might be willing to change are mushy. This is the fourth element of mushiness. In the context of buying, many brand purchases are motivated by emotion, habit or peers. Not by a brand’s positioning or performance. This means mushy. When attitudes are mushy, there is nothing to firm up resistance to negative pressures like boycotts. In the sort of charged environment that we have today, even iconic brands are at risk from such actions if attitudes about them are anything less than firm. Mushiness is a barometer of risk.

None of this is what Dan Yankelovich had in mind with his Mushiness Index. Nonetheless, the concept of mushiness and the elements that predict it continue to have relevance.

Not every question or issue is equally definitive in people’s minds and thus not every survey result either. High brand ratings are great, unless they’re mushy. It’s the extra step to measure mushiness that tells us whether our results are a mirage of certainty or a sure finding.