• by Jon Last , Columnist, August 11, 2015

It wasn’t that long ago that marketing research and consumer insights were largely afterthoughts in the world of sports marketing. Who needed research? Because most of us in the sports industry pursued career paths that closely aligned with our personal passions, we saw ourselves as the customer. We thought we knew everything. Fortunately, our profession has evolved, and as I’ve addressed in past posts, we’ve recognized that most of our customers, while still sharing a passion for sports, aren’t as immersed in that world as we are. Those sports marketers that take the time and effort to recognize this and seek out objective feedback can be rewarded with insights that help our brands truly resonate amidst a cluttered marketplace.

But just because sports marketers have increasingly recognized the need to look beyond our colleagues when seeking customer perspective, it doesn’t mean that research efforts are being carefully planned and executed. The discipline of “sampling” involves selecting a representative base of research respondents. It’s a rigorous science that we propeller heads who conduct marketing research take seriously. Just as properly framing survey questions and discussion guides is best left to professional practitioners, proper sampling recognizes that there are lots of bumps in the road that can render even well-designed research data collection instruments a source of misinformation, rather than a pathway to actionable insights. Here are some common sampling mistakes to steer away from. 

Convenience Samples

Many of the more common and potentially dangerous sampling errors involve “respondent self selection” or convenience samples. We’ve all seen the bingo cards to fill out or the place-based kiosks positioned around sporting venues. One assumes that since these are made accessible to all fans or customers, that we are properly framing our sample. But such an approach assumes that those who actually participate in the survey are reflective of the full target audience. Countless research on research has shown that this is all too often not the case. Convenience samples can yield dichotomous response.

In other words, your respondents are often those who feel strongly one way or the other, or are the easiest to reach. What about the otherwise silent majority? The best research doesn’t leave sampling to chance. Rather, it is actively designed to provide for discrete quotas of those who are truly reflective of the target audience. A random and representative sample, by definition, means that everyone in the desired population universe has an equal chance to be represented, and it also builds in safeguards to protect against “ballot box stuffing.” This can be actively managed, typically with frequent and diligent human interaction, assuring that the resulting participants are actually reflective of not just those who want to talk.

The Lunatic Fringe

Self-selecting samples may provide early warning of problematic situations, but they also can potentially over-amplify the extent of a situation. Such is often a shortcoming of being overly reliant on feedback loops like message boards or social media monitoring. That’s not to say that such research doesn’t sometimes yield valuable directional feedback, but just like the old suggestion box or letters to the editor, beware the lunatic fringe. Simply troll through various discussion boards and fan forums and you won’t need an advanced degree in marketing research to recognize that those posting may not be reflective of the full population. 

Professional Respondents 

Perhaps you’ve come across pop-up ads promising the opportunity to win prizes or supplement your income by actively and frequently participating in surveys and focus groups. Sad to say, but there are many people out there who try to rig the system by taking on what they hope will be the right persona to qualify for a study. Professional researchers know how to obfuscate the qualifications they are seeking, by carefully wording questions, providing placebo options, and building in other patterned response data quality protocols to keep these people from corrupting the results of a study. This can often involve “knowledge diagnostics” that help validate that a respondent knows what they are talking about. For example, if we are seeking fans of a particular team, it’s not enough to qualify potential respondents by simply asking who they root for. 

It’s great to see so many sports organizations recognizing the value of good consumer insight. But for those doing so, it’s incumbent that sampling shortcuts don’t compromise the investment.