• by Mario Carrasco , Op-Ed Contributor, July 7, 2016

As Hispanic market spend continues to grow and more companies are looking to capture a piece of this growing consumer, Hispanic research has been on the same exponential growth trajectory. More and more research companies are marketing themselves as equipped with Hispanic research capabilities and Hispanic online sample providers have been growing at the same rate. As a research company dedicated to researching multicultural consumers, we have welcomed the growing number of panel companies in the space since the need for Hispanic online sample is so great. However, a recent study from Pew Research has made us wary of the Hispanic online sample in the market place outside of our own assets. 

What is Hispanic sample?
Before we delve into the issues the Pew report highlighted regarding Hispanic and African-American samples, let’s define what Hispanic sample is. Marketers have relied more heavily on Hispanic online research in the past five years than ever before, moving away from more traditional methodologies such as telephone interviews and in-person intercepts. Online research respondents are typically sourced from online panels. Respondents sourced from online panels are referred to as online sample. 

What is a Hispanic online panel?
A Hispanic online panel is a website that is created to attract Hispanic respondents to register and take surveys on a regular basis in exchange for some sort of incentive. The backend of the website is typically research software that allows you to profile and invite respondents based on certain demographic criteria gathered during registration. 

Pew Research Center Analysis

To better understand the current landscape of commercially available online non-probability samples (most online panels), Pew Research Center conducted a study in which an identical 56-item questionnaire was administered to nine samples supplied by eight different vendors.

The full report is over 60 pages long and goes into depth on various measures of data quality but for those of us in the Hispanic marketing world, this is the most salient point of all:

Widespread errors found for estimates based on blacks and Hispanics. Online non-probability survey vendors want to provide samples that are representative of the diversity of the U.S. population, but one important question is whether the panelists who are members of racial and ethnic minority groups are representative of these groups more broadly. This study suggests they are not. Across the nine non-probability samples, the average estimated bias on benchmarked items was more than 10 percentage points for both Hispanics (15.1) and blacks (11.3). In addition, the online samples rarely yielded accurate estimates of the marginal effects of being Hispanic or black on substantive outcomes, when controlling for other demographics. These results suggest that researchers using online non-probability samples are at risk of drawing erroneous conclusions about the effects associated with race and ethnicity.

Note, none of the companies measured were panels focused specifically on Hispanic or African-American respondents, but rather General Market panels meant to be reflective of the U.S. population as a whole. For marketers that have been making business decisions on behalf of their clients for their Hispanic and African-American marketing efforts, this data is troubling. Ten percentage points could be the difference between success and failure for key marketing metrics such as purchase intent or awareness. 

This report highlights the fact that not all “Hispanic” respondents are created equal. It’s not enough for a panel company to ethnically tag respondents acquired through their general recruitment campaigns and deem them representative of others in their ethnic groups. Properly recruited Hispanic research respondents should be invited into a panel in-language and in-culture. Properly balanced quotas and weighting are also necessary to ensure that ethnic online samples are as representative of their larger offline communities as possible.

Getting the Hispanic part of a nationally representative survey wrong may have had limited consequences when ethnic minorities were truly a minority of the population but as we move towards becoming a ‘minority majority’ nation, getting this part of the research right is the whole game. Taking the time to vet your Hispanic research partner and sourcing sample from a panel that has in-language recruitment, culturally appropriate incentives, and a bilingual portal are essential to conducting both Hispanic and nationally representative research that will yield results reflective of the population you are trying to understand.