Commentary
What 2020 Election Polls Might Indicate About Consumer Research
- by Maarten Albarda , Featured Contributor, November 13, 2020
So the polls got it wrong. Very wrong. And that got me thinking…
There are already some initial published postmortems on the “why” of the epic pollster fail, 2020 edition. What do they reveal?
In 2016, the consensus (per the New York Times) was that poll data underrepresented non-college-educated voters, and they voted Trump. When the pollsters adjusted this omission, they found their 2016 Trump numbers were underrepresented by 4 points, which to a large degree explained their “miss.”
The Times concludes that in 2020, the issue was perhaps not underrepresentation of any voter groups, but rather that certain groups simply did not want to participate in polls, while others were over-eager.
One group of non-participants are the oft mentioned “Hidden Trump voters” a group of people who vote Trump without disclosing this to most anybody else. And there is an assumption that many other Trump voters simply did not want to participate, because the polls came from the “fake news” outlets they distrust so much.
advertisement
advertisement
Then there is the “resistance” group, i.e., all people that felt strongly anti-Trump. These people are so vocal in their anti-Trump feelings that they made their voices heard at any opportunity they got. When the pollsters came a-knocking, these people were super-motivated to participate, while their counterparts were far less inclined to do so. So one side got overrepresented, and the other underrepresented.
Kent Harrington over at MediaVillage explains that “with smart phones that the law says can't be autodialed, caller ID that makes pollsters easy to ignore, and Millennials and Gen Zs who would rather text than talk, the good old days of easy outreach to the man-in-the-street are long gone.”
It is perhaps a tad ironic that with the increase of technology and data, the accuracy of our polling data is decreasing. Still, in that respect it follows the same rule as my “Law of Marketing Data and Understanding,” which I shared in 2014: In the case of political polling, with each increase in technology and data, we lose an equal amount of actual voter insight and understanding.
And that got me thinking about consumer research. I ‘m not referring to data tracking that happens via cash registers, credit card readers, online transactions, etc. Those are real, perhaps comparable to what happens on election day in the voting booth. What I am referring to are research efforts into consumer perception, consumer motivation, etc., which depend on respondents voicing their opinion.
I can totally imagine there are banana haters out there, ready to be vocal about their disgust at the mere mention of bananas as an acceptable food or snack. I myself am such a person (#bananasareevil -- Google it!). So if someone would call on me for my opinion on bananas, I am ready and primed to give them a solid piece of my mind. But what about the quiet banana lovers out there? Would they come out completely underrepresented in the research?
It is a known fact that researchers are finding it harder and harder to recruit people for actual in-person opinionating. The alternative is to use electronic surveys made available on the device of choice. Given how that worked out for polling, I am now wondering if we should rethink consumer research -- just as we’re rethinking polling.
Hi Maarten,
I like your Law of Marketing Data and Understanding. Makes a lot of sense.
Also, many of the polls are purposely slanted to sway the voters, the herd mentality. I would think that's also a part of the challenge.
I happen to believe that if the would- be respondent is immediately aware of the true purpose of the survey and has some aversion to the subject ---or is most interested in it----that this influences cooperation rates and, to a degree, the answers one receives. For example, when a TV rating panel is recruiting people to join their operation and supply information for a sustained period of time---the purpose is stated at the outset---with the "cover" claim that the information supplied will help TV programmers make better shows---- but participation is clearly a burdeon that some may opt for while others won't. My hunch is that heavy TV viewers---who like TV--- are probably more inclined to cooperate than light viewers---but to an unknown extent. As you can't account for the resulting bias simply by demographics---a fair percentage of young people happen to be heavy viewers---it's not just old folks---you live with it---so long as the results are not incorrectly skewed to favor
one network over another as far as you can tell. Or ----if no one asks about it.
I wouldn't be surprised if this isn't what's happening in the election polls. Even if they ask if the person is a likely voter, that still doesn't account for the potential bias of those favoring Biden being more likely to participate while those who are suspect of such polls either lie for privacy reasons or just to mess them up. If this happens 5-10% of the time that's more than enough to skew the findings away from reality by a distressing margin.
I'm not suggesting that this theory explains everything---it probably doesn't. But it might be interesting to see what happens if these polls disguised their true purpose starting on another subject and asking people a few non political questions at the outset---then, once they have secured cooperation, follow with the political stuff. At least that might draw more people in and, as a rule, once a person starts to cooperate with another human being in a telephone survey some bonding takes place and you have a better chance of getting honest answers.
Maarten, I think that there are other factors at play.
First, this is the first PPP I can recall - Pandemic Political Poll. Access was affected which could be a contributing factor. I suspect that the more reputable pollsters may have made allowance that others may not.
So Guy, I think the incidence of 'slanted' polls is less of an issue than 'lazy' (aka .. quick and cheap) polls.
But a huge factor is that many of these 'polls' work to an sample threshold. That is, they "push" out the poll and the first (say) n=1,000 who respond are accepted as being 'representative'. This 'push' tends to be SMS, email, 'phone etc. Yep - convenience polls. I suspect that this has an urban bias, which likely favours urban areas. If is for reasons like this that research such as TV ratings use stratified panels (random within strata) which provide reduced bias but also longitudinal data (is it really a swing or is it chance).
But the last time I looked, the polls were predicting a Biden win, and I think January 20 will show that as correct.