An alternative title for this piece could be: “The media should have known better despite insufficient cautions from pollsters.”
Yes, confidence levels for all the major
polls were almost always reported.
Large, truly representative probability samples without significant non-response bias are rare if not impossible to achieve. Only surveys with these
rare properties can support the traditional margin of error calculation.
Consequently, the margins of error identified on various polls were unfounded. The media should have
recognized and reported that statements such as: “Results are +/- 3.5% accurate at the 95% confidence interval” on any survey or poll were specious.
Add to
this hidden biases that must be accounted for in any survey approach, whether quantitative or qualitative.
As Ed Papazian of Media Dynamics Inc. reminded us in Media Post recently,
based on well-established media consumption studies, an unknown percentage of respondents will misrepresent their actual behavior or mask their opinions so as to be more in step with what they
perceive to be politically correct.
Sometimes called the “social desirability” or “lying factor,” this aspect will subsequently confound almost any close poll
result. We suspect this bias was significant, based on the nativism ultimately reflected by much of the population during this election.
So were the numerous projections from so
many reputable research companies and their media partners that far off the mark?
No! Especially when you throw in all the targeted dark posts (by the Trump campaign) on Facebook; The
James Comey announcements so close to Election Day; and the anticipated Gary Johnson vote in a tight race, especially in critical swing states.
Regarding the Facebook influence, this
chameleon is a tech company, a research company and surely a media/advertising company that poses an array of issues for those in media and marketing. In her article “The Secret Agenda of
a Facebook Quiz,” New York Times, Nov. 19, 2016, McKenzie Funk stated: “If Mr. Zuckerberg takes seriously his oft-stated commitments to diversity and openness, he must grapple
honestly with the fact that Facebook is no longer just a social network. It’s an advertising medium that’s now dangerously easy to weaponize.”
The majority of
polling averages had Clinton winning the popular vote by ~2%-4%. She won by ~1%, despite the wonky error margin for any of the individual polls. This raises the second anomaly. How could a
~2%-4% Hillary lead translate into a 60% -75%+ likelihood to win based on polls with multiple and known flaws and the Facebook, Comey and Johnson dimensions?
The third anomaly for the
polls was they also “missed” in most of the key battleground states, notably NC, WI, PA and Michigan, where the final result is still to be declared. Again, the difference between
the poll and the result was generally less than ~4%-5%, well within any truly pragmatic error estimate.
Tom Anderson, Founder OdinText, has suggested the real problem may be with
quantitative polling itself. “It just is not always a good predictor of actual behavior.”
He believes the failure of the polls was an over reliance on
simplistic-aided questions. While easy to collect and analyze, such quantitative polls lack the richness of unaided, qualitative type questions. Likert-scales, a series of questions with a
set of answers or a ratings scale, typically used in virtually every poll and quantitative survey, tend to fail when used to study behaviors that are not concrete or
“quantifiable.”
OdinText asked a straightforward unaided, top-of-mind response via an open-ended question: “Without looking, off the top of your mind, what
issues does [insert candidate name] stand for?”
The OdinText qualitative results strongly suggested that Hillary Clinton was in more trouble than polling data indicated. The
No. 1 most popular response for HRC - the high perception of dishonesty/corruption versus the No. 1 and No. 2 most popular responses for Donald Trump, relating to platform - immigration, followed by
pro-USA/America First and only a third by perceived racism/hatemongering.
Tom stated: “If I want to understand what will drive actual behavior, the surest way to find out is
by allowing you to tell me unaided, in your own words, off the top of your head.” But even his surveys will reflect the vagaries and biases of an imperfect science despite sophisticated
analytic software.
The media should have been reporting, or more aptly “interpreting,” the polls as “too close to call” well before the actual election
night!
The ARF & GreenBook are sponsoring “Predicting Election 2016” - Tuesday Nov.
29.