Commentary

What Are Prediction Markets And Why Should You Care?

1. Ed Papazian from Media Dynamics Inc, July 24, 2023 at 8:06 a.m.

   Very interesting, Ed. But I'm not sure that I buy the idea that random probability surveys, when properly executed, give you a less reliable reading than a "crowd" survey as, in theory, the random probability method is meant to represent the total population's views or activities.
   
   The basic reason for random probability research is cost. You can't afford to do anything approaching census level research for most projects---so as a matter of affordability, much smaller but  hopefully representative samples are employed. If, instead, you go "the crowd" route does this  mean that you get your information from a much larger but readily available sample of folks who may or may not represent a true crossection of the population. In other words, is a large sample---even if it's membership may not represent all  segments of the population better than a much smaller sample which is designed to do just that?
   
   Generally speaking, most of the random probability surveys have been proven reasonably accurate over the years---meaning that while they are off by a few percentage points or even less from the truth, they come close enough to provide useful data relative to their cost. The political polls are frequently cited as exceptions to this but when their tiny samples are accounted for---allowing a variability of several points in the findings---they are usually predictive when used in a sensible manner. Inother words a fairly conducted poll of 1000 likely voters which has POTUS candidate A beating B by two percentage points is usually cited as an example of a hopelessly wrong poll, implying that a sample of 50,000---or a "crowd" survey----would have done better. Maybe so, maybe not, but as the parties involved usually can't afford "crowd" surveys we seem stuck with smaller sample research. The question being whether the design is fair and the study is properly executed----wherein lie most of the problems.

   Reply
Ed DeNicola from MediaLytics, July 24, 2023 at 10:37 a.m.
Hi Ed -- First off, I want to say that you're very smart and everyone benefits from your commentary.

Regarding your feedback here, you have to remember that prediction markets aren't based on probability theory. They're based on the theory of the wisdom of crowds. As a result, sample size and representation aren't necessarily key factors when it comes to harvesting good predictions.

One of the really interesting findings from my time at Media Predict was that woman were just as good at predicting the success of male-oriented shows as men were at predicting the success of female-oriented shows. The demographics didn't matter. Moreover, the people who were bad at making predictions would predict poorly in both directions and cancel each other out.

If you were going to use a prediction market for research, you don't have to pull a sample. You would use the existing panel and pay to have a question added to the market. It doesn't have to be more expensive to use a crowd methodology.

This op-ed isn't to bash polls or surveys. I still look at them. However, when available, I look at prediction market data as well.
Reply
Ed Papazian from Media Dynamics Inc, July 24, 2023 at 11:19 a.m.
Thanks, Ed, appreciate your comment.

About the wisdom of"crowds" though---if I understand you---it doesn't necessarily matter what kinds of people you get opinions/predictins from so long as there are lots of them----hence your example of men correctly evaluating something of primary interest to women or the reverse. While I, too have seen examples such as you describe and there are plenty of built-in biases in many "probability sample" studies---especially about how the questions are posed---I remain somewhat skeptical about the "crowd" appraoch---though that doesn't mean that it wont work---especially when dealing with personally relevant or highly emotional subjects like politics.
Reply

CommentRead more comments >

Next story loading