Prediction Markets Posted by: Dale Franks
on Tuesday, November 21, 2006
Just prior to the election a few weeks ago, the Tradesports futures market was predicting a Democratic House, and a Republican Senate. Clearly, that turned out to be wrong, prompting unseemly glee from people like Kos and Atrios, who promtply declared that the idea that reliable predictions could come from the futures markets was just BS.
But, as Prof. Cass Sunstein and Bo Cowgill write in The New Republic, that view is based on a misunderstanding of statistics. It is unseemly, because it is the result of ignorance.
But the doubts are unwarranted; they are based on serious confusion. Even for the latest elections, the performance of prediction markets has been truly impressive. On July 31, long before the pundits reached a consensus, the prices on the Iowa Electronic Markets suggested that the Democrats were likely to take the House of Representatives. The Republicans had a brief market surge on September 16, but the Democrats were seen as increasingly probable winners from October 9 until Election Day. Even better, in every single Senate race at TradeSports.com, the markets' overall favorite defeated his or her opponent.
Still, the predictions markets did forecast a Republican Senate. Should we dismiss them for getting that one so wrong?
If you think so, then you're probably forgetting the principles of probability. Prediction markets do not make absolute predictions about electoral outcomes, economic developments, product success, or anything else. Instead, their predictions are mere probabilities. That's one reason why the markets expected the Senate to stay in Republican hands, even though so many Democratic candidates were favored. Senator-elect Jim Webb had roughly a 60 percent probability of success, and Senator-elect Bob Casey Jr. had roughly a 70 percent chance of success; but these numbers do not suggest that both candidates were likely to win simultaneously.
Prediction markets cannot be judged by asking whether a single prediction turned out to be right. The real question is how their probability judgments compare with the actual outcomes. And on this count, prediction markets have a fantastic track record in many domains, including elections, company performance, and economic developments. If you look at the set of outcomes estimated to be 80 percent likely, about 80 percent of them happened; events estimated to be 70 percent likely happen about 70 percent of the time; and so on.
This is what it means to say that prediction markets supply accurate probabilities. No one thinks that all events whose probabilities are above 50 percent should come true. That's most unlikely.
Whether it's Tradesports, or the Iowa Electronic Markets, the futures markets reflect the probability of an outcome occurring. It's important to remember, though, that probabilities are not certainties. For instance, let's say that you have 100 unrelated events, each of which has a 70% probability of occuring. The chances that all 100 events will occur is vanishingly small (0.7100 = 3.2344765096247579913446477691002e-16). In fact, even if you have 100 unrelated events, each of which has a 99% probability of occurring, the chances that all of them will occur is only 36.6% One notes, however, that, as Messrs. Sunstein and Cowgill relate, Tradesports was spot on for every single senate race. I other words, at the individual race level, Tradesports accurately reflected the outcome of the election, and did so precisely. That is an enormously impressive record.
It was the overall market for Republican control that came out wrong. It's especially important to remember, therefore, that the overall result contract for GOP control was a separate market, not an aggregation of the individual races. It measured the probability that the senate would remain Republican, based on the aggregate assessments of the individual investors. Investors, by the way, who may not, in fact, be the same investors that were buying contracts on individual races. Let's remember that there were very few people who expected the Senate to change hands. And those that did, probably based that prediction more on wishful thinking,than on any rigorous methodology.
The fact that the futures market wasn't magically correct, as Atrios derisively comments, doesn't mean what Atrios evidently thinks it means. Apparently, Atrios and Kos believe that if the futures markets aren't 100% successful, then they're 100% worthless. But nothing is 100% successful in making predictions. By that standard, we should throw out polls completely, too, since polling has a far worse record of prediction than futures markets.
At the end of the day, futures markets are just tools, and tools whose relability is based on—and limited to—the aggregate judgements of individual investors. No one touts them as being perfect, they merely have a superior track record to polling, in terms of their predictive ability. If they are useful tools, that provide a generally reliable prediction of outcome probabilities, why shouldn't they be used? Why denigrate them?
Frankly, one wonders if Atrios and Kos hold views on the futures markets that are based in a general distrust of markets as a whole. Why else would they seize with such glee on the fact that one 70% probability didn't occur, despite the extraordinary accuracy of the individual predictions? Why else, in the face of massive peer-reviewed evidence to the contrary, are they so certain that futures markets are BS? Why are they so seemingly afraid of markets? Could it be that, if people come to realize how useful markets are in deriving dependable outcomes in one area, they might start to wonder if there are other areas where markets are superior to government control? Why, that might call some of the unquestionable verities of "progressive" economic policy into question.
Shame on you for assuming that Atrios and Kos have any clue how futures markets work. They are "street fighting men" and nothing more. All that is missing is the flashy outfits and paramilitary ranks. But they march lockstep with banners and slogans.
Kos and Atrios scoff at the markets but they sure put all of their faith in exit polling- they’re still holding a candle for that Kerry win that the early polls (they’re an exact science they tells ya!) "predicted"
The problem with futures markets is that they do a great job of predicting things that have already happened. It’s foolish to ignore political markets entirely, of course, but in the bottom line they reflect conventional wisdom and publicly available opinion, nothing more. I have a feeling that if you had frozen bets on, say, George Allen vs. Jim Webb in July, the market would have missed badly. Similarly, the political futures markets will never put you ahead of obvious trends, as far as I can see.
As for the polls, Larry Sabato’s accuracy was around 98%. I can’t think of a Senate race where the polls consistently showed someone losing in the week before election day, and they won. So I’m not sure you’ve made your point that futures markets are somehow superior to polls. Why should they be? They’re similar mechanisms. In both cases, you’re basing opinions on the guesswork of a random sampling of the population.