This is a background paper on several of the important concepts in prediction markets that may be learned from the game of craps (and other dice games). While the prediction markets discussed are ridiculous (given that the outcome is completely unpredictable), I believe the concepts are well demonstrated. The concepts in this paper will have direct relevance to my next paper that concerns prediction market accuracy (actually, failure) in public prediction markets.
The betting game of craps is played by rolling a pair of dice. Each roll is a random event, with known probabilities. No one knows which number will come up on the next roll, but everyone knows the distribution of outcomes over a larger number of rolls. Sound familiar? It looks like this:
A Perfectly Calibrated, Accurate Prediction Market (that is Useless for Decision-making)
Now, let’s suspend logic for a minute and run a hypothetical prediction market to predict the outcome of one roll of the dice. We would expect the distribution of bets (or trades) to form a distribution that is very well calibrated with the actual distribution of outcomes. For now, let’s assume that it is perfectly calibrated. At closing, the prediction market indicates that ‘7’ is the most likely outcome, which should occur once in every six rolls on average. As a decision-maker, relying on the prediction market forecast, you would choose the market prediction as your best guess about the outcome. When the dice are rolled two sixes come up, making the value ‘12’.
Chris F. Masse (Midas Oracle) is unhappy (maybe not), because the prediction market failed to predict the actual outcome. Jed Christiansen is somewhat happy, because the market is perfectly calibrated. He might even be ecstatic if the prediction market involved as few as 12 participants. (I’m joking here, sorry Jed) Professor Ipeirotis says, “What did you expect?”
While the prediction market is perfectly efficient and perfectly accurate (calibrated), it is also perfectly useless for the purpose of predicting a future discrete outcome. All relevant information is contained among the market participants (information completeness). The market’s failure to accurately predict the next roll is caused by the randomness of the outcome. The market is, however, perfectly useful for the purpose of betting (i.e. craps), as the odds are calibrated, perfectly, with the outcomes, making it a fair game. As in the game of craps, we are dealing with random outcomes, which by definition are unpredictable.
Now, let’s drop one die and continue to explore the properties of these prediction markets…
Calibration Loss Caused by Information Incompleteness
To the right is the distribution of all potential outcomes for rolls of a single die. If we were to run another hypothetical prediction market on the outcome of a single die roll, the distribution of trades should perfectly match this distribution, assuming a fair die is used and all participants know this. That is, the market has perfect, complete information about the die roll. The resulting distribution is perfectly calibrated with the distribution of actual outcomes over a large number of trials.
Now, let’s add a twist.
All participants are told that the die being used is “loaded”, such that it will turn up one number more often than any of the others. Everyone has accurate information, but it is incomplete: no one knows which number is more likely to be rolled. What will the prediction market distribution look like? It should be identical to the first case, because the participants would be expected to evenly spread their trades across all possible outcomes. In this case, we have accurate, but incomplete information, and the resulting market distribution will not be well-calibrated with the distribution of actual rolls. The market can be said to be efficient, accurately reflecting the information held by the participants, but it is not an “accurate” prediction market, because it is not well-calibrated.
Calibration Restored, with Completeness Overcoming Information Inaccuracy
Let’s try another variation. One of the traders is told that the die is loaded to turn up the number ‘4’ twice as often as any of the other numbers. All other traders are kept in the dark (i.e. their information is inaccurate and incomplete). Assuming the knowledgeable trader has sufficient wealth to move the market (i.e. the market is “efficient”), the prediction market distribution should be accurately calibrated with the actual outcomes. In a perfectly efficient market, we would expect to see the following distribution.
Even though almost all participants had inaccurate information, the market does contain all of the information necessary to determine the true distribution. If the prediction market did reveal this distribution, we could say that it operates as an efficient mechanism for revealing the true, complete information held within the group.
- Markets must be efficient to accurately reflect the information available within the market.
- Not all market participants need to have accurate or complete information, so long as the market is efficient and the market, collectively, holds complete information.
- These conditions are necessary for a prediction market to provide a distribution that is well-calibrated with that of the actual outcomes.
- Even a perfectly calibrated distribution, based on perfect, complete information may not be useful for predicting an outcome. This is particularly true when dealing with discrete outcomes.
- A prediction market is even less likely to be useful, when there is a significant randomness inherent in the process of generating an actual outcome.