I agree with your summary. In addition, I think prediction markets offer several additional benefits, including: faster predictions, continuous updating, a measure of uncertainty surrounding the prediction, and reasonable cost. I’m sure there are a few more, but for now, this should do.
I am strongly in favour of using prediction markets to complement existing forecasting methods, especially where they are able to quantify uncertainty. Your example of project milestones is another excellent use for prediction markets.
Although your research indicates that as few as 15 participants can achieve calibrated results, I am skeptical. All of the market maker mechanisms will ensure that there is market liquidity, but I believe they influence trading behaviour (generally making traders quite risk-seeking), which undermines the accuracy of the predictions. Such market maker mechanisms were designed to allow markets to operate with smaller numbers of participants, but doesn’t this degrade the “crowd” precondition for successful market predictions? You aren’t likely to have a very diverse group with a small number of participants.
The basic theory behind prediction markets is that each trader has a piece of information combined with an error factor, and the aggregation method adds the pieces of information together, with the error factors cancelling out (more or less), resulting in more, accurate information. Having a smaller number of traders not only means having fewer pieces of information to aggregate, but also, the error factors will not “cancel out” properly.
As I see it, one of the major stumbling blocks is getting enough people to be interested in each market (and staying interested). The greatest benefit of prediction markets comes from forecasting the outcome as far in advance as possible, but this runs counter to the traders’ need to know the outcome “immediately” (or at least soon). You can see this in most marketplaces where very few people trade in long-term markets. Most trade in the markets that will close in the next few hours, days or maybe a week. These very short term market predictions have little value to a decisionmaker.
If prediction markets are to gain more widespread acceptance in the business community, they will need to focus on ways to ensure their predictions are useful. That is, by providing predictions well in advance of the outcome, accurately (diverse crowd, properly motivated) and with a very reasonable cost.
Much more work needs to be done in the area of motivating traders.
Just my thoughts for now.