Posted by: Paul Hewitt | May 5, 2009

The Future of Prediction Markets – Part I

We would like to be able to run a prediction market to predict the future adoption of prediction markets (public or private), but we can’t.  There is no way to verify the outcome to determine which option would pay off.

Based on my research to date, this is where I think prediction markets are heading and where I think they should be heading.  In this paper, I will focus on Enterprise Prediction Markets.  A subsequent paper will cover Public Prediction Markets.

Private (Enterprise) Prediction Markets

In my view, these provide the most promise for future adoption, despite the almost insurmountable problems they have gaining acceptance in the corporate setting.  I am optimistic, however, because I believe prediction markets do have the potential to be better predictors of the future than other forecasting methods, at a lower cost.

My review of the literature and case studies (that have been published) indicates that prediction markets have improved the accuracy of forecasts, but the improvements have not been great enough to encourage widespread (or even minimal) acceptance.  Furthermore, these studies like to average their results over a number of markets, disguising the fact that some markets improve forecasts, while others fail to do so.  Some studies look at average absolute errors, covering up the fact that some predictions were underestimating the true outcome and others overestimating it.  This means the real errors are as much as twice as large as those reported.  Few, if any, explanations for the failures are ever presented. This raises the issue of consistency.  In case studies such as these, where there is no clear under- or over-estimation tendency, for which a correction may be made, the prediction errors are just too great.

Clearly, if similar prediction markets do not provide consistently accurate forecasts, they will not be relied upon for any important business decisions.

Businesses make estimates and forecasts in virtually everything they do.  Every decision model accepts inputs which are estimates, predictions or forecasts of likely scenarios for future conditions, events and actions.  Decisions made are only as good as the model used and the accuracy of the data being used.  “Garbage-in, Garbage-out” doesn’t just apply to computers.  There is a clear profit incentive for companies to improve their decision-making, by improving the quality of the data relied upon.  Traditional forecasting models have a spotty track record for accuracy.  Prediction markets may be a good alternative to, or add value to, traditional forecasting methods.

To be useful in the corporate world, prediction markets must provide forecasts that are more accurate than traditional methods, or be a cheaper alternative of providing equivalent forecasts.  Only if this pre-condition has been met, can we look at the other potential benefits.  It makes no sense to talk about how quickly or cheaply a prediction market gives a forecast, if the forecast is wrong!  Therefore, the focus must be on accuracy.  Let’s get it right, first.  Then, we can make it better or more efficient.

Once the accuracy and consistency issues have been met, prediction markets can be relied upon to provide a measure of the uncertainty surrounding the forecasts.  It does this with an objective distribution of “votes” around the mean prediction.  It is a particularly useful measure, with applications in risk management and contingency planning.

Assessment of Enterprise Prediction Markets (EPMs) to date:

  1. they have some ability to improve the accuracy of forecasts in specific situations;
  2. an ability to reduce (and measure) uncertainty of the forecast;
  3. perform a relatively fast aggregation of traders’ predictions, and
  4. are a relatively cheap forecasting method.

EPM Deficiencies:

  1. We don’t know why they don’t work in some cases (even with similar markets);
  2. Most forecasts are not significantly better than traditional methods (yet);
  3. They lack consistency;

Future Research (just a few):

  1. Prediction markets require a crowd of people, with as much diversity as possible, holding privately-generated independent information.  Future research must focus on how to achieve these characteristics.  Too often the research has focused on how to get around the need for a “crowd”, seemingly forgetting that reducing participation will also reduce diversity and completeness of the information contained in the crowd.  Mistake.
  2. We need to know the determinants of accuracy and consistency.  Find out what makes some markets work well, while others fail.  Find out why there is a lack of consistency in the predictions obtained from similar markets.  Then correct for these deficiencies.
  3. Find out which types of issues are best suited for prediction markets, and discard those that will never provide accurate, consistent predictions.
  4. Find out what makes a good “crowd”.
  5. Find out how to get a good crowd and keep them motivated to reveal their private information.

Of course there are many other issues related to EPMs, but I believe these are the crucial, must solve ones.  Without accuracy and consistency, EPMs will be nothing more than a novelty.

Advertisements

Responses

  1. […] Enterprise prediction markets have *no* benefits for businesses. Written by Chris F. Masse on May 5, 2009 — Leave a Comment Paul Hewitt: […]

  2. […] other traditional forecasting methods, and hopefully, make predictions at a lesser cost.  Here, reliability means consistency.  The same type of prediction market must consistently provide more accurate forecasts than other […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: