Posted by: Paul Hewitt | April 5, 2009

Practical Enterprise Prediction Markets

Lately, there has been a lively discussion on-line regarding the slow adoption of prediction markets in the corporate world.  It seems that the major researchers and academics believe that it is just a matter of time until the corporate world wakes up and sees the incredible value of these markets.  Others, like Chris Masse (Midas Oracle), are more than a bit skeptical.

At first, I was very optimistic about the value of prediction markets and their eventual highly esteemed place in the corporate forecasting world.  The logic behind the basic theory of prediction markets makes a lot of sense.  You take a “crowd” (lots) of people, each with his own set of information and opinions, let them make choices (independently), and aggregate those choices.  Each person holds a piece of information with an associated error factor.  The law of large numbers ensures that the aggregated error will be quite small, leaving a combined chunk of “information” that is better than any individual’s piece of information.  Designing sophisticated markets would be able to reveal not only the most likely forecast outcome, but also the expected distribution of outcomes (or uncertainty).  And, all of this could be done very cheaply.  Seemed like a sure winner to me.

There were two major stumbling blocks in the corporate areana – anti gambling and insider-trading laws.  These are still issues, but I won’t get into them here, because I don’t think these were the main reasons holding major corporations back from incorporating prediction markets in their forecasting processes.

Prediction markets have been available for many years, yet the number of publicized, successful implementations is really quite small.  Many have been run as short-term “pilot” projects, which rarely seem to achieve a permanent place in the corporate forecasting process.  When you consider that most of the major international consulting firms (McKinsey, et al.), leading academics/consultants (Hanson, et al.) and several prediction market software providers, it is really quite amazing that there are so few bona fide enterprise prediction markets.

Here are my thoughts as to why they haven’t caught on:

Failure to follow First Principles

Unless firms (and their consultants) fully understand all of the prerequisites (first principles) for proper functioning of a prediction market and make sure the implementation addresses all of these requirements, the market is more likely to fail or provide inaccurate predictions.

For example, prediction markets need a large number of participants (and diverse ones at that).  Several academics have come up with innovative methods of facilitating trades through market maker mechanisms.  These have provided market liquidity that allows prediction markets to function (i.e. facilitate trades) even with a relatively small number of participants.  It is a neat little “trick” to make the market seem larger than it is in reality.  Unfortunately, the market maker mechanism allows the “crowd” prerequisite to be violated.  In addition, a smaller crowd lessens the diversity of the participants, at least partially undermining another key prerequisite.  As a result, a smaller crowd has the distinct potential to compromise the accuracy of the predictions.

The various market maker mechanisms also introduce a market distortion, which influences trading behaviour.  More work needs to be done on this, but it is my belief that market scoring rules create highly lucrative potential trading opportunities.  Combined with a “play money” market (where there is little to lose), I believe this creates disproportionate incentives for traders to undertake very risky investment decisions.  Few companies operate with a high risk profile, which calls into question the use of predictions based on risk-seeking traders.

It is interesting to note that the various software providers promote the ease of getting started in prediction markets.  True, it is easy to set up a market using the software.  The difficult part is making it function properly.  The software is merely a tool for aggregating the traders’ opinions.

Public Nature of Forecasts

Judging by the types of enterprise prediction markets that have published results, it appears that many companies have not been focusing on serious, high value forecasting issues.  Perhaps it is the public nature of the resulting prediction that is holding them back.

For example, in many cases, management has a vested interest in creating a forecast for the “market” that may not bear much resemblance to the “true” forecast.  The (“public”) existence of the “true” forecast would undermine their promotion of the official forecast for public consumption by the markets.  A bad situation, I know, but there is more than ample evidence that this is widespread phenomenon.

Existing forecasting practices utilize senior management and consultants to determine the official forecasts.  This group of strategic planners can be trusted to keep the forecasts confidential.  Prediction market forecasts are much more widely known throughout the company.  Most often, the forecasts are based on what they need to show, as opposed to what they might reasonably expect.  Then, of course, the forecast (budget) is pushed down to the lower levels to do whatever is necessary to hit the numbers.  As we see (rather frequently), this often results in many seriously wrong actions taken within companies.

If management is mildly concerned about prediction market results becoming public, it is highly unlikely that they will tackle the most important forecasting issues in this manner.  Perhaps the best way to break into the market is to operate in parallel with existing forecasting methods until prediction markets prove their worth and companies figure out how to minimize the public disclosure of these forecasts.

Practical Usefulness Issues

In order for companies to incorporate prediction markets into their forecasting systems, they need to prove their usefulness.  I think it is obvious that prediction markets have the potential to be extremely useful in this regard, but it is all in the implementation.

As discussed above, software companies make it sound so easy to implement a prediction market, but that is only a small part of the process.  There are a number of major issues that make it difficult to implement effective prediction markets, and the literature has not been particularly useful in resolving them.  While many of these issues have been raised in the literature, the discussions have been very general and sorely lacking in the practical implications.  I guess that’s where the consultants come in, but it also means that a great deal of education is required in order to “sell” the concept.  This needs to change.

Advance predictions & Incentives

In order to be useful, an accurate prediction must be determined well in advance of the actual outcome.  It makes little sense to run a market where you obtain the prediction just before the actual outcome occurs.  This sounds obvious, but it is actually quite difficult to achieve, because traders want to know how their “investment” (bet) turned out, fairly quickly.  This runs counter the the corporation’s need to know the prediction in advance.  So, innovative incentives have to be designed to encourage traders to adopt patient investment strategies and be rewarded for investing in longer-term outcomes.  Not only do they need to make investment decisions well in advance, as new information becomes available, they have to be encouraged to continue trading in the market.  This provides corporations with dynamically updated predictions, which yield valuable information on trends, level of uncertainty, and may indicate the strength of various factors influencing the outcome.

Sufficient, appropriate traders

As discussed above, companies need to have a sufficient number of traders for each market, to ensure that the “crowd” prerequisite is met.  Where necessary, these traders will need to be trained in trading on prediction markets, and the incentive systems need to be explained (and preferably tested), to ensure appropriate trading behaviour is encouraged.

Focus on Valuable Variables

Management needs to determine those conditions, events and actions (variables) that are most valuable to predict, and they must know what to do with the resulting prediction when it is determined.  Again, this sounds obvious, but it isn’t something that can be determined in a few minutes (as suggested by several of the software providers).

Dynamic Analysis

One of the major benefits of prediction markets over other forecasting methods is that they provide a built-in mechanism for continuously updating their predictions.  Assuming the appropriate incentives are in place to promote continuous trading, movements in the prediction over time provide valuable information to management.  Similarly, the distribution of “investments” in the prediction options provides a measure of uncertainty in the outcome, and changes in the distribution will indicate changes in uncertainty, providing management with an early warning system for evaluating forecasting issues.

Bottom Line

I do think that enterprise prediction markets will eventually reach a tipping point, but a lot of work needs to be done.  The academic literature is good, but it is becoming too technical and theoretical.  This has to scare the corporate types.  The focus needs to be on practical implementation issues.  It needs to get away from sweeping generalizations with respect to implementing prediction markets.  Consultants need to step up and focus on rigorous implementation planning that never forgets the first principles that make prediction markets work.  Then, we can be useful helping forward thinking executives run their companies better.

Comments?

About these ads

Responses

  1. [...] Lean Execution added an interesting post on Practical Enterprise Prediction MarketsHere’s a small excerptThis provides corporations with dynamically updated predictions, which yield valuable information on trends, level of uncertainty, and may… [...]

  2. [...] that prediction markets have improved the accuracy of forecasts, but the improvements have not been great enough to encourage widespread (or even minimal) acceptance. Furthermore, these studies like to average their results over a number of markets, disguising the [...]

  3. [...] Link: Paul Hewitt’s blog on prediction markets Posted in All Best Posts Ever, Analysis (Accuracy & Precision), Analysis (Meta), Collective [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: