Posted by: Paul Hewitt | March 15, 2012

Status Quo Predicting

If you’ve been reading my blog, you know that I am taking part in the Good Judgment Project, where a number of us are predicting future world events.  Now, I wouldn’t consider myself to be all that knowledgeable about world events, but I am doing quite well in predicting outcomes.  In fact, I haven’t got any prediction wrong, except for one fluke that caught almost everyone by surprise.  So much so, that the Project excluded this question from the rankings.

While I may not be an expert in world events, I do have some knowledge in most areas, which I gain by reading The Economist every week, reading local papers, and watching CNN.  So, how am I doing it?

I think the main reason I have been successful is my perspective on the world in general.  We have experienced a huge recession (still are in most parts of the world), financial turmoil in the EU, a U.S. election year, and the Arab Spring.  As a result, except for a few specific instances, I have predicted that most events will not occur.  That is, the status quo will prevail.

Here are a few examples (percentages relate to likelihoods of the status quo outcome):

Will Saif al-islam Gaddafi face trial before March 31?  I bet no at 13% and it now stands at 86%

Will a civil war break out in Syria before April 1?   I bet no at 37%.  It’s now at 69%

Will South Korea announce a policy of reducing Iranian oil imports by April 1?  I bet no at 23% and it’s now at 76%.

Will Asif Ali Zardari lose the Presidency of Pakistan by June 1?  I bet no at 29% and it is now at 41%.

Will the U.N. Security Council pass a resolution regarding Syria by March 31?  I bet no at 27% and it’s now at 69%.

Will the U.N. Security Council pass a resolution regarding Iran before April 1?  I bet no at 41% and it is now at 86%

Will Greece remain in the EU at June 1?  I bet yes (status quo) at 50% and it’s now at 82%.

Most of my other no “bets” were placed at 50% and are now in the 75% – 94% range.

The only exception to the status quo bets was a major bet I placed on whether Wade would be re-elected in Senegal.  I bet that he would not be re-elected at 18% and it is now sitting at 80%.  I decided against the status quo on this issue, because Wade is old (he could die), there had been a lot of political unrest in Senegal, and if Wade did not receive at least 50% of the vote during the election, a run-off election would be called to decide the winner.  In my opinion, Wade will lose the run-off, as the opposition candidates will come together and elect Macky Sall instead.  It’s looking better every day.

Based on this anecdotal evidence, it appears that many of the participants in my group display an initial bias towards change, which lowers the early likelihood of the status quo outcome.  In fact, it appears that most events will not occur before the deadlines, meaning that there should be a natural bias towards the status quo.  By betting the status quo after the early “chimps” have bet on change, one can be very successful, indeed.  I expect to rocket up the leaderboard once the results are announced in early April.  I’ll let you know.

Update April 5, 2012:  Well, I did do quite well on the questions that closed around the end of March, “earning” about $40,000.  This “rocketed” me up the leaderboard from #13 to #9.  The top predictor broke past $100,000 in winnings.  Once you get behind, there’s no way to catch the front runners, because no one loses a bet in this game (at least they shouldn’t).

About these ads

Responses

  1. The percentages you provide suggest that the “Crowd Belief” shown is restricted to one’s group, not the entire “team”. In my group, current status quo predictions are:

    Saif al-islam Gaddafi trial – 69%
    Civil war in Syria – 56%
    South Korea reducing Iranian oil imports – 66%
    Asif Ali Zardari losing the Presidency – 47%
    U.N. Security Council resolution on Syria – 80%.

    And so on – obviously different. In my group, we are not on a “market” and we were not given any formal training. I got a sense that at least one other group does not even have “Crowd Belief” shown. It would be really something if organizers totally fake “Crowd Belief” bets for some of the experimental conditions – I find it hard not to pay any attention to it and it would be really interesting to see how strong an effect herd mentality has in this case.

    How does your group’s setup look like?

    It has been apparent for quite some time that status quo betting is successful strategy. But in large part it is so because of the early deadlines.

    • I described most of the design setup in my earlier posts – The Good Judgment Project, Good Judgment Project Performance, and Prediction Market Design Issues, on this blog.

      The organizers don’t really tell anyone much about how and why they are conducting each group’s forecasting. Mine is some sort of modified prediction market. You can’t “sell” an investment to make a gain prior to the closing, but you can “withdraw” a trade and get all of your money back. Odds don’t seem to update with each trade. I think they update the odds once or twice a day, based on cumulative trades.

      I don’t know how your group works, but I find it very odd that your group’s odds are much different from my group’s! We’re all dealing with the same information. The odds should be very similar. This is a typical problem with prediction markets. I wrote about this in Why Public Prediction Markets Fail.

      Most markets show a lot of uncertainty until near the close of the market. I think there is a significant herd mentality that takes place, here. Traders who are on the wrong side see that time is running out and they withdraw their bets, increasing the odds for the correct outcome. One has to question the accuracy of the predictions for any significant time period prior to the market close. This has been a major criticism of mine about the accuracy of prediction markets in general.

      Thank you for your comments. If you’d like to post details about how your group is set up, I think everyone would appreciate it.

      • I don’t find it odd that our groups’ odds are different. They are not *that* different, with overall direction being the same. First, N is relatively small, 200-250, and the effective N is even smaller because of the certain percentage of “dead wood” (the kind of people who don’t seem to be actively involved, still giving 5% to Medvedev to be inaugurated president or 18% to Le Pen dropping out). Second, and perhaps more importantly, I understand that psychologically we are in a very different positions. You are on a market, buying and selling positions, earning “money”, etc. We don’t have any of that. I might be wrong but I imagine that your group suffers from the “frequent trader” syndrome. Of course, it’s also possible that my group, where we have nothing to lose or gain, suffers from being negligent and lax. In the crudest from this can be compared looking at final scores for the closed questions. I list some of them below for comparison.

        Here is how my group works:
        We were not trained in any way and we are asked to evaluate the odds as best as we can. We also must provide “an honest assessment of your knowledge of this question”, on a scale from 1 to 5 (this is difficult to judge to me). The vote can be changed at any time and we are urged to update the odds whenever we feel that they change (it’s an equivalent of your “withdrawn my bet only to repurchase the same position with a better payoff” except that re-purchasing a position costs nothing). By default, “Crowd Belief”, an average of the groups’ current evaluations, is prominently displayed on the right. It is my understanding that for days when one does not provide the odds, the group’s average is assigned. One can click a button to “withdraw” an answer and, again, from that point onward, group’s average is used in place of the actual answer. Since we have no money, that’s all that it does – effectively a decision to stop betting (but past performance is not erasable). That’s about all there is. Other than the final Brier scores and ranking, there is no feedback and no interaction with others.

        Here are my group’s Brier scores for the last 20 closed questions:

        #1028 Moody’s Greece downgrade? 0.85
        #1030 UN Security Council on Syria in October 2011? 0.30
        #1031 U.S. Congress disapproval F-16 deal with Taiwan? 0.07
        #1032 Japanese announce jet fighters decision 0.45
        #1033 Tunisian Ennahda announce interim coalition government 0.94
        #1034 Japan becoming a member of TPP? 0.64
        #1038 Hamad bin Isa al-Khalifa remain King of Bahrain? 0.06
        #1039 al-Assad remain President of Syria through 31 January 2012? 0.25
        #1041 Berlusconi vacate office before 1 January 2012? 0.14
        #1042 Will Papademos be the next Prime Minister of Greece? 0.22
        #1043 Lucas Papademos resign before 1 March 2012? 0.33
        #1044 UK Tehran embassy 0.15
        #1048 S&P downgrade of EFSF 0.43
        #1060 Christian Wulff 0.83
        #1066 Yemen’s next presidential election? 0.54
        #1070 Israel release Aziz Duwaik? 0.20
        #1072 Serbia EU candidacy 1.17
        #1074 Libyan government regain control of Bani Walid 0.40
        #1075 Russian presidential run-off? 0.13
        #1077 Egypt lifting travel ban on Americans 0.59

        An average is 0.43, just barely better than 0.5 hedge (although I disagree with appropriateness of using unweighted means in this case). Just how different are they from your group’s?

  2. Oh, one more comment: your bets of 13%, 29% and so one totally crack me up. Such a precision! As if there is enough information on any of the questions to distinguish between 13 and 15, 29 and 30. All of my probabilities are rounded to 5 and even that, I feel, is quite a bit over what I can rationally evaluate.

    • These percentages are provided by the prediction market, based on trading. I agree that it is hard to understand what 13% means when you are making a trade. However, the lower the percentage the higher the eventual payoff, if you are correct.

      Also, I don’t make trades based on whether I think the true percentage likelihood is 25% instead of 13%. I make the trade, based on whether I think the outcome will actually occur. I do seek out low percentage markets that I think are incorrectly predicting the eventual outcome.

      Also, in our group, you have to constantly monitor the markets, because the odds do change. Many times, I have withdrawn my bet only to repurchase the same position with a better payoff.

      • One more thing. Now that I saw your old posts and how your group’s betting works, I should more clearly explain how it works in my group: We simply give % to Yes and No for a total of 100. For multiples like #1079 (Venezuela primary), we simply distribute odds: a) 3%, b) 90%, c) 3%, d) 2%, e) 2%.

      • Now I see what you mean by the percentages. That seems to be a very odd way of predicting, although, if you can predict one alternative accurately, why wouldn’t you be able to predict the others (in a multiple-option market).

        I see that you rate your own knowledge of the subject as a proxy for how sure you are of your prediction. We, on the other hand, show our confidence by placing higher bets (subject to our available funds).

        The advantage of our system is that by keeping track of our success with cumulative earnings, we are weeding out the “chimps” and giving more weight to the “experts” (ha ha). Which begs the question, how do you know how well you are doing? Where are your incentives for gathering information to improve your predictions?

  3. […] change bias, Good Judgment, Hewitt, prediction markets, status quo, TGJ Blogger Paul Hewitt presents his strategies for doing well in the Good Judgment project, one of the other ACE teams. In particular, he […]

  4. Paul,

    I am in your group, and I think it will be hard for you to rocket up the leaderboard once the questions close in April. The rational strategy in this type of market is to have all your money bet, and you simply haven’t won enough yet to keep pace with people who are betting larger amounts on each question. The only real question you could make much money on (listed above) was Gaddafi, and I am guessing you didn’t overweight that bet at that time because it was still uncertain.

    Its easy to make capital once you have capital, and the leaderboard represents a race for capital which is ever accelerating as bet amounts grow.

  5. I usually have all of my money invested. Of course it will be hard to catch the leaders, with their large balances, but I do stand to “earn” over $39,000 on April 1, if my investments work out. Since that is more than double what I already have, I think I will “rocket” up the leaderboard. I got a late start in the game, losing the opportunity to build funds at the start, unfortunately.

    Part of the value of prediction markets (and this is a variant of one) is to identify the top predictors. Unfortunately, there are a few abnormalities that make it possible to build large wins without knowing too much. It’s not exactly an efficient market mechanism.

  6. 39K would be a solid month of performance. I do think though that we’ll see the leaderboard change a bunch at this close, I think there have been some opportunities to go over 100K for the month with these questions with the right betting strategy, and I suspect some people might have some very lucrative bets out there. We’ll see what happens.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: