The New York Times blog 538 keeps a running tally of a carefully designed model it uses to predict the popular and electoral vote based on recent polls. As of this afternoon, the model states that Obama has a 63% chance of winning re-election.
InTrade is a political predictions market in which people invest real money in futures of whether a particular event will happen or not. The thought is that knowledgeable people with proper incentives end up being pretty good predictors of discrete events. Right now, InTrade is predicting about a 53% percent chance of an Obama re-election.
From where I sit, the difference between a 53% chance of something happening and a 63% chance of it happening are pretty substantial. So, which prediction do you think is more accurate, and why?
Intrade seems to basically just track conventional wisdom, which in turn (somewhat illogically) seems to track polls. So if I had to bet, I’d bet InTrade is underestimating the chance of an Obama victory by a few points.
I’d agree. 538’s predictions seem to come from polls. InTrade’s prediction seem to come from how people feel about the polls. One has one less layer of subjective thinking than the other.
It comes down to whether or not you think polls are an accurate gauge of what people really think and how they will really vote (or more precisely, what people really will think and how they really will vote in November).
Silver’s model is as objective as he can make it, which is pretty damn good; but it also precludes him from making judgements about “people are saying X, but they don’t really mean it.” That kind of perceptive judgement is real, and some people base their whole careers on it (though usually at the individual, not societal, level).
Of course, we don’t know if intrade bettors are more or less perceptive than the average person. If 90% of them were ad execs, psychologists and car salesmen, I’d give it a lot of weight. As it is … maybe, maybe not.
538’s economic variables don’t weigh the odds of a downturn sufficiently. Economic forecasters tend to provide median forecasts, which don’t reflect events like, say, Greece leaving the Euro. More prosaically, forecasters aren’t especially talented at predicting recession. I opine that InTrade’s take is closer to the mark. I suspect that during more typical economic conditions, that the two sources would agree more.
The other unmeasured factor is Romney’s Super-Pac money. I discount that, or rather I think it’s offset by the personal unattractiveness of the presumptive GOP nominee.
Finally, InTrade incorporates 538’s forecasts. The opposite is not the case.
ETA:
The difference is worthy of a thread, but offhand if one person thinks Obama’s odds are 50-50 and another thinks they are 60-40, then I really don’t believe they disagree too much. If they are loss-averse a 55-45 bet would not be attractive to either.
Does InTrade actually take money? Because if they do, the odds are just as much about providing a market with a roughly equal number of people on each side, as they are about predicting who wins. That said, you would expect their money odds to skew more towards the underdog since that will even out the betting.
There have been times in the past when Intrade’s odds on political events have fluctuated wildly due to the actions of one individual bettor, or a small number of them. The concept of crowdsourcing information, which a betting site like Intrade depends crucially upon, breaks down when a single outlier can swing the results so far. And you’ve also got to consider the demographics of the crowd that Intrade is averaging over: There’s a natural tendency for people to overestimate “their side”, and Intrade probably attracts a more conservative audience than the country as a whole.
brickbacon, the way that InTrade is set up is as a forum for bets between private individuals (with the site taking a cut, of course), so they inherently have the same number of bettors on each side. The prices that those individuals agree on can then be straightforwardly converted into probabilities, if you assume that the bettors are rational and that they collectively have access to the true probabilities.
As I understand it, the theory that election-oriented prediction markets are based makes two pretty questionable assumptions:
[ol]
[li]Everyone who participates is betting sincerely (i.e., in line with their actual beliefs).[/li][li]The participants are representative of the electorate as a whole.[/li][/ol]
The latter is definitely an issue here, and the former is somewhat questionable given the money that’s up for grabs. Nate wrote a short piece about irregularities at Intrade during the 2008 election which is worth taking a look at.
If I claim that an event has a 90% chance of happening and it doesn’t, what’s my track record?
But the participants aren’t expressing their preference in the election market. They don’t have to be representative, they only have to be smart, at least collectively. Of course if there is partisan bias in their judgments, then the composition of the population will matter, absent “Smart money” that can take advantage of such bias.
Bias probably exists: according to Intrade Ron Paul has a 1.9% chance of being elected President. Even given speculation of a third column of RP delegates, that seems high.
Intrade is subject to wild swings. But those typically self-correct as profit takers push prices back to a more reasonable zone.
Intrade thinks that the odds of neither Romney nor Obama being elected is currently 3.8%. The punters seem to give a hefty weighting to wild-card events.
I understand that there is a literature devoted to scoring algorithms. Presumably a substantial sample of predictions is required.
Yes, 538’s methodology is fairly mechanical. This is not necessarily a bad thing but it does mean that it’s not really good incorporating something like a belief that there is a 30% probability of a complete Euro meltdown which triggers a financial crisis in the US. If you look at Intrade’s history, Obama was at around 60% until the last bad jobs report. This lends support to the idea that Intrade investors have a more pessimistic view of the economy and how it impacts Obama.
Having said that my personal view tends towards 538’s; I think Obama has a 60-65% chance of winning. I think the Greek election and the new bailouts have seriously reduced the probability of a Euro catastrophe in the next six months. It may be a case of kicking the can down the road but that’s all Obama needs from an electoral perspective. I think a mediocre economy is already priced into his poll numbers so it won’t really hurt him more. A silver lining for him is that a mediocre global economy will lead to lowish oil and gas prices something which is already happening just in time for the US summer driving season.
The other big variable: IMO Obama is fundamentally a better campaigner than Romney something that will make a big impact when the actual campaign heats up in the fall. Romney made an awful lot of unforced errors during the primaries against weak opponents. I expect similar or even bigger errors in the heat of a close general election against a much better opponent.
I think money will largely be a wash but organization could be another moderate advantage for Obama. I have no doubt that Romney will build a fine organization but Obama’s team have done it all before and will probably have a much richer data set from 2008.
Finally I expect Obama to continue using his agenda-setting powers as President to box Romney into awkward positions just like he did with immigration.
From what I can gather, 538 predicted that Obama would receive 349 EVs in the 2008 election, and InTrade predicted 364. The actual was 365. 538 wasn’t around in 2004, but I have read that InTrade was correct on its predictions for every state. 538 was correct on every Senate race in 2008, but I’m having trouble finding other data on Senate races.
InTrade is not infallible, of course. It seems that InTrade predicted Obama would win New Hampshire in the 2008 primary, it appears by a significant margin, but I can’t find the exact numbers. Here and here are some Beltway Bandit types discussing the pros and cons of InTrade, although not in great depth.
I’ll lay it out there: I think the InTrade model just isn’t good. Although I follow politics closely, I believe I’m more knowledgeable about world events, and the predictions made on InTrade about the probabilities of various events (e.g., bombing Iran) are almost ludicrous in their lack of connection to reality. I’m exaggerating here, but it as if Obama mentions Iran in a speech and the market suddenly spikes with John McLaughlin-like metaphysical certainty that Iran is going to get bombed in a few days… until it doesn’t happen. Basically, I think the main customers of InTrade don’t know very much.
However, in this case, at this point in the race, I think InTrade is a better reflection of where the race stands. Not that political prognosticators are infallable either, but Charlie Cook, the highly regarded analyst, recently called the race between Obama and Romney a “coin flip.” If I were to put a number on it, I feel like 55% chance of Obama re-election is about right, right now.
The stock market reacts wildly to any rumor or poll or election or event that might even conceivably have some effect, good or bad, on any business interests. Investors’ anxiety about/hopes for their money seems to be an enormous psychological force-multiplier. It creates a situation where a given stock’s market price at the moment sometimes bears no conceivable relationship to how well or badly the corporation is doing at the moment.
Not really. There are fairly standard methods of measuring goodness of fit in binary models. For example one can calculate the correlation coefficient between predicted probabilities and outcomes. E.g. you create a variable with value 1 if the event happens and 0 if it doesn’t and then calculate the correlation with the predictions you make. This is more fine-grained than just looking at correct predictions because for example you score higher if you correctly predict something with probability 90% than with 60%. Of course as mentioned by MoM you need a reasonable sample for such goodness of fit measures to make any sense.
I don’t think there is a large enough sample of common predictions by Intrade and 538 but in a few election cycles it should be possible to calculate which one performs better.
Perhaps it’s a risk measure? Given the odds, such an event could shut them down, while 538 would probably just readjust its models. It’d be in their interests to under-estimate any third party populists opposed to gambling in any scenario, since they’d lose out whether they were elected or not.
Not directly on topic, but InTrade blew it on the health care mandate. For the past month, the bettors at InTrade believed that there was greater than a 70% chance that the mandate would be unconstitutional.