Never mind Nate Silver, "Yahoo Signal" should get the biggest props

Politics & Elections
21

In your last post you said that was what Silver predicted. It is not what he predicted.

That’s not an accident: I don’t think caveats and hedges are a bad thing, as you seem to. I think you are giving Yahoo Signal extra credit for not hedging because it makes them look more confident, and I’m saying you shouldn’t do that. Silver could’ve deleted all the hedging and just said 332 EVs for Obama was the most likely outcome, but it wouldn’t make his predictions more or less reliable - he’d just be relating less information. That’s a false kind of convidence. One of Silver’s virtues as a public science figure is that he explains how he is doing what he’s doing and why he’s doing it and why he could be wrong. Yahoo Signal didn’t do that, largely because they were doing a one-off prediction and not a daily tracking of the polls, and that’s a different way of evaluating how the election might turn out. You could bash Silver for hedging or you could look at it as additional context. In the end all of these outlets came up with very similar results. I think it could have been enlightening if Yahoo Signal had explained some of those issues with its own model, but they didn’t.

But if you wanted only the most likely outcome, that was it.

I don’t really care what goes in your book as a prediction. According to Silver’s model Obama was more likely than Romney to win Florida and the most likely outcome of the election was 332 electoral votes for Obama and 206 for Romney. All of that was correct.

Wang’s modeling was very similar. At this point I think you’re just judging this arbitrarily.
The model said 332 EV. Unlike Yahoo Signal and Wang, that model returned the correct

22

That was supposed to say that Silver got Florida right while YS and Wang didn’t. They all agreed the state was a tossup.

Reading Wang’s final prediction again, he said his prediction of 303 EVs had “a 22% chance of being exactly correct.” That’s pretty much the same probability as Silver’s forecast of 332 EVs. And like Silver, he calculated a range of probabilities. For the most part they did the same things and came up with similar results. I know Wang criticized Silver for adding extraneous information and perhaps hedging his bets a little, and the strength of Wang’s results in the popular vote and the Congressional races suggests he had a point. But there’s not a whole lot of daylight between the two of them and it sounds like you are criticizing Silver for doing things they both did.

23

Except for the fact that it does.

24

Yahoo Signal deserves props in my view. And Nate Silver’s model was over-rated: taking a simple average of the state polls in late October could have called the election.

That said, Nate deserves the most props. Not for his model -lots of people were modeling. But for his column, his frequent updating and user-friendly display of the probability of a Romney loss. Sam Wang did a similar exercise -but his probabilities bounced around more than I would like.

During the election, you would have done well to ignore most of the horse race reporting and just read Nate Silver and Charlie Cook. And the latter was quite frankly off his game in 2012, though at least he was numerate so that he could be read for balance. I trust that somebody can assemble a better model than Nate. But I know of no better reporter of the horse race and most of his competition is laughable.

During the next election, I aspire to focus more on policy.

25

This is splitting hairs to say the least. It was his “forecast”. From dictionary.com (emphasis mine):

Few people are as long-winded as Nate Silver, granted; but they did do a fairly extensive writeup–they didn’t just throw up a number and some charts or whatever. But more can be found here.

Snarky, I wasted some time scanning manually all down the sidebar. Then I realised a simpler way to check your claim: Ctrl-F, 332, Enter. “0 of 0”, no results found. So, again: the **number **332 (a three, followed immediately by another three, followed immediately by a two) does not appear on the sidebar. However, the most prominent position on that sidebar–at the upper left, right under the word “forecast”–is reserved for the number 313.0.

It’s ironic that the unexpected vehement opposition I’m encountering ITT is so reminiscent of the talk from the “skewed polls” crowd.

ETA: Measure for Measure, good post–just want to be clear that I wasn’t referring to you, even if I don’t agree 100% with every detail of your particulars.

26

I don’t know what you are trying to argue at this point. You said you understand that the 313 figure is just a mean, so I thought you understood it was not the result Silver expected from the election. There was no plausible combination of states that would have given Obama 313 electoral votes. According to Silver’s model the most likely total for Obama was 332, followed by 303 and 347. Wang said the most likely result was 303 and 332 was the next most likely. They had similar confidence levels.

27

It’s unfortunate that this number is so prominent in the sidebar, but he has explained the meaning many times over. It is not his exact prediction and he has never claimed it was. In fact he stated the exact opposite. But what is in the sidebar are the state by state predictions. Add up the associated electoral votes and you’ll find that the total is exactly 332. The 313 basically reflects the non-negligible possibility that one or more swing states that chose Obama would have gone for Romney instead. (Yes, I know it’s a bit more complicated than that.)

28

No, no it doesn’t. It’s an average. The senate forecast was 52.5 seats for the dems. It does not represent the non negligible possibility that half a person is voted in.

29

I’m quite aware that it’s an average. The fact that the average is less than 332 reflects the fact that while no result was more likely individually, of the other realistic results, it was more likely that Obama would do worse than he did rather than better.

30

Ok, I misunderstood what you were saying. We’re on the same page.

31

The 313 average pretty much reflects the odds of Florida flipping back and forth.

32

Actually, the most likely figure for the Senate was approximately 52.5 seats for the Democrats, counting the independent guy from Maine as a half a Democrat.

33

Are you sure about that? It doesn’t make a lot of sense- King isn’t half a Democrat; he was either going to caucus with the Democrats or not. Silver’s forecasts just dealt with “seats controlled,” and since King was always expected to caucus with the Democrats I thought he was counting him that way.

34

Yet 313 is significantly closer to 303 than to 332. How do you explain that?

And I’ll say again: Wang at least predicted a plausible number. To claim that Silver did as well, and that it was 332, is tortured logic when it does not appear in the “forecast” prominently featured on the sidebar. Calling that number’s prominence there “unfortunate” seems like a strange characterisation given that Silver presumably has complete control over that sidebar!

I think the 313 is in reality a pretty mushy figure that reflects the hedging I complained about–he wanted to be somewhere not too far away from whatever the final number turned out to be. Even the probability number was, especially before the last few days, kind of a have-it-both-ways deal. Does it really look, in retrospect, that Romney ever had any significant chance of winning? I am increasingly convinced this win was “baked in the cake” long ago, as long as Obama’s team did what they needed to do and the candidate didn’t self-immolate.

35

I just don’t get this characterization of hedging–how did he hedge? He shows his numbers, and the numbers line up with the final number he gave.

And why are you arguing numbers when the source of this page got the number wrong, yet you seem so amazed by them?

36

Dude, on that exact same sidebar he has a graph that shows the probability of every possibility of Obama’s EV total. The highest probability outcome according to his model was exactly right. To claim that Silver didn’t predict the actual outcome is asinine. I guess you can’t control+F a chart though so it’s not your fault that you didn’t see the completely obvious.

37

Very short version? The chance Romney would win Virginia. Silver’s model reflected a wide variety of different possible results, but basically, if you look at his most likely outcome (332 EV for Obama) and the next few most plausible outcomes (Obama 303, losing Florida; Obama 290, losing Florida and Virginia; Obama 347, winning North Carolina) you see they do hover in the neighborhood of 313.

Your objections have pretty much ceased to make sense at this point. It sounds like you don’t care for Nate Silver personally (which is your right, of course), so you’re complaining about a sidebar and his phrasing rather than his actual statistical work. The bottom line is this: Silver said the most likely outcome for the election was 332 electoral votes for Obama, and that was the outcome of about 20% of his simulations. Wang said the most likely outcome was 303 EV, and that occurred in 22% of his simulations. So their levels of confidence were basically identical. They both agreed Florida was just about 50-50, with Silver saying Obama would win it and Wang saying it would go to Romney.

I’m not sure what your issue is with this 313 thing. I get the sense you either don’t understand that it’s an average or you don’t want to treat it as one because you think it makes it look like Nate Silver didn’t stick his neck out and make a realistic projection. Except Silver and Wang both did the same exact things. They forecast a range of outcomes, said the final result would be very similar, and said there was about a 20% chance their most likely forecast would happen.

It isn’t.

And some people have criticized Silver for that kind of thing. I think Wang did, in fact. But that criticism was based on his overall forecast (91% chance Obama would win), not the 313 average - because other people also constructed averages. Wang said Silver added some noise to his model and hedged his bets a bit, but by Election Day, Wang said there was a 99% chance Obama would win and Silver said there was a 91% Obama would win. I’m not seeing that as have-it-both ways. Even if he did hedge, Silver got his share of criticism from dopes who said this election couldn’t possibly be anything but a tossup.

No, this is not how you do this kind of thing. You don’t go back and rewrite the election afterward to say nothing else could have transpired. That’s what pundits do, and you get a lot of garbage that way. Wang and Silver were analyzing polls of voters and basing their forecasts on analysis of those polls. At Obama’s lowest point in the polls, Silver said there was still about a 60% chance he’d win.

And you could argue that Wang and Silver’s work reflects this just as Yahoo Signal’s does; they always gave Obama a much better than even shot at winning the election. Late in the game Silver even had a blog entry titled something like “The race returns to the fundamentals,” meaning that in late October the polls looked pretty much the same way they did in June.

38

313? Really?

I’ll just redirect you to my earlier comments rather than repeat myself in arguing why that was an impressive prediction. I will say though that what you are saying (even if I were to concede, for the sake of argument, that 538 got the number more accurately than Yahoo Signal–which I don’t) amounts to being more impressed by someone going five-for-five from the free throw line than by someone going four-for-five from half court.

Not at all correct. I was nothing but a 538 booster until the last couple days. You can look at my Twitter feed (@SlackerInc) for confirmation, or at the comments I posted at 538 in the last week before the election, which got enough reader “mojo” to appear in the top 10 or 20 if you select “Reader Picks”. For that matter, just look at what I posted here the day after the election, in the “Nate Silver was right” thread. But even in that thread, you’ll notice that I made it clear in a subsequent post that I put a high value on predictions made early, months before Election Day.

So call me fickle if you like, but now that I’ve rediscovered the Yahoo Signal prediction, Silver just doesn’t look nearly as impressive any more. Furthermore, as YS was not based on actually asking people who they would vote for, it would be fair to say that I’ve had a paradigm shift as a result. It now seems like Silver and his ilk are just amalgamating data that describes what people are actually doing (planning to vote for a certain candidate) rather than digging down under the surface and figuring out how to predict what they *will *do, before they even know they’re going to do it. I find the latter much more impressive, and far more interesting.

The reality is that I think I’m just more immune to confirmation bias than most people; I don’t stubbornly cling to shibboleths if there is evidence going the other way. If I see new information that undermines even long-held beliefs, I will turn on a dime. And that is what has happened here, simple as that.

Irrelevant, a straw man really. No one here (I suspect) is sticking up for Joe Scarborough. I certainly am not: I’ve disliked that blowhard since he was in Congress. (I will say that at least the “skewed polls” guy took his lumps and said flat out that he had been wrong.) This is, or should be, more like an argument about Dawkins vs. Gould rather than Dawkins vs. [crazy Creationist of your choice].

Maybe you don’t, but please don’t tell me what to do. Kthx.

To be clear, though, I didn’t exactly say nothing else could have transpired. I said, essentially, that as long as Axelrod and Plouffe did their jobs, and Obama made no major missteps (more major than the Denver debate), and the economy didn’t crater, Romney’s campaign team faced a challenge about equivalent to that the coach of my local Div. II football team would have in trying to lead his squad to a victory against the St. Louis Rams. Theoretically possible given a perfect storm, but very, **very **unlikely.

60% is “much better than even”? That’s somewhat subjective, but we’ve discussed on this board before (referring to weather forecasts) that people have a tendency to overrate percentages in the 60-90 percent range, and it sure looks like you’re guilty of that here. I’m a poker player, and there are many situations in slow structured, deep stack tournaments (where, just like in elections, you don’t get a second chance unless it’s a rebuy) that I’d pass on putting all my chips in with what looks likely to be a 3-2 edge for me, and wait for a better spot or a later, more desperate situation.

39

He did. This is really not arguable.

That’s exactly what they are doing.

Like I said a bunch of times, Silver/Wang and Yahoo Signal did different things. They made predictions based on different types of data. It’s fine if you find one more interesting than the other, but if you think everybody should abandon analysis of polls to focus on economic data, I think that’s a little myopic. We’re better served to have people analyzing a wide variety of data.

Based on this thread, I am not sure about that.

It’s relevant because you’re criticizing Silver for hedging when he always had Obama as the winner - with confidence of more than 90% on Election Day. It sounds like you’re saying 99% is sticking your neck out and 91% is hedging your bets. I don’t think that makes sense.

I agree, but I don’t think that’s how you’re addressing it. Your criticisms of Silver compared to Wang have been totally inconsistent, and while I’m no math PhD, you seem to be struggling with the concept of an average and you’re putting more emphasis on the layout of the 538 blog than what Silver’s model actually forecast.

No, I don’t, because that isn’t a good way to evaluate things.

You’re in the million-to-one shot range here. I don’t think Yahoo Signal or Silver or Wang would go that far. Many people (me among them) would have told you that the odds favored Obama as long as there wasn’t a major disaster and the economy continued to slowly improve, and that is pretty much how it played out. That doesn’t mean Romney never had a chance and it doesn’t mean you can go back and discard everything else, nor does it mean that non-economic analysis is no longer useful. People have been making forecasts based on the economy since forever - some forecasts are good and some aren’t. Yahoo Signal’s forecast was certainly good, but it does need to be viewed in context of a broader body of predictions. That’s why I wish I could do something with the state-by-state forecasts. And I don’t think they said anything about other elections, which helps in looking at Wang and Silver’s work.

40

I think people are missing an important point, here: Even though Yahoo and Silver/Wang used two completely different methods to get their answers, the answers all three of them got were very close to each other and to the real world. This is a sign that all three of them did what they were doing pretty well. We sometimes refer to the reality-based community. Well, one of the hallmarks of reality is that it’s consistent.