For those of you who don’t know, Nate Silver was the founder of 538 and devised the mathematical models which they used to make election predictions. After the 2022 midterms, he sold the brand, but kept the IP rights to the election model. The highly accurate model that we’ve known and loved since 2008 now resides on his Substack, which you should all subscribe to (the above link is to a non-paywalled post).
The 538 predictions are now done by a guy named G. Elliot Morris, who Silver clearly has little respect for.
The take-home point is that Morris believes that polls have very little predictive value until about 75 days before the election. Currently, the 538 model is based almost entirely on economic fundamentals and historical experience – meaning that all 538 is telling us at this point is that Biden is an incumbent and the economy is OK, so he should be in good shape. I think most of us feel that historical experience is unreliable as a predictor of this particular election.
In the last month, Trump has gained 2 points in the national polling average, but because some economic indicators have improved, Biden has gained a point in the 538 projections!
Silver’s conclusion:
As I wrote in the piece that introduced the Silver Bulletin forecast, there’s a fine line between an “objective” statistical model and “just some dude’s opinion”. All models inescapably include some degree of educated guesswork. But the 538 model falls somewhere on the wrong side of this line, in my view. It’s basically just Morris’s opinion that incumbency is still a powerful advantage, and you shouldn’t look at the polls until after Labor Day. He’s entitled to those opinions, although I think he’s wrong on each of those points. But intentionally or not, he’s designed his model in such a way as to be nearly impervious to contrary evidence.
This is important because people point to “the 538 predictions” as evidence that “Biden’s polling isn’t that bad” – but the 538 predictions aren’t even taking the polls into account!
Maybe once the polling kicks in their model might be worth something. But as of right now it looks like they’re just irresponsibly peddling hopium to Democrats who desperately want to think the race is a toss-up.
While there is a lot of science to polling there is also a great deal of art to it. I have a hard time judging these arguments. IIRC Silver got it wrong in 2016. Polling is difficult.
To me (and this is just my WAG) I would read a poll today as, “If the election happened today, this is what is likely to happen.” To me, its predictive value for what will happen four months from now is like predicting the weather four months from now. It may show each candidate some clues on where they are weak and strong but there is too much time to say for certain.
Again, just my WAG. I have no expertise in this beyond reading about it.
He got it “wrong” in that he predicted Clinton would probably win, but he only had her as a 2:1 favorite, whereas many other analysts thought she was at 10:1 or better. Technically, you can’t get a probabilistic forecast “right” or “wrong”, but he was certainly less wrong than most of his competitors.
So when polls and models show Trump leading it’s unquestionably true and it’s proof that Biden should drop out, but when polls and models show Biden leading it’s hopium.
It appears that you did not read the OP carefully. The issue is that there are not “polls and models” showing Biden leading, there are only “models”, and models with no previous track record to evaluate.
Well, sure, but the flaw of a “fundamentals” based approach is that it assumes that all elections are equal. For example, the 538 model, as I understand it, assumes that the polls will likely change about as much between now and November as they have in the average election since 1948. In almost all of those elections, though, partisanship was much lower and the undecided vote was much larger. As a result, 538’s model is only 95% certain that the final margin will be somewhere between Trump winning by 20% and Biden winning by 30%. If you are willing to stake me 1:20 odds that the margin will fall outside that zone, I will literally wager everything I own and can borrow.
The broader point, though, is that his model might turn out to be brilliantly correct, but it’s not polling-based, and therefore people shouldn’t use it as evidence that Biden’s polls aren’t bad.
Silver seems to have gone off the rails during COVID. Now he’s virtually a conspiracy theorist. He’s been a good statistician and modeler but a terrible pundit.
I have no opinion on whose model is better right now; we won’t know until the actual votes have been counted.
SILVER: I think people shouldn’t have been so surprised. Clinton was the favorite, but the polls showed, in our view, particularly at the end, a highly competitive race in the Electoral College. We had him with a 30 percent chance, and that’s a pretty likely occurrence. Why did people think it was much less than that? I think there are a few things. One is that I don’t think people have a good intuitive sense for how to translate polls to probabilities. In theory, that’s the benefit of a model. But I think people thought “Well, Clinton’s ahead in most of the polls in most states, and I remember that seems similar to Obama four years ago, and therefore I’m very confident that she’ll win.” It’s ad hoc and not really very rigorous, that thought process.
Explain how this demonstrates that “Silver got it wrong in 2016”. You may use the front and back of the page.
That is literally not what Silver (or anyone else) has said but feel free to misattribute and misinterpret at will.
I think not being able to attend poker tournaments in Las Vegas is what sent him over the edge but he’s always been a bit of an unctuous prick when challenged, and not the best at explaining probabilistic modeling (and the limitations thereof) to the general public. But in my opinion, FiveThirtyEight.com started really tanking when ABC forced the firing of Clare Malone, and I pretty much stopped paying any attention to it shortly thereafter. Regardless of what you think of Silver, at least he had a fairly consistent methodology for evaluating the quality of polls and integrating them into a predictive model; it is clear that it has now been completely hollowed out and replaced with a PDOOMA ‘model’ that even the creator admits is not of any real utility.
Regardless of how crazy he is, Silver is clearly a bad messenger for this because it’s extremely personal between him and 538.
However, his arguments are pretty easy for anyone to see even if you aren’t a professional statistician.
It’s hard to disagree with the overall point Silver is making here. And if the model relies that little on polling that this is possible, it’s probably just pointless to put out the model until it has more confidence in polling.
I don’t think there’s a conspiracy to overestimate Biden’s chances by the liberal media. I think it’s more likely due to the fact that ABC slashed the size of 538 and may be just trying to sell the name recognition without the same effort behind it. Not to mention the old 538 was probably an anomaly–Nate Silver is a good enough analyst that he could have been working directly for a political campaign and not giving out his data to the public, and the old 538 of him running the show with a massive amount of resources behind him just so political nerds could look at a number every day was an anomaly.
I think you got to separate Silver’s commentary from his model. I don’t agree all his opinions, but his model is pretty empirically driven. For instance he is way more bearish on Biden then his model, but the model still gives Biden a reasonable chance. Silver is also pretty forthcoming on what makes up his numbers, so if you disagree with his priors you can make mental adjustments. The biggest problem with 538 right now is that they are a black box. You can’t tell why they show Biden winning.
I also disagree you have to wait for results to make an opinion. You can have a crappy model that happened to be right. Doesn’t mean it is a good predictor going forward.