# 538 is No Longer a Credible Source **URL:** \ **Category:** Politics & Elections\ **Tags:** trump\ **Created:** [July 19, 2024, 4:58am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449 "2024-07-19T04:58:27Z")\ **Posts on this page:** 20\ **Page:** 8
**Author:** ![DrDeth](https://avatars.discourse-cdn.com/v4/letter/d/b487fb/32.png) [@DrDeth](https://boards.straightdope.com/u/DrDeth)\ **Post date:** [August 2, 2024, 11:13pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/142 "2024-08-02T23:13:54Z")
> [@Maserschmidt](#): > > I had decades of involvement with modeling, and I can’t ever recall us talking about a model being “right” or “wrong”, just weak or strong (or robust)(or “really awful” or “useless”, I suppose). However, as many have pointed out, if you have a bunch of estimates from a model, you can look at the overall performance and gauge its strength. Right. Not “right” not 'wrong" but the overall performance can be measured. > [@Stratocaster](#): > > e.g., that there isn’t a sufficient sample size to assess the strength of Nate’s modeling). I did not claim that, there certainly could be. But do we have enough with Silver on his own, without the whole 538 team to back him up? 538 during the Silver years probably does. That doesnt mean anyone prediction is right or wrong, of course. ---
**Author:** ![Chingon](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chingon/32/17645_2.png) [@Chingon](https://boards.straightdope.com/u/Chingon)\ **Post date:** [August 2, 2024, 11:16pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/143 "2024-08-02T23:16:51Z")
> [@DrDeth](#): > > Which means- he was wrong. 🤔 Hmmmmm ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [August 3, 2024, 12:41am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/144 "2024-08-03T00:41:00Z")
Simon Rosenberg (_Hopium Chronicles_ Substack) gets in a dig at Nate Silver – and in doing so, **[peels back the curtain a sliver](https://www.hopiumchronicles.com/p/why-im-optimistic-about-november/comment/64176765)** regarding Silver ‘losing’ FiveThirtyEight (my emphases): > As for Nate, I adopted the Hopium brand because in the final week of the 2022 election he went on a wild tirade against me on his podcast for claiming there would be no red wave. He said I was smoking Hopium and misleading people. Of course he completely blew the 2022 election. **It is because of his arrogance and poor management of 538 that** “Hopium” became a thing, and **he lost control of the site he built**. So no Nate is a not really a friend of ours here at Hopium, and I prefer other sources of data and analysis. Silver “_lost control of the site he built_”? He left unwillingly? ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [August 3, 2024, 2:44am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/145 "2024-08-03T02:44:28Z")
> [@DrDeth](#): > > Which means- he was wrong. What was he wrong about? He said she had a significant chance of losing, and she did. > [@DrDeth](#): > > He predicted Hillary would win- she didnt. Where did he predict that? He didn’t put that prediction up on his website. > [@DrDeth](#): > > Each one by a different %, and each one was a single event. So he hasn’t predicted a dozen races, because each individual race was a single event? What do you call a whole bunch of single events? ---
**Author:** ![flurb](https://avatars.discourse-cdn.com/v4/letter/f/51bf81/32.png) [@flurb](https://boards.straightdope.com/u/flurb)\ **Post date:** [August 3, 2024, 3:02am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/146 "2024-08-03T03:02:47Z")
> [@bordelond](#): > > Silver “_lost control of the site he built_”? He left unwillingly? He sold 538 to Disney a while ago but stayed on as editor in chief. Last year he announced that his contract would not be extended at the same time that Disney laid off half his staff. ---
**Author:** ![Tamerlane](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tamerlane/32/7602_2.png) [@Tamerlane](https://boards.straightdope.com/u/Tamerlane)\ **Post date:** [August 3, 2024, 3:21am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/147 "2024-08-03T03:21:56Z")
> [@flurb](#): > > Last year he announced that his contract would not be extended at the same time that Disney laid off half his staff. Yes, my understanding was he left in protest after Disney gutted the 538 staff in a money-saving move and took his models with him. If that was truly what happened, then I can’t blame him. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [August 3, 2024, 3:30am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/148 "2024-08-03T03:30:55Z")
FWIW I found [this](https://statmodeling.stat.columbia.edu/2024/07/19/my-comments-on-nate-silvers-comments-on-the-fivethirtyeight-election-forecast/) compare and contrast of the Fivethirtyeight, the Silver, and the current Economist models from while Biden was still in it, and Silver’s critique. It seems to make sense to me. > [@](#): > > The Fivethirtyeight model expresses a lot of uncertainty because it’s allowing for large, Dukakis-versus-Bush-in-1988-style swings in public opinion, whereas the Economist’s model and Nate’s model are making stronger predictions by assuming that swings during the campaign will stay in the narrow range that we’ve seen in recent national elections. > > Nate frames his differences with the Fivethirtyeight forecast of 2024 as a disagreement of how much to trust the fundamentals, and that’s part of it. The other part is the model’s uncertainty about potential changes in public opinion. > > As Nate points out, Fivethirtyeight’s wide uncertainty about national swings translates to wide uncertainty about state forecasts, for example, “in Pennsylvania, 538’s 95th percentile forecast covers outcomes ranging from roughly Biden +18 to Trump +17.” Bottom line to me is that each start off with best guess assumptions and decisions about which history is your precedent. I suspect that their probabilities for Harris v Trump will converge fairly quickly. ---
**Author:** ![Gordon\_G](https://avatars.discourse-cdn.com/v4/letter/g/3da27b/32.png) [@Gordon\_G](https://boards.straightdope.com/u/Gordon_G)\ **Post date:** [August 3, 2024, 4:18am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/149 "2024-08-03T04:18:53Z")
> [@Whack-a-Mole](#): > > Pedantry aside you could just read about it but you seem uninterested so are you here to nitpick word choices? > > Fine, I phrased that poorly and could have written that sentence more clearly. My apologies. I skimmed the first paragraphs and Silver just repeats the obvious point that others have tried to explain to you: A predictor _should_ get 30% of its “70% predictions” _wrong_. I wonder if prediction markets might be the most “reliable” predictor we have. At Polymarket Trump peaked at 70% about when he put a bandage on his ear, but the gamblers now show Trump over Harris by only 53 to 44. I’d put little faith today in models. Models are trained with past data, but “Toto, we’re not in Kansas anymore.” No matter how detailed your records of the animals in a particular zoo may be, that data loses relevance when the animals have escaped and are on the rampage. ---
**Author:** ![Kolak\_of\_Twilo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/kolak_of_twilo/32/73_2.png) [@Kolak\_of\_Twilo](https://boards.straightdope.com/u/Kolak_of_Twilo)\ **Post date:** [August 3, 2024, 5:14am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/150 "2024-08-03T05:14:41Z")
Never mind. I misread and this post has been deleted. ---
**Author:** ![DrDeth](https://avatars.discourse-cdn.com/v4/letter/d/b487fb/32.png) [@DrDeth](https://boards.straightdope.com/u/DrDeth)\ **Post date:** [August 3, 2024, 6:57am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/151 "2024-08-03T06:57:11Z")
> [@Chronos](#): > > What was he wrong about? He said she had a significant chance of losing, and she did. He said when Silver said such and such as a good chance of happening - and it did- that means Silver was right. But when Silver said Hillary had a good chance of winning- but she didnt- he claims that if you just give a %, you can’t be wrong. That last bit does have logic. If someone predicts a batter will get a hit 25% of time time- but the batter does get that hit- the predictor is neither wrong- nor right, since he was predicting ODDS. So yes, if you want to take they were all % odd- thye they could not be wrong. But nor could they be right. As was said- > [@Maserschmidt](#): > > I had decades of involvement with modeling, and I can’t ever recall us talking about a model being “right” or “wrong”, just weak or strong (or robust)(or “really awful” or “useless”, I suppose). So, yeah, in Silvers Hillary prediction he just predicted the odds- so he could not be wrong. But neither could he be right. > [@Thing.Fish](#): > > So I’m counting 2008, 2010, 2012 and 2018 as cases where he called one party’s chance of winning as being _much_ greater than the other’s, and he got them all right. But then he said I was innumerate as I said Silver was then 'wrong" in calling Hillary Clinton/trump. Sure, in giving odds you cant be 'wrong" but then you cant be 'right" either- and Thing.Fish stated Silver was 'right" You can have it one way or the other, not both. > [@Chronos](#): > > Where did he predict that? He didn’t put that prediction up on his website. He said Hillary had a 60% chance of winning. You have not been reading these last few posts, have you? The whole debate about Right and Wrong has been going on for a while now. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [August 3, 2024, 11:25am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/152 "2024-08-03T11:25:51Z")
> [@Gordon\_G](#): > > I wonder if prediction markets might be the most “reliable” predictor we have. No, prediction markets are terrible predictors. There are too many irrational bettors out there, and it only takes one or two to skew things. > [@DrDeth](#): > > So, yeah, in Silvers Hillary prediction he just predicted the odds- so he could not be wrong. But neither could he be right. We can’t say that Silver was right or wrong about Clinton, because we can’t say that he was right or wrong about any single individual race. We can say that a forecaster is right or wrong when we look at a great many predictions they make. Silver has made a great many predictions, and he’s been right on those about as often as his percentages say he should be. In other words, there is no way to say that Silver is wrong, but there is a way to say that he’s right. Or to distill it down to two words, “he’s right”. ---
**Author:** ![pulykamell](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pulykamell/32/3166_2.png) [@pulykamell](https://boards.straightdope.com/u/pulykamell)\ **Post date:** [August 3, 2024, 2:46pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/153 "2024-08-03T14:46:05Z")
The thing is, if he were to be always correct in his predictions in terms of picking a winner (whoever has the highest probability), his model would likely be wrong, and underestimating win probabilities. If there’s fifty races running 70-30, and you end up with almost all fifty wins in those races, you need to check your model. People just don’t seem to under and probabilities very well. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [August 3, 2024, 3:06pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/154 "2024-08-03T15:06:37Z")
To evaluate that you need a meaningful n, a meaningful comparison with how other forecasters have also matched their percentages, and the analysis done by an independent neutral third party. I know Silver claims to have done that and shared his work, but I have great faith in his ability to choose the exact methods that confirm how good he is. Plus the forecast made a week before and the forecast made three months before are very different beasts and should be evaluated separately. By the week of election the forecasters all rely more on polling and less on “fundamentals.” Assumptions of future poll volatility become less important as the data is basically all in. The difference between the models then are based on how each models rank possibilities of different sorts of polling errors to predict behaviors and use some polls to better deduce signal from noise on others. And how much faith they have in those methods. But the difference between 538’s approach and Silver’s, big at the point of Biden’s debate, will be little by then. 538 stood out of the crowd because they assumed the longer run of history’s poll volatility and therefore went with a fundamentals as priors heavy approach, while Silver and The Economist went the other way. There was no way to really _know_ which set of assumptions was the better one, even if we all had our own conclusions about it. ---
**Author:** ![DrDeth](https://avatars.discourse-cdn.com/v4/letter/d/b487fb/32.png) [@DrDeth](https://boards.straightdope.com/u/DrDeth)\ **Post date:** [August 3, 2024, 7:41pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/155 "2024-08-03T19:41:47Z")
> [@Chronos](#): > > We can’t say that Silver was right or wrong about Clinton, because we can’t say that he was right or wrong about any single individual race Exactly my point. ---
**Author:** ![RickJay](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@RickJay](https://boards.straightdope.com/u/RickJay)\ **Post date:** [August 4, 2024, 1:33am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/156 "2024-08-04T01:33:03Z")
> [@Dr.Strangelove](#): > > Maine and Nebraska have split electoral votes, so you’d only need to win one district in one of those. Still pretty unlikely, but not as unlikely as winning the whole state. One must also consider the possibility of a faithless elector throwing RFKJr a meaningless vote. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [August 4, 2024, 1:37am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/157 "2024-08-04T01:37:06Z")
Yeah, **bordelond** mentioned that possibility. But frankly, if we’re counting that, I’d put the odds at more like 50%. This is going to be a weird election. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [August 4, 2024, 2:57am UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/158 "2024-08-04T02:57:45Z")
> [@Chronos](#): > > No, prediction markets are terrible predictors. There are too many irrational bettors out there, and it only takes one or two to skew things. I would ask you to back this up if I didn’t already know that you can’t. ---
**Author:** ![LSLGuy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lslguy/32/5813_2.png) [@LSLGuy](https://boards.straightdope.com/u/LSLGuy)\ **Post date:** [August 4, 2024, 6:13pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/159 "2024-08-04T18:13:05Z")
To my mind the larger question about the prediction market is: “Are there more or fewer irrational bettors vs. irrational voters?” ISTM the real wild card in every election since about 2012 is the ever increasing fraction of just plain insane voters driven by gosh-knows-what particular long laundry list of batshit irrationalities. The crazies, like the poor, will always be with us. But we seem to have gotten a lot better at manufacturing them of late. Or at least _somebody_ has gotten a lot better at manufacturing them. ---
**Author:** ![Dr.Strangelove](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr.strangelove/32/6613_2.png) [@Dr.Strangelove](https://boards.straightdope.com/u/Dr.Strangelove)\ **Post date:** [August 4, 2024, 7:23pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/160 "2024-08-04T19:23:48Z")
> [@LSLGuy](#): > > To my mind the larger question about the prediction market is: “Are there more or fewer irrational bettors vs. irrational voters?” The thing about prediction markets is that it only takes a small percentage of rational players to override the rest (by taking their money). Voting is just an averaging mechanism. But if a prediction market is way off from what it “should be”, then it only takes _one_ rational player to correct that by making a large bet. Which isn’t to say they’re perfect. There’s the known favorite-longshot bias which amplifies unlikely scenarios. The reason is probably that it’s hard to make money betting against these since it means making a 99:100 odds bet or the like. So what should be an 0.01% scenario becomes a 1% scenario. But aside from certain biases like these, they tend to do well. Not necessarily as well as one good analyst. But probably better than any given random analyst with a unknown success rate. ---
**Author:** ![Thing.Fish](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/thing.fish/32/9168_2.png) [@Thing.Fish](https://boards.straightdope.com/u/Thing.Fish)\ **Post date:** [August 4, 2024, 10:23pm UTC](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449/161 "2024-08-04T22:23:12Z")
> [@RickJay](#): > > One must also consider the possibility of a faithless elector throwing RFKJr a meaningless vote. No, the model doesn’t attempt to predict the possibility of faithless electors, only the popular vote results. [Previous page](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449.md?page=7) [Next page](https://boards.straightdope.com/t/538-is-no-longer-a-credible-source/1004449.md?page=9)