Any chance the image below is what you were looking at on Silver Bulletin? If so, that top line is national polling percentage (a proxy for % of popular vote), not percentage of Electoral College votes.
No, he doesn’t, he publishes his results on his Substack, “Silver Bulletin”.
No, on the Substack there is a list of “Crazy and Non-Crazy Scenarios”, listing the likelihood of various events happening. “Kennedy receiving at least one electoral vote” is at 5.5% as of right now, while Kennedy’s projected popular vote percentage is only 3.9%.
But he gives Kennedy only about a 0.5% chance of getting as much as 15% of the vote, with 20% an absolute ceiling. Seems to me that’s probably about the minimum threshold you need nationally in order to have a prayer that maybe in your best States, you might get double that and, if you’re lucky and the other candidates split equally, you could win a State with 35% of the vote or something. So I’m mystified at how he calculates his chance as being that high, it seems like it should be well under 1%.
Exactly. So, sure, you can say that if you just predict a % for a single occurrence, you werent wrong, I will concede that.
But then, you can’t be right either.
We dont have enough data to say his model works very well, you would need dozens if not hundreds to tell that. And then you cant look at any one election and say he was wrong or right, you’d have to look at all the elections he ever gave odds for , and do some advanced math - because he doesnt give the same odds for every election.
So, yes, technically if someone says “the batter has a 25%” chance of getting a hit- but he does get that hit- the odds prediction isnt wrong- but if he doesnt get a hit, it is not right either. Once you get into a whole season, then you can say right or wrong.
We have tons of data, he’s predicted every Congressional, Senate, and Governor’s race since 2008 in addition to the Presidential race, as well as a great many sporting events.
And the math isn’t super advanced, you just look at all the things he thought were 50% likely to happen and see if 50% of them happened (note that getting 70% right would be just as wrong as getting only 30% right). Then repeat for every other percentage. Silver has done this and showed his work.
Maine and Nebraska have split electoral votes, so you’d only need to win one district in one of those. Still pretty unlikely, but not as unlikely as winning the whole state.
Each one by a different %, and each one was a single event.
Any chance at all Silver’s looking at past elections and coming up with mathematical odds for a chaotic event like “a faithless elector casting an EC vote for RFK, Jr.”?
Good point. I knew there was at least one faithless elector in 2016… it turns out there were ten, with seven of them successful:
Colin Powell, Kohn Kasich, Ron Paul, Bernie Sanders, and Faith Spotted Eagle were among them. 5% might actually be low for RFK Jr getting one.
If you were to take all possible outcomes of all elections that he predicts, you could arrange them all in some way (e.g. from one where all Democrats win on the one side to all Republican wins on the other, or from how many races he predicted correctly to how many he got wrong) on a line.
If you were to simulate the odds that he gave, you’d produce a set of outcomes that favor a particular range of territory on the above lines.
The actual world results should fall in his preferred territory and - if he’s more accurate - it should fall more towards the middle across all different ways that you choose the arrange the results.
You can also do things like betting money on each race and seeing how much money you win, using his predictions, both in the real world and in computer simulation. The total money amount should come out relatively similar and become even closer the more races that you run (on average).
See, so note that Thing.Fish didnt claim 538 did better than average with their predictions- that is quite possible. But he claimed Nate Siler- alone- was “right” four times, and that he wasnt 'wrong" in 2016. See the difference?
Claiming 538 or Silver has a good percentage- sure, if someone does the math and claims that I will buy it. But he claimed Silver was 'right" but never wrong. So, if I am innumerate., then that looks like being … well…
The math of probability theory has no problem with dealing with this situation. In fact, simply evaluating the Mean Squared Error (MSE) is usually good enough. People who make better predictions will have lower MSE, whether that’s predicting 100% for a guaranteed success or 50% for a true 50/50 chance.
I think that, although it’s not technically “wrong”, it would be fair to say that if that if you say something is 90% likely to happen and it doesn’t happen, it looks bad. In the four elections that he had a high degree of certainty about the winner, he was right.
In the ones where he didn’t have a high degree of certainty, they were all close elections; therefore, he was right that having a high degree of certainty would have been inappropriate.
Sure. But that is not what was claimed. Silver or 538s overall % may well be pretty good. But that does not make any one single prediction “right” or “wrong”.
But you would be accused of being innumerate.
If I say that there’s a 50% chance of flipping a head on this here fair coin and, through billions and billions of flips, the precise ratio of heads to tails is exactly 50%, then I was right.
My one prediction was right. It wasn’t 50.2% one way and 49.8% the other. I was exactly on it.
Yeah, and so? This isnt a coin flip. You can look at a coin and note two sides.
However, there are one sided trick coins…
And with a one-sided trick coin, my one prediction of 50% will be wrong. After billions and billions of flips, the result will be 100% heads and 0% tails.
There are degrees of wrongness. It’s subjective where one sets the threshold. Nevertheless, we can say that Silver predicting a 33% Trump victory is much less wrong than one predicting 1%.
We can still ask if that one prediction was a fluke. We can’t rerun the election, but we can look at how the model did in other cases. And it did quite well overall.
I had decades of involvement with modeling, and I can’t ever recall us talking about a model being “right” or “wrong”, just weak or strong (or robust)(or “really awful” or “useless”, I suppose). However, as many have pointed out, if you have a bunch of estimates from a model, you can look at the overall performance and gauge its strength.
That said, we worked with large datasets with repeatable outcomes; in my opinion, election forecast models have limited value because of the low number of outcomes, lack of repeated events, and the potential strength of exogenous effects (for example the Comey letter, according to Silver).
There are some inaccurate assertions in the thread, to be sure (e.g., that there isn’t a sufficient sample size to assess the strength of Nate’s modeling). That aside, this seems like one of those frequent SDMB arguments that are essentially semantical—i.e., how do you define “wrong”?