I have never looked at a poll and felt confident I knew the winner. That would be foolish. They might make me optimistic or pessimistic, but they don’t convince me that I know the outcome.
No more than odds of a sporting event convince me of the outcome before the event concludes.
Meteorologists seem to have a better track record in my experience. Weather is a lot easier to predict than people.
You might then ask, what is the point of a poll? I don’t think the conclusion (the odds) is all that important, because it is just a guess. I think what matters is how that conclusion is drawn.
Let’s say a candidate polls at 30%. Why? If it’s lack of experience, a scandal, a tendency to not honor promises, if it’s being a bad public speaker, all of those things matter. That 30% figure on its own isn’t more than a curiosity to me. Maybe it means more to someone else but not me.
Here is Silver looking at his results in 2017. He has no regrets:
I think this is a key quote:
Emphasis added.
This is perhaps the crux of the debate in this thread. I think the site, through its structure and presentation, was misleading on this point (though there were hedgey things to be found), and I think Silver could have been more emphatic about how that 30% chance Trump had ought to be perceived.
But in order to be totally fair to Silver, I would need to go back to his media appearances in 2016 and see what he was saying on that account. E.g., whether he needed to correct others’ false impressions that the Hillary’s 70%+ chance on the site meant other than it did, etc.
Well, I was accused of making up a statistic, but I won’t accuse you of making up a conversation that didn’t happen.
We’d have to go back and look at what Silver was saying in the media up to election day.
By the way, I think the general assumption in this thread (since this is the Dope) is that I am an idiot, and therefore was also an idiot in 2016, and I therefore totally didn’t understand 538, never saw the hedgey messages, etc. Actually, I did understand what he was doing, and I read those messages. I understood that it was a tight race and that success in the dodgy institution known as the Electoral College was necessary. I actually read 538 on a daily basis, not just looking at the numbers but reading a lot of the blog-like content on the site.
After Hillary lost, my conclusion is that site wasn’t useful to me. I never thought about it again and certainly never had a debate about its usefulness with anyone.
In a tight race, in a model like Silver’s, you’re going to get things like that 70/30 split. Is that cognitively useful? In a race that isn’t tight, you won’t really need the site in the first place. That’s my issue with it.
It means you’ve sidelined this thread with your continued hijack about the 2016 election, and are now tossing out insults and trying to bait people into response, so there is now a Pit thread for you to do exactly that.
Meteorologists do no not make forecasts that are valid for more than a few days. ‘Long Term’ forecasts are usually just regression-to-the-mean (Seasonal average is 16, it’s 12 today. We forecast it will be moving towards 16 in the longer term, but for now we can make a better short-term forecast based on system movements, radar, weather in nearby areas, etc."
Weather forecasters learned the lesson of the futility of predicting the future. That’s the domain of the Farmer’s Almanac.
To feed the insatiable desire of people to know the future. See also: macroeconomists, the CBO, and fortune tellers. All of whom are approximately the same in accuracy over a 10 year period - zero. No better than a coin flip. Yet we will listen to them when they prognosticate about the future, because we have a burning desire to know. The fact that we can’t possibly know doesn’t enter into it.
People have a huge desire to know what’s going to happen, and the media and various academic types are happy to supply it. Those that are unwilling to predict the future soon find they don’t get interviewed by the media any more, don’t get book deals, aren’t celebrated by partisans, etc. All the incentives push people to try to predict the future. The fault is on those who believe they can actually do it and pay attention to them.
The Fortune tellers are the smart ones. They make a gazillion predictions every year, confident that through sheer luck one or more will come true. Then they memory-hole the wrong ones and hype the accurate ones, making them look prescient. Come to think of it, that’s what pollsters do, too.
Well that’s good because A) I didn’t make the accusation, and B) calling me out on a clearly fictional conversation would be silly. That’s why I used “Forecaster” and “Reporter” rather than actual names, to make it clear that it’s an example, not a literal conversation.
I didn’t get that at all. I don’t think anyone on this board thinks you are an idiot. They may be having a hard time understanding your argument in this case, however.
If other pollsters are saying 95% and Silver says 70%, that seems like an interesting difference worthy of analysis - especially if you are involved in the campaigns.
One of the problems pollsters have is that they are giving information to partisans who love to attack the messenger. I seem to recall Silver taking a lot of flak for not agreeing with other pollsters that Hillary was a lock. And it was Democrats who were angry with him. The interesting thing is that Democrats should have ben happy with him, because he might have helped kill the complacency that was partly what did Hillary in. It’s way better from a motivational standpoint to under-represent the support of your candidate than to over-represent it. But Democrats got angry, probably because the average person sees such a poll as bad news they don’t like, or representative of something they refuse to believe (widespread support for Trump).
We’d all be a lot better off if we focused on resiliancy in the face of an uncertain future rather than turning to our ‘thought leaders’ to tell us what the future will be. Because the future is still going to be unpredictable no matter what they say, but at least if we don’t listen to them we might have a better grasp of the risks of an uncertain future.
Again, I largely agree. Thank you for debating in a polite manner!
I would refer you and others again to the link I just posted. It gets into the things he was saying before election day and how he hedges everything he says so he can never be wrong.
Saying 70% instead of 95% is preferable in the relative sense, yes, but is it virtuous in the absolute sense in terms of both statistics and communications? That is a difficult question. Re statistics, there are PhDs who criticize his methods (I can’t say who is right; I don’t know enough). Re communications, I have rendered my opinion, but in order to present my opinion more strongly (and hold it myself more forcefully), I’d really have to go back and study all of the site content from the time and things Silver was saying in the media. I do feel strongly, however, that the front page of the site was lacking in clear caveats that could give a layperson a better understanding of what the big numbers presented up top meant or didn’t mean.
While with things like elections, you aren’t going to get enough data to really check that accuracy of the final prediction, what you can look at is what assumptions went into the model and how did those pan out. The basic modeling that most people were using was to estimate the voter precentages in each state, and in an an estimate for variability and then put all those together to determine the likelihood of individual scenarios which result in one or the other candidate winning. The main difference that I saw in Nate predictor versus the others, is that Nate took into account wave phenomena, where the results of individual states weren’t independent coin tosses, but instead were correlated, if Trump did well in one state he is more likely to do well in another state.
Looking at the states results in detail, Nate looks very good, all of the states polls were within the margin of error of where he said they would be, but they all had a general red-ward shift from the average, exactly as Nate’s model suggested might happen. This is why I say that even though his model suggested that Clinton was more likely to win than Trump, the election supported the accuracy of his model.
So, @Aeschines , if I’m reading you correctly, your criticism is that Silver shouldn’t have given the most accurate form of information possible in his field, the probability of each candidate winning, because people are likely to misinterpret accurate information, and that he should therefore have given less accurate information?
Yes, 538 regularly assesses their overall performance and shares the data. They do a pretty good job; of all the things they give a 75% chance of happening, about three out of four of them do in fact happen.
That’s how you have to judge probabilistic predictions; for any one prediction you are correct, there is no way to ever meaningfully say it was “wrong”.
Forecasts have always been a core part of FiveThirtyEight’s mission. They force us (and you) to think about the world probabilistically, rather than in absolutes.
We’re not trying to pick winners, though; we’re trying to model the games, which means including in our predictions all of the randomness inherent in baseball.
(Note that although preview panel says “4 Apr 2019”, the metrics are routinely updated.)