Sanders supporters do like to point to that Michigan primary as an instance of Silver’s fallibility, and that’s legit: I believe Silver had all but guaranteed victory for Clinton, only Sanders ended up on top.
OTOH, Michigan was one of the few states he got wrong, and the vote totals in MI were extremely close. So the takeaway of “he’s a fraud/he’s biased” really aren’t justified.
Silver was also very early to argue that Sanders didn’t have a chance at the nomination. In the primaries and caucuses between Arizona and New York, when Sanders was doing very well, Sanders’s campaign talked a lot about momentum. In contrast, Silver looked at the results in context and concluded that the states in question, by random draw, happened to be states that Sanders was likely to win; once the calendar got to NY and PA, he said, things would change and Clinton would win easily. He was right, but some Sanders people felt he was too quick to dismiss their guy’s chances.
I don;t know about an NYT poll, but there was an issue with a CNN poll recently in which Sanders didn’t do very well. Some Sanders supporters said that the poll didn’t talk to any young people and had too few respondents; Silver said that neither was the case and called the complainers “badly misinformed,” which also did not endear him to those folks.
Right; as someone posted above, if you’re going to do something data driven, you have to trust at some point in the accuracy of your data. I don’t know why some Sanders voters used this occasion to turn on Silver to the exclusion of other pundits or the pollsters themselves…but for whatever reason, some did.
Suppose that a forecaster predicted the outcome for every state, and in each case, said that they were 98% confident in their predictions. 98% confidence is pretty strong, right? If someone says they’re 98% confident in something, and they turn out to be wrong, doesn’t that call them into doubt?
Well, no. Because if you’re making 50 different predictions with 98% confidence, you’d expect that you would, in fact, get about one of them wrong. That’s what 98% confidence means. And if your confidence is lower than that (as it usually was, for Silver’s predictions), then you’ll expect to get even more of them wrong. And that doesn’t mean that you’re doing a bad job.
Incidentally, another thing that Silver uses, especially in sparsely-polled races, is correlations with other similar races. And he’s got a big fat database of all sorts of ways in which states are similar to or different from each other. Not only can he say simple things like “Mississippi and Alabama will probably vote in similar ways, because Mississippi is a lot like Alabama”, but he can also say “Candidate A is favored in states with high proportions of Southern Baptists, states with high proportions of people in this age bracket, and states with people in these income brackets, and Candidate B is favored in states with large numbers of African-Americans, states with people in these other age and income brackets, and states with a high proportion of immigrants. State X has this percentage of Southern Baptists, African Americans, and immigrants, and these age and income demographics, therefore, A is likely to get this percentage”. And putting together, using, and interpreting those models is difficult. In fact, most of the other prognosticators (who all did much worse than Silver) failed so hard because they assumed no correlation at all between states.
**Chronos **-- excellent points, though IIRC Silver had thrown his usual caution to the winds in the case of that MI primary and pegged the odds at 99.7% or some such. It’s worth noting, as to your second point, that no states much like MI had voted at that point. OH, PA, and WI were later in the schedule. So Silver didn’t have the kind of direct comps he likes to have. (Of course, WI went easily for Sanders while OH and PA were easy wins for Clinton, so who knows how he’d have interpreted that info…)
This is the question of the chicken and the egg. Which one came first? Without any media, there would be no rise. Once there was a rise (from media), they chased it.
While I understand the paradox of choice argument, who should be allowed to choose who gets the soapbox and why?
Since I disagree with you about how that interest was generated, I guess my question above is moot. IMO, it’s back to the media choosing.
My point about aggregated polls was in trying to understand why iiandyiiii would claim that Nate Silver’s reputation was so far above everyone else when 3 other people had the same success with aggregated polls.
Yes, the aggregator chooses weights to each poll and sometimes which polls to include. That’s why there’s room for bias.
Here’s the video of the TYT crew talking about Nate Silver’s bias back in May 2016. They were saying that in order to miss Trump’s win in the primary, Silver had to dismiss the polls that showed him in the lead. They said that Silver’s bias [in excluding certain polls] could lead to “the establishment” relying too much on his analysis and not taking the threat of Trump seriously enough. I wasn’t watching them at the time, so I don’t know what the controversy was at the time. But looking back, it looks a little prescient.
OK. Not to be argumentative, but whether Nate’s record is the best is not an opinion. That’s measurable, like the records of stock predictors. Stock predictors get ranked and rated for accuracy based on pure prediction value. I haven’t seen a similar list for political predictors.
Analytical predictive models work well when they use historical trends to predict future outcomes if the environment is relatively stable. In volatile environments, the predictive value breaks down.
The current political environment is pretty volatile. That may be some of the reason for the lack of predictive value.
It would be helpful if people were able to separate his model driven analysis and his punditry and weight their comments accordingly. I don’t see that happening much.
Thanks for that. Silver’s argument is that Iadarola should not have put so much emphasis on the small sample size because there was another poll with data that corroborated the one with the small sample size. But the one that corroborated was an online poll.
I’m not buying that argument. Yang has online polls with at least 8K people voting showing him in the lead. If someone called up 10 young people from Reddit and asked who they were voting for, and they said Yang, I doubt anyone would be convinced that he was in the lead. I’m sure the argument would be that Biden has many more polls, but they’re the ones picking which polls.
Thanks. Boy, Bernie supporters go feral at the drop of a hat.
I thought the problem with Michigan primary polling in 2016 was related to the shitshow in 2008. The pollsters were working with bad models because it had been so long since a proper primary was held there.
Not all online polls are created equal. A lot of online polls are garbage, because it’s really easy to make a garbage online poll. But there are a few online pollsters who put a lot of effort into getting it right. Do they still have flaws? Yes; all polls have flaws. But if you aggregate together a whole bunch of different kinds of polls, with different kinds of flaws, you’ll probably get something closer to a true overall picture.
Heffalump, I believe you are conflating legitimate, scientific polls conducted online (which are increasingly important as people become less likely to answer the phone), with unscientific “polls” that anyone can gang up and vote on (these used to go gangbusters for Ron Paul, and now Yang is the guy). Silver calls the latter “clickers” to distinguish them from real polls. 538 does give letter grades to pollsters, but they don’t grade “clickers” or use them as part of their model in any way.
As others have noted, Silver really wasn’t wrong about Trump. To think he was is frankly an indication of innumeracy.
On the 538 politics podcast, they recently discussed an audit of 538’s predictions over the years. The verdict was that they did very well for general election political predictions, less so for primaries, and they varied on sports predictions.
Something I really like about Silver’s attitude toward this is that while some forecasters would probably think they were doing great if every time they predicted a candidate had a 75% or greater chance of winning, they did in fact win. But Silver’s view, which I strongly endorse, is that if he predicts 100 candidates have a 75% chance of winning, and 95 of them do win, that’s actually worse than if only 60 of them win. You will of course never get the general public to understand that, so I admire his sticking to his guns. Around here we should be among the select few who do understand it.
It drives me totally nuts, in light of this, when people say he got the election wrong. He judged Trump to have about a 1 in 3 chance (which is huge: picture Russian roulette with two bullets in the six-shooter) of winning, and Trump won very narrowly in a few states while losing the national vote by three million votes. If instead Hillary had won in a blowout (which I honestly expected, and thought before the election that Silver had lost his touch), his 1 in 3 prediction would have looked very suspect.
I’ll put it this way: if I want to guess which candidates have a good chance of winning, I’m reading Silver. Doesn’t mean he’s an Oracle, but he’s as credible a pollster and data cruncher as there is.
I know this is not scientific - like at all - but I occasionally do some “trending” on Google. The two biggest names on Google among the Dem field are Bernie Sanders and Kamala Harris. Obviously, Biden is still clearly the front runner, so the trends in Google don’t have direct correlation, but I do wonder whether or not they reveal interest and activism.
FWIW, the search term “Bernie Sanders” seems to have the most activity among all candidates, and he dominates all others in many states. However, Kamala Harris is clearly getting interest in delegate-rich California and also, as one might expect, the Deep South, from Louisiana to the Carolinas and Virginia. Something tells me that if she doesn’t stumble and fizzle out before the primaries, Kamala Harris could be a major story on Super Tuesday.
I did a quick search on legitimate online political polls and came up with Nate Silver’s primer on how to tell if a poll is legitimate. He writes in Fake Polls Are A Real Problem
This was in response to a poll indicating that Kid Rock could win a Senate seat. The press ran with it and created a bit of a stir at the time…
In the case of the CNN poll, John Iadarola was doing his due diligence in looking at the poll as Silver suggests for a representation of age, education, etc. Iadarola was noting that the data wasn’t age representative. But for some reason, the rules that Silver put out on how to determine if a poll is legitimate or not in this article, doesn’t apply to the CNN poll. The CNN poll only needs to have other online polls agree with it.
That’s where I question how much bias plays a part in the process. I’m looking forward to a time when AI can predict the results more accurately than the pundits.
I’m also wondering how anyone knows whether the demographic information in online polls is accurate. It seems trivially easy to create a bot to post a new vote with a different profile every time. Right now, people are accused of making up fake accounts to do this. I would think it would be easy for a bot to do this as well.
I haven’t seen anyone dispute this. TYT is careful to say that Silver was more right than anyone else about Trump on this issue.
We don’t need Google trends to know that certain political compounds are seriously getting into it online that is unattached to the general public.
I can’t describe how disheartening it was to talk to my life long friend who’s gone full Bernie Bro. Every disagreement s a sign of enemy action. Every loss caused by trickery, subterfuge and a monolithic establishment’s machinations. Sanders has really tapped into a cynical almost nihilistic group of men and gave them something to “hope” for. I use scare quotes because they haven’t really changed and don’t expect Sanders to win. They eagerly await his defeat so they can have a nice tidy proof that the forces of evil shall never be defeated.
My WAG: The large crowded field of Democrats still helps Biden and it becomes Biden vs. the other 22 divvying up the attention among themselves, and what might have been a tough race eventually turns into an easy walk for Biden.
I don’t really know who these folks are, but it’s pretty easy to look back after the nomination is clinched and say, “Hey, you used the wrong polls.” I expect Silver did that himself. The real trick is to say at the time, “Hey, you’re using the wrong polls; here are the polls you ought to be using.” Did they do that?
As for being “prescient,” it’s been explained more than once that Silver was just about the only data-driven prognosticator who gave Trump much of a chance. If people didn’t take Trump seriously enough in the general election, that really cannot be blamed on Silver.
Um. Did you miss the part where Silver says “every other national survey”? That’s not cherry picking, that’s all of them. Or do you know of a legitimate (see below) poll that gave a different result? If so, I hope you’ll let Mr. Silver know about it.
Again, the CNN poll actually has ALL other polls agree with it, not just one–and not just an online one.
More to the point: Looks like you are still misunderstanding the difference between the Morning Consult poll and the types of polls in which Andrew Yang is the winner, despite the best efforts of some other posters. Morning Consult does its thing online, yes, but they essentially “reach out” to people to make sure they have a roughly representative sample of voters, and then they work with the data to make sure any underrepresented groups are given extra weight–the process is essentially the same as a telephone survey. Could people be lying about their demographics in the Morning Consult survey? Sure–but they can lie in a telephone survey too. Anyway, there are safeguards to ensure that people get interviewed once and only once, and that each person is weighted appropriately. Perfect? No, of course not. In line with standard polling principles? Absolutely. It’s a legitimate poll with legitimate methods.
Contrast that with the standard idea of an online poll in which Andrew Yang wins (or as SlackerInc points out, Ron Paul). These are the equivalent of leaving a ballot box in the middle of the mall and telling only your friends about it. You don’t have a representative sample at all, you don’t have any way to ensure that people don’t lie about who they are, people will pass the information on to like-minded folks in tweets and emails to help stuff the box for their candidate… This is why I used the word “legitimate” above. These types of polls are not legitimate. Or are you arguing that they should be given the same weight as a poll along the lines of the Morning Consult one? Because really, that is not a position that anyone should be taking.
Actually we can answer this empirically. The poll surge came first.
We have actual data on cable news coverage. Buttigieg’s bump happened after a long period with the same lack of media attention that others polling 0 to 2% were getting (see Castro, Inslee, Hickenlooper, Yang …) until the last week of March when he started to get mentioned lots more building up from there.
What happened that week to cause the media to start paying attention to him? This Emerson poll of Iowa in the field 3/21 to 3/24 suddenly having him in third at 11%.
The media chased the polling without question. If Inslee surged from 0 to 2% to 11% in the Emerson Iowa poll then Inslee would have been getting the media attention.
Yes their chasing that Iowa polling result got Buttigieg national exposure and people nationally liked what they heard and saw, so national polling increases followed. But end of March more than half still never heard of Pete Buttigieg. Still those who had were more likely to like what they heard than others with similarly low name recognition.
The media is motivated by eyeballs, earpans, and clicks. That is how they sell ads and make money. They will pay attention to who they believe will deliver that. If they feature someone and it results in less viewership/clicks and more yawns (cough Yang cough) they won’t keep the coverage up. And someone who keeps 'em watching (see Trump) they will keep featuring.
Of course polls are not the only way someone gets in the news. Buttigieg also created newsworthy news cycles for himself - for example he baited Pence into a back and forth that was able to advertise his take on Christian faith and contrast it with the RR version. He created a story that would hold the eyeballs, earpans, and get the clicks, that at the same time helped define his brand.
Interest of the audience drives the media, pure and simple. Fair or not. But Yang does not deserve a soapbox as big as Biden’s or Sanders’ just because he is running in order to be fair.
Imagine how your friend must feel to get to that point. I sometimes wonder if people who don’t live in this country can really understand why some people in the US are looking for political solutions to problems they experience every day. Can someone who lives in a country with universal health care know what it feels like to be one accident away from possibly losing your job and then losing everything you own?
I’ve written privately to people from Germany, Sweden and Canada about politics. There’s a fundamental disconnect between how they see the political situation and how it feels to be in it. At some point, it’s difficult to bridge that gap.
All good points, DSeid. But do you really think that if the poll in question had shown, say, Gillibrand or Hickenlooper shooting up to 11% in Iowa, that the media would have given either of them as much attention as they have lavished on Mayor Pete? I really think they were already intrigued by him as a Rhodes Scholar who served in Afghanistan, is married to a man, and taught himself Norwegian to read otherwise unavailable works from an obscure author. So then when they got a poll result to give them an angle, they were primed to go with it.