There’s the military saying “once is happenstance, twice is coincidence, three times is enemy action.” In reality it means it’s OK to ignore one-off events, but you definitely don’t wait for a two-off to turn into a three-off.
2016 was the one-off. People could have ignored it but they did investigate it. In 2020 it looks like those investigations were not enough to compensate, but remember the polls didn’t do too badly in 2016.
It seems likely to me that Trump voters (or chaos voters, as I think we should call them going forward), may be engaged in a concerted effort to corrupt the polls. I don’t think it takes a big conspiracy theory, it just takes a few people saying “hey everybody we should own the libs by lying to polls”. They’ve proven that they have no political consciousness other than owning the libs, so it’s very reasonable to assume they’re mostly lying, most of the time.
I’m not sure I’m totally following what you’re saying here, but note that you can’t just shift all the polls by the average error rate and count the electoral votes to determine accuracy. The error rate is not monolithic, so some will shift by more and some by less. And tens of electoral votes end up decided by tens of thousands of votes, so it’s not a linear process.
Silver has said over and over again that the 2016 results were within a smaller-than-average polling error. It remains to be seen if that’s true for 2020, but if it is, the takeaway should not be that polls are useless, but that they are not perfect, and a probabilistic approach like 538’s is a really good way to understand them.
Is there any evidence that any of the errors are due to outright deception?
I think the under-analyzed factor in how polls are off is how quickly our use of communications technology changes. That is: people don’t answer the phone as much. In the past 4 years, the number of spam calls I’ve gotten has increased dramatically. At the margin, many more people won’t answer an unknown caller than they would 4 years ago, and that change happens differently in different demographic groups. The pollsters all have sophisticated models that try to account for that, but behavior is changing faster than they can keep up.
Once upon a time you could call a landline at dinnertime and like 50% of households would answer it and respond to your call. These days you get like 1 < 1% response rate. Getting a random sample is really hard and continues to get harder.
538 is at the mercy of the polls, and the polls were really really bad. So if Nate’s methodology involves assessing each poll for accuracy, he’s got a lot of reassessment to do. The underlying question is: why were some polls more accurate than others? Dunno, but a lot of people are going to be working that one.
But the number of bad polls led 538 to a horrifically bad day. He ground that garbage data and produced forecasts that gave Biden a 90% chance of winning, with 400 electoral votes being the most likely outcome.
He gave Biden a 69% chance of carrying Florida. He also painted several other states blue that are still in doubt.
And he gave the Democrats a 75% chance of flipping 4 Senate seats. They flipped 2.
All those Republicans who claimed that if they had known how Trump would turn out they never would have voted for him, all those Republicans who claimed that they were undecided, all those Republicans who claimed that they would just sit this one out and all those Republicans that claimed they would vote for Biden…versus the fact that Trump didn’t lose very many(if any) voters this election.
Oh for fuck’s sake. Please show us where he is inserting his opinion on their worthiness. He has a fucking model. He fucking models data with it. This is fucking tiring. The vast majority of the weight his model grades on is based on the Predictive +/- scores. Hell, here’s the way too detailed version of how this shit works, why they do it that way, etc. It’s a model. It might be wrong, it might have large scale bias, it might even have coding errors. What is doesn’t have is “Nate doesn’t like Rasmussen so he dings them an extra letter grade”.
I would think if they conspired to lie to pollsters, there would be evidence of this being discussed. The pollsters interview a wide range of random people. If there was a unified effort to lie to the pollsters, somehow that information would have to secretly get out to a huge segment of the population. I can’t really see that happening without everyone knowing about it.
My confusion about polling in the modern area is about who they get to actually answer the polls. The set of people who answer the phone from an unknown number and also answer the questions is not necessarily representative of the population as a whole.
I think this is one of those “the plural of anecdotes is not data” things. I read a lot of op-eds like that too, and I bet that the op-ed writers did not in fact vote for Trump. But people writing think-pieces aren’t polls.
What, exactly, is the agenda in being consistently wrong? Isn’t that like trying to get yourself fired?
The problem isn’t that the models were incorrect- it’s that the data coming in is suspect. Republicans have discovered that they can get an electoral advantage by having their supporters lie about how they’re going to vote.
As an aside, I have never, not once, been polled. If I had been, I would’ve made the models even more off by responding honestly. That’s not a systemic error, it’s a data error- one side is deliberately skewing the results.
I don’t think there was any kind of planned or coordinated effort. Right-wingers eagerly transmit sketchy gossipy call-to-arms that are easy to overlook as irrelevant chatter.
I understand why most people don’t spend time in the online Fox-brain fever swamps. Dripping with hatred, derangement, and lies. But that sort of space, being too unsavory for reputable journalists to monitor, is exactly the place to seed a buzzy faxlore campaign like “destroy the polls.” I haven’t searched it and I don’t want to, but it seems plausible to me.
I didn’t even bring those people up. I’m talking about people on numerous message boards(including this one), people I know personally, people who talked to me when I was a Census worker, people I’ve seen on television and heard on the radio and, yes, Republicans that were polled.
hmm…all this talk of conspiracy to corrupt the polls by lying…
Sour grapes, methinks.
Does anybody remember just a few days ago there was talk about “shy voters” who won’t admit to pollsters that they support Trump… Everybody on the SDMB agreed that there was no evidence that shy voters exist…it’s just an excuse for the conservatives to deny the fact that they are going to lose…deny reality and create a make-believe world where facts don’t matter.
So now I see that it’s the Democrats turn to deny reality, and create a make-believe world, where evil conspirators who don’t exist mislead us all.
When times are tough, everybody need a good fantasy. It’s one of the five steps of grief, isn’t it? Denial, …anger…etc
What I blame 538 for is pretending their manipulation is somehow “more scientific”. It’s a bunch of pseudo-scientific garbage, as you can see by the poor results. And the dodge of “calculating probability” is pretty worthless as well. The probability of a single event is meaningless. What he’s really calculating is how much polling error leads to what event, and if he presented it more correctly, I would probably not be so annoyed by it.
What are you going on about here? People are trying to understand how we got as large of a polling error as we did. Chaos voters and shy Trumpers are two theories. I don’t know where you’re getting this ‘sour grapes’ / ‘denying reality’ nonsense is coming from.
Sure, but did those people lie, or do you have a non-random sample? Aside from the “Republicans that were polled” category, all the groups you listed look like a very non-random sampling to me. People who post on message boards you frequent is very much not a random sampling of Republicans, nor is people who are on the radio or television.
Among actual Republicans polled, how do you differentiate between “Republicans polled lied about their intentions” and “Republicans polled were not a random sampling of Republicans”?
I was a registered Republican in 2016. I didn’t vote for Trump then (in the primary or the general) and I didn’t vote for him this year (I’m now a registered Democrat). But OMG I was not a typical Republican and my vote is absolutely not representative. I wasn’t polled, but if I had been I would have made the polls even less representative, but not by lying about anything.
It seems that the most likely reason for the discrepancies in the results is that the pollsters aren’t as good as taking representative polls now as they were in the past. The accuracy of the polls is based on a random sampling of voters. That’s what the statistics for polling depend on. But if the sampling isn’t random, then the results won’t be statistically representative. I can certainly believe some people lie to the pollsters, but it doesn’t really seem likely that lying happened on a large enough scale for that to have influenced the results. It’s my guess that the pollsters did not really get responses from a random sampling and their polls reflect the bias of their sample rather than the general population.
I personally have only answered polls in the days before caller id. Back then, the phone rang and you picked it up. If the caller was taking a survey, most people would do it. But now people have caller id and are wary of scammers. Not only will I not answer the phone in the first place, I won’t answer the pollster’s questions because I don’t know if they are an actual pollster or if they are a marketing scammer trying to build a marketing profile of me. I certainly saw a host of pollster-sounding names come across my caller id this season, but I didn’t answer any of them.
Yep, I was thinking the same thing, how lax regulations have resulted in ever increasing numbers of scammers, robocallers and annoying and aggresive salespeople. Also, even internet survey responders are abysmal, it seems. One guy on the 538 election feed said that out of something like 1200 who clicked the link, only a total of 6 actually completed the survey. Non-face-to-face polling in the digital age may need some work.
This is akin to predicting the weather; they can say that there’s a really good chance that the temperature will be X, or that it’ll rain on X day in the afternoon, or whatever. Or even say that the hurricane will make landfall somewhere within a “cone of uncertainty” (not sure of the technical term). But they can’t predict the exact trajectory, time and amount of rain, or even if it’ll rain definitively.
Election prediction is the same kind of thing- they can say that something will happen within a certain “cone of uncertainty”, but they can’t really say that it’ll definitely go a certain way. And in this case, the cone says that it’ll be close, and that the cone mostly lies over Biden win territory, but that it does go over Trump win territory. Just because that line down the center of the cone goes over Biden win territory, doesn’t mean that they failed if Trump wins- the only failure in this analogy would be if the cone was totally over Biden territory and Trump won anyway.
And FTR… Trump’s nonsense declarations last night are basically him coloring the cone with a sharpie again.
Honestly, this is 80% of the issue. Nearly all the major polls you read about in the paper have response rates of about ~10%. Anyone willing to take a political poll in 2020 is by definition an unusual person.
Cellphones/caller ID are a huge part of that.
Add onto it that polls are increasingly long : 20, 30, 40 questions; calls lasting half an hour or more. Who has time for that?
Add onto that that people are more aware than ever that their info can get shared; they don’t want to get on some mailing list for fundraising or worse. Everyone knows that googling “backpacking trip” leads to weeks of targeted ads for hiking boots – are you going to enter political preferences into an online poll?
This is especially true among working-class voters, who have the least time or inclination to talk to pollsters. In 2012, they broke for Obama, which is why the polls overestimated Romney, In 2016 and 2020, they went for Trump.
Polling needs a fundamental rethink, but nobody’s going to admit that unless they have no choice. And they very much do have a choice, so long as the media keeps paying for them.
While it’s a right wing conspiracy theory, the idea that the pollsters deliberately overestimated Bidens chances to make Republican voters not vote is just as valid as the current “Massive right wing conspiracy to deliberately lie to pollsters that absolutely nobody on the left actually has any proof of”. At this point either of those are valid.