# So will anybody ever trust the polls again?
**URL:** https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714
**Category:** Politics & Elections
**Created:** [November 4, 2020, 5:49am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714 "2020-11-04T05:49:40Z")
**Posts on this page:** 20
**Page:** 6
### Author:  [@Acsenray](https://boards.straightdope.com/u/Acsenray)
#### Post date: [November 5, 2020, 3:26am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/104 "2020-11-05T03:26:44Z")
I say racehorse polls are inherently bad during an election. That’s different than gathering data to formulate policy.
---
### Author:  [@Thudlow\_Boink](https://boards.straightdope.com/u/Thudlow_Boink)
#### Post date: [November 5, 2020, 3:51am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/105 "2020-11-05T03:51:48Z")
Has anybody in this thread yet suggested that maybe the polls are right, it’s the elections that are wrong? i.e. that the polls predicted the results we would have gotten without some sort of shenanigans (rigged voting machines, selective counting, etc.)?
---
### Author:  [@robby](https://boards.straightdope.com/u/robby)
#### Post date: [November 5, 2020, 5:13am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/106 "2020-11-05T05:13:13Z")
> [@Thudlow\_Boink](#):
>
> Has anybody in this thread yet suggested that maybe the polls are right, it’s the elections that are wrong? i.e. that the polls predicted the results we would have gotten without some sort of shenanigans (rigged voting machines, selective counting, etc.)?
I’m starting to wonder. For one thing, I wish all states had a paper backup for each and every vote so that true independent recounts could be conducted for auditing purposes.
---
### Author:  [@Deeg](https://boards.straightdope.com/u/Deeg)
#### Post date: [November 5, 2020, 6:20am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/107 "2020-11-05T06:20:37Z")
I’m assuming you voted for Trump because apparently you can’t read. Your own cite says:
“Our overall method is [largely the same as in 2010](https://fivethirtyeight.com/features/pollster-ratings-v40-methodology/). That year, for the first time, we introduced a consideration of a poll’s methodological standards in addition to its past accuracy. We think the case for doing so has probably grown stronger since then, but you can find a number of versions of the pollster ratings based on past accuracy alone if you prefer them.”
Their aggregate includes a rating that depends on the methodology of the pollster, among other things. Otherwise all they would be doing is tracking past performance, which isn’t the best predictor. I will admit to overstating the individual dimensions.
Now, if you want to continue the snark go ahead but maybe we can have a calmer discussion.
I will say that as the final votes are coming in it’s not as bad as it was originally. I’ll only be slightly miffed at 538.
---
### Author:  [@Deeg](https://boards.straightdope.com/u/Deeg)
#### Post date: [November 5, 2020, 6:25am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/108 "2020-11-05T06:25:15Z")
I suppose it’s possible but… Ohio and Florida were off by 100s of thousands. I can’t imagine how you could pull off that kind of fraud without getting caught.
---
### Author:  [@Czarcasm](https://boards.straightdope.com/u/Czarcasm)
#### Post date: [November 5, 2020, 6:54am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/109 "2020-11-05T06:54:45Z")
> [@Deeg](#):
>
> I suppose it’s possible but… Ohio and Florida were off by 100s of thousands. I can’t imagine how you could pull off that kind of fraud without getting caught
By having politically biased officials not bother to investigate.
---
### Author:  [@Asuka](https://boards.straightdope.com/u/Asuka)
#### Post date: [November 5, 2020, 8:44am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/110 "2020-11-05T08:44:01Z")
Instead of the polls being wrong or the election being wrong, maybe the Straight Dope Message Board is wrong?
---
### Author:  [@Acsenray](https://boards.straightdope.com/u/Acsenray)
#### Post date: [November 5, 2020, 10:57am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/111 "2020-11-05T10:57:33Z")
About what exactly?
---
### Author:  [@Gyrate](https://boards.straightdope.com/u/Gyrate)
#### Post date: [November 5, 2020, 12:21pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/112 "2020-11-05T12:21:43Z")
> [@Thudlow\_Boink](#):
>
> Has anybody in this thread yet suggested that maybe the polls are right, it’s the elections that are wrong? i.e. that the polls predicted the results we would have gotten without some sort of shenanigans (rigged voting machines, selective counting, etc.)?
Me, virtually in those words:
> [@Gyrate](#):
>
> Before you decide whether you trust the polls, you need to decide whether you trust [the result.](https://boards.straightdope.com/t/tinfoil-time-if-the-voting-machines-were-hacked-would-we-know/924724)
>
> And potential hacking aside, note that the 538 model explicitly did not factor in all the voter suppression and nullification efforts - legal and illegal - the Republicans were openly engaged in across the country, from fake ballot dropboxes to intimidating robocalls to severely limiting polling sites etc etc etc - there’s no doubt it had a significant effect (and, sadly, proved to the GOP that it was effective - look for more and more each election).
>
> It’s all very well to talk about the odds-on favorite of the Kentucky Derby but if there’s a couple of guys in the bushes with rifles sniping at the horses it’s a little unfair to blame the bookies for getting the odds wrong.
I don’t know if it’s true, and I don’t know if the mechanisms exist to prove it even if it were true. Which is in itself a problem. But it needs to at least be considered.
---
### Author:  [@Paul\_was\_in\_Saudi](https://boards.straightdope.com/u/Paul_was_in_Saudi)
#### Post date: [November 5, 2020, 12:32pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/113 "2020-11-05T12:32:03Z")
I like numbers. I like everything laid out clearly.
After this I am giving up both Arabic numerals and decimals. We have proven they do not work. I am switching to a system based on quipu.
I had an strange experience. I listened to the pre-election morning _Morning Joe_ program. They were so confident! They were so sure! They were so wrong. It is something I will never forget.
---
### Author:  [@Gyrate](https://boards.straightdope.com/u/Gyrate)
#### Post date: [November 5, 2020, 1:53pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/114 "2020-11-05T13:53:51Z")
> [@Paul\_was\_in\_Saudi](#):
>
> I am switching to a system based on quipu.
Know your knot.
---
### Author:  [@Paul\_was\_in\_Saudi](https://boards.straightdope.com/u/Paul_was_in_Saudi)
#### Post date: [November 5, 2020, 1:57pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/115 "2020-11-05T13:57:40Z")
> [@Gyrate](#):
>
> Know your knot.
Do we have an upvote button?
---
### Author:  [@kayaker](https://boards.straightdope.com/u/kayaker)
#### Post date: [November 5, 2020, 2:26pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/116 "2020-11-05T14:26:09Z")
> [@Paul\_was\_in\_Saudi](#):
>
> Do we have an upvote button?-
⤴
---
### Author:  [@Pantastic](https://boards.straightdope.com/u/Pantastic)
#### Post date: [November 5, 2020, 2:59pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/117 "2020-11-05T14:59:39Z")
The events of 2016 could have been a fluke, but in 2020 pollsters were aware of what happened in 2016 and (most likely) went out of their way to push errors in favor of Trump just to be safe and actually did a worse job at predicting the election. Polling has never been a scientific endeavor, but this makes it clear that the ‘art’ part of it is more guesswork than most of us previously thought, and is clearly subject to huge, systemic errors that could not be simply accounted for. Like the OP, I certainly won’t be trusting political polls in the future, it’s clear that they are fundamentally broken in ways that the people using them to make predictions can’t work around.
I think (like others here) that changes in communication are a big part of why this has happened. When I get an ‘unknown number’ phone call, I generally just don’t answer it, and if I do it needs to identify itself as someone I know or a business I work with in about five seconds or I say ‘put me on your do not call list’ and hang up. And I’m not an outlier, a lot of people don’t answer strange calls, and if they do don’t want to spend a half hour giving answers to a stranger about a lot of often-sensitive questions. (How do you know that it’s really a pollster and not someone who has an ulterior motive?) That means that people who do answer calls are outliers, and generalizing the few who do answer to the whole population clearly isn’t working. Weirdly, regulating away all of the robocalls saying ‘you’ve won!’ or ‘we’ve been trying to reach you about your car’s warranty’ and their habit of displaying a fake number on caller ID might massively improve the ability to gather data about politics in this country.
Here’s an article about polling that I found relevant:
> **[Pollsters Learned Nothing from 2016](https://www.acsh.org/news/2020/11/04/pollsters-learned-nothing-2016-15129)**
>
> Polling is more art than science. Once again, American pollsters have shown that they aren't as good as we thought.
The author has some other articles, notable he points out that Nate Silver’s super-precise percentages are nonsense:
> **[Nate Silver's 'Chance of Winning' Election Forecasts Are Complete Nonsense](https://www.acsh.org/news/2016/11/07/nate-silvers-chance-winning-election-forecasts-are-complete-nonsense-10414)**
>
> Nate Silver, statistician and election forecaster, told ABC News that election forecasts that gave Hillary Clinton a 99% of chance of winning didn't "pass a common sense test." That is certainly true. What he left unsaid -- possibly because it...
---
### Author:  [@Fotheringay-Phipps](https://boards.straightdope.com/u/Fotheringay-Phipps)
#### Post date: [November 5, 2020, 3:11pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/118 "2020-11-05T15:11:46Z")
Re 538, the key question is how his predictions stack up against the cruder straight-up RCP average. IOW, is his sophisticated analysis really adding anything. (I don’t know, but would be interesting if someone looked.)
> [@PhillyGuy](#):
>
> I’m not convinced that the polls this cycle were unusually far off on average. Maybe they were, but it is too soon to say this.
>
> How about – in a close election, you can’t trust the polls.
I think this is it. I don’t think the polls were particularly bad this election. But if polls are projecting that so-and-so is ahead by 12 points and he wins by 8, no one pays attention. It’s when it’s striking distance of changing the outcome that it gets a lot more attention.
I believe the margin of error for polls is a lot higher than the numbers that are typically published. This is because, as noted earlier in this thread, there is a lot of data manipulation that goes on when publishing the results of a poll. I’m pretty sure that the published margin for error only includes the population-size driven possibility of random fluctuation, but does not include the possibility that the data adjustments themselves were off. (The reason I think so is because the adjustments don’t have a known and measurable variance, so it’s hard to imagine how these would be reflected.) But in reality the adjustments themselves do offer some possibility of error as well. To some extent this is mitigated by aggregating numerous polls, but it’s not total, and there’s some herding as well, so hard to know.
> [@Acsenray](#):
>
> I say racehorse polls are inherently bad during an election. That’s different than gathering data to formulate policy.
I think the opposite is true. The polls during the election are as good as you can get, and the “gathering data to formulate policy” polls are inherently bad. The polls during the election are straightforward yes/no questions (unless they’re push-polls by dubious pollsters), but policy question polls are highly dependent on the way they’re presented and phrased and are far less reliable.
---
### Author:  [@Acsenray](https://boards.straightdope.com/u/Acsenray)
#### Post date: [November 5, 2020, 3:23pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/119 "2020-11-05T15:23:25Z")
By “inherently bad” I mean “bad for us as a country,” not “of poor quality.”
There’s no good reason to feed the public racehorse polls during an election. It just makes things worse.
---
### Author:  [@DMC](https://boards.straightdope.com/u/DMC)
#### Post date: [November 5, 2020, 3:29pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/120 "2020-11-05T15:29:31Z")
> [@Deeg](#):
>
> The NYTIMES disagrees with you: [https://www.nytimes.com/2020/11/04/us/politics/poll-results.html](https://www.nytimes.com/2020/11/04/us/politics/poll-results.html)
Please stop replying to shit without quoting which portion of a post you are replying to, especially if you are going to cherry-pick which part to respond to. I’m going out on a limb and assume that you are responding to me stating that the Trafalgar group has mediocre historical performance and that their grade is based on the numbers behind that performance. If so, they can disagree all that they want. They’d be wrong, but that’s on them.
But, since this is the hill you have chosen, okay:
He has polled only 11 states since the beginning of October, so I’ll stick to the most recent versions of the polls for each state (I’m not cherry-picking, he just seemed to focus on these, most of which are battleground states). Here are the picks:
Georgia - Trump
Nevada - Trump
Florida - Trump
Pennsylvania - Trump
Ohio - Trump
Michigan - Trump
North Carolina - Trump
Arizona - Trump
Minnesota - Biden
Wisconsin - Biden
Louisiana - Trump
While I’ll gladly revisit this if the outcome changes (and it appears possible that Arizona could flip back), the current consensus with numbers people is that PA and GA are both about to flip for Biden, while NC is likely not going to make that switch.
If that happens, he’ll have 6 out of 11 picks correct, for 55% of races called correctly. Stellar.
Here’s the 538 take on the same states:
Georgia - Biden
Nevada - Biden
Florida - Biden
Pennsylvania - Biden
Ohio - Trump
Michigan - Biden
North Carolina - Biden
Arizona - Biden
Minnesota - Biden
Wisconsin - Biden
Louisiana - Trump
If the above outcomes pan out, that’s 9/11, or 82%. Again, feel free to call me out when this is over if everything above changes.
Also, they are acting a bit shady right now and hiding things that make them look bad:
>
>
> Nate Silver @NateSilver538
It should also be noted that the guy behind Trafalgar, Robert Cahaly is pretty clearly a Trump guy, so it shouldn’t come as a shock that he scored well in 2016. [Here’s his take on the polls in PA](https://www.washingtonpost.com/outlook/2020/11/01/meta-divide-over-predicting-2020-election/):
> I believe Pennsylvania to be the No. 1 state that Trump could win and have stolen due to voter fraud.
Does that sound like a pollster or Fucker Carlson?
Again, they’re not banned, so they are absolutely included in the 538 results, they just have a shitty (maybe me calling them mediocre was too generous) pollster score. Rightfully so. The good news is that the scores are based on actual data, not Nate’s opinion, so if they get their shit together, their score will improve.
---
### Author:  [@iamthewalrus\_3](https://boards.straightdope.com/u/iamthewalrus_3)
#### Post date: [November 5, 2020, 3:32pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/121 "2020-11-05T15:32:39Z")
Susan Collins outperformed her polling average by like 8 points, in Maine, a state Trump lost.
That’s not explained by voter intimidation or suppression. Susan Collins is one of the most moderate Republian Senators, so it’s not explained by “shy” voters or purposefully lying to pollsters.
Polling is hard!
---
### Author:  [@Acsenray](https://boards.straightdope.com/u/Acsenray)
#### Post date: [November 5, 2020, 3:34pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/122 "2020-11-05T15:34:16Z")
> [@Pantastic](#):
>
> The author has some other articles, notable he points out that Nate Silver’s super-precise percentages are nonsense:
I agree that Silver’s whole “under these circumstances Biden wins 60 percent of the time” is just useless statistical wankery. So what if running a model produces X result Y times? What does that mean for us as the political public? Zero. Nothing. We should just ignore that crap. It’s not informative in any meaningful sense.
---
### Author:  [@Acsenray](https://boards.straightdope.com/u/Acsenray)
#### Post date: [November 5, 2020, 4:02pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/123 "2020-11-05T16:02:04Z")
Slate: Election forecasts are useless. Ban them.
> **[The Problem Isn’t That the Polls Were Wrong. It’s That They Were Useless.](https://slate.com/news-and-politics/2020/11/fivethirtyeight-upshot-polls-useless.html)**
>
> Ban election forecasts, or at least ignore them.
> It’s not all that comforting to Democrats today to know that 9 out of 10 times this election happens in the greater multiverse, Biden will win it. As former FiveThirtyEight writer Mona Chalabi [put it](https://twitter.com/MonaChalabi/status/1323981536962748418/photo/4), assuming the voice of FiveThirtyEight’s much-derided Fivey Fox mascot, “No matter what happens, I will find a way to say ‘I told you so! That’s how probabilities work!’ ”
> … an entire industry of pundits and soothsayers have turned polling analysis into something more like a religion while proclaiming it a science. Meanwhile, it is increasingly unclear why these projections are useful at all.
> In truth, the polls only reveal one sliver of the reality of elections, and especially this election. But unlike in baseball, which Silver first made a name for himself analyzing before shifting to politics, this game doesn’t always have a predictable set of rules that all players abide by. There’s much more noise in the signal that can interfere with an algorithm.
> The polling gurus portray themselves as objective number-crunchers, unswayed by human bias or emotion. But in truth, they are in the reassurance business. … But they’re hawking a false sense of certainty—and, presumably, racking up earth-shattering levels of web traffic in the process.
[Previous page](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714.md?page=5)
[Next page](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714.md?page=7)