I was going to post this in GQ, however my last question ended up here… So it’s up to the moderator.
I understand poser important, but how much can we trust them? Are they still using land-lines? Who the hell are they polling? Are there any unrepresented people in the polls, such as the youth?
Thanks!
Some of the polls use landlines, some use other phones too. Obviously young people don’t have landlines and don’t answer the phone if they don’t know the number. Nate Silver’s predictions have been getting slowly worse over the years, because he relies on the polling, and it just can’t be relied on any more.
Nate Silver would be surprised to hear that. Here’s his final forecast for 2018, which was pretty much pinpoint (suggesting an average gain of 39 seats for the Democrats, who in fact gained 41 seats).
People think the polls were way off in 2016, but that’s because most people don’t really understand what the polling actually indicated (a very, very close election, with Clinton about three points up in the popular vote). What we got was a very, very close election with Clinton about two points up in the popular vote. 538’s final forecast was that Trump had about a 29% of winning, but stuff that has a 29% chance of happening happens more than a quarter of the time – it shouldn’t really be a surprise when it does happen. (More detailed discussion, from Nate himself, is here.)
There are definitely some issues with reaching younger people and other unrepresented groups (generally, reliable pollsters call cellphones as well as landlines, but it’s still true that some demographic groups may be more likely to pick up than others, and that you have to call a LOT of numbers these days to get a response). They generally try to compensate by weighting by demographic group, so, for instance, if African-Americans are 1/3 of the voting population but only 1/6 of your sample, you count the responses you do get more heavily. This isn’t perfect, especially if you have a small sample to begin with (if you only manage to reach two 18- to 29-year-old Asian-Americans and one of them happens to be a huge Trump fan, you’re probably getting an inaccurate picture of how that demographic votes as a whole). But it works better than you’d think. The NY Times did a whole series of polls in competitive Congressional districts where you got to see all the calls and responses in real time, and it was really interesting to watch the responses “settle” as the calls went on; for the first 100 or so responses, you’d get some wild swings, and then by 350 or so, a clear picture would start to emerge. They didn’t all get it exactly right, but they were mostly pretty accurate (scrolling through the first dozen or so results, I believe the only ones where the NYT failed to show the eventual winner as ahead were Barr-McGrath, which their polling showed as essentially a tie, and Brat-Spanberger). Getting it right about 5/6 of the time is about as well as you can expect a poll in a competitive election to do – it isn’t an exact science, and last-minute events can affect results in ways no poll can measure.
Tl;dr: yes, it’s getting more difficult to reach voters, but pollsters are also getting better and better at compensating and interpreting results, and so far it doesn’t seem to have affected accuracy that much.
Pretty much depends on how much you paid for it. If you paid top dollar, they will tell you exactly what you wanted to hear.
Tris
If your first source doesn’t tell you what you wanted to hear, just pay a different source.
I trust polls but I also realize the MOE is important.
I’m not a conspiracy theorist, but I would not be surprised if the actual vote from the precinct level was altered during the 2016 vote. Trump pretty much had to run a gauntlet of states that he was down 6 -10 pts and won every single one of them. I know it’s uncouth to talk about such a thing being a possibility, but damn, I wish someone had the balls to investigate it.
It all depends on sample size and what we’re talking about.
I think if you want the most accurate polling, the government ups it’s cyber security and hosts a giant server to have every identified American be able to vote online. The people you see in these polls are mostly older people because they don’t spend majority of their time researching things on the internet or playing minecraft.
Seems like you’re talking about head to head election polling. You could also consider issue polling. The latter is inherently dicey and that’s nothing new. Issue opinions often depend on how you ask the question which ‘who are voting for’ doesn’t as much. Because people care to widely varying degrees about particular issues even if they have some opinion. The actual legislative process though depends a huge amount on the people who care a lot about that issue. Polls can try to correct for that but it’s quite difficult, always has been.
And whatever selection issues there are with head to head election polls, they apply all the more to issue polls. There’s seldom any screening comparable to ‘how excited are you about voting in the upcoming election?’.
As far as head to heads, you can torture the statistics till they confess there was no issue with polls in 2016, but there obviously was. Even if the hallowed Nate Silver no longer predicted for the NYT, and he gave Clinton ‘only’ IIRC something like 70% chance of winning, the replacement prediction team at the NYT, still a very major outlet in setting the agenda for discussion at least on one side of the politico-cultural Grand Canyon in the US, said the election was over basically with Clinton as virtually certain winner. That tells you there was a problem. It doesn’t mean polls don’t tell you anything for any type of election. They are obviously still something to consider, and could turn out right on the money in a given case.
It is an open question as to whether the media has gotten over its “a Trump victory can’t happen, it’s just not possible” syndrome from 2016. That is arguably the reason why so few thought Trump had a solid chance of winning in 2016 despite the polls showing him being within striking distance; it struck the media as being like Whoopi Goldberg becoming president; too laughable to be imagined. Had it been someone conventional like Romney or Jeb, but with the exact same polling data as The Donald, the media would have taken the chances of victory much more seriously.
In November 2020, if Trump is trailing in the polls by 2-3%, I’m not sure if the media will think his reelection is more possible than his first election in 2016 or not.
Some things to consider:
Polls are not intended to be predictions of the election outcome. Polls are only estimates of the potential voters’ sentiment at the time the poll was taken. You can use polls as a basis for predictions, but the poll itself isn’t the prediction.
Polls have large margins of error, and the actual error can be larger than the margin. And there can be systematic error, so you don’t always get the correct value by averaging multiple polls.
National polls don’t correspond to any elections because all elections are local, including the Presidential election. So a specific party or Presidential candidate being ahead by 10% doesn’t mean much, it all depends on where the votes are distributed.
I.e. polls are very useful, but also easy to misinterpret or put too much trust in. If you understand all this and construct a model, using state-by-state / district-by-district polls and taking into account the probability of systematic errors, then you can make decent predictions. Nate Silver’s model did pretty well in 2016 - it took into account the probability of systematic errors in state polls (“is it possible that all state polls are off by 0.5%, and if so, how likely is that?”) and concluded that Trump has a ~30% chance of winning, which was much higher than most people thought.