Reply
 
Thread Tools Display Modes
  #1  
Old 08-01-2019, 11:53 AM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862

Polling as November 2020 nears: Will it be more accurate than before November 2016?


Last time around, the "shy Trump voter" effect was blamed for the polling being wrong - the idea being that many Trump voters weren't willing to be truthful to pollsters.

I don't see how the pollsters can fix that in next year's election either, because being a Trump voter is still as being just as socially bad as it was in 2016, if not in fact even worse. Hence, Trump voters have even more incentive to lie to pollsters than before. So are the pollsters going to add an artificial 2-4% Trump boost to their polling in order to cover the expected discrepancy?
  #2  
Old 08-01-2019, 12:02 PM
Chingon is offline
Guest
 
Join Date: Dec 2006
Location: the hypersphere
Posts: 563
It was?
  #3  
Old 08-01-2019, 12:08 PM
iiandyiiii's Avatar
iiandyiiii is offline
Member
 
Join Date: Aug 2010
Location: Arlington, VA
Posts: 34,948
Do you have a cite for this? In my understanding, the polling in 2016 was only slightly less accurate than usual. The national polling was quite good, and all states except for a few were fine, in my understanding. So no, I don't expect pollsters will add any sort of boost to any candidate; doing so would likely result in reduced accuracy.

If you're not following Nate Silver with respect to polling, you're doing it wrong.
  #4  
Old 08-01-2019, 12:12 PM
dalej42 is online now
Charter Member
 
Join Date: Feb 2004
Location: Chicago
Posts: 14,290
The polling was fine. You don’t fire a weather forecaster for being off by one degree in their forecast. Trump won by the narrowest of margins in three critical states.

And, of course, most of the polls weren’t able to take that damn Comey letter into effect. How many reluctant Clinton voters threw their hands in the air and said, ‘none of the above!’
__________________
Twitter:@Stardales IG:@Dalej42
  #5  
Old 08-01-2019, 12:14 PM
JKellyMap's Avatar
JKellyMap is offline
Member
 
Join Date: Mar 2005
Location: Wisconsin
Posts: 9,476
Polling was overall quite accurate, despite inherent difficulties in this cell-phone era. The aggregate models of FiveThirtyEight, for example, accurately foresaw a rather high chance that Trump would win the electoral vote — close to 30% chance, as I recall. There was some “shy Trump” voting going on, but as I recall this was less a factor than (among other things) the portion of voters to non-voters being somewhat different than expected within certain demographic groups. Polling methods continue to evolve and improve slowly, but the basic approach needn’t change. The same goes for aggregate modeling methods — there are always lessons learned to incorporate, but the “shy Trump voter” is a small part of of all this, and no one needs to categorically add some fixed percentage to account for it.
  #6  
Old 08-01-2019, 12:25 PM
asahi's Avatar
asahi is offline
Guest
 
Join Date: Aug 2015
Location: On your computer screen
Posts: 10,574
The polling nationally was actually very accurate. The reason why pollsters were off is the fact that Trump won specific states by very thin margins, which was the difference in the race. But the polling had Clinton winning anywhere from 1-3 percent nationally, and that's pretty much what happened. Unfortunately, she lost the EC and it was a surprise because there wasn't nearly as much good polling data at the state level, which was one of the reasons why Nate Silver disagreed with Sam Wang and said that there was a lot more uncertainty in the polls than people believed at the time.

I don't think there were "shy" Trump voters. More like unaccounted for Trump voters, and voters who were showing up to vote for the first time in several election cycles, and in some cases the first time ever. There were also Clinton voters who simply failed to show up at the polls. Moreover, there were 3rd and even 4th party candidates who, while receiving less than 5% of the vote, were enough to swing the races in some states.

Last edited by asahi; 08-01-2019 at 12:28 PM.
  #7  
Old 08-01-2019, 12:44 PM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Quote:
Originally Posted by JKellyMap View Post
Polling was overall quite accurate, despite inherent difficulties in this cell-phone era. The aggregate models of FiveThirtyEight, for example, accurately foresaw a rather high chance that Trump would win the electoral vote — close to 30% chance, as I recall.
FiveThirtyEight was the closest, but even then, it put Trump's chances of winning in the minority, not the majority.


Other sources were wildly off:

"Hillary has a 99 percent chance of winning"
"Hillary has 98 percent chance of winning, and Democrats stand a strong chance of capturing the Senate"
"Hillary has 91 percent chance of winning"
"Hillary has 99 percent chance of winning" (again)
  #8  
Old 08-01-2019, 12:46 PM
ElvisL1ves is offline
Charter Member
 
Join Date: Jul 2000
Location: The land of the mouse
Posts: 50,109
And that leads to suspicion that it wasn't the polls that were off.
  #9  
Old 08-01-2019, 12:48 PM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Quote:
Originally Posted by ElvisL1ves View Post
And that leads to suspicion that it wasn't the polls that were off.
Well, OK, sure, but what are the predictors going to do to ensure that their predictions of win % probability are better this time than in 2016?



I feel like I should have reworded my thread title to "election forecasting" instead of polling. The raw polling data itself might have been okay, but the way the forecasters predicted all-but-guaranteed victory for Hillary sure wasn't.
  #10  
Old 08-01-2019, 12:49 PM
ElvisL1ves is offline
Charter Member
 
Join Date: Jul 2000
Location: The land of the mouse
Posts: 50,109
There's no way any statistician can account for tampering. The process assumes the data are genuine.

Last edited by ElvisL1ves; 08-01-2019 at 12:50 PM.
  #11  
Old 08-01-2019, 12:59 PM
JKellyMap's Avatar
JKellyMap is offline
Member
 
Join Date: Mar 2005
Location: Wisconsin
Posts: 9,476
Quote:
Originally Posted by Velocity View Post
FiveThirtyEight was the closest, but even then, it put Trump's chances of winning in the minority, not the majority.
You misunderstand. They ran their model a thousand times each week, and in that final week, in THREE HUNDRED out of those thousand runs, Trump straight-out WON. Reality happened to then coincide with one of those THREE HUNDRED runs. Not surprising in the least.

If there were ten identical situations — ten contests where the exact same 538 model happened to apply, and the exact same polling numbers were generated each week as they were in this instance — and the “Hillary” candidate won all ten races, THAT would indicate serious PROBLEMS with the model. You’d EXPECT her to straight-up lose around three of those times. Which she did.
  #12  
Old 08-01-2019, 01:17 PM
guizot is offline
Member
 
Join Date: Mar 2005
Location: An East Hollywood dingbat
Posts: 8,634
The problem isn't that polls "weren't accurate" (it's the predictions some made which weren't accurate), but that people in general don't understand statistics, (as the premise of the OP demonstrates).
  #13  
Old 08-01-2019, 01:21 PM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Quote:
Originally Posted by ElvisL1ves View Post
There's no way any statistician can account for tampering. The process assumes the data are genuine.
Are you saying Trump did not get the 62.9 million votes, or that someone messed with the polling data?
  #14  
Old 08-01-2019, 01:23 PM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Quote:
Originally Posted by guizot View Post
The problem isn't that polls "weren't accurate" (it's the predictions some made which weren't accurate), but that people in general don't understand statistics, (as the premise of the OP demonstrates).
When multiple sources claim a 98-99% chance of victory for Hillary, then something did in fact go wrong with the modeling or assessment.

If they had claimed a 70-80% chance of victory, one could claim that Trump simply happened to be the lucky red card drawn out of a shuffled hand that included three or four other blue cards, but when they go with a 99% prediction for Hillary, that's way out of mere chance.
  #15  
Old 08-01-2019, 01:32 PM
E-DUB's Avatar
E-DUB is offline
Guest
 
Join Date: Mar 2013
Posts: 4,668
If you have three states, each of which has a one third chance of going for trump, getting all three of them sure sounds like a real long shot, one the order of one in twenty-seven. Which does calculate out to about a three percent chance. The problem with this mode of thinking is that it regards them as discrete events, like coin tosses. Except they aren't discrete events, they're inter-related so that something (or someone) that helped trump in one has an effect in the others.

I have said that I think that there are trump voters who lie to pollsters, either from shame or perversity and I don't know how you could compensate for that as even a larger sample might simple include more of them folks.
  #16  
Old 08-01-2019, 01:47 PM
guizot is offline
Member
 
Join Date: Mar 2005
Location: An East Hollywood dingbat
Posts: 8,634
Quote:
Originally Posted by E-DUB View Post
Except they aren't discrete events, they're inter-related so that something (or someone) that helped trump in one has an effect in the others.
Like Manafort's list, maybe. It would include all three.

And really, those three situations are why Trump is president. It was such a small percentage of those states, let alone the country as a whole. And yet, people took it as some kind of huge, ignored part of the country. That narrative is bullshit. The rest of Trump's electoral wins were mostly people just automatically voting Republican, as they always do.
  #17  
Old 08-01-2019, 02:00 PM
ElvisL1ves is offline
Charter Member
 
Join Date: Jul 2000
Location: The land of the mouse
Posts: 50,109
Quote:
Originally Posted by Velocity View Post
Are you saying Trump did not get the 62.9 million votes, or that someone messed with the polling data?
Given the reliability of previous polling processes in most previous elections, a sudden and massive failure is not best explained by a failure of the process.

You know about Russian hackers. You knew about Diebold in earlier elections. You know about Republican vote suppression tactics both before and after the fact. You don't know what else you don't know. Do you still trust that the announced vote totals accurately reflected who all the voters thought they were voting for?
  #18  
Old 08-01-2019, 02:23 PM
sps49sd is online now
Member
 
Join Date: Feb 2009
Posts: 454
Maybe you should start a poll on this question.

Quote:
Originally Posted by ElvisL1ves View Post
Given the reliability of previous polling processes in most previous elections, a sudden and massive failure is not best explained by a failure of the process.

You know about Russian hackers. You knew about Diebold in earlier elections. You know about Republican vote suppression tactics both before and after the fact. You don't know what else you don't know. Do you still trust that the announced vote totals accurately reflected who all the voters thought they were voting for?
Yes. Russian 'hackers' were trying to influence people via social media and furnishing truthful data

Last edited by sps49sd; 08-01-2019 at 02:25 PM.
  #19  
Old 08-01-2019, 02:26 PM
ElvisL1ves is offline
Charter Member
 
Join Date: Jul 2000
Location: The land of the mouse
Posts: 50,109
You think that's all? Let's hope so.
  #20  
Old 08-01-2019, 02:30 PM
sps49sd is online now
Member
 
Join Date: Feb 2009
Posts: 454
Maybe you should start a poll on this question.

Quote:
Originally Posted by ElvisL1ves View Post
Given the reliability of previous polling processes in most previous elections, a sudden and massive failure is not best explained by a failure of the process.

You know about Russian hackers. You knew about Diebold in earlier elections. You know about Republican vote suppression tactics both before and after the fact. You don't know what else you don't know. Do you still trust that the announced vote totals accurately reflected who all the voters thought they were voting for?
Yes. I do.

Russian 'hackers' were trying to influence people via social media and furnishing truthful data concerning the Hillary email server. Diebold is fuzzy. Any effects of voter suppression would not likely end with a voter voting incorrectly.

(Sorry, my post was partially eaten)
  #21  
Old 08-01-2019, 02:32 PM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Quote:
Originally Posted by ElvisL1ves View Post
Given the reliability of previous polling processes in most previous elections, a sudden and massive failure is not best explained by a failure of the process.
IMHO, one reason many sources picked Hillary so overwhelmingly was because the idea of Trump winning sounded ludicrous to their ears. If it had been any other candidate - say, Romney vs. Hillary, but with the exact same data - they wouldn't have given Clinton 99%; they would have lowered her odds. But the idea of Trump winning sound like the idea of President Whoopi Goldberg or President Kanye West; unfathomable.


But anyway, I didn't mean this thread to be another postmortem analysis about 2016; we've had many of those - I wanted to ask what the pollsters and analysts will do different this next upcoming time around.
  #22  
Old 08-01-2019, 02:51 PM
iiandyiiii's Avatar
iiandyiiii is offline
Member
 
Join Date: Aug 2010
Location: Arlington, VA
Posts: 34,948
Quote:
Originally Posted by Velocity View Post
But anyway, I didn't mean this thread to be another postmortem analysis about 2016; we've had many of those - I wanted to ask what the pollsters and analysts will do different this next upcoming time around.
It wasn't pollsters, again. It was predictors. The ones with sense will rely on the models with proven records, like Nate Silver's.
  #23  
Old 08-01-2019, 02:51 PM
enalzi is online now
Guest
 
Join Date: Apr 2008
Location: Chicago, IL
Posts: 8,005
Quote:
Originally Posted by E-DUB View Post
If you have three states, each of which has a one third chance of going for trump, getting all three of them sure sounds like a real long shot, one the order of one in twenty-seven. Which does calculate out to about a three percent chance. The problem with this mode of thinking is that it regards them as discrete events, like coin tosses. Except they aren't discrete events, they're inter-related so that something (or someone) that helped trump in one has an effect in the others.
Yep, this is why Nate was giving Trump 30% when everyone else was giving him 1%. They were all saying "He would have to win PA, MI, AND WI! What are the odds of that?" He was saying "If he wins PA, he's probably going to win MI and WI too."

I'm actually very curious to see how models will change in reaction.
  #24  
Old 08-06-2019, 06:05 PM
Thing Fish is offline
Guest
 
Join Date: Sep 2001
Location: Chicago (NL)
Posts: 3,354
Quote:
Originally Posted by Velocity View Post
Well, a lot of people are dumb.

Your first link is to a story about a study done by Sam Wang, who was getting a lot of media attention in the last election but is an idiot. The only real problem with this story is the clickbait headline; the story itself explains that other predictors were less optimistic.

No idea what the "HuffPo Model" is, but clearly it sucks.

The third link discusses prediction markets, not polls.

And the fourth is to a student project which commits the basic error EDUB explains in post 15. At least, unlike HuffPo, they explain their methodology clearly enough that it's easy to see where they went wrong.

The problem is that a lot of the media folks who are reporting the poll results aren't good at math. I don't see this significantly improving in the near future, but we can hope.
  #25  
Old 08-06-2019, 06:24 PM
Thing Fish is offline
Guest
 
Join Date: Sep 2001
Location: Chicago (NL)
Posts: 3,354
Quote:
Originally Posted by Velocity View Post
IMHO, one reason many sources picked Hillary so overwhelmingly was because the idea of Trump winning sounded ludicrous to their ears. If it had been any other candidate - say, Romney vs. Hillary, but with the exact same data - they wouldn't have given Clinton 99%; they would have lowered her odds. But the idea of Trump winning sound like the idea of President Whoopi Goldberg or President Kanye West; unfathomable.


But anyway, I didn't mean this thread to be another postmortem analysis about 2016; we've had many of those - I wanted to ask what the pollsters and analysts will do different this next upcoming time around.
I think it's absolutely true that a lot of media coverage was biased due to reporters being unable to imagine that Trump could actually win; Nate Silver had a series of articles about this last year. The New York Times was particularly egregious. But that shouldn't be an excuse if you're claiming to be using mathematical models to interpret polling data; the subjective opinion of the person running the models shouldn't affect those results at all.
  #26  
Old 08-06-2019, 06:56 PM
CarnalK's Avatar
CarnalK is offline
Guest
 
Join Date: Jul 2000
Posts: 17,917
Quote:
Originally Posted by Velocity View Post
None of those are polls. Those are people interpreting polls and trying to guess odds. The fact that polls were way out is a myth.
  #27  
Old 08-06-2019, 08:34 PM
DSeid's Avatar
DSeid is online now
Guest
 
Join Date: Sep 2001
Location: Chicago, IL
Posts: 22,318
So established - national polling was actually quite good, with a few critical states being a little off. Many pundits were not good at speculating odds based on the fairly good polling.

Will pundits and talking heads in aggregate do a better job of understanding how to interpret the numbers this time?
  #28  
Old 08-06-2019, 08:40 PM
jshore is offline
Charter Member
 
Join Date: May 2000
Posts: 6,963
FWIW, I think what Nate Silver found in his post-election analysis that the systematic bias in polls is that they tended to underestimate Trump's strength in states with a larger percentage of white, non-college-educated voters and tended to overestimate Trump's strength in states with a smaller percentage of white, non-college-educated voters. So, overall, nationwide, it was pretty much of a wash. But, it turned out in the states that mattered, i.e., the battleground states, the proportion of white, non-college-educated voters was higher than for the nation as a whole.

Last edited by jshore; 08-06-2019 at 08:40 PM.
  #29  
Old 08-07-2019, 12:28 PM
RTFirefly is offline
Charter Member
 
Join Date: Apr 1999
Location: Maryland
Posts: 39,244
Quote:
Originally Posted by DSeid View Post
So established - national polling was actually quite good, with a few critical states being a little off. Many pundits were not good at speculating odds based on the fairly good polling.
Excellent summary.
Quote:
Will pundits and talking heads in aggregate do a better job of understanding how to interpret the numbers this time?
Let's just say my expectations aren't high.
  #30  
Old 08-08-2019, 12:03 AM
JKellyMap's Avatar
JKellyMap is offline
Member
 
Join Date: Mar 2005
Location: Wisconsin
Posts: 9,476
Nate Cohn talks about shy Trump voters (there seem to be few of them now) in this piece:
https://www.nytimes.com/2019/08/07/u...gtype=Homepage
  #31  
Old 08-08-2019, 12:29 AM
Heffalump and Roo is offline
Guest
 
Join Date: Nov 2006
Posts: 4,249
People are biased. People are blind to their biases. No, I don't expect that to change any time soon.
  #32  
Old 08-08-2019, 09:48 AM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Tangential question about polls: Are they ever subject to being audited by an outside entity? What prevents a survey group from simply fabricating BS polling data? (claiming, "we surveyed 2,765 likely voters in an effective sample size by telephone" when in fact they did no such thing and just spun up BS stuff on the spot). As long as their "data" isn't too far off from all of the other Gallup polls, who would question it?
  #33  
Old 08-08-2019, 10:18 AM
enalzi is online now
Guest
 
Join Date: Apr 2008
Location: Chicago, IL
Posts: 8,005
Quote:
Originally Posted by Velocity View Post
Tangential question about polls: Are they ever subject to being audited by an outside entity? What prevents a survey group from simply fabricating BS polling data? (claiming, "we surveyed 2,765 likely voters in an effective sample size by telephone" when in fact they did no such thing and just spun up BS stuff on the spot). As long as their "data" isn't too far off from all of the other Gallup polls, who would question it?
What would be the point in making up a bunch of data if they're just going to match what everyone else is saying?
  #34  
Old 08-08-2019, 10:28 AM
DSeid's Avatar
DSeid is online now
Guest
 
Join Date: Sep 2001
Location: Chicago, IL
Posts: 22,318
Some anecdotes (both 538 as source) ...

Research 2000.

More broadly about the problem of fake polls.
Quote:
... Maybe they conducted some polling and maybe they didn’t, but their lack of transparency and shady behavior make it easy to disqualify their work. But let’s say they were more transparent, and their data looked more legitimate. Even then it’s not clear whether news outlets should take their results seriously.

FiveThirtyEight has traditionally accepted any poll from any firm so long as we don’t have evidence the poll or pollster is an outright fake. But ...
  #35  
Old 08-08-2019, 10:38 AM
Velocity is online now
Guest
 
Join Date: Jun 2014
Posts: 14,862
Quote:
Originally Posted by enalzi View Post
What would be the point in making up a bunch of data if they're just going to match what everyone else is saying?
Save trouble and energy, money and resources - and also have the opportunity to further skew things in favor of or against someone.
  #36  
Old 08-08-2019, 12:24 PM
iiandyiiii's Avatar
iiandyiiii is offline
Member
 
Join Date: Aug 2010
Location: Arlington, VA
Posts: 34,948
Quote:
Originally Posted by Velocity View Post
Tangential question about polls: Are they ever subject to being audited by an outside entity? What prevents a survey group from simply fabricating BS polling data? (claiming, "we surveyed 2,765 likely voters in an effective sample size by telephone" when in fact they did no such thing and just spun up BS stuff on the spot). As long as their "data" isn't too far off from all of the other Gallup polls, who would question it?
Nate Silver rates polling outfits based on history, methodology, and transparency. If a brand new poll came about with no history, methodology, or transparency, he'd likely pay it very little attention and give it an F rating.
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 03:52 PM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.

Send questions for Cecil Adams to: cecil@straightdope.com

Send comments about this website to: webmaster@straightdope.com

Terms of Use / Privacy Policy

Advertise on the Straight Dope!
(Your direct line to thousands of the smartest, hippest people on the planet, plus a few total dipsticks.)

Copyright © 2018 STM Reader, LLC.

 
Copyright © 2017