These polling organizations are selling themselves as someone who can reliably predict the future based on what a targeted audience is currently thinking. Better than their customers can, anyway. The majority of polls pointed to a Hillary win. They were proven wrong.
538 is a polling aggregator which relies on the results of other polling organizations in order to make a living. They add their own spin to the mix, and then offer odds as to who will win what.
However, the term “garbage in - garbage out” still applies.
When the other polls are failing to accurately reflect/predict/guess as to what the voters are currently thinking, any polling aggregator which uses those faulty numbers will fail to produce accurate results.
*"In an appearance on ABC’s “This Week,” the FiveThirtyEight chief claimed Clinton is a “2-to-1 favorite,” but noted that recent polls show Republican presidential nominee Donald Trump with a slight edge in electoral college-heavy states like Florida, Ohio, and North Carolina.
“The electoral college math is actually less solid for Clinton than it was for Obama four years ago, where four years ago we had Obama ahead 320-some electoral votes. Clinton has about 270,” Silver said.*
Clinton has about 270, except when she doesn’t. Oops.
Do you work for a probability-based analyses firm?
Just to clarify, if the individual poll numbers were wrong, then anyone who uses those faulty numbers to create a new guesstimation will also be wrong.
I had a similar reaction. The aggregate polling there and elsewhere had Hillary as very, very likely to win for months. I can’t tell you how disappointed I was to see them get it so very, very wrong. Ugh.
Yes the majority of polls pointed to a Hillary win and did not happen. The RCP average of national polls looks to have been about 3.0% too optimistic about Hillary. Then again that average was 3.2% too optimistic about Romney’s actual vote (final RCP avg of national popular minus actual vote margin). I take facts like this to common knowledge. The mean of national polls were off significantly the last two times, in different directions as to which party they favored, compared to the several times before that.
But in case of poll analyzers like Silver, I think there’s still a basic way in which you and those responding to you are talking past one another. And it’s reflected in the quote you gave. Silver is talking about probabilities, 2 to 1. He is speaking and thinking in a probabilistic way. You seem to be part of the large population IME who simply does not or refuses to think probalistically. ‘Tell me what will happen, or what use are you?’ But again nobody can tell you what will happen. Whether they pretend to be able to or not is secondary. Silver definitely does not.
And by the same token your fairness in quoting Silver as saying the odds were 2 to 1 for Clinton, not ‘Clinton will win’, undercuts your criticism of him saying ‘she has 270’. Since according to the previous statement in the quote, and every statement pretty much Nate Silver ever makes, meant the expected value of her outcome is 270, not ‘she will get 270 and not any other number’.
Which again gets back to the valid criticism of 538’s model. Just because Trump won does not invalidate a statement that he faced 2 to 1 odds. Again you have to really willfully ignore the whole concept of odds and probabilities to say that. OTOH, the fact that he won certainly doesn’t prove that the probability was 2 to 1. You’d need lots of trials in similar elections to see if a call of 2 to 1 odds, ~65% chance of Clinton winning as of the time Silver said that, actually panned out around 2/3’s of the time.
But again, I’m getting the feeling you just don’t accept the concept of there being any such thing as odds or a probability of an election outcome. You seem rather to think there is a final outcome, so some really smart person should be able to figure it out beforehand, for sure. You could be even be right some pollsters think that way and think they are that person (though they are not, no one could be). But Silver definitely does not look at things that way.
This is a tempting narrative, but it doesn’t square with the fact that Trump got fewer votes than Romney, despite there being 10 million more Americans now than there were in 2012. The story is not the Sean Trende “missing white voters” of 2012 rising up in 2016. It’s the millions of missing Democratic voters who caused her to lose, barely, in the Electoral College, while still winning the popular voye comfortably. It’s still true that no non-incumbent Republican has won the national popular vote since the 1980s.
You do realize that she missed getting 270 by less than 100,000 votes out of 120 million cast?
I’m not saying 538 is a biased, unreliable, head-up-their-ass polling organization. I’m saying that 538 relies on a collection of other polling organization’s data. Something that even 538 admits to doing. Since many/most of those other polling organization were wrong, 538 used faulty data to arrive at their conclusion. Garbage in - garbage out. Capisce?
According to 538, they take four major steps to arrive at their predictions.
*Four major steps
All versions of the model proceed through four major steps:
Step 1: Collect, weight and average polls.
Step 2: Adjust polls.
Step 3: Combine polls with demographic and (in the case of polls-plus) economic data.
Step 4: Account for uncertainty and simulate the election thousands of times.*
On Nov 8th, the NYT showed that 9 out of 10 polls indicated a Hillary win. (9 out of 10 polls were wrong.)
Polls - Dates - Type, Respondents - Clinton - Trump - Margin
YouGov/Economist New - 11/4 - 11/7 - Online3,669 - 45 - 41 - Clinton +4
IBD/TIPP New - 11/4 - 11/7 - Live Phone1,107 - 41 - 43 - Trump +2
Insights West New - 11/4 - 11/7 - Online940 - 45 - 41 - Clinton +4
Bloomberg/Selzer New - 11/4 - 11/6 - Live Phone799 - 46 - 43 - Clinton +3
Lucid/The Times-Picayune New - 11/4 - 11/6 - Online931 - 45 - 40 - Clinton +5
Fox News New - 11/3 - 11/6 - Live Phone1,295 - 48 - 44 - Clinton +4
Monmouth University New - 11/3 - 11/6 - Live Phone748 - 50 - 44 - Clinton +6
ABC News/Washington Post New - 11/3 - 11/6 - Live Phone2,220 - 47 - 43 - Clinton +4
New York Times/CBS News New - 11/2 - 11/6 - Live Phone1,426 - 47 - 43 - Clinton +4
Rasmussen New - 11/2 - 11/6 - I.V.R./Online1,500 - 45 - 43 - Clinton +2
Garbage in, garbage out. It’s up to 538 to adjust it’s methods and procedures to compensate for faulty data.
Not only admits it, but was criticized for (what we now know in hindsight) accurately quantifying the errors in the polling. He was telling us how stinky the garbage was the whole time.
Doorhinge, I think the fundamental disconnect here is that I’m not sure you even understand that we are saying that if Nate Silver says Trump has a 35% chance of winning the election, and then he does win the election, that does not mean he made an incorrect prediction. In fact, if Nate Silver predicted things to happen 65% of the time 100 times in a row and they happensed 95 of 100 times, that means his prediction was wrong, most likely. He would want it to go the other way 35% of the time, because otherwise the true probability was probably closer to 95% and therefore 65% was incorrect… So although the sample size is too small to say for sure, there’s no reason to think that he predicted anything wrong at all. We are just in the 35% chance.
Look at it this way. If I am about to cut the deck to reveal a random card, and I say thereis 75% chance that it will be something other than a spade, and then I cut the deck and it’s a spade, that doesn’t mean what I said before was wrong. We just happened to hit the other 25%.
NB:As of Friday, Nov. 11, 2016 at 11:59 p.m. CST, I’m unsubscribing from all political threads and will no longer participate in discussions in the Elections board, nor in political discussions in the Pit or MPSIMS. If you reply to a political post of mine after that point, I will not see it; please do not PM me to try to pull me back in to the debate. Thanks!
Tell that to the people who disagreed with my previous post. 538 fucked up by relying on faulty poll numbers taken from other polling organizations. Garbage in, garbage out. 538 never had a chance of accurately predicting the winner. :smack:
I have been saying that 538 could not produce accurate results because 538 relied on faulty information taken from other polling organizations. Garbage in, garbage out. This isn’t rocket surgery.
But it was accurate, in the sense that the results were within the uncertainty limits they computed. A better complaint is that their uncertainty was large (high imprecision), but that large uncertainty was justified, because as you say, the input polls were poor. Your complaint against 538 is not fair, because they had the model that actually captured the problems of the polling.
Your complaint is very valid against Sam Wang and PEC, because they gave a high confidence to the polls that was wholly unjustified. And many of us were complaining about that even before the election.
To put it another way, if Silver was just flat out wrong, full stop, how come he was engaged in a bitter argument the day before the election with a HuffPo prognosticator who gave Hillary more than a 95% chance to win? What was the substance of their disagreement, if they are both to be now categorized as simply wrong in the same way?
Thanks.
NB: *As of Friday, Nov. 11, 2016 at 11:59 p.m. CST, I’m unsubscribing from all political threads and will no longer participate in discussions in the Elections board, nor in political discussions in the Pit or MPSIMS. If you reply to a political post of mine after that point, I will not see it; please do not PM me to try to pull me back in to the debate. Thanks![/I
I was polled on Sunday. I don’t have a landline, just a cell. We don’t get polled much here in Oregon, so it was unusual for me to get polled. Ohio residents have a different experience. Many of them stop talking to pollsters.
I suspect the main reason the polls were wrong is that Trump supporters stopped talking to pollsters at a higher rate than Hillary supporters. That built a bias into their polls that they were unaware of.
That and there were a lot more undecided voters than usual, at least according to Nate Silver
I read an article that Trump supporters didn’t trust polls and therefore didn’t answer polls. So they were part of the problem. Maybe if they can learn to cooperate more, they will be more represented in the polls. Kind of ironic overall
Those who were voting for Trump as a protest against the “system” would see the polls as part of what they were protesting against. Not cooperating with them would be expected.
I haven’t read this yet, but Nate Silver himself is obviously frustrated with exactly the same kind of thing we are seeing here. I just heard him on his podcast say that “if we predicted a 30% chance of an earthquake hitting Seattle, and everyone else said ‘No, you’re an idiot, the chance is actually only 1%’ and then the earthquake hits…they can’t turn around and say ‘but hey, you said there was a 70% chance of no earthquake.’” :smack:
This whole argument reminds me of something I read that Chuck Klosterman wrote about probability in one of his books. It initially enraged me with its absurd wrongness, but after I mulled it I decided (in part because I always liked Klosterman so much, and didn’t want to start hating him) that he was pulling his readers’ legs, and didn’t really mean it:
Although even if he’s just putting us on, I do think it was a genuine error to say that 50:50 is the same as 2:1, when of course it is actually 1:1.
NB:As of Friday, Nov. 11, 2016 at 11:59 p.m. CST, I’m unsubscribing from all political threads and will no longer participate in discussions in the Elections board, nor in political discussions in the Pit or MPSIMS. If you reply to a political post of mine after that point, I will not see it; please do not PM me to try to pull me back in to the debate. Thanks!
Part of which problem? The polling organizations failure to accurately predict Hillary’s loss? If the polling organizations wish to make a profit selling polling data, which is their primary goal, the polling organizations have to adapt to the current situation, not the other way around.
There are no rules, regulations, or laws that require the targeted audience to cooperate with what they consider to be intrusive pollsters. If people consider polls to be faulty, biased, or horseshit, they aren’t going to cooperate.