# Doubting the statistical analysis at 538 **URL:** \ **Category:** Politics & Elections\ **Created:** [September 2, 2016, 10:09pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819 "2016-09-02T22:09:00Z")\ **Posts on this page:** 20\ **Page:** 4
**Author:** ![elucidator](https://avatars.discourse-cdn.com/v4/letter/e/8edcca/32.png) [@elucidator](https://boards.straightdope.com/u/elucidator)\ **Post date:** [September 6, 2016, 4:46am UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/61 "2016-09-06T04:46:13Z")
Well, let’s say I bet ten thousand on Trump and he wins, how much would that be in Canadian money? ---
**Author:** ![asahi](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/asahi/32/8693_2.png) [@asahi](https://boards.straightdope.com/u/asahi)\ **Post date:** [September 6, 2016, 10:26am UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/62 "2016-09-06T10:26:11Z")
> [@septimus](#): > > Or, you can accept that the 28% figure is reasonable and ask _ **Why?** _ There are various reasons why polls might be inaccurate. Some pollees may be ashamed to admit they’re going to pull the Trump lever when alone — it would be like admitting they’re the one who laid a juicy fart at the important meeting. “Likely” voters might be upside-down with the usual bunch staying home exasperated, while red-neck yokels proudly cast the first vote of their lives. Trump has a history of going through disastrous stretches of a week or two, before miraculously recovering and regaining his strength. Tomorrow night’s Commander-in-Chief forum might be the first real test to see Trump’s staying power in the rest of this race. If he makes a critical unforced error, he could seal his own fate. I expect, however, a watered-down Trump, like what we’ve seen the past few weeks. > [@septimus](#): > > Some will have a brainstorm when they see Gary Johnson’s name on the ballot. And have the burned Bernie-Brats come to their senses yet? It’s not really clear to me how the 3rd party vote is going to affect races, but if there’s a year in which they make an impact, it’s this one. In 1992, the third party hurt Bush. In 2000, it hurt Gore. This year, I suspect Clinton has more to lose with a successful 3rd party bid as Johnson seems to draw many of the independents who would support either candidate and Stein absolutely draws from candidates who would help Hillary. ---
**Author:** ![Northern\_Piper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/northern_piper/32/5304_2.png) [@Northern\_Piper](https://boards.straightdope.com/u/Northern_Piper)\ **Post date:** [September 6, 2016, 11:02am UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/63 "2016-09-06T11:02:08Z")
[Electoral-vote.com](http://Electoral-vote.com) critiques 538. I love it when stats guys get down and dirty! > **[FiveThirtyEight: An Assessment](https://electoral-vote.com/evp2016/Feature_stories/538.html)** > > Track the election with a red/blue map of the US updated daily using the latest state polls. ---
**Author:** ![CarnalK](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/carnalk/32/486_2.png) [@CarnalK](https://boards.straightdope.com/u/CarnalK)\ **Post date:** [September 6, 2016, 12:34pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/64 "2016-09-06T12:34:36Z")
> [@Northern\_Piper](#): > > [Electoral-vote.com](http://Electoral-vote.com) critiques 538. > > I love it when stats guys get down and dirty! > > [http://www.electoral-vote.com/evp2016/Feature\_stories/538.html](http://www.electoral-vote.com/evp2016/Feature_stories/538.html) That has a nice paragraph that sums it up: > [@](#): > > Beyond that, the whole analysis is not telling us what is happening, merely what could happen if Trump pulls even in the polls. But he’s rarely done so this year, has never done so since the Democratic convention, and there’s currently no particular reason to believe that he will do so in the future. And finally, even if the swing state polls do end up a dead heat across the whole country, we must recall that Clinton needs considerably fewer of those swing states than Trump does. If we hypothetically imagine each of the 12 or 13 swingiest states as a pure coin flip, Clinton may only need two or three coin flips to go her way, depending on which states we’re talking about. In fact, about 86% of the various coin flip combinations end with a Clinton win, about 12% end with a Trump win, and about 2% end in a tie. So, in fact, if the polls do tighten, it is exceedingly likely that the Electoral College will save Clinton. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [September 6, 2016, 1:16pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/65 "2016-09-06T13:16:48Z")
I think the biggest reason why Silver’s numbers for Trump are higher than the other statisticians’ is that he’s putting a bigger weight on black swan events. You can run down a list of things that might happen that could affect the election, assign a probability to each, and bake that into your model… but what if something happens that you didn’t think to put on your list? Those things happen, too, and while it’s often obvious in retrospect that you should have considered them, it wasn’t obvious at the time. All you can do is estimate the probability that there will be _any_ black swan event, and try to figure out how big one is likely to be. That said, though, a black swan could be in either side’s favor. Even just taking the principle of indifference, it would be a 50-50 shot to help or hurt Trump. Giving Trump 1 in 4 odds of winning the whole thing means that you’re first of all assuming a 50% chance of there not only being a black swan, but one large enough to overcome Clinton’s lead, and then assuming a 50% chance that that large black swan is in Trump’s favor. I can’t see either of those being correct: Black swans are by definition not that common, and it seems a heck of a lot more likely for a surprise to hurt Trump than Clinton. ---
**Author:** ![BobLibDem](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/boblibdem/32/3149_2.png) [@BobLibDem](https://boards.straightdope.com/u/BobLibDem)\ **Post date:** [September 6, 2016, 1:28pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/66 "2016-09-06T13:28:19Z")
The news networks and the analysts like Silver all have a vested interest in acting like the race is much closer than it will wind up being. Hillary is going to win and it isn’t going to be particularly close. The only Obama state that Trump might pick up is Iowa, whereas Hillary is competitive in GA and AZ now. PA is a lost cause for Trump, and to be honest FL is not at all likely going red this time, either. Take those two off the board and Trump simply has no path. 90% of us already know who we’ll vote for, probably 85% knew which party had our vote even before the primary season. There simply aren’t any blue states ripe for the picking for Trump, everybody knows that but the media will cover it like a horse race even though one horse is headed for the glue factory. ---
**Author:** ![septimus](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/septimus/32/410_2.png) [@septimus](https://boards.straightdope.com/u/septimus)\ **Post date:** [September 6, 2016, 3:34pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/67 "2016-09-06T15:34:31Z")
> [@Chronos](#): > > That said, though, a black swan could be in either side’s favor. It’s too bad the “black swan” got tagged with “black” becuase there are many colors of black swan! I’m not sure what sort of black swan you envision — a Katrina-scale disaster? the FBI indicting Hillary? a stock market crash? race riots in Milwaukee with hundreds of deaths? I’m afraid the most likely “black swan” is an international crisis deliberately caused by one of Trump’s supporters — Putin, Assad, Xi Jinping or Kim Jong-un. Even ignoring that such a deliberate crisis would be directed at helping Trump, a _random_ foreign crisis is also likely to help Trump: Any increase in American fears and hatred will tend to benefit the candidate for whom Fear and Hatred are the major platform planks. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [September 6, 2016, 5:16pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/68 "2016-09-06T17:16:28Z")
> [@](#): > > Quoth **septimus** : > > I’m not sure what sort of black swan you envision Neither am I-- That’s the whole point. ---
**Author:** ![Dob](https://avatars.discourse-cdn.com/v4/letter/d/b2d939/32.png) [@Dob](https://boards.straightdope.com/u/Dob)\ **Post date:** [September 6, 2016, 5:29pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/69 "2016-09-06T17:29:06Z")
my brain hurts reading this thread… But i’m with the OP in thinking that +70% odds of winning just seem to low. For us as a country to ensure we dont get more “Trumps” in the future I think the margin of victory has to be a landslide… something so bad that nobody like him would try again. I fear if we dont, we will see the same asshats in 4years. While I know debates dont really swing voters (remember Obama’s first debacle agains Romney?) I am very interested in the foregin policy town hall tomorrow. I wish, just wish, they would ask them the captial and leaders of like the 10 largest economies in the world. Would just love that… ---
**Author:** ![Salvor](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@Salvor](https://boards.straightdope.com/u/Salvor)\ **Post date:** [September 6, 2016, 6:10pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/70 "2016-09-06T18:10:35Z")
> [@Dob](#): > > my brain hurts reading this thread… > > But i’m with the OP in thinking that +70% odds of winning just seem to low. For us as a country to ensure we dont get more “Trumps” in the future I think the margin of victory has to be a landslide… something so bad that nobody like him would try again. I fear if we dont, we will see the same asshats in 4years. > > While I know debates dont really swing voters (remember Obama’s first debacle agains Romney?) I am very interested in the foregin policy town hall tomorrow. I wish, just wish, they would ask them the captial and leaders of like the 10 largest economies in the world. Would just love that… My brain hurts after seeing a poll today that put Trump up 2% nationally. It disturbs me to the deepest recesses of my soul to think of what so many people support. ---
**Author:** ![RickJay](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@RickJay](https://boards.straightdope.com/u/RickJay)\ **Post date:** [September 6, 2016, 6:18pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/71 "2016-09-06T18:18:11Z")
> [@Sterling\_Archer](#): > > That seems understandable for a polls-plus analysis, but in polls only, we are starting at a certain % chance for Hillary to win in each state. If you aggregate those probabilities, you get a 1% chance for Trump, not a 27%. Mine is a simple stats and math analysis… Too simple. You’re making a rudimentary error; you’re treating each state as an independent event. That’s wrong, and is now how 538 (or Sam Wang, or anyone else) does it. The mistake you’re making is an easy one to make; you are assuming that each swing state is it own random number generator, and so if Trump has only a 33 percent chance in Florida, a 33 percent chance in Ohio, 33 in North Carolina, and you’re multiplying those percentages and coming up with one percent. The fault in this approach is obvious; if Trump improves in one place, he will generally improve in ALL of them. Trump is not going to gain six points in North Carolina but not gain anywhere else. His polls numbers will general rise, fall, or sink in most-to-all states at the same time. You are assuming the 538 chances of each state have to be multiplied together to get his chances, but that’s not how it works; they ARE his chances, because they’ll all change more or less together. (Note that, mathematically, you can quickly prove your approach is wrong by doing the same thing for Hillary Clinton’s chances in all swing states and getting a number of about two percent.) To use an easy example, according to 538, Trump has a 20 percent chance of winning Virginia. If on election Day I were to allow you access only to Virginia results, and you saw that he had won Virginia, you would know that if that were the case, it is virtually certain he has ALSO won North Carolina, because there is just no way Virginia swings towards Trump but North Carolina magically remains the same. That information also tells you that there’s no chance Clinton has flipped Georgia, and probably tells you Florida and Ohio have gone Trump. A rising Trump tide will raise all boats. Talking about EV margins at this point is just not the correct way to look at this. What matters is the candidate’s broad support. If Trump loses the popular vote by four percent, as 538 now sees him as doing, he will lose. 538’s analysis is based on many things but the primary one is the probability that Trump will not lose the popular vote by four percent - that there is a combination of pollin accuracy and Trump comeback that will have him surpass Clinton in the popular vote, in which case he will almost certainly win the most electoral votes. ---
**Author:** ![iiandyiiii](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/iiandyiiii/32/7924_2.png) [@iiandyiiii](https://boards.straightdope.com/u/iiandyiiii)\ **Post date:** [September 6, 2016, 6:21pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/72 "2016-09-06T18:21:11Z")
> [@RickJay](#): > > Too simple. You’re making a rudimentary error; you’re treating each state as an independent event. That’s wrong, and is now how 538 (or Sam Wang, or anyone else) does it. > > The mistake you’re making is an easy one to make; you are assuming that each swing state is it own random number generator, and so if Trump has only a 33 percent chance in Florida, a 33 percent chance in Ohio, 33 in North Carolina, and you’re multiplying those percentages and coming up with one percent. The fault in this approach is obvious; if Trump improves in one place, he will generally improve in ALL of them. Trump is not going to gain six points in North Carolina but not gain anywhere else. His polls numbers will general rise, fall, or sink in most-to-all states at the same time. You are assuming the 538 chances of each state have to be multiplied together to get his chances, but that’s not how it works; they ARE his chances, because they’ll all change more or less together. > > (Note that, mathematically, you can quickly prove your approach is wrong by doing the same thing for Hillary Clinton’s chances in all swing states and getting a number of about two percent.) > > To use an easy example, according to 538, Trump has a 20 percent chance of winning Virginia. If on election Day I were to allow you access only to Virginia results, and you saw that he had won Virginia, you would know that if that were the case, it is virtually certain he has ALSO won North Carolina, because there is just no way Virginia swings towards Trump but North Carolina magically remains the same. That information also tells you that there’s no chance Clinton has flipped Georgia, and probably tells you Florida and Ohio have gone Trump. A rising Trump tide will raise all boats. > > Talking about EV margins at this point is just not the correct way to look at this. What matters is the candidate’s broad support. If Trump loses the popular vote by four percent, as 538 now sees him as doing, he will lose. 538’s analysis is based on many things but the primary one is the probability that Trump will not lose the popular vote by four percent - that there is a combination of pollin accuracy and Trump comeback that will have him surpass Clinton in the popular vote, in which case he will almost certainly win the most electoral votes. If I understand you correctly, I don’t believe you’re right about 538 – IIRC, Silver has explained that his model does not treat the states as independent, but rather tied together such that a surge in one is likely to also indicate a surge in other states. ---
**Author:** ![CarnalK](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/carnalk/32/486_2.png) [@CarnalK](https://boards.straightdope.com/u/CarnalK)\ **Post date:** [September 6, 2016, 7:03pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/73 "2016-09-06T19:03:03Z")
> [@iiandyiiii](#): > > If I understand you correctly, I don’t believe you’re right about 538 – IIRC, Silver has explained that his model does not treat the states as independent, but rather tied together such that a surge in one is likely to also indicate a surge in other states. That is exactly what RickJay said. ---
**Author:** ![Sterling\_Archer](https://avatars.discourse-cdn.com/v4/letter/s/f1d935/32.png) [@Sterling\_Archer](https://boards.straightdope.com/u/Sterling_Archer)\ **Post date:** [September 6, 2016, 7:38pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/74 "2016-09-06T19:38:48Z")
> [@RickJay](#): > > (Note that, mathematically, you can quickly prove your approach is wrong by doing the same thing for Hillary Clinton’s chances in all swing states and getting a number of about two percent.) Hillary doesn’t need to win all of the swing states in my model, she only needs 27 EV’s from the swing states. Her chances of achieving that are 99% in the 200 scenarios I ran. If Hillary’s chances of winning all the swing states is about 2% how does that prove anything wrong? I get that there is co-variance between the state results, but what poll-moving events are being factored into those estimations? I don’t see the point of a polls-only analysis that also takes into account outside events that might shift the polls. That’s not polls-only in my book. Also, national polls really have nothing to do with the results. That’s like accounting for the weather on Election Day. Yes, it could have an impact but how is that polls-only? ---
**Author:** ![Sterling\_Archer](https://avatars.discourse-cdn.com/v4/letter/s/f1d935/32.png) [@Sterling\_Archer](https://boards.straightdope.com/u/Sterling_Archer)\ **Post date:** [September 6, 2016, 7:49pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/75 "2016-09-06T19:49:09Z")
Oops, 21 EV’s not 27. I also disagree somewhat with the strength of the covariance of the states. There is more independence there than many of you are giving credit for. It is not impossible that Clinton could lose Iowa and Nevada but win NC. The states do not move in lockstep with each other, there are entirely different populations, with different demographics and priorities. (And different polling effectiveness too) ---
**Author:** ![RickJay](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@RickJay](https://boards.straightdope.com/u/RickJay)\ **Post date:** [September 6, 2016, 7:50pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/76 "2016-09-06T19:50:09Z")
> [@Sterling\_Archer](#): > > Hillary doesn’t need to win all of the swing states in my model, she only needs 27 EV’s from the swing states. Her chances of achieving that are 99% in the 200 scenarios I ran. Again, if you are treating each state as an independent event, your analysis is fundamentally, irretrievably flawed, because that isn’t how elections work. Indeed, it’s precisely the error next to everyone made in assuming mortgage bonds would not fail; they assumed the likelihood of each mortgage failing was an independent event, when of course that is obviously not true. Consequently, they calculated the likelihood of failure the way you’re calculating it, and therefore near-impossible, when it fact it was quite possible. > [@](#): > > I get that there is co-variance between the state results, but what poll-moving events are being factored into those estimations? I don’t see the point of a polls-only analysis that also takes into account outside events that might shift the polls. That’s not polls-only in my book. Also, national polls really have nothing to do with the results. Given the rather strong correlation between national polls and winning the Presidential election, I’d say they do. As to what poll-moving events are being factored in, I don’t believe Silver factors any particular “event” in, inasmuch as that would be nuts. He’s simply looking at the polls and running election simulations over and over and over. (The difference between the Polls Only, Nowcast and Polls Plus projections are so small now as to be kind of pointless.) Given how good he has been at this sort of thing, I am strongly inclined to believe his methodology works. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [September 6, 2016, 7:51pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/77 "2016-09-06T19:51:25Z")
> [@Sterling\_Archer](#): > > I get that there is co-variance between the state results, but what poll-moving events are being factored into those estimations? I don’t see the point of a polls-only analysis that also takes into account outside events that might shift the polls. There are no outside events being taken into account only covariance. This is necessary for the model to be accurate since the things being modeled are covariant. The polls only model takes in only polls as input. Polls plus takes in economic data in addition to poling data. ---
**Author:** ![CarnalK](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/carnalk/32/486_2.png) [@CarnalK](https://boards.straightdope.com/u/CarnalK)\ **Post date:** [September 6, 2016, 7:53pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/78 "2016-09-06T19:53:29Z")
It is almost certain that Silver takes the demographics into consideration when developing a covariance model. ---
**Author:** ![iiandyiiii](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/iiandyiiii/32/7924_2.png) [@iiandyiiii](https://boards.straightdope.com/u/iiandyiiii)\ **Post date:** [September 6, 2016, 7:56pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/79 "2016-09-06T19:56:48Z")
> [@CarnalK](#): > > That is exactly what RickJay said. Then I read it wrong – when he said “that’s exactly how 538 does it”, I thought he was saying they made this mistake. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [September 6, 2016, 7:59pm UTC](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819/80 "2016-09-06T19:59:48Z")
> [@Sterling\_Archer](#): > > I also disagree somewhat with the strength of the covariance of the states. There is more independence there than many of you are giving credit for. Many people in this thread are oversimplifying the concept of covariance to make a point. Nate Silver however has carefully measured covariance between states and uses that in his model. [Previous page](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819.md?page=3) [Next page](https://boards.straightdope.com/t/doubting-the-statistical-analysis-at-538/764819.md?page=5)