# So will anybody ever trust the polls again? **URL:** \ **Category:** Politics & Elections\ **Created:** [November 4, 2020, 5:49am UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714 "2020-11-04T05:49:40Z")\ **Posts on this page:** 20\ **Page:** 14
**Author:** ![Monocracy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/monocracy/32/4493_2.png) [@Monocracy](https://boards.straightdope.com/u/Monocracy)\ **Post date:** [November 10, 2020, 5:32pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/265 "2020-11-10T17:32:44Z")
> [@Lance\_Turbo](#): > > No one is saying that. I am saying that when you compute the MSE across all 56 electoral vote awarding entities for the 2020 presidential election for both 538 and PredictIt models, PredictIt outperforms 538. I don’t know what any of that means. Because I would assume that Georgia is an electoral vote rewarding entity and that 538 outperformed PredictIt on that market. ---
**Author:** ![iamthewalrus\_3](https://avatars.discourse-cdn.com/v4/letter/i/258eb7/32.png) [@iamthewalrus\_3](https://boards.straightdope.com/u/iamthewalrus_3)\ **Post date:** [November 10, 2020, 5:49pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/266 "2020-11-10T17:49:29Z")
Yes, you can certainly cherry pick a few places where 538 outperformed PredictIt. But if you look at all the predictions, PredictIt did better (is what Lance\_Turbo is claiming, and he’s been making a solid argument so I tend to believe him, though I haven’t analyzed the data myself). ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 10, 2020, 5:50pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/267 "2020-11-10T17:50:05Z")
Yes Georgia is an electoral vote rewarding entity. One of 56 electoral vote rewarding entities. If one were to look solely at Georgia, 538 did indeed have a better prediction than PredictIt, but there is no reason to look only at Georgia. When you look at all 56 electoral vote rewarding entities, PredictIt does better than 538 on average despite being slightly worse in Georgia. Comfortably so. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 10, 2020, 6:26pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/268 "2020-11-10T18:26:21Z")
Here are two forecasts of the same 56 events. Which forecast is better? How do we decide? | Index | Forecast A P(X) | Forecast B P(X) | Result (1 = X) | | --- | --- | --- | --- | | 1 | 0.0002 | 0.0003 | 0 | | 2 | 0.0006 | 0.0016 | 0 | | 3 | 0.0003 | 0.0102 | 0 | | 4 | 0.0007 | 0.0057 | 0 | | 5 | 0.0008 | 0.0056 | 0 | | 6 | 0.0007 | 0.0072 | 0 | | 7 | 0.0007 | 0.0165 | 0 | | 8 | 0.0028 | 0.0056 | 0 | | 9 | 0.0007 | 0.0289 | 0 | | 10 | 0.0015 | 0.0142 | 0 | | 11 | 0.0015 | 0.0241 | 0 | | 12 | 0.0030 | 0.0145 | 0 | | 13 | 0.0015 | 0.0325 | 0 | | 14 | 0.0014 | 0.0456 | 0 | | 15 | 0.0016 | 0.0519 | 0 | | 16 | 0.0033 | 0.0356 | 0 | | 17 | 0.0036 | 0.0849 | 0 | | 18 | 0.0083 | 0.0393 | 0 | | 19 | 0.0064 | 0.0637 | 0 | | 20 | 0.0064 | 0.1000 | 0 | | 21 | 0.0182 | 0.1495 | 0 | | 22 | 0.0178 | 0.1803 | 0 | | 23 | 0.1480 | 0.3987 | 0 | | 24 | 0.1818 | 0.3825 | 0 | | 25 | 0.1994 | 0.4533 | 0 | | 26 | 0.3550 | 0.5815 | 1 | | 27 | 0.4145 | 0.5726 | 0 | | 28 | 0.3615 | 0.6910 | 0 | | 29 | 0.4775 | 0.6393 | 0 | | 30 | 0.5527 | 0.6802 | 1 | | 31 | 0.6382 | 0.8431 | 1 | | 32 | 0.8147 | 0.7169 | 1 | | 33 | 0.7819 | 0.9468 | 1 | | 34 | 0.7940 | 0.9436 | 1 | | 35 | 0.8737 | 0.8781 | 1 | | 36 | 0.8751 | 0.8909 | 1 | | 37 | 0.8528 | 0.9581 | 1 | | 38 | 0.9743 | 0.9012 | 1 | | 39 | 0.9808 | 0.9684 | 1 | | 40 | 0.9760 | 0.9776 | 1 | | 41 | 0.9961 | 0.9728 | 1 | | 42 | 0.9890 | 0.9816 | 1 | | 43 | 0.9822 | 0.9902 | 1 | | 44 | 0.9923 | 0.9920 | 1 | | 45 | 0.9933 | 0.9916 | 1 | | 46 | 0.9954 | 0.9937 | 1 | | 47 | 0.9911 | 0.9986 | 1 | | 48 | 0.9904 | 0.9999 | 1 | | 49 | 0.9965 | 0.9960 | 1 | | 50 | 0.9939 | 0.9993 | 1 | | 51 | 0.9946 | 1.0000 | 1 | | 52 | 0.9955 | 0.9992 | 1 | | 53 | 0.9961 | 0.9992 | 1 | | 54 | 0.9958 | 0.9995 | 1 | | 55 | 0.9960 | 0.9996 | 1 | | 56 | 0.9992 | 1.0000 | 1 | ---
**Author:** ![naita](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/naita/32/5862_2.png) [@naita](https://boards.straightdope.com/u/naita)\ **Post date:** [November 10, 2020, 7:03pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/269 "2020-11-10T19:03:09Z")
> [@Lance\_Turbo](#): > > I am using a standard comparator for probabilistic forecasts. It’s a comparator that 538 uses to [evaluate its own forecasts](https://projects.fivethirtyeight.com/checking-our-work/president-election-night/). > > What do you suggest? How should we answer the question, “Which probabilistic forecast was more accurate for the 56 electoral college awarding entities in the 2020 presidential election? 538’s poll driven model, or a model derived from PredictIt using the methodology from Rothschild’s 2009 paper (linked above)?” What comparator should we use? I withdraw my complaint. I didn’t read your posts carefully enough and made a rash judgement based on my misinterpretation of your position and arguments. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 10, 2020, 7:51pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/270 "2020-11-10T19:51:10Z")
I appreciate this. Thank you. ---
**Author:** ![Fotheringay-Phipps](https://avatars.discourse-cdn.com/v4/letter/f/d9b06d/32.png) [@Fotheringay-Phipps](https://boards.straightdope.com/u/Fotheringay-Phipps)\ **Post date:** [November 11, 2020, 2:22pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/272 "2020-11-11T14:22:23Z")
> [@iamthewalrus\_3](#): > > I’m saying that the market is not liquid, so we can’t put much faith in the prices it comes up with. > > In no universe is there currently a 13% chance that Trump wins the Presidency. If PredictIt would let people bet against it, that price would drop dramatically. But they won’t, so it doesn’t. > > I don’t know when the PredictIt market became illiquid, but I know I couldn’t bet on it the day before the election. This seems illogical and contrary to the facts. If in fact PredictIt didn’t let you bet the day before the election because “the maximum number of people already participating in the market” had already been reached, it doesn’t make sense that this would make the market illiquid. It might make sense that the day before the election, when interest in presidential bets was at an all-time high, that perhaps the PredictIt systems were overloaded and they had to cut participation. But it wouldn’t make sense that PredictIt would keep barring entry after the election and to the point that they would create an illiquid market - which would go completely against their own interests. (At a minimum, you should try “tak[ing] those people’s money” today and see if you still can’t before making such claims.) Further, the post-election pricing on PredictIt is not consistent with an illiquid market. An illiquid market is characterized by volatile and inconsistent prices. But PredictIt has had Trump’s chances in the low-to-high teens all week. And consistent with that, Biden’s chances have been in the high-to-low 80s. And comparable pricing on the “Republican Party” and “Democratic Party” electoral votes have been consistent with the pricing on Trump and Biden. As has been the pricing on state races for supposedly unsettled states like PA et al. In sum, by every logic and indication the current pricing on the presidential race has nothing to do with illiquidity and everything to do with PredictIt consistently overpricing Trump’s chances. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 3:46pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/273 "2020-11-11T15:46:05Z")
This post-election PredictIt discussion is interesting and all, but it really has nothing to do with the fact that a forecast derived from pre-election PredictIt was more accurate than 538’s pre-election forecast. ---
**Author:** ![Fotheringay-Phipps](https://avatars.discourse-cdn.com/v4/letter/f/d9b06d/32.png) [@Fotheringay-Phipps](https://boards.straightdope.com/u/Fotheringay-Phipps)\ **Post date:** [November 11, 2020, 3:54pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/274 "2020-11-11T15:54:39Z")
It has everything to do with it, specifically with _why_ the pre-election PredictIt was more accurate. The post-election PredictIt is the same methodology as the pre-election PredictIt. (Probably most of the same bettors too.) If there is a pro-Trump skew to the post-election PredictIt, then it makes sense to assume that there was also a pro-Trump skew to the pre-election PredictIt. And if that’s the case, then the increased accuracy in the PredictIt projection is an artifact of that skew combined with Trump beating his poll numbers, and has very little applicability to races involving other candidates. This thread is not about who gets a prize for accuracy in the 2020 presidential race; it’s about whether anybody will trust the polls “ever again” i.e. in other races. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 3:58pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/275 "2020-11-11T15:58:18Z")
> [@Fotheringay-Phipps](#): > > It has everything to do with it, specifically with _why_ the pre-election PredictIt was more accurate. Great. You acknowledge that PredictIt was more accurate. You believe that PredictIt was right for the wrong reasons, and you may even be right about that, but you are not disputing the mathematical rock solid fact that PredictIt was more accurate. ---
**Author:** ![Monocracy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/monocracy/32/4493_2.png) [@Monocracy](https://boards.straightdope.com/u/Monocracy)\ **Post date:** [November 11, 2020, 4:02pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/276 "2020-11-11T16:02:25Z")
> [@Lance\_Turbo](#): > > Great. You acknowledge that PredictIt was more accurate. You believe that PredictIt was right for the wrong reasons, and you may even be right about that, but you are not disputing the mathematical rock solid fact that PredictIt was more accurate. I will acknowledge that. But why can’t you acknowledge that PredictIt has a heavy pro-Trump lean considering the insane prices that are happening in the markets right now? ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 4:10pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/277 "2020-11-11T16:10:58Z")
> [@Monocracy](#): > > But why can’t you acknowledge that PredictIt has a heavy pro-Trump lean considering the insane prices that are happening in the markets right now? I have made no comments on what’s happening in the markets right now. Things happening after the election have no value in evaluating a forecast. It’s not relevant. If the question is where should I have looked to get the best picture of how the election was going to shake out in the months or days leading up to the election, PredictIt or 538? The answer is clear. PredictIt. Whatever is going on after the election does not change that. My irrelevant opinion on what is going on at PredictIt right now adds nothing so I can’t imagine why you care about that. ---
**Author:** ![Monocracy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/monocracy/32/4493_2.png) [@Monocracy](https://boards.straightdope.com/u/Monocracy)\ **Post date:** [November 11, 2020, 4:16pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/278 "2020-11-11T16:16:16Z")
> [@Lance\_Turbo](#): > > If the question is where should I have looked to get the best picture of how the election was going to shake out in the months or days leading up to the election, PredictIt or 538? That’s not the question. The question is who is a better predictor overall. So what is happening right now on PredictIt is entirely relevant. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 4:22pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/279 "2020-11-11T16:22:41Z")
> [@Monocracy](#): > > So what is happening right now on PredictIt is entirely relevant. It is not. PredictIt, and everything else, is not useful for forecasting after the election primarily because after the election it fails the ‘fore’ part of forecasting. PredictIt’s pre-election forecast was superior to 538’s. There’s no such thing as a post-election forecast by definition so there’s no reason to discuss it. ---
**Author:** ![Monocracy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/monocracy/32/4493_2.png) [@Monocracy](https://boards.straightdope.com/u/Monocracy)\ **Post date:** [November 11, 2020, 4:29pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/280 "2020-11-11T16:29:21Z")
> [@Lance\_Turbo](#): > > PredictIt, and everything else, is not useful for forecasting after the election primarily because after the election it fails the ‘fore’ part of forecasting. The voting may be over, but the election isn’t. We are still in the forecasting phase. ---
**Author:** ![Fotheringay-Phipps](https://avatars.discourse-cdn.com/v4/letter/f/d9b06d/32.png) [@Fotheringay-Phipps](https://boards.straightdope.com/u/Fotheringay-Phipps)\ **Post date:** [November 11, 2020, 4:31pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/281 "2020-11-11T16:31:54Z")
> [@Lance\_Turbo](#): > > Great. You acknowledge that PredictIt was more accurate. You believe that PredictIt was right for the wrong reasons, and you may even be right about that, but you are not disputing the mathematical rock solid fact that PredictIt was more accurate. PredictIt’s pre-election projections for this election were closer to what ended up happening than 538. That seems obvious and AFAICT has not been disputed by anyone in this thread. It’s also not what’s relevant to the discussion here. What’s relevant is whether this implies that PredictIt is more reliable than polls or 538-type analysis in general. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 4:31pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/282 "2020-11-11T16:31:58Z")
> [@Monocracy](#): > > We are still in the forecasting phase. 538 isn’t. They haven’t updated their forecast since the morning of 11/3. Should we compare the current “forecasts” of the two. 538 will get crushed even with the post election weirdness going on over at PredictIt. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 4:33pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/283 "2020-11-11T16:33:41Z")
> [@Fotheringay-Phipps](#): > > PredictIt’s pre-election projections for this election were closer to what ended up happening than 538. Great. We’re done. ---
**Author:** ![Monocracy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/monocracy/32/4493_2.png) [@Monocracy](https://boards.straightdope.com/u/Monocracy)\ **Post date:** [November 11, 2020, 4:40pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/284 "2020-11-11T16:40:09Z")
> [@Lance\_Turbo](#): > > 538 isn’t. They haven’t updated their forecast since the morning of 11/3. > > Should we compare the current “forecasts” of the two. 538 will get crushed even with the post election weirdness going on over at PredictIt. 538’s current prediction for a Trump win in Pennsylvania is 0%. PredictIt has it at 16.7% currently. We can run these numbers after PA is resolved to see which is better. ---
**Author:** ![Lance\_Turbo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lance_turbo/32/6156_2.png) [@Lance\_Turbo](https://boards.straightdope.com/u/Lance_Turbo)\ **Post date:** [November 11, 2020, 4:43pm UTC](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714/285 "2020-11-11T16:43:04Z")
[538’s ‘current’ Pennsylvania forecast](https://projects.fivethirtyeight.com/2020-election-forecast/pennsylvania/). Trump 16% Biden 84% 538 froze their forecast on 11/3. [![Google Photos](https://lh3.googleusercontent.com/1eqh2xxYhm6Q2hSuGh6aM7isnBEIDv1reXcSVAFc0sjfaL-eTaiOr_yPlbQXl7rIjgx90yZZgBSU3el2lJcvMOl7A2qyEvLohspJH06wlUbrYKjmD9YOkC8bAoXd0IemN2TUh0dX_A=w600-h122-p-k "New item by K S") ](https://photos.app.goo.gl/wfFajnAurahtUrEeA) [Previous page](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714.md?page=13) [Next page](https://boards.straightdope.com/t/so-will-anybody-ever-trust-the-polls-again/924714.md?page=15)