# Polling is broken, right? Or is it the news? Or all of the US? **URL:** \ **Category:** Politics & Elections\ **Tags:** trump\ **Created:** [November 27, 2023, 5:30pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779 "2023-11-27T17:30:18Z")\ **Posts on this page:** 20\ **Page:** 24
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 15, 2024, 4:22pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/466 "2024-10-15T16:22:27Z")
Easy to do. I quickly looked at the NYT aggregation for PA. And interesting right now mainly because the NYT poll is the big outlier. That poll has H+4. A WSJ sponsored poll and Emerson have T+1. Going back there is a Quinnipiac H+2. The partisan ones are in line with WSJ and Emerson. Overall H+1. Which is the number ignoring the partisan ones. Same big polls in 538 and they come up with H+0.7. Eyeballing the GA ones is the same. The biggest +T is Quinnipiac, T +5. Which brings the aggregation slightly Trumpward less than 1. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 4:37pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/467 "2024-10-15T16:37:25Z")
> [@DSeid](#): > > Easy to do. I quickly looked at the NYT aggregation for PA. I’m working in a different time frame – specifically, Monday, 9/30 through Friday, 10/11. I’ve started with one state, Pennsylvania. I downloaded FiveThirtyEight’s **[presidential general election polling spreadsheet here](https://projects.fivethirtyeight.com/polls/president-general/2024/pennsylvania/)** (Scroll down to “ **Download the data** ”). I made an extra column to break out the red-leaning pollsters. Here are the pivot tables of the results. Pennsylvania only to start: [![](https://i.imgur.com/EwWZS4i.jpeg) ](https://i.imgur.com/EwWZS4i.jpeg) I’m satisfied that 1.7 points makes a meaningful difference to the narratives around “the state of the election” and/or “the state of polling”. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 15, 2024, 5:45pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/468 "2024-10-15T17:45:49Z")
Fabrizio Lee and Associates **GBAO** is the highly respected WSJ poll, not “red wave”. The GBAO side is Democratic partisan. I guess they act as checks on each other? In any case the WSJ poll should be in the respected group, not the partisan one. And the difference to measure is between that, including the WSJ poll, but not the red wave ones, and their reported 0.7. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 7:36pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/469 "2024-10-15T19:36:46Z")
> [@DSeid](#): > > Fabrizio Lee and Associates **GBAO** is the highly respected WSJ poll, not “red wave”. The GBAO side is Democratic partisan. I guess they act as checks on each other? In any case the WSJ poll should be in the respected group, not the partisan one. That changes the numbers to: | PA without red-wave pollsters (7 pollsters out of 18) | | | --- | --- | | Harris | 48.1 | | Trump | 46.7 | | **Harris** | **+1.4** | ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 15, 2024, 7:49pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/470 "2024-10-15T19:49:28Z")
Their reported out aggregate number (also includes out of state factors) is H+0.7. So numbers are within 0.7. If you think less than 1 is significant we will just have to disagree. ---
**Author:** ![BobLibDem](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/boblibdem/32/3149_2.png) [@BobLibDem](https://boards.straightdope.com/u/BobLibDem)\ **Post date:** [October 15, 2024, 7:53pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/471 "2024-10-15T19:53:31Z")
I don’t trust a lot of pollsters or those that interpret polls. Rachel Bitecofer is one. Another is the guy that runs the YouTube channel Voting Trend. If you’re hung up on the polls and shit your pants every time 538 comes up with new averages, check out [this guy.](https://www.youtube.com/watch?v=SWyfSpvWyEo&t=110s) He’s definitely left leaning but his analysis of polls I believe is sound. The rest of us don’t have the time or expertise to dig in and look at what goes into the polls but he does. Check out a few of his videos and you won’t be as convinced about the inevitability of the orange goon. I saw a talking head on television, I can’t recall his name, say that the right is now flooding the zone with trends showing the country shifting red. Take the polls with a shaker of salt. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 7:54pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/472 "2024-10-15T19:54:03Z")
> [@DSeid](#): > > Their reported out aggregate number (also includes out of state factors) is H+0.7. I’m not following. I’m only working within the data set I got from FiveThirtyEight’s website. Further, I’m using raw poll numbers – no out-of-state factors, weighing, scoring, etc. Just raw poll results. Other rules I’ve applied to my analysis: - Use polls that include all candidates when available. Not sure if this is more right or more wrong … but I had to pick a lane, and there are third parties on the ballot. - Use likely voter polls where available. Only a handful used registered voters only (e.g. Patriot Polling). ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 7:55pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/473 "2024-10-15T19:55:41Z")
> [@BobLibDem](#): > > Another is the guy that runs the YouTube channel Voting Trend. If you’re hung up on the polls and shit your pants every time 538 comes up with new averages, check out [this guy.](https://www.youtube.com/watch?v=SWyfSpvWyEo&t=110s) He’s definitely left leaning but his analysis of polls I believe is sound. The rest of us don’t have the time or expertise to dig in and look at what goes into the polls but he does. Check out a few of his videos and you won’t be as convinced about the inevitability of the orange goon. If he explains red-wave polling and zone-flooding well … he could save me a lot of time 😃 I’m still going to repeat the Pennsylvania exercise above for some more swing states. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 7:58pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/474 "2024-10-15T19:58:54Z")
> [@DSeid](#): > > So numbers are within 0.7. If you think less than 1 is significant we will just have to disagree. It’s not significant for election-outcome prediction. But that’s probably not even the main thing polls are used for these days – especially when all the major players are raising both hands screaming “COIN FLIP! CAN’T STAKE MY REP ON A FIRM PASS/FAIL CALL!” So … what _are_ the polls used more for these days if not prediction? _Answer:_ **Narrative driving.** Creating a “reality” because you can point to “numbers!” And small nudges are enough. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 8:01pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/475 "2024-10-15T20:01:31Z")
> [@BobLibDem](#): > > I saw a talking head on television, I can’t recall his name, say that the right is now flooding the zone with trends showing the country shifting red. Take the polls with a shaker of salt. This is _all over the place_ on liberal Substack, and is not seriously contested. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 15, 2024, 8:11pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/476 "2024-10-15T20:11:04Z")
What’s all over those substacks not contested there is that 538 and aggregators in general are gamed by flooding and their numbers essentially propaganda. The number they produce is H+0.7 in PA. The number you produce taking out the flood of “red wave” polling is H+1.4, not significantly different. The aggregate is not successfully gamed. Yes selected polls are used by various pundits to drive narratives to drive clicks. The aggregate is still a fair reflection of where good polls are. Better for including out of state data historically. What are you not getting? ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 8:20pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/477 "2024-10-15T20:20:22Z")
> [@DSeid](#): > > The number you produce taking out the flood of “red wave” polling is H+1.4, not significantly different. Just a raw 1.4, though. Wouldn’t surprise me if it were more like 2.0 - 2.5 after whatever factors FiveThirtyEight applies. I don’t know how to apply Morriss’ model, however. Besides, there’s kind of a “_$19.99 feels way cheaper than $20.00_” effect at work. Taking (say) a 1.5-2.0 point lead from your opponent and replacing it with a 0.7 point lead _feels_ impactful even if the math isn’t all that different. A lot of this stuff, frankly, defies arithmetic and logic. And we know people in the voting booths aren’t Vulcans – they’ll often vote by the preponderence of “_somethings_” that appealed to the non-logical side of their brains. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 15, 2024, 9:55pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/478 "2024-10-15T21:55:28Z")
> [@bordelond](#): > > I’m still going to repeat the Pennsylvania exercise above for some more swing states . OK, I’ve done five swing states. Probably will stop right here. For state-level presidential election polls aggregated at [FiveThirtyEight.com](http://FiveThirtyEight.com), Monday 9/30 through Friday 10/11: [![](https://i.imgur.com/iZRNZuL.jpeg) ](https://i.imgur.com/iZRNZuL.jpeg) I think the point is generally made. Arizona is a straight flip from a “_Harris lead_” to a “_Trump lead_” – the details be damned on Xitter. “Statistical tie” just sounds like weasel words to many. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 16, 2024, 12:14am UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/479 "2024-10-16T00:14:18Z")
> [@bordelond](#): > > Just a raw 1.4, though. Wouldn’t surprise me if it were more like 2.0 - 2.5 after whatever factors FiveThirtyEight applies Huh? Neither should it surprise you if it went the other way. The result is still including more than the most recent sets of polls and throwing out every pollster that you decide to call “red” you come up with a similar number to the 538 aggregated number that is somehow being gamed and per those substacks just GOP propaganda. > [@bordelond](#): > > Arizona is a straight flip from a “_Harris lead_” to a “_Trump lead_” Honestly I have no comprehension of your analysis here. Arizona looking just at the most recent of the big name pollsters, widely believed to as good as it gets, is: NYT T+5; Emerson T+3; and WSJ (again the doudleteamed one) H+2. Average is T+2. 538 aggregation inclusive of the red flood and their out of state factors if any? T+1.6. (Throw in an oldish Morning Consult and you get T+0.75.) The number only looking at the most recent results of the top pollsters is about the same as the number including the “red flood”. It isn’t so complicated. Your method does weird things including over counting repeated data (MC, known to bias D ward relative to others, reports overlapping sets for example) that the aggregators know not to do. ---
**Author:** ![Dr.Drake](https://avatars.discourse-cdn.com/v4/letter/d/ad7895/32.png) [@Dr.Drake](https://boards.straightdope.com/u/Dr.Drake)\ **Post date:** [October 16, 2024, 1:07am UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/480 "2024-10-16T01:07:28Z")
Does each poll have the same margin of error, and, if not, are those factored in when aggregating? I would assume so, but I don’t know. ---
**Author:** ![JKellyMap](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/jkellymap/32/15880_2.png) [@JKellyMap](https://boards.straightdope.com/u/JKellyMap)\ **Post date:** [October 16, 2024, 9:17am UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/481 "2024-10-16T09:17:21Z")
Almost, and yes it’s factored in. The margin of error (which isn’t the only source of error) is mostly about number of people polled. Once you get above around 900, it tends to improve only incrementally. That’s why so many of the polls poll around 1,000 people. ---
**Author:** ![TeroSunbear](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/terosunbear/32/14748_2.png) [@TeroSunbear](https://boards.straightdope.com/u/TeroSunbear)\ **Post date:** [October 16, 2024, 11:50am UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/482 "2024-10-16T11:50:33Z")
Harris ahead in PA from NYT/Siena. 49/45. Oct 7-10. And that does not even include Trump “dancing” for 39 minutes including the gay anthem YMCA. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 16, 2024, 12:24pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/483 "2024-10-16T12:24:15Z")
The NYT has explained how many other houses are using weighted recall this year and they are not, and why; the latest rounds of polling shows big differences resulting. This NYT result is the only recent poll with Harris ahead in PA. The WSJ and Emerson both have Trump up +1. Aggregate is H up under one, but damn, one model is going to be more on the money than the other. Flip side is that the NYT approach has her farther behind in Sun Belt states, such as Trump up 6 in AZ … Both models are nerve wracking in the different narratives they tell. ---
**Author:** ![bordelond](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bordelond/32/150_2.png) [@bordelond](https://boards.straightdope.com/u/bordelond)\ **Post date:** [October 16, 2024, 2:50pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/484 "2024-10-16T14:50:20Z")
> [@BobLibDem](#): > > I saw a talking head on television, I can’t recall his name, say that the right is now flooding the zone with trends showing the country shifting red. Take the polls with a shaker of salt . Required reading. A lot of this stuff condensed down into one quick-read Substack, courtesy of guest writer Jay Kuo and **[The Big Picture Substack](https://thinkbigpicture.substack.com/p/2024-trump-red-wave-polls)**. > **[Weaponized Polling Is More Dangerous Than Ever](https://thinkbigpicture.substack.com/p/2024-trump-red-wave-polls)** > > Right-leaning outfits are skewing the polls for Trump for quite sinister reasons. > Election analyst Simon Rosenberg recently noted that of the last 15 general election polls released for Pennsylvania, a state viewed by both sides as key to any electoral victory, 12 have right-wing or GOP affiliations. > > Rosenberg warned, > > > “Their campaign to game the polling averages and make it appear like Trump is winning—when he isn’t—escalated in [the] last few days.” > > … > > Legacy media should be pushing back hard against this narrative, having been bamboozled by it before [_2022 midterms - b_]. Instead, headline after headline is repeating the “vibe” generated by crap, partisan polling: “Trump is on the move, the race is narrowing.” > > But that’s not true. The race, both nationally and in the battlegrounds, hasn’t appreciably moved in the past few weeks at all. Harris still has a narrow lead both in national polls and in the battleground states. My emphasis below: > … the flood of GOP pollsters [_during the 2022 midterms - b_] had taken a toll. It demoralized Democrats and shifted dollars away from races that the polls claimed were unwinnable. Chief among these was the nearly successful Senate campaign of Wisconsin Lt. Gov. Mandela Barnes. Those corrupted polling averages had him trailing the incumbent Republican, Sen. Ron Johnson, by as much as five points. Democratic donors believed the numbers and decided Barnes was likely to lose. Money shifted to places like John Fetterman’s race in Pennsylvania—a race he actually won quite handily. > > But Barnes was much more competitive than the corrupted polls showed. He lost to Johnson by less than one percent. The Senate would have been a very different place over the past two years if we could have ignored the corporatists like Sens. Manchin and Sinema because we had 52 instead of 51 seats in that body. > > … > > Note in particular attempts to sink Sen. Jon Tester, the Democratic incumbent in Montana, by making it seem like he can’t win. This is the Mandela Barnes story rebooted. It’s an attempt to drive national dollars away from Tester and depress turnout in Montana for him. > > On Sunday, October 13, Rosenberg reported with a stunning update: > > > “With three weeks to go we should note that Rs have dropped more than 60 polls into the polling averages over the last few weeks to red wave 2024.” > > **Some observers, such as Jon Favreau, have noted that these GOP-leaning polls haven’t been able to move the actual averages that much, maybe a point or so. But even that can be significant in a tight race. And besides, it’s more than just the averages. It’s the constant drumbeat of “Trump is winning” or “the race is essentially tied” headlines. They are affecting the media reporting while dampening the national mood with Democrats and raising expectations undeservedly with Republicans.** As I’ll discuss later, that all matters a lot. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [October 16, 2024, 6:03pm UTC](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779/485 "2024-10-16T18:03:28Z")
Timely. Yeah some circles will dismiss it. > **[Are Republican pollsters “flooding the zone?”](https://www.natesilver.net/p/are-republican-pollsters-flooding)** > > And what would our forecast look like with only high-quality surveys? > [@](#): > > For the flooding-the-zone theory to hold water (pun somewhat intended), polling averages and forecasts would have to just toss these polls in the average without any adjustment. But that isn’t happening. Here at Silver Bulletin, for example, we weight polls based on [pollster quality](https://www.natesilver.net/p/pollster-ratings-silver-bulletin) and adjust them [based on pollsters’ house effects](https://www.natesilver.net/p/which-polls-are-biased-toward-harris). And every other high-quality polling average does something similar. > > What’s the result? The polling averages say pretty much the same thing, regardless of which polls they choose to include or exclude. Nationally and in the battleground states, the biggest difference in Harris-Trump margin between the Silver Bulletin average and averages from [538](https://projects.fivethirtyeight.com/polls/president-general/2024/national/), [Split Ticket](https://split-ticket.org/2024-presidential-polling-averages/), [The New York Times](https://www.nytimes.com/interactive/2024/us/elections/polls-president.html), and [VoteHub](https://polls.votehub.us/) is 0.5 points. In Pennsylvania — the likeliest tipping point state — our average is Harris +0.6. Split Ticket has the race as Harris +1, 538 has it as Harris +0.7, VoteHub has it as Harris +0.7, and The New York Times has Harris up by less than 1 point. > > The important thing here is that these averages have somewhat different philosophies on which polls they use. For example, [Split Ticket excludes Rasmussen Reports and Trafalgar](https://x.com/lxeagle17/status/1843366816267284632); we include them, but automatically designate them as Republican partisan polls. 538 uses polls from Big Data Poll, Quantus and SoCal Data and ActiVote but we don’t.[2](https://www.natesilver.net/p/are-republican-pollsters-flooding#footnote-2-150316460) And VoteHub [only uses high-quality nonpartisan polls](https://polls.votehub.us/). But because we’re all weighting and adjusting the polls in reasonable ways, we all end up in about the same place. [Previous page](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779.md?page=23) [Next page](https://boards.straightdope.com/t/polling-is-broken-right-or-is-it-the-news-or-all-of-the-us/993779.md?page=25)