Have there been some polls favorable to Tester in Montana? That’s going to be the tipping point senate race, unless Colin Allred pulls off a shocker and knocks off Cruz.
Polls for this race seem thin on the ground, but here’s a few. Tester was doing very well in February, but Sheehy seems to have caught up going strictly by the very few polls available. I wouldn’t draw any conclusions yet.
The most recent poll is from the end of March, so an update is sorely needed.
I wish this op-ed had been around when the OP was written. From the Boston Globe, no paywall.
The problems with political polls are multiple:
- The dominance of cell phones and caller ID programs on landlines has made what statisticians call the “response rate” plummet. A generation ago, few Americans would let their phones ring without answering. Now many calls go unanswered as we ignore unrecognized numbers. Such screening means that pollsters must place thousands more calls to reach enough respondents, which increases their costs and adds pressure to shrink their sample sizes. On top of that, folks who do answer skew older than the overall electorate, making it even harder to gather a survey group that accurately represents the nation’s complex and shifting demographics.
- There are too many political pollsters conducting too many polls. When John F. Kennedy and Richard Nixon waged a historically close race in 1960, Gallup was virtually the only national polling organization tracking the contest. Three years later Louis Harris started surveying political opinion. For decades they remained the dominant political pollsters, with few competitors. Today there are dozens of polling groups using different methodologies. Nathaniel Rakich, a FiveThirtyEight senior elections analyst, in March 2023 examined hundreds of poll results going back to 1998 in elections for president, the Senate, the House, and governor. During their final three weeks, pollsters predicted the winner only 78 percent of the time. In 2022, they got just 72 percent of the races right, a steep drop from the 88 percent high-water mark of 2008. Focusing on the 2022 elections, Rakich found big disparities among 33 pollsters. Only six called at least 90 percent of their contests correctly.
- Now, 80 or 90 percent accuracy might seem like pretty high scores, given the volatility of many elections. But consider that three firms, SurveyUSA, the University of New Hampshire, and Alaska Survey Research, were 100 percent accurate in 2022. Why can’t more firms reach that level? Also consider that 10 groups got less than two-thirds of their predictions right, which should not be a passing grade. And bringing up the rear were Morning Consult at 8 percent correct and Ipsos at 17 percent. Journalists vet their sources carefully, but these numbers suggest that they should be more careful in covering polls.
- The internet, with its voracious appetite and greatly expanded space for new information no matter how incremental, has made some political journalists less discriminating and fueled more questionable polling. The financial pressures on news organizations have increased the need for digital readers, which has led to election articles that are little more than click-bait.
- Issue polls have their own problems. Immediately after the Supreme Court overturned the constitutional right to abortion in June 2022, many newscasters and reporters predicted the ruling would have minimal electoral impact. They cited old surveys showing that majorities of voters ranked the economy, health care, crime, and other concerns ahead of reproductive rights. Yet those polls failed to measure the intensity of anger the high court ruling would prompt among scores of women, whose increased turnout in subsequent elections has proven decisive.
Indeed. Thanks!
Except for the judicious captain obvious conclusion, the Boston Globe op-ed linked two posts above is mostly misleading. Polls tell where the race is now. Pollsters do not make the broad claims James Rosen sees himself to be debunking.
As documented earlier in the thread, final pre-election polling averages are about as accurate, or inaccurate, as they ever have been. Rosen claims otherwise by taking a few data points from a long 538 article he links to while ignoring the thesis.
As for the three pollsters Rosen points to as being accurate in 2022, consider this by Nathaniel Rakich in the linked article:
It doesn’t matter that the pollsters aren’t making these claims. Other parties are. Popular media use this poll or that one to sell eyeball-and-click-attracting narratives. And this cycle, for whatever reason … “Trump’s an easy shoo-in – the polls say so!” drives A LOT of traffic.
Everyone’s favorite liberal election guy, Simon Rosenberg, made a good point in his Substack this past Sunday: Pay attention to the specific “likely voters” polling more than “registered voters” or “all respondents”.
A new ABC News poll has Biden leading Trump 49%-45% (+4) with likely voters. This result is similar to this week’s NPR/Marist poll which had Biden up 52%-47% (+5) with likely voters:
ABC News Biden 49%-Trump 45% (+4) [graphic below]
NPR/Marist Biden 52%-Trump 47% (+5)
In both polls Biden does signifcantly better with likely voters than the broader electorate of registered voters.
…
Trump had a nice week last week in the RealClear Polling average as the two polls with Biden furthest ahead aged off while a Trump +6 and +4 were added in. That got Trump up 1.5 points in the average. Today, though, RealClear picked up the two Biden-leading polls Rosenberg links above. Right now, that puts Trump ahead in the RCP average 1.1 points, with a big Trump-leading Rasmussen poll potentially dropping this week.
EDIT: For some reason, RCP doesn’t show the number of respondents and the margin-of-error for the recent ABC News/Ipsos poll linked above. ABC News’ link doesn’t have that information, either, but Ipsos has it on their website – 2,260 respondents and a margin-of-error of 2.1%.
That’s why turnout is important. If you have a district with 50 Biden supporters and 50 Trump supporters, but 45 of the Biden voters are motivated to vote while only 35 of the Trump voters are, Biden wins by 10 votes, despite not having more supporters.
A general poll would show Biden having no advantage.
I took a quick look at the NPR poll. It appeared to me that it was an overall nationwide poll. Unfortunately, the election is not nationwide - it’s state-by-state. Yes, I feel much better that Biden is 4-5% over Trump, but that nagging little voice in my head says "How is he doing in Pennsylvania? In Michigan? In Wisconsin?
Rosenberg addressed these states specifically last week. The words after the colon in the post preview below are Rosenberg’s.
Most pollsters only switch to likely voter screens closer to the election. The reason usually given is that such screens only work closer to the election.
Last time, Trump polled better with likely voter screens.
I can think of one reason why likely voter screens may favor Biden this time. Suppose your screen is primarily based on asking whether the voter voted in 2020. Biden won that year!
As a Democrat, I do not believe in seeking reasons to be optimistic. But it is a positive sign for a candidate to do better with a likely voter screen.
ABC News shared their ‘likely voter’ methodology:
Likely voters
The poll tests a few versions of likely voters, e.g., those who say they are registered to vote or will register and are certain to vote in November; and those who fit that definition and also say they voted in 2020, if old enough to have done so.
Trump is +2 points among all adults (excluding nonvoters) while Biden is +4 among likely voters. While neither is a statistically significant difference, the reason for the Biden bump is that more educated people are more likely to vote, and, as noted, Biden leads Trump by 19 points among college graduates. Among all adults in the survey, 35% have a college degree. Among likely voters, it’s 45%.
The reality is likely to be somewhere in between. In 2016 College degree or higher was 37% in 2020 it was 40%. So 45% is likely too high, and 35% likely too low.
Which tracks with the race pretty much begin a tie nationally. Which will not be enough - Clinton won by 2% nationally and Biden by 4%, and we know both of those were extremely close in the EV.
What I’m still struggling to understand when it comes to polling is the extent to which Biden is running behind Democratic Senate candidates. In some cases I think it’s just that the candidates are pretty strong (Casey, Baldwin, Rosen, even Gallegos), but the gaps are significantly larger than what we’ve seen in recent strongly-partisan elections.
I would be very surprised, for example, if Biden loses PA by 4 points while Casey wins by 4 (which is what the AARP poll out today shows). That’s a level of ticket-splitting we just don’t see these days.
Maybe Biden is a uniquely weak candidate (in some ways he surely is), but Trump is as well so it’s a bit of a movable object v. stoppable force situation.
This gets us back to the title of the thread: Polling is broken, right?
Posed another way: Can aggregated 2024 poll numbers be accepted at face value?
Liberal commentator Jay Kuo threw this into his recent column about former Georgia lieutenant governor Geoff Duncan’s endorsement of Joe Biden:
For Duncan, voting for Biden is a “binary choice” in an election and a practical necessity to save our country from Trump. He cited polls showing that Trump is leading nationally (he actually isn’t, even according to the polling averages) and in the battleground states (again, not necessarily). But if these misleading polls can convince Republicans who are frightened of a second Trump term to cast their votes for Biden, then so be it.
Kuo doesn’t show his work in this particular column, but addresses the faults of 2024 polling in some of his other recent pieces. Some of it is quoted upthread – could probably search this thread for “Kuo” and capture it all at one shot.
EDIT: See also James Rosen’s Boston Globe column upthread about the pitfalls of polling today.
I love this!
Simon Rosenberg is shaking the pom-poms again, getting ready to drop an article specifically about the unreliability of polling-numbers aggregation. Today’s teaser is not about the 2024 presidential election, but looks back at the 2022 U.S. senate polling:
Be Careful of Real Clear Politics and Polling Averages - In the coming days I’m going to write about what I think is questionable utility of polling averages. To whet your appetite a bit, let’s revisit Real Clear Politics last Senate map from 2022 (below). These predicted results were based on RCP’s polling averages in each state. As we all know Republicans didn’t get to 54 seats in the Senate. They got to 49. What went wrong? More soon.
I appreciate that polling is far from an exact science. But isn’t this stacking one inexact factor (how accurately “likely voters” are identified) atop another (the poll results themselves)?
Polling is an inexact science; poll aggregating moreso. And the way it is viewed by the public is wrong.
Nate Silver was absolutely correct in 2016. Remember - all vote projections have a “margin of error”, and the results were within that margin. There are massive recent shifts in polls - accuracy, responses, etc. Any unbiased poll agency is constantly looking at past polls vs. actual results and using that analysis to modify their current polling results. Same thing with the aggregators. It’s tough, though, when those indicators are constantly changing.
Election polling is actually very similar to weather forecasting. Both rely heavily on statistics and interpreting factors. Both give you an expectation within a margin of error. Both are more correct the closer to the actual event. And, the American public, not understanding statistics, howls when the results are in the “unlikely” end of the probability zone. 90% of the time they get it right, but the perception that “wrong” happens half the time. It doesn’t, it just feels that way.
ETA - for years, pollsters relied on phone polls. With the move to cell phones and people not answering if they don’t recognize the number, phone polls are now skewed. Polling agencies are working on how to massage the results of existing tactics while finding new, more exact tactics. Might take a while.