I wish this op-ed had been around when the OP was written. From the Boston Globe, no paywall.
The problems with political polls are multiple:
- The dominance of cell phones and caller ID programs on landlines has made what statisticians call the “response rate” plummet. A generation ago, few Americans would let their phones ring without answering. Now many calls go unanswered as we ignore unrecognized numbers. Such screening means that pollsters must place thousands more calls to reach enough respondents, which increases their costs and adds pressure to shrink their sample sizes. On top of that, folks who do answer skew older than the overall electorate, making it even harder to gather a survey group that accurately represents the nation’s complex and shifting demographics.
- There are too many political pollsters conducting too many polls. When John F. Kennedy and Richard Nixon waged a historically close race in 1960, Gallup was virtually the only national polling organization tracking the contest. Three years later Louis Harris started surveying political opinion. For decades they remained the dominant political pollsters, with few competitors. Today there are dozens of polling groups using different methodologies. Nathaniel Rakich, a FiveThirtyEight senior elections analyst, in March 2023 examined hundreds of poll results going back to 1998 in elections for president, the Senate, the House, and governor. During their final three weeks, pollsters predicted the winner only 78 percent of the time. In 2022, they got just 72 percent of the races right, a steep drop from the 88 percent high-water mark of 2008. Focusing on the 2022 elections, Rakich found big disparities among 33 pollsters. Only six called at least 90 percent of their contests correctly.
- Now, 80 or 90 percent accuracy might seem like pretty high scores, given the volatility of many elections. But consider that three firms, SurveyUSA, the University of New Hampshire, and Alaska Survey Research, were 100 percent accurate in 2022. Why can’t more firms reach that level? Also consider that 10 groups got less than two-thirds of their predictions right, which should not be a passing grade. And bringing up the rear were Morning Consult at 8 percent correct and Ipsos at 17 percent. Journalists vet their sources carefully, but these numbers suggest that they should be more careful in covering polls.
- The internet, with its voracious appetite and greatly expanded space for new information no matter how incremental, has made some political journalists less discriminating and fueled more questionable polling. The financial pressures on news organizations have increased the need for digital readers, which has led to election articles that are little more than click-bait.
- Issue polls have their own problems. Immediately after the Supreme Court overturned the constitutional right to abortion in June 2022, many newscasters and reporters predicted the ruling would have minimal electoral impact. They cited old surveys showing that majorities of voters ranked the economy, health care, crime, and other concerns ahead of reproductive rights. Yet those polls failed to measure the intensity of anger the high court ruling would prompt among scores of women, whose increased turnout in subsequent elections has proven decisive.