What's the most wrong state/national polling has ever been?

I’m getting the impression that many out there (not just here) don’t trust the polls that say Trump has catching up to do against Clinton.

Yes, it’s early, and things could change.

Yes, it’s good not to get complacent.

Yes, there is a very non-negligible chance Trump could be our next President.

But the way I’m reading some of those who worry, it seems to me like they think that all the polls could show Clinton with the lead going into Election Day, and all of a sudden, BOOM, Trump.

Has that ever happened? What’s the closest it’s gotten to happening? Because I think that if Trump does win, or even comes close to winning, we’ll see it ahead of time in the polls, perhaps very far ahead. Am I wrong here?

Ever hear of President Dewey?

Beat me to it.

assuming you mean outside of the margin of error, Dewey is probably the most recent example.

While in 2000, some polls had Gore ahead, some had Bush, same with Kerry in 2004, its rare the polls are really ever wrong for general elections.

The classic example was the* Literary Digest* poll of 1936 that confidently predicted Alf Landon would beat FDR.

Roosevelt got 60.8% of the popular vote, carried 46 of 48 states and won 523 out of 531 electoral votes.

Well it depends on how big that lead is and how much confidence one has that the likely voter screen apply to a potentially atypical election.

Polls are not always perfect.

Final 2008 polls missed the spread by 4% (Obama won by 7 instead of 11%).

Off by 3 in '04.

Off by 3 in '96.

Off by 6 in '92.

Off by 5 in '88.

Off by 7 in '80.

So on.

No, none of these had called the race to a different winner, they just missed the margin, but if the race is moderately close being off by 3 to 6 one way can cause a different result than was expected.

Is there any reason to be concerned that likely voter screens may be off this time? Well yeah. Trump’s path depends on getting typically less likely voters to turnout sizably. I doubt they will but there lies the rub: I suspect the pollsters’ likely voter screens doubt they will either and what if we are wrong and they do?

Of course! Those of a certain age saw it in The Chicago Tribune with our very own eyes!

If you saw it in the Chicago Tribune, it must be true! (No, contrary to what I’ve long believed, this isn’t the same newspaper that published “Yes, Virginia, there is a Santa Claus”. That was The [New York] Sun.)

Do Trump voters have shame? I wonder sometimes if its a telephone poll or something if a large number of people who are voting for Trump are simply embarrassed to say it out loud to another person. Also I wonder how poll results are skewed by the types of people that actually participate in polling.

The 2000 election had Bush ahead consistently in October and November. Gore got at least three percentage points more than predicted and in fact won the popular vote.

Extremely close elections are impossible to properly predict ahead of time. Most polls’ margin of error will be at least three percent. Larger margins close to the election will almost certainly be correct these days for a presidential race, as indicated in DSeid’s post.

Polling is an inexact science, but the statistical methodology (especially for how you choose your sample) has evolved greatly over the years. At a fairly elementary level, it’s taught as part of any decent Introductory Statistics course.

Here’s how the big errors arose in the Roosevelt vs Landon polls, or maybe it was Truman Vs Dewey, as I recall reading about it:

This was in the fairly early days of mass political polling. It was also in the days when telephones were far from universal, and typically only the upper-middle to upper classes had phones. Mere everyone else had party lines, if they had phones at all.

The polls were conducted by telephone, that being the new mass media darling fad of the time. Hey, telephones were so new-fangled! That meant the sample was skewed towards the wealthier segments of the population. That’s why the Republican candidates did so well in those polls.

Pollsters have learned a whole lot over the last 70-some years about how to choose population samples that are most likely to be well-representative of the population as a whole. Thus, polls have gotten better and better.

To be sure, the same understandings that tell us how to select unbiased samples also tell us how to choose biased samples, if one wants to conduct a poll to get some pre-determined desired result. That’s how, for example, Fox News or Breitbart can get their polls to show Trump, Cruz, etc., to come out looking good.

Yes, it was the Roosevelt/Landon election of 1936. No, telephones were not new-fangled, since they had been around for 50 years. True, telephones were expensive and their distribution skewed to wealthier families, who tended to vote Republican. This was not necessarily a bad technique since Literary Digest had correctly predicted the 1932 election that Roosevelt won. (They had also called the 1928, 1924, and 1920 elections correctly, although those were all easily won by Republicans.) No, the polls were not conducted by telephone; they sent out ballots to be returned via mail. Nor were the names taken only from telephone directories; they came from a variety of sources, although it’s possible that those were also biased against the very poorest since they included automobile registrations and magazine subscribers.

The real issue is that there was more to the mistake than just selection bias. They also had response bias.

There was nothing new about this technique. That’s why people were so surprised when it failed. It had been right for so long. In fact, it called Roosevelt’s vote to within 1% in 1932. George Gallup made his name in 1936 by getting it right through proper sampling, although his prediction was low by 5 percentage points for Roosevelt.

Later studies have determined that both selection and response biases needed to combine to get this wrong an answer. That’s why it was never tried again. Well, that and Literary Digest going out of business in 1938.

I think you are speaking of a version of the Bradley Effect, or social desirability bias. There are probably a significant number of Trump supporters who are too ashamed to profess their views out loud, so maybe Trump will indeed get a few million more votes than the polls project. But it don’t get him the election.

Clinton proved in the primaries that polls don’t matter?

Well, they don’t matter so much if you control who counts the actual votes.

People don’t like hearing it, but it is relatively easy to steal an election in a primary compared to a caucus. I don’t know if that happened, but there actually are somewhat credible accusations of conspiracy in a couple of major population centers. :confused:

Of course, general elections are different. I don’t think Clinton can steal the election from Trump unless he wants her to do so.

But he might be unserious enough to be happy with that.

So, Democratic confidence rests on Hillary stealing the election, Trump letting her, and the two of them not being pushed out by legal troubles before that time. I feel reassured, don’t you?

Got any specifics to share with the room?

That was what happened in 1936. I still recall during the war that my grandmother was one of the relatively few people on her block who even had a phone and she would frequently get calls for one of her neighbors who didn’t.

By 1948, polling was considerably improved, but they made two errors. First they didn’t state their margin of error and it was close. But most importantly, they assumed that the undecideds would split the way the decideds did. They didn’t. There also may have been a bit of “I’m for Truman, but I’m embarrassed to admit it”.

Polling has improved a lot, but it has gotten worse recently because they are not allowed to robocall cell phones and increasingly the younger people are eschewing land lines. Also a poll 4 1/2 months before the election is none too predictive. Hell, the negative ads haven’t hit the fan yet.

Three errors. They stopped polling, IIRC, two weeks before the election. Dewey’s win seemed assured, polling is expensive, and they didn’t realize how many people would make up their minds at the last moment.

Of course not. He’s still trying to justify that Bernie wasn’t as popular as Hillary.

A big problem with thinking Hillary somehow outperformed polling (and it’s the first part in a long series of “deeply held beliefs” you must have to buy into the Hillary stole the primary conspiracy theory) is a highly selective remembrance of the aggregate polling data. The biggest actual result that aggregate polling got wrong versus the actual results, in the Democratic primaries, was Michigan–and that error favored Sanders, as Hillary was projected as having something like a 90% chance of winning a majority of the Michigan votes, and by something like a 10% margin.

The rest of the aggregate polling on state campaigns was fairly close, for primaries. Caucuses had a greater margin of error but they’ve always been that way–and caucuses again, actually favored Bernie more than Hillary.

The big things that I saw happening during the campaign that I think contributed to Bernie fans confusion was they would often conflate national polls with state polls. And they would also conflate single-off polling results with aggregate poll numbers. And they would not pay attention to the pollster. Bernie fans hated FiveThirtyEight and called it a Hillary shill site–because FiveThirtyEight kept predicting Bernie defeats, and got them right essentially every time other than Michigan (at least for primaries.)

It’s actually not super easy to get a “good” primary poll, that’s why the quality of the pollster if very important. Some polling outfits would do online polls 5 days out that would show Bernie winning. These polls had no ability to realistically filter on things like “who is actually able to vote in the upcoming primary” and “how many people in this state voted early or absentee, back when Hillary was up by 20 points?” The better polls tried to take into account early votes and who could actually vote int he primaries, but there were a lot of polls from the “bad” pollsters, some so bad FiveThirtyEight and RCP don’t even include them. If you chose to cherry pick those polls instead of the much more reliable polls from respected pollsters, that were then aggregated, then it’d have been very easy to run from one state election to the next convinced Bernie would win, and then see him losing over and over.

They also focused a lot on exit polls. Some exit polls would have a small sample size, and again are just one poll, what we’ve learned in the last 8 years or so about poll aggregating is it is much more accurate that trusting a single poll. I don’t believe a single state in the Democratic primary had multiple independent exit polls (in fact I think many didn’t have exit polls at all), and one exit poll of a few hundred election day voters has a pretty good margin of error, especially compared to a half dozen polls taken by different pollsters and given weighted values based on the quality of that pollster’s operations and other factors like the age of the poll and sample size. The oft-repeated claim that exit polls are the way to “prove” election fraud, simply isn’t true, and there is no form of polling that can be 100% reliably with only a few hundred people asked, and for exit polling we know for a fact that it suffers from things like “enthusiasm bias” (a candidate whose voters are more enthusiastic are probably more interested to stop on their way out to talk to a pollster, a less enthusiastic voter is more likely to just brush by the pollster to get home.)

So take all these misunderstandings about how polling actually went during the election, and then combine it with how States run elections. States run elections very poorly. And they frankly, always have. Not only are there essentially no national standards on running elections, there aren’t necessarily state ones, either.

Go watch the docudrama “Recount”, which is about the 2000 election recount in Florida. Or just read up on that situation. What you’ll learn is that Florida essentially had 67 different mechanisms of conducting elections, one for each county.

You know how Bernie fans were up in arms about people being purged from voters rolls? That happened to 20,000 voters or more in Florida, many of them improperly. Conspiracy to give the election to Bush? Eh, I think “Recount” actually covers it pretty well. They definitely used Katherine Harris to push their hands on the scale, but most of these problems can be found in all fifty states. Florida may actually be among the very worst in how it runs elections, and thus why it was especially unfortunate (in terms of not having a national crisis) for it to be the deciding state in a Presidential election. None of the “election problems”, be it voters being turned away, voter purges, long lines, people being sent home after a certain time even though they shouldn’t have been etc–all of those are 100% found in every election. Not just Presidential elections, but State elections too. Go back to election 2004, there were tons of articles about this same topic back then, in Ohio.

But that’s because Ohio was the last State to be decided in 2004; Ohio wasn’t special. It was just a swing state. The media usually doesn’t even focus on the deep red or blue states, but they are full of electoral cluster fucks every time people go and vote.

When you have no national standards, often times no state standards–an amazing thing about the Florida recount is how much power county canvassing boards had, with the Florida Division of Elections actually only legally able to “advise” the canvassing boards on issues like how to design a ballot or how to conduct the hand recounts, it’s simply impossible to not have elections be a clusterfuck. Frankly, I do find it both scandalous and sad that it only gets attention every four years, in a close election, and usually targeted in a few competitive states. The grim reality is every time we vote in a state or local election, we’re entering a shit-show of a process that is badly run and straight out of the 1800s.

However, to get back to this year’s election–you can’t take that shit that happens constantly and is mostly just evidence of really poorly ran elections and claim that is any kind of evidence of fraud. Some of the most prominent examples that have come out in the Democratic primary have actually been ones that likely hurt Hillary. Like the infamous voter purge not long before the election in Brooklyn, that was a heavy Hillary district, so it’s more likely than not those voters would’ve helped her, not hurt her. [Plus there’s not been any real evidence most of those voters purged had not, in fact, moved, died, or been otherwise validly removed–removal required you to miss a few consecutive elections and most people that normally vote who miss a few consecutive elections have a reason for doing so, like no longer being alive or no longer living in the precinct.]

When a campaign loses an election by several million votes (in this case, by “election” I am referring to the primaries as a whole), there are two things that its supporters can do:

*They can (option A) recognize that a significant majority of voters rejected their guy in favor of his opponent, and they can resolve to do better next time;

*Or, they can (option B) refuse to recognize that voters might legitimately have reasons to vote for somebody else, which results in saying that the whole thing was illegitimate whether or not there is any actual evidence that this is the case.

I don’t think most or even a large minority of Sanders supporters are joining you in picking option B. Regardless, option B is *not *the way to go. If the Sanders “revolution” is to accomplish anything, it can’t sit in a corner with its collective thumb in its mouth, complaining that it just isn’t fair and that there were shenanigans (especially when those so-called shenanigans have been shown to be untrue–read Martin Hyde’s excellent post, or others in threads over the last several months). Instead of asking “How can we prove that the Clinton campaign stole the election?” they need to be asking, “How can we change our tone, our focus, to attract the votes of people who didn’t vote our way?” That’s the way to win elections and change the way people think. To focus on other stuff, in an election decided as convincingly as this one was, is to set the movement back.

Thanks for clarifying the myth (which I myself have vectored) about it all being down to phone and car registrations, thus excluding a notional vast majority of voters both carless and phoneless. They were a vast section of the electorate all right, but not a majority.