Read the link. No one is conflating two stories. It was two different events, one in 1936, and one in 1948, but the problem was basically the same: relying on a biased sample.
Of course they get your name. They already have your name before they even call you.
No, I’ve also sone some polling for short term temp work, and I was given phone numbers to call, and no other information. I didn’t see any way to connect the poll results with an individual name. The only information that I attached to the phone number was if they didn’t ansers (the number would be called again later), or declined to participate. If they did participate, the number wasn’t attached to the response.
Of course, other pollsters might do it different ways.
Actually the 1948 case was more a case of jumping the gun. Wiki entry. The Chicago Tribune was a Republican paper, and issued the headline after the East Coast results and before the West Coast results, and based on the wide expectation that Dewey would win.
I think Literary Digest was unbiased - they just screwed up, though the same polling method had worked before.
I’ve sat in on more than one class on sampling which discussed the 1936 one, but none have ever talked about the 1948 one.
1948 was about like if Karl Rove wrote the headline for the results of the last election before all the votes were in.
If just a few states swung the other way, Dewey would have won. The polls were wrong, but not as wrong at the Literary Digest poll. (FDR won in a landslide).
In any case, the fame of the 1948 case was not that the polls were wrong, but that one paper did the wrong headline based on early results and wishful thinking. I don’t think they had exit polls back then anyway.
Yeah, real polls seeking to gain real information don’t have names attached to them, because there’s no need. But push polls bear only the most superficial resemblance to real polls, and usually do have names attached to them.
And also of course, the fame of the 1948 case was surely due, in large part, to that famous and widely published photograph.
Recent Elections threadabout the currents problems with polls as per the assessments of professional pollsters and the likes of Nate Silver.
The current political entertainment industry creates a huge market demand for polling product but the polling industry has not yet figured out how to create cost effective and accurate product in a world that has rapidly, but in uneven and so far unpredictable by demographic and political lean ways, given up land lines and is less likely to answer calls from numbers they do not know. And even are the people who agree to participate in a poll way before an election a self-similar group to those who agree to participate as the election draws nigh?
As Nate Silver put it (referenced in that thread in a more complete quote and with link)
The political entertainment industry needs numbers to keep the horse race narrative going. But at this point they are essentially clickbait.
Please anyone correct me if the following isn’t true:
The thing that’s hard to convince yourself of is that if you take a truly random sample of a population, the number of people you need to get a sufficiently accurate sample is the same regardless of how large the entire population is. And yet it’s true. So you want to get a sufficiently large sample of a city of 50,000 to get a certain accuracy for just that city, it would be the same as the size of a sufficiently large sample for all 7,000,000,000 people in the world. Approximately, the size of the truly random sample you need to get an accuracy of about 1 in x is about x squared. So to get an accuracy of about 1%, which is 1 in 100, you need a random sample of about 10,000, since that is 100 squared. And that’s true regardless of whether you’re talking about the 50,000 people in one city or all 7,000,000,000 people in the world.
The real problem is getting a truly random sample of the population. If you could just take everyone in that city or everyone in the world and throw them in a huge bowl, mix them up thoroughly, and then draw out a person at a time, 10,000 people being drawn out would give you the same approximate accuracy in each case. But of course you can’t throw everyone in a city or in the world in a bowl. You have to create a way of picking the 10,000 people that you survey. And any normal way of picking those 10,000 people for the sample makes it hard to keep the sample truly random.
I think the first part is not true for most here anyway. Pretty much everyone here understands that the issue is not so much sample sizes but getting a true representative random sample. It’s why aggregation won’t help: non-random non-representative in is still crap with a bigger sample size.
I think the first part is true for the OP. He asked why we don’t see pollsters on every corner, just like we see McDonalds on every corner. The answer is that we don’t need that many people polled to get a random sample of the American population. The OP seems to think we need a larger sample because there are so many people in the U.S. But of course that’s irrelevant. If you could somehow get a truly sample of all the 310,000,000 people in the U.S., you’d only need, say, 10,000 of them for sufficient accuracy. That’s only one person in 31,000.
Well your statement assumes “a truly random sample of a population” … Given a large, not “well stirred” very heterogenous mix (which is the United States voting pool), a small sample from one or two corners (so to speak) is more at risk to not be “a truly random sample of a population” than a large sample taken from many different corners. A small well mixed pool is more likely to have such a true random sampling with smaller n.
I can’t speak for the op but I read it not as a question of the size of the n but as asked: “who are the ones polled?”
And that is a real and increasingly significant problem according to the experts. The people who are polled are increasingly the people who are standing on the corner where the light is good. Getting a sense of what is going on with the people who are not where the light is good, which is more and more of the voting population, is both increasingly difficult and something the polling companies are well incentivized to do, even if they could figure out how to do it. Those who happen to be at that corner may not reflect the whole country.
Who is where the light is good and is polled?
Those with land lines who answered the computer generated call whose caller ID they did not know and took the time to answer the questions.
Those hand dialed on cell phones who answered an unidentified caller ID and took the time.
And those who respond to internet polls.
Of all those called the numbers have gone from, once upon a time, that being about 80% of all those called participating, to 36%, to recently 8% or less. The pollsters can weight the responses such that the responses “equal” the demographics of however they define the likely voter population but that still does not mean that the White 50 to 65 year old woman who responds is necessarily reflective of all White 50 to 65 year old likely voting women. My guess would be that those who pick up the phone and take the time to participate, and those who respond to internet polls, are a non-random self-selecting sample, especially farther away from an elections. But we have no way to know, and no way to do anything more than speculate in which ways they would be non-random.
The Straight Dope on polling, from the experts, is that they are producing more product but that the product is of poorer and poorer quality.
Meanwhile more and more of the political discussion is focused around the polling numbers than on issues.
Yes, of course my post assumed a truly random sample of the population. That was the point of my post. If it were really, actually possible to somehow find a random sample of the U.S. population, you would only need about 10,000 people in that sample for a reasonably accurate poll of that population, even though the sample would be only about one person in 31,000. As I said above, the real point is how difficult it is to get a truly random sample of the U.S. population. If you could throw everyone in the U.S. in a giant bowl, mix up everything in the bowl very well, and then draw people from the bowl one by one, you would get a truly random sample of the population. Unfortunately, that’s impossible. In practice, all the usual polling methods are hopelessly biased. Stopping people on the street, calling them on the telephone, setting up a website and asking people to post to it, going from house to house - none of them give a truly random sample.
Beyond sample size, achieving a random sample and the problem of actually reaching people by phone (land or cell), I think there is another huge problem with political polls. The problem I see relates to the truthfulness or sincerity of those polled. I don’t truly believe that all of the people who say they favor Trump would actually vote for him. At this stage, 14 months before the general election or five to eight months before primary elections, it’s easy to tell some stranger on the phone that you support Trump. Actually pushing the button for him in the voting booth is another thing altogether.
I know when I’m called by a pollster - and it has happened - I am just as likely to screw around and answer misleadingly as I am to tell the truth. I think many, if not most, people are so disgusted with politics and the media that they feel no duty to answer pollsters honestly.
This little bit just in on the way polls have been used this cycle to determine who is in debates and who is out.
I’ve made survey calls for the last couple years. If you think getting calls is a pain, you should try making the calls. People who cite the “do not call” list are morons, as that is only for commercial calls (selling you something, etc). Your own bank and others you already deal with are exempt from “do not call”
Most survey calls are political these days and some people who get calls aren’t happy about it. “Where did you get my number???” Duh, if you vote, it is from the public voter registration.
Of course, there is the rudeness factor. Tops on my list are those who hang up the second I say who I am, etc or mention the word survey. Survey calls I make can be answered in 30 seconds and I mention this when calling, “Would you take 30 seconds to answer a few question” and some people take more than 30 seconds to tell me they are just going out, getting ready for dinner, etc. Then there was the guy who said he couldn’t talk cause he works during the night and needed to get some sleep. We’re not supposed to be rude, but I said, “I wouldn’t want to take 30 seconds from your nap time”
Of note, I don’t do survey for PEW or other national survey for “public opinion”. Rather, it is for a national organization that advocates for fiscal conservative issues. Due to IRS classification, they can not advocate for any particular candidate of party. Since, they can’t advocate for “John Smith”, what they do is find candidates (local, state or federal) who holds similar beliefs. The calls are to determine how people think. Hypothetically, do you like the color purple or green? After they identify all the people who like purple (assuming similar belief of a candidate they like), the calls then go into “get out the vote” mode before an election. If purple people are the desired voters, it’s not likely the green people get a reminder to vote.
Done on a massive scale, it’s possible to move the needle several points one way or the other in an election or on a ballot issue.
Last comment, since the surveys identify political leaning, I’ve found that those holding liberal views are the rudest. They also tend to be younger whereas conservatives are older and more pleasant to talk to.
A response from talking to an 84 yr old woman had me in stitches. How do I know she was 84? Again, Duh, it’s public record on voter registration.
One of the most common survey questions deals with a particular piece of national legislation on health care, with the question, “do you approve or disapprove?”.
This lady said, I hate everything about that @##@#@ (a person who holds office whose name is often used to describe this particular “health care”
Far from being a back woods hick, she was well spoken and proper. Hearing the F-bomb come out of her mouth put me in stitches. Other survey questions dealing with government taxes and spending elicited similar colorful expressions.
Did you say “I take that as disapprove?”
Here is an article about the current state of US polling-and one person who still does it correctly.
Hint: she works hard to get random samples and works hard not to bias the results. Her work is very good and very expensive.
Oops, I didn’t see DSeid’s second post before I provided this link. Still, the story is a good one.
It’s actually pretty complex. As this article notes, bad polling techniques hurt. But also notice how narrow the margin for a supposed “lock election” was-- a measly six points, within margins of error. It should have been called a near tossup. Plus factor in the facts polls took a lot longer back then (up to three weeks) and that voter turnout that year was much lower than normal that year and why the pollsters got it so wrong is now obvious.