Especially during election time, the press is overly concerned with polls. Having worked for a market research firm, I have learned quickly to distrust polls.
Let’s consider a telephone poll. In a telephone poll you must meet the following basic criteria:
You must own a telephone with an operable telephone line
You must be home at the time they call
You must answer your telephone. No answering machine or privacy manager.
There must be a representative to take your call once you answer
You must continue to stay on the line, despite of the small delay. This is because you get accustomed to the delay being a telemarketer.
You cannot be deaf or mute (market research is not often done with TTY/TTD)
You must be able to speak fluent English (or on some rare occasions Spanish is acceptable)
You cannot have a mental handicap that hinders your ability to communicate clearly with the representative.
You must have the time available and willingness to complete the survey
Even if you get through these you often need to meet other criteria. For example, you must be a Republican, Democrat, or non-committed voter. Another requirement could be that you are of a certain gender, race, or age range.
Once you get through all the requirements, your answers are finally recorded and reported.
Hence, a senior citizen that is lonely with plenty of time to stay home and talk on the phone is counted in a survey. A busy businessman who is away from home a lot of the time is not counted.
Is it fair to even talk about these bogus values we call polls? Can the +/- x% values really offset the disparity?
I’d say they’re inherently inaccurate. News 4 in Detroit always quotes polls that seem to have a very liberal sample base. For example, they have Gore in a dead heat with GWB at a time when most other polls show Bush widening his lead. You see similar biases in many other issues like labor and education. If the issue is one that will be voted on, you can bet money that News 4’s polls will be nothing like what the voting results are. It all depends on who you ask, and that information is almost never revealed.
I worked for a market research firm years ago, and I would never trust polls. The questions in polls are almost always written with a specific agenda in mind - any market research grunt can figure out very quickly who’s paying for the poll by the way the questions are worded. It’s very easy to put words in people’s mouths.
Also, the people doing the polling (the phoners) are not without their own biases and will sometimes change responses and/or scrap surveys that don’t fit into their own particular view of the situation. I’ve seen left-wing student types do this, and bible-thumping Christians.
This is all true, but to a certain extent it can be corrected for. For example, they probably have some data that says “10% of the people don’t have phones, and those people are 50% more likely to vote for party A than people with phones.” It still won’t be accurate, but at least it removes systematic errors. In Japan there’s a more extreme case where people don’t like to admit that they vote for the Communist Party. So raw poll results severely underestimate the Communist Party votes, and they know to correct for it.
Also, you will notice that their “margin of error” numbers are pretty huge. People tend to overlook this and discuss 1% differences in 5% margin statistics.