Are trick questions fair?

In My Humble Opinion
1

Okay everybody- no cheating!

Trick questions are those that, according to my personal definition, are written in such a way as to lead you to select a wrong answer. Sort of the opposite of a leading question, for the trick question is truely misleading.

Are they fair? I guess your opinion depends heavily on context. On a formal quiz, say in Entomology:

  1. Which of the following is not an insect?
      a. spider
      b. millipede
      c. worm
      d. tick

The answer is, of course, that none of them are insects. Is this a fair question? The question strongly suggests that there is a correct answer here, when in fact all of the choices are correct. If the exam rules do not specifically state that questions may have multiple correct answers, this question (even though it is easy) is not fair IMHO. If you fail to select all of the choices, you will get this question wrong even though you knew that a spider is not an insect (satisfying the question literally).

  1. Raincoats are:
      a. yellow
      b. plastic
      c. long sleeved
      d. waterproof

This is a slightly different sort of trick question. I’ll let you guys select the “correct” answer. Is this a fair question? How do you feel about exams that are full of these kinds of questions?

2

My opinion of multiple choice questions is that they are designed to make it easy to grade the paper. In order for multiple choice questions to test levels of intelligence about the “duh” level they have to be tricky, arbitrary and “yes” unfair.

3

Yes. No. I mean - dammit, is that a trick question?

d&r

4

Thinking back to my college days (yes, we had trick questions in the Stone Age too), I hated trick questions. There was one exception – I had one professor who would give you credit if you missed a trick question if you could come up with a creative reason for why your answer was also a correct answer, which I thought was sporting.

All raincoats are waterproof, right? Otherwise they wouldn’t be raincoats. Not all raincoats are yellow, nor are they all plastic, and I guess you could have a short sleeved raincoat if you wanted to. Or it could be one of those poncho style raincoats that doesn’t have any sleeves. Ugh, just knowing I was expected to tackle a trick question made me anxious.

5

Because Waterproof is the only answer that is common with all Raincoats and the only answer that is in the definition, it is the correct answer. The other three answers are all answers of what raincoats can be - but don’t have to be. If you substituted Waterproof with heavy, then you could select all answers. I’ll parallel it to this question.

Humans are
a) tall
b) primates
c) athletic
d) black
A human has to be a primate, but a human doesn’t have to be black, athletic, or tall.

6

I think the question is not so much “is this question fair?” but “is this question valid?”

When I was taking instructor’s courses in the Army that rule about writing tests was very simple; no trick questions, ever. The official policy of the university I went to around exams: No trick questions.

The problem with trick questions is that they test your ability to take tests, rather than your understanding of the subject matter. If the purpose of today’s exam is to test you on the knowledge you’ve gained in four months of Geography 225, the questions should be designed to measure your understanding of Geography 225. A trick question, however, to some extent, measures your ability to spot trick questions and guess what the trick is - not your understanding of the course meterial.

I don’t understand your comment, kniz. I’ve taken AND written multiple choice tests that were quite difficult and included not a single trick question; just one clearly right answer and three clearly wrong ones for each question. But they were hard.

7

No, they’re *##! not! This may not be the kind of trick question you were thinking of, but it’s something that happens a lot in my church - and I’ve seen it used in lectures too. It’s just an attention-grabber, really, but the speaker will say something like “Who wants more of God?” At which in the sheepish crowd atmos of a church, of course, 95% of us raise our hands and a couple of folk go “Amen!”

And then the preacher goes, “Well you’re all WRONG, because doctrinally you’ve already got as much of God as you’re ever going to get! Ha!” What troubles me isn’t the doctrines at issue, it’s the way we’re forced into making the wrong move, which somehow gives the guy a mandate to hassle us for a bit. Crowds just aren’t as smart as individuals, especially when they’re berated just as often for not responding at all, and he’s playing on that. It’s a stupid game to play with people when you need to be taken seriously. Really disreputable speakers can use these moments as occasions to get thoroughly angry at a congregation. Proper ones just use it to get attention, and move on.

I also saw it done by a guy lecturing on bees and beekeeping, to an audience of complete laymen.
“Do you think there are four billion bees in that hive?”
“Umm… sure.”
“What kind of an idiot are you? There’s four thousand!”

Cue ritual humiliation. Like we ever had a chance to answer for real.

Hacken? Hacken? Sassen frassen rassen…

8

[QUOTE]
*Originally posted by RickJay *
**

Amen! Trick questions are simply wrong. The purpose of a test is to query the knowledge of the person taking the test, on the subject matter of the class. They are not supposed to test how well a student can detect trickery.

9

I disagree completely. A trick question at least tests your alertness. And a person who can answer a trick question correctly shows full mastery of the subject. Something like Attrayant’s first example (but with proper instructions beforehand) tests not only that you know what insects are, but what they are not. This sounds trivial, but it really isn’t.

Here’s an example of a trick question from physics: Is light made of waves or particles? Scientists debated the question for years. No one expected the real answer - that Nature doesn’t distinguish between the two at that level. This answer changed our understanding of physics at its very foundation and literally changed the world. It eventually led to nuclear bombs and lasers, among other things.

I tend to agree for this particular type of question. But for trick questions in general, notice at the beginning of the course should be enough. I have never had an exam that was “full of” trick questions. I have only encountered such questions occasionally.

Tangentially, you might be interested in a PC-based multiple-choice test of my competence in computer programming that I had to take a couple of years ago. It was a truly hard multiple-choice test. In the instructions it stated plainly that any number of the choices could be correct, including none. In addition, some wrong answers would get you partial credit, and other wrong answers would be penalized. Their example went something like this:

“a” would get partial credit (because it’s close), and “b” would be penalized (because it’s not even in the ballpark).

It was the most grueling test I ever took. The final scoring was based on all the people who had previously taken the test. I got the equivalent of an A.

10

I write all of my own tests, both for the college courses I teach, and for my 5th grade class. Multiple choice questions are an entirely valid form of question, if they are well constructed. I’ll get back to this in a moment.

Question 1 is clearly unfair. It should be written as an alternate response question, like this:
Place an I in front of the animals that are insects, and N in front of those that are not insects.
___1. spider
___2. millipede
___3. worm
___4. tick

Question 2 isn’t a trick question, but it is a very difficult question, and the question itself is poorly worded. The question should be changed to “Which is characteristic of ALL raincoats?” It is a “best answer” type multiple choice question.

There are two types of multiple choice question. The first is the correct answer type question. These have only one correct answer, with two to four incorrect answers. These can vary from easy to very difficult. The second type is the best answer question. This type may include more than one answer that could be considered correct for the question, but has one answer that is better than the others. These are more difficult than correct answer type questions.

Question difficulty generally depends on the number of “distractors” that are included. A distractor is a wrong answer that could seem right to someone who doesn’t know the material well. Easy questions don’t have any distractors. More difficult questions have 1 or 2. The most difficult questions may have distractors that are at least partially right, making the question into a “best answer” question. A good test will consist of range of questions from easy to difficult, with a majority of the questions being somewhere in the middle. A question with one distractor is considered moderately difficult.

For example, suppose the question was this:

Where is the Eiffel Tower located?

The answer sets might be

  1. A. Paris B. Moon C. Atlantis D. Carnegie Hall
  2. A. Paris B. London C. Atlantis D. Carnegie Hall
  3. A. Paris B. London C. Rome D. New York
  4. A. Earth B. Europe C. France D. Paris

Set 1 is the easiest; there is only one correct answer, and no distractors; all of the wrong answers are obviously wrong.
Set 2 has one distractor (London), Set 3 has three.
Set 4 has four correct answers, but one (Paris) which is better than the others.

All of these are fair questions of increasing difficulty. On my tests, most of MP questions will have answer sets like #2, with a few like #1, and a few like #3. I only use answer sets like #4 in my college classes.

Trick questions are those in which what the question is actually asking is different from what it seems to be asking, and are unfair. A question can be very difficult without being a trick question. To determine fairness, one needs to ask, “Will someone who knows the material well be able to answer the question correctly?” If the answer is yes, the question is fair. If a question is constructed in such a way that people who know the material well are led away from the correct answer, it is unfair.

11

I think there is nothing wrong with them. If you truly know your material and you are concentrating on the test, you will catch them. If you are uncertain of your material and you are relying on tricks like ‘Which of these looks the least wrong?’ then it will get you. Anybody who was thrown off by that insects trick question does NOT know the subject matter (which I’m assuming would be the taxonomic classification of invertebrates).

What pissed me off were teachers who would give nothing but multiple choice quizzes and you could always count on 2 or even 3 of the 4 being VERY obviously wrong. You could tell they just didn’t want to fail anyone.

12

I’ve never taken a course in alertness. I’ve never heard of a course in alertness. So why should an exam include a question that tests alertness? This strikes me as being about as logical as including a physical fitness test in a math exam. A test should be as transparent as possible - it should measure understanding of the subject matter, not test-taking ability.

Having questions that are different from the norm is fine if they aren’t tricks. If the exam starts off with the instruction “Some of these questions don’t have right answers, or have more than one right answer” they aren’t trick questions.

That’s why the insect question was a bad one; it deliberately led the student to believe that one of the answers was right when in fact none were. What’s the right answer to that question? Just leaving it blank? The question’s being there implies an answer must be chosen, unless the test instructions specifically say some questions have no right answer (and I’d gather from the OP they didn’t.) Giving the student a conflict over how to write the test distracts from the job of measuring the student’s understanding of the subject matter. It was a stupid question.

On the other hand, I liked the raincoast question. There was one right answer. It’s a tough question, but fair. Fair tests can still be hard.

13

My only comment here is that if presented with an obvious “trick” question, such as #2 in the OP, I’d have a leg to stand on if I were to get it marked wrong and wanted to fight it.

Multiple choice tests I hate are the ones where every single question includes:
[li] e) none of the above[/li][li] f) all of the above[/li]
Grrr, I hated those.

14

Heh. That’s how I wrote all my quizzes. The multiple choice format is easy to grade, and including “all of the above” and “none of the above” removes any advantage the format offers to the student. Bwah ha ha!

By the way, you should reverse e & f . . . If f) is “all of the above” that would technically include e). That little paradox is guaranteed to land you a student screaming at you in office hours.

I believe that the answer to a properly-designed multiple choice question is unambiguous, even if it’s a trick question. The student, upon hearing the answer, should find that it’s obviously true, even if he thinks it’s not “fair.” No exam should have more than one or two “trick” questions–something to separate the men from the boys, to differentiate the A+ from the A, not to ruin the grade of an earnest student who is just flustered.

I also think that every test should include clear instructions–“Choose the answer to each question that is most correct,” or something equally transparent. Otherwise it’s not fair at all. If a student doesn’t bother to read the instructions to find out that some questions have no correct answer and should be left blank for full credit, well, then boo hoo. I screwed up an exam I was taking because I didn’t read the instructions once. ONCE.

15

I agree with everything you say here. I refuse to give any oral instructions when giving a test, in both my college and 5th grade classes. Students need to learn to read the instructions. When a student comes to me with a question about the instructions, I have him/her read the relevant instructions out loud to me, and they nearly always understand them then.

I do the E: all of the above thing also. Unfortunately, the optical-scan answer sheets we use only have A-E, so my instructions tell students to leave blank any question with no right answer. My quizzes and finals are all multiple choice.

On unit exams, which are of the write-in-the-correct-answer type, I like to use Corrective True-False questions, which the students also tend to dislike. On these, if the statement is true, they just mark a T, but if it is false, they must mark F, and rewrite the statement in such a way that it is true. For example:

___1. The Pulitzer Prize is a prestigious national award given each year for the outstanding children’s book.

To get this one right, the student would have to mark it F, and rewrite the statement, substituting either Newberry Award or National Book Award for Pulitzer Prize.

16

Why can’t I mark it “F” and stick the word “not” after the word “is”? :smiley:

17

I think the bolded words are the only ones you can change, otherwise I could say “The Pulitzer Prize is my favorite award” and have a true statement.

18

Bingo! I always bold the key part of the statement, and only that part may be altered to correct it.

19

I pretty much agree with what Podkayne said.

You have taken a course in alertness. Everybody has. It was very short and informal. It occurred the first time you came across a trick question. Everybody gets refresher courses, too (myself included). :slight_smile:

Concerning Attrayant’s insect question, I originally thought it was borderline unfair. But after reconsidering it, I now think it’s borderline fair. The question was not “Which one of these…” - it was “Which of these…” This careful reading would suggest the correct action to an alert student. Attrayant said it was a quiz in capital-E Entomology, from which I infer that this is a college course. I’ve never heard of a high school offering a course with such a narrow focus, but I could be wrong. (Admittedly, it’s a rather basic question for a college course.) College students should be able to pick up on things like this. The question would be even fairer if it were scored as one point for each selection chosen. I personally would tell the students about the possibility of trick questions at some time prior to their appearance.

Maybe this sort of analysis is what is meant by “test-taking ability”. But let’s look at a slightly different case: you misunderstand a question and choose the wrong answer. You are just as wrong as if you didn’t know the subject. This is generally true outside of an educational setting as well as within it.

20

Referring to the OP’s insect question I would probably only pick one of the choices and argue with the teacher if it was marked wrong. The Question is “Which of the following IS not an insect?”(Capitalization added) and a spider(or any of the other choices) is not an insect, therefore it is a correct answer. The question does not read “Which of the folliwing ARE not insects?” And any question that has more than one correct answer and asks you to pick the best answer is at least on some level subjective in my view at least.