Are trick questions fair?

In My Humble Opinion
21

I remember a worksheet we did in 4th grade. It had 30 problems on it and the instructions said to read through all 30 of them before begining the worksheet.
The problems were things like standing up in class and saying your name. Counting to ten out loud. Doing a math problem. Finding the noun in a sentence.

#30 said: only complete problems 4, 12, and 17 on this worksheet.

Two people in the class finished the assignment in five minutes and sat there smiling. The rest of the class finished in around half an hour.

Fair? I’d say so.

22

Of course “best answer” questions are at least partially subjective. This does not, by itself, make them either unfair or trick questions. The ability to make subjective judgements is a higher order thinking skill which everyone needs to develop. Take my example above. Given the Question “Where is the Eiffel Tower located?” and the choice of Earth, Europe, France, and Paris, Paris the best answer to the question because it is the most specific. This question requires the student to know both where the Eiffel Tower is, and to exercise some critical thinking skills.

23

Trick questions as far as riddles are concerned are supremely evil and not to be tolerated. Once a friend told me:

Q: How do you get an elephant into the fridge?

  1. Open door.
  2. Insert elephant.
  3. Close door.

Q: How do you get a giraffe into the fridge?

  1. Open door.
  2. Remove elephant.
  3. Insert giraffe.
  4. Close door.
    and I had to smack him.
24

With that example of the location of the Eiffel Tower I don’t really see how with no context Paris is any Better of an answer than France. If you asked for the most specific answer Paris would be Better, but if you asked for Best answer and you were talking to someone planning which countires to visit in Europe I would choose France as the best answer. Just depends on the context.

25

If I came across that insects question on a test and I was not told ahead of time that there were some questions where there was no correct answer, I would bring it to the attention of the teacher, let them know that none of the answers were correct, and ask what to do. There is no way I would ever choose an answer I knew was incorrect because all the choices given were incorrect.

26

If you want to visit the Eiffel tower it isn’t enough to go to France, you have to go to a specific place in France, namely Paris. Figuring out that the most specific answer is the best answer is one of those critical thinking skills I referred to.

27

Wow this thread has taken off. I’m not quite sure what to do about that :slight_smile:

First, some context. The insect question was observed on television, possibly the discovery channel. I wasn’t paying close attention until the narrator went into quiz mode and asked the question. Feeling superior, I was making the mistake that so many students make when taking oral quizes- I was listening with my answer idling. As soon as the choice a (spider) was presented, I said “ah HA!” A spider is not an insect. What kind of a moron do they think I-- then the 2nd choice was shown: millipede. “Hmm…”, I questioned myself, "I didn’t think a millipede was an insect either. Let me think about that spider choice again.

In short, the narrator was causing me to doubt myself. After all choices were presented, I wrinkled my forehead & mumbled “what a load of nonsense! NONE of them are insects!”. I then sat back in my chair just waiting for the narrator to give the answer and tell me which was the correct answer (i.e., which is not an insect). Of course I had to roll my eyes when the answer was given (all choices are correct because none of them are insects).

So I was initially incensed because the narrator made me question & doubt myself on such an easy question. On a written quiz, this clearly would have been an unfair trick question unless you were told beforehand that multiple correct answers are permitted.

Now, on to the issue of what the quiz is actually measuring. Clearly, at least IMHO a test measures a person’s mastery of the subject matter and the person’s ability to take a test. When I used to teach math and electricity/electronics classes, I had many students who I knew had a high level of mastery of certain subjects but never the less did only C/C+/B- on quizes. When I talked to these students face-to-face, they didn’t miss a trick. They could walk me through complicated circuit diagrams and explain signal processing as if they were journeymen in the trade. I could not believe these students were not making straight A’s on my quizes.

Then when I sat down and reviewed the quiz with them, it came to light that some questions were focusing more on tricking the student into picking a wrong answer than on just evenly and fairly measuring the student’s knowledge.

To make an analogy, let’s say I want to test your juggling ability. You believe yourself to be a fine juggler and agree to take my test. On the morning of the test, you show up and I ask you to put on ice skates and stand on an ice rink while juggling angry ferrets. And because this is a mid-term juggling test, I’m going to make things a little harder by playing Ride of the Valkyries at maximum volume over the rink’s low-fidelity PA system.

Am I really testing your juggling skills? Amd I fairly testing them?

28

Q: The Lion (King of the Jungle) invites all the animals to a feast. Who doesn’t attend?
A: The Giraffe - it’s in the fridge

Q: How do you cross a crocodile infested river?
A: Swim across - the Croc’s are at the Lion’s feast.

:slight_smile:

29

According to an education professor I had in college when I was studying to be a high school math teacher, a multiple-choice test is not only a valid type of test, it’s the ONLY valid type of test. According to him, it’s the only type of test you can construct in which you can do a statistical analysis on the results and truly analyze whether the questions are valid.

As for how you were to apply this technique to math tests, that didn’t matter, because he felt that math was a worthless subject anyway. But that’s another story.

I don’t like trick questions, but then again I’ve never liked multiple choice tests either. They’re easy to grade and yes, easy to analyze, but I don’t think they’re really good as a test of knowledge. I think my main reason against MC tests is just that they’re so easily abused. Either you have trick questions, obvious distractors that raise the success rate of guessing, stupid distractors that actually could be valid answers, or, and I’ve seen this more often than I feel comfortable with - incorrect answer keys. I had one prof who gave the same MC tests year after year even though his keys were wrong.

30

I hate trick questions in general, but especially this kind. Last night, a friend emailed me a bunch of riddles and trick questions. Here’s an example of one:

You’re probably thinking, like I did, that the answer is 4100. But no, you’re wrong! The answer is 2030, because the total starts over with the third “1000”. The way the question is written implies you sum all of the numbers, unless you’re very picky about the semantics.

31

Most have probably seen this http://www.pressanykey.com/cgi-bin/cgiwrap/pak/jquiz.cgi for those that have not check it out.

32

Trick questions are, by definition (I think - I haven’t consulted Webster’s) unfair. It’s a trick!

33

The critical thinking skill it exercises is the skill of figuring out what answer the teacher wants to see, and figuring out that this is more important than other considerations. This is a valuable lesson for business (Who cares what’s right, just do what the boss wants) but I’m not sure it’s appropriate for 5th grade. You should at least wait for high school before demonstrating that those in authority can make arbitrary decisions which affect others.

If you ask where the Eiffel Tower is, and allow people to choose from Earth, Europe, France, and Paris, you shouldn’t mark any of the choices wrong. They’re all absolutely correct. Now, if the instructions for the test said “If there is more than one correct answer, choose the one listed that is both correct and most specific” it’s still arbitrary, but at least the students know what your arbitrary criterion is.

Also, you say that Paris is more specific an answer than France, but have you ever been through Paris, Texas? There are plenty of places called Paris. Is Paris really more specific than France?

And if France is an “incorrect” answer, then isn’t Paris also incorrect? There are more specific descriptions of the Eiffel Tower’s location. What about a street address? What about exact latitude/longitude coordinates?

34

You need to read more carefully. First, I said that I use such answer sets only with my college students. Second, this is a criterion-based judgement, not an arbitrary one, and I identified the criterion (specificity). Third, I teach my students test-taking skills, so as to reduce that as a factor in determining their grades. That a more specific (or more detailed) answer is usually a better answer is one of the test taking skills that I teach.

I said that all of the answers were correct. The instructions on the final that I gave to my summer session Children’s Lit class read: “For each question, there may be no correct answers, one correct answer, or more than one correct answer. If there is no correct answer, do not mark any answer. If there is only one correct answer mark that answer. If there is more than one correct answer, mark the best answer.”

The question refers to the Eiffel Tower. Anyone who associates the Eiffel Tower with Paris, TX or any Paris other than the one in France deserves to miss the question. I notice that, in your comment above, you assumed that I meant Paris, France.

First, you’re making Straw Man argument. Where did you get the idea that I thought “Paris” was the only correct answer? I specifically said that all of the answers in that answer set were correct. Second your alternatives were not among the answer choices, so they are irrelevant.

That being said, you might be right. This question may be unfair. It may be that a large number of students would choose France over Paris, and if they did so, I would throw the question out. I would have to give it to a class and then conduct an item analysis (which I do on every question on every quiz or test I give) to determine whether the question was fair.

It being a hypothetical, we’ll never know how many students would choose Paris as the best answer. Therefore, I will present two questions that did actually appear on one of my tests, both of which have more than one answer that is at least partially correct, forcing students to choose the best answer:

  1. Who is the protagonist in Charlotte’s Web?

A. Charlotte
B. Wilbur
C. Fern
D. Templeton

Referring to Maniac Magee

  1. Where did Jeffrey feel most at home?

A. the band shell
B. the buffalo pen
C. with the Beales
D. with his aunt and uncle

Both of these questions have more than one answer that is at least partially correct. After performing an item analysis, I threw out one of them, but kept the other. I’ll come back later and explain which question and why it was thrown out.

35

I had a history teacher in high school who went even further. She had multiple choice questions such as:

George Washington was:

a) born in one of the 13 colonies
b) a white male
c) a slave owner
d) the first President of the United States
e) the man who invented basketball
f) all of the above
g) none of the above
h) both a and b
i) both c and e
j) both a and d
k) some of the above

Some of the above??? WTF? And yes, she would literally have 10 or 12 choices for each question. I did very poorly on my tests that year in history class.

36

Damn, Jeanie, and I thought my tests were tough. On your sample, A,B,C,D,H, and J are all correct, but not E,F or I. I would mark K on this question, but to be sure, “Some of the above” is a flawed response, as it is too vague.

OK, to return to my previous post. I have bolded the best answer to each question. I then give my rationale for picking the answer I did as best, and the item analysis.

  1. Who is the protagonist in Charlotte’s Web?

A. Charlotte
B. Wilbur
C. Fern
D. Templeton

Rationale: To some extent, Charlotte, Wilbur, and Fern are all protagonists, so the question then becomes, which character is the lead character, and which are important supporting characters. We can eliminate Fern first, as she fades into the background after the first few chapters. Charlotte appears well into the story, and is gone at the end. Wilbur gets the most screen time, and the story revolves around him, so the best answer here is Wilbur. Charlotte is the most important supporting character.

Item Analysis: Out of 44 students taking the test,A: 19; B: 9 C: 11; D: 5–20% correct. Of the top 10 students, 6 chose A and 4 chose B, 40% correct. This told me that the question was flawed, leading students who did and did not know the material well to the wrong answer, namely A. As a result of this item analysis, I changed the answer criteria and accepted A as well as B.

Referring to Maniac Magee

  1. Where did Jeffrey feel most at home?

A. the band shell
B. the buffalo pen
C. with the Beales
D. with his aunt and uncle

All of these are places Jeffrey calls home at some time during the story, so all are too different degrees correct. He runs away from his aunt and uncle’s house, never to come back so that is a poor answer. He leaves the buffalo pen several times, to live with the Beales, and in the Band shell, so that seems to be more of a waystation than a home. He leaves the band shell after the groundskeeper dies, even though he could have stayed unnoticed. The only home he leaves, and then returns to is the Beales. He leaves the Beales’ home initially to protect them. He ultimately makes the Beales’ home his permanent home.

Item analysis: Whole Class: of 44 students A: 14; B: 12; C: 16; D: 2–36% correct. Top 10 A: 8; B: 1; C: 1; D: 0–80% correct. This told me that the question was difficult, but fair, as the best answer was the most chosen, and the students who knew the material best overwhelmingly chose C as the best answer.

“Best-Answer” type questions can be fair and are useful for testing higher-order thinking skills, but any question should be constantly under review for whether it is fair. There is a difference between hard and unfair.

37

I remember that test. Those of us that read the directions would very amused by everyone else. However, a few went beyond the call of duty on this one. One of the instructions read, if you are the first one to reach this point, stand up and say, I am the first person to reach this point. When he stood up and said it, I chuckled. When the second guy did it, I laughed. When the third guy did it, I just shook my head.

38

[QUOTE]
*Originally posted by Attrayant *

I’d like to see an example of this.

Let me modify your juggling analogy a bit. The juggling test starts, and I start heckling the juggler, shining light into his/her eyes, etc. No, this is not just testing juggling ability. It’s testing the overall ability to be a juggler. Hecklers are inevitable, and a juggler who can’t handle them and loses concentration will have a very hard time. (This is how Tiger Woods’ father trained him.) But I personally wouldn’t include this in a test without advance notice.

39

You’re right, I do need to read more carefully, you did in fact say that you didn’t use this sort of question on your 5th graders. Sorry 'bout that.

If a decision is based on a specific criterion then it is not arbitrary. However, the criterion itself can be arbitrary. But it doesn’t matter, since you’ve explained that you do tell your students what the criterion is.

No one has to associate the Eiffel Tower with Paris, Texas for “Paris” to not be specific. It refers to several geographically disparate cities.

I don’t think it counts as a straw man if I’m arguing with you about somthing I actually thought you said. But, you did say all the answers were correct. My mistake. As for your second point, I heartily disagree, but there’s no sense going into it since we’re arguing about something you never actually said. :slight_smile:

Hmm. I think this is a good question, personally. Although that depends on what you’re trying to test for. If you’re trying to see if students did the reading, it’s not a great question. If you’re trying to see if students know the difference between a protagonist and a title character, it’s a great question. Provided, of course, that you’ve taught them what a protagonist is.

Anyway, it sounds like you’re a fair teacher, given that you do give sufficient understanding of what is expected and you’re willing to discard bad questions. (“Fair” meaning equitable, not mediocre) Sorry I was argumentative, it’s just in my nature to argue with every teacher I see.

40

Senglelat: Thanks for the feedback. As I said, the Eiffel Tower question might be flawed; I’d have to give it to a class first. The purpose of the Charlotte’s Web question (this was again my college sophomore level children’s lit class) was to test both whether they’d done the reading, and if they understood what a protagonist was. This is what I was trying to show with my Eiffel Tower question; that a single multiple choice question can test more than one thing at once. I guess I used a poor hypothetical.

I thought the question was fair on its face, too, but the item analysis told me that I had somehow not taught the class the concept of what a protagonist is well enough, and this enabled me to go back and fill in that gap in my instruction.