# Real life examples of biased test questions? **URL:** \ **Category:** Miscellaneous and Personal Stuff I Must Share\ **Created:** [April 17, 2012, 10:56am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022 "2012-04-17T10:56:17Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![Meyer6](https://avatars.discourse-cdn.com/v4/letter/m/cc9497/32.png) [@Meyer6](https://boards.straightdope.com/u/Meyer6)\ **Post date:** [April 18, 2012, 1:50am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/41 "2012-04-18T01:50:52Z")
> [@dataguy](#): > > You should re-read my post. The TEACHER felt that this was a biased question. Sorry, you said you agreed with her assessment so I was just wondering if your opinion would be changed by whether or not the word had been previously taught to the kids. I wasn’t trying to call you out. I guess I just don’t see why ‘a plane goes in a hangar’ is an unacceptable thing to expect kids to know but ‘Argentina is in South America’ isn’t. Surely both of those things could have been mentioned in school, as vocabulary and geography respectively. A kid who has been to Argentina is going to be at an advantage on the geography question just like a kid who has been on a private plane will be at an advantage on the other. Now, if either of those things have never been mentioned at school, then I can get behind the idea of bias, at least a little bit. ---
**Author:** ![robert\_columbia](https://avatars.discourse-cdn.com/v4/letter/r/e79b87/32.png) [@robert\_columbia](https://boards.straightdope.com/u/robert_columbia)\ **Post date:** [April 18, 2012, 2:20am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/42 "2012-04-18T02:20:52Z")
> [@Irishman](#): > > … > Wait, children who aren’t from affluent homes don’t know that a pumpkin has seeds? That celery is a stalk? > … I am doubting the entire idea that children from affluent homes necessarily gain more knowledge from home life. Consider all the rich families who have their BMW’s and Lexuses professionally serviced at the dealership and where the kids never have to deal with the fact that they can’t go to the park because the (spark plugs are fouled, alternator failed, radiator is leaking, etc), while the kids from lower class families get to watch Dad drink beer and show the kids how to adjust a carburetor on their 1982 Honda Civic and manually check each spark plug. They are going to have a greater understanding and possible appreciation of auto maintenance. Also, considering the different supposed types of intelligences, can anyone give an example of a question that can be graded _objectively_ that measures interpersonal or musical intelligence? What is musical intelligence anyway? Is it just book learning related to music such as being able to identify and classify musical instruments according to the standard classifications (e.g. “that’s an oboe and an oboe is a woodwind instrument”), being able to identify specific keys or chords, and being able to memorize specific fingerings associated with specific notes or chords, or does it represent more of an inner intimacy with music and an ability to not only play notes _right_, but to play with spirit and impress audiences? I’m not sure “Play your choice of relaxing music. Your grade is based on how calm your teacher believes they feel afterward.” is objective. ---
**Author:** ![california\_jobcase](https://avatars.discourse-cdn.com/v4/letter/c/9d8465/32.png) [@california\_jobcase](https://boards.straightdope.com/u/california_jobcase)\ **Post date:** [April 18, 2012, 3:07am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/43 "2012-04-18T03:07:23Z")
My dad had a stroke. He couldn’t speak. A therapist asked him to look at a page and answer a question. On the page was a picture each of a lion, chicken, dog, and horse. The question was “Three of these animals have something in common the other doesn’t have. Which is the other animal?” Anyone care to guess the therapist’s “correct” answer? ---
**Author:** ![robert\_columbia](https://avatars.discourse-cdn.com/v4/letter/r/e79b87/32.png) [@robert\_columbia](https://boards.straightdope.com/u/robert_columbia)\ **Post date:** [April 18, 2012, 3:16am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/44 "2012-04-18T03:16:55Z")
> [@california\_jobcase](#): > > My dad had a stroke. He couldn’t speak. A therapist asked him to look at a page and answer a question. On the page was a picture each of a lion, chicken, dog, and horse. The question was “Three of these animals have something in common the other doesn’t have. Which is the other animal?” > > Anyone care to guess the therapist’s “correct” answer? Lions, dogs, and horses are mammals (and have hair/fur and possibly other things that only mammals have), but a chicken is a bird? ---
**Author:** ![california\_jobcase](https://avatars.discourse-cdn.com/v4/letter/c/9d8465/32.png) [@california\_jobcase](https://boards.straightdope.com/u/california_jobcase)\ **Post date:** [April 18, 2012, 3:18am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/45 "2012-04-18T03:18:49Z")
Nope. That’s what my dad answered by pointing, and what I would have answered with my biology degree and all, but the _correct_ answer is the lion- it doesn’t live on a farm. ---
**Author:** ![Tarwater](https://avatars.discourse-cdn.com/v4/letter/t/9de0a6/32.png) [@Tarwater](https://boards.straightdope.com/u/Tarwater)\ **Post date:** [April 18, 2012, 3:20am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/46 "2012-04-18T03:20:19Z")
I would’ve said the chicken was the only animal with two legs. ---
**Author:** ![ToeJam](https://avatars.discourse-cdn.com/v4/letter/t/dfb087/32.png) [@ToeJam](https://boards.straightdope.com/u/ToeJam)\ **Post date:** [April 18, 2012, 3:21am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/47 "2012-04-18T03:21:57Z")
> [@california\_jobcase](#): > > Nope. That’s what my dad answered by pointing, and what I would have answered with my biology degree and all, but the _correct_ answer is the lion- it doesn’t live on a farm. I woulda said the Chicken- it has 2 legs, the rest have 4. ---
**Author:** ![robert\_columbia](https://avatars.discourse-cdn.com/v4/letter/r/e79b87/32.png) [@robert\_columbia](https://boards.straightdope.com/u/robert_columbia)\ **Post date:** [April 18, 2012, 3:29am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/48 "2012-04-18T03:29:42Z")
> [@california\_jobcase](#): > > Nope. That’s what my dad answered by pointing, and what I would have answered with my biology degree and all, but the _correct_ answer is the lion- it doesn’t live on a farm. In my opinion, that is not an example of a biased question, it is an open-ended discussion/thought experiment question or a meaningless question. You could say that the answer is “horse” because it is the only one that people ride. ---
**Author:** ![TonySinclair](https://avatars.discourse-cdn.com/v4/letter/t/49beb7/32.png) [@TonySinclair](https://boards.straightdope.com/u/TonySinclair)\ **Post date:** [April 18, 2012, 3:30am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/49 "2012-04-18T03:30:40Z")
Not exactly what you’re looking for, but the lack of test prep screwed me up when taking the SAT, many years ago. I was an excellent math student, and could solve polynomial equations at a glance. But for whatever reason, all of my math teachers and math books had always said, “Find the solutions of this equation” or “Find the zeroes of this polynomial,” etc. They never said, “Find the roots,” unless it was a day I was sick or something. But that’s what about a dozen of the questions on the SAT’s math section said, and I had no idea what to do with them. To me, a root was a square root or cube root, and that got me nowhere, so I scored about 200 points lower than I should have, based on every other math test I’ve ever taken. And where I lived, you only got one shot a year, and it never occurred to me to take it over. ---
**Author:** ![kittenblue](https://avatars.discourse-cdn.com/v4/letter/k/ba8739/32.png) [@kittenblue](https://boards.straightdope.com/u/kittenblue)\ **Post date:** [April 18, 2012, 3:34am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/50 "2012-04-18T03:34:13Z")
> [@dataguy](#): > > I had a friend who was an elementary school teacher. > I remember once she showed me examples of questions from a standardized test that she thought showed class bias. > > The one I remember was something about where do you keep an airplane. The correct answer was in a hangar. I agree with her assessment. Unless you are from an environment where private planes are common, I doubt this is something a typical 3rd grader would know. I never owned a private plane, but I would have known this answer when I was in 3rd grade because my dad worked for United Airlines and took us to work sometimes to see the planes, and talked at the dinner table each night about things that happened in the hangar. And because of the whole “open your mouth here comes the airplane” technique of feeding babies. And because of a popular children’s book of the day about an airplane that mentioned hangars. So there is more than one way to hear about hangars than being rich enough to own a plane. ---
**Author:** ![Boyo\_Jim](https://avatars.discourse-cdn.com/v4/letter/b/87869e/32.png) [@Boyo\_Jim](https://boards.straightdope.com/u/Boyo_Jim)\ **Post date:** [April 18, 2012, 4:10am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/51 "2012-04-18T04:10:59Z")
An example of a biased question: “Do you still beat your wife?” ---
**Author:** ![Sleel](https://avatars.discourse-cdn.com/v4/letter/s/73ab20/32.png) [@Sleel](https://boards.straightdope.com/u/Sleel)\ **Post date:** [April 18, 2012, 4:33am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/52 "2012-04-18T04:33:28Z")
> [@Zsofia](#): > > Okay, I looked up the famous “regatta” question from the SAT - it’s an analogy: > > runner : marathon :: > A) envoy: embassy > > B) martyr: massacre > > C) oarsman: regatta > > D) referee: tournament > > E) horse: stable > > I think that’s a pretty clear example of a biased question - all the other choices are common words for high school students. Regatta is on the same level of common words for high school students… who go to Exeter. I disagree. The pattern is — participant : event. Even if you have no idea what a regatta is you’d be able to guess that A, B, and E don’t fit at all. The only questionable one would be D, but it also doesn’t fit the pattern because a referee is an official, not a participant. Elimination would give you the correct answer even if you had no idea at all what a regatta was. Bias or no bias, it’s a question you could reasonably expect a person with a high-school level vocabulary to answer correctly. ---
**Author:** ![Amblydoper](https://avatars.discourse-cdn.com/v4/letter/a/fbc32d/32.png) [@Amblydoper](https://boards.straightdope.com/u/Amblydoper)\ **Post date:** [April 18, 2012, 4:52am UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/53 "2012-04-18T04:52:52Z")
Seems like a simple solution exists: If they whole point of standardized testing is to measure the success of the school in teaching the subject matter, then the test questions should cover the subject matter. Teach the kids about fruit and vegetables in Science class, Health class and Home Economics… Teach them about countries in South America in Social Studies and Geography. Cover occupations and tools of those occupations in Social Studies and Home Ec. All the Math questions should probably be covered in Math class. Make sure that “conserve” and “resource” are both taught in the vocabulary part of English class, and later they use those words in Science class and Social Studies. Give me a fucking break. Somewhere, there is a list of the 10,000 things that need to be taught in school, and it should be someone’s job to make sure the kids are learning those things. If the pumpkin vs. celery bullshit is really proving to be a problem, then the standardized test DID WHAT IT WAS SUPPOSED TO DO in discovering that _fruit and vegetable identification_ needs to be added to that list! ---
**Author:** ![Irishman](https://avatars.discourse-cdn.com/v4/letter/i/b487fb/32.png) [@Irishman](https://boards.straightdope.com/u/Irishman)\ **Post date:** [April 18, 2012, 4:22pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/54 "2012-04-18T16:22:13Z")
> [@doreen](#): > > I can easily imagine a child here or there who doesn’t know what a pumpkin is, but I can’t imagine that huge numbers of kids of any socioeconomic status have never seen the bags of pumpkin seeds sold right next to the sunflower seeds on the potato chip racks. Uh, I didn’t know pumkin seeds were edible until I was an adult, and I don’t think I’ve ever noticed them in a grocery store right next to the sunflower seeds. Of course, I’m not typically paying attention for sunflower seeds, either. (Of course, I should have known it, since pumpkins are squash, and squash you typically eat the seeds. But that takes three steps of logic, and – squirrel!) > [@](#): > > Of course, these questions should not be used to evaluate the quality of instruction - but that’s because they are usually unrelated to the instruction , not because they are biased. They \*\*might \*\*be appropriate questions in a cooking or shop class They’re being used in the phases of test looking at vocabulary and at reasoning. Is reasoning something that schools can teach, or is it more of an innate trait? > [@robert\_columbia](#): > > Also, considering the different supposed types of intelligences, can anyone give an example of a question that can be graded _objectively_ that measures interpersonal or musical intelligence? No. Certainly not something that can go on a standardized, scantron test. > [@](#): > > What is musical intelligence anyway? … or does it represent more of an inner intimacy with music and an ability to not only play notes _right_, but to play with spirit and impress audiences? I’m not sure “Play your choice of relaxing music. Your grade is based on how calm your teacher believes they feel afterward.” is objective. “Musical intelligence” would be far more about the latter. The ability to understand music on a personal level, an ability to feel it and play it and compose it. An ability to convey emotion to others through music. Having a good sense of rhythm, being able to hold a steady beat, being able to sing and match pitches, play/sing by ear. It might be possible to create some sort of standardized musical intelligence test that involves testing rhythm, beat, singing to match pitches, etc. But that test is also going to combine training with ability. Somebody musically inclined is likely to get exposed to it, discover that talent, and explore it more than someone who isn’t. Speaking for typical American experience, where every child has some exposure to music via school, or parents hobby shopping with their kids. (Who didn’t have either a dance class, piano lessons, or equivalent exposure to some level of musical training at some point?) With that kind of test, my sister and father would fail miserably. My mother and I would do middling well. But that kind of test won’t be scoreable on a scantron a-b-c-d kind of test. It would have to be evaluated by someone directly. I’m not sure how that kind of thing could be objectively scored. And it’s not something that could be done in a classroom/lecture hall evironment, everyone testing at once. It requires individual testing. > [@california\_jobcase](#): > > My dad had a stroke. He couldn’t speak. A therapist asked him to look at a page and answer a question. On the page was a picture each of a lion, chicken, dog, and horse. The question was “Three of these animals have something in common the other doesn’t have. Which is the other animal?” Already explained - I was going to point out both possible answers. That isn’t really a biased question so much as an ambiguous question. There are multiple “right” answers based upon interpretation and rationalization. That is still a lousy question, because of the ambiguity. Unless the point of the question isn’t so much which animal you pick, but your reasoning method. But I take it that wasn’t the point. The “right” answer could be any of them. The “other” animal could be dog, because it’s the only common house pet. It could be “chicken”, because it’s the only one sold in fast food restaurants. It could be “lion”, because it’s the only one from Africa. It could be “horse”, because it’s the only one you ride. ---
**Author:** ![njtt](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@njtt](https://boards.straightdope.com/u/njtt)\ **Post date:** [April 18, 2012, 5:50pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/55 "2012-04-18T17:50:31Z")
On the other hand, a kid who has grown up in abject poverty with a minimum of education can win the Indian version of “Who Wants to be A Millionaire?” provided exactly the right set of unlikely coincidences have happened to him over the course of his life. The specific things that individuals know or do not know are subject to all sorts of uncontrollable contingencies, and it is therefore impossible to construct truly unbiased tests. What about the math test that some math-smart child fails because he had an awful stomach ache on the day some crucial concept was covered? The take-home message should not be that all conceivable bias needs to be removed from tests (which is not possible); it should be that tests are very blunt, unreliable instruments to use in assessing someones actual potentialities, and we should not place too much reliance upon them. ---
**Author:** ![grude](https://avatars.discourse-cdn.com/v4/letter/g/e47774/32.png) [@grude](https://boards.straightdope.com/u/grude)\ **Post date:** [April 18, 2012, 5:56pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/56 "2012-04-18T17:56:17Z")
> [@california\_jobcase](#): > > My dad had a stroke. He couldn’t speak. A therapist asked him to look at a page and answer a question. On the page was a picture each of a lion, chicken, dog, and horse. The question was “Three of these animals have something in common the other doesn’t have. Which is the other animal?” > > Anyone care to guess the therapist’s “correct” answer? I would have chosen the lion, because it is the only non-domesticated animal on the list. ---
**Author:** ![delphica](https://avatars.discourse-cdn.com/v4/letter/d/e9a140/32.png) [@delphica](https://boards.straightdope.com/u/delphica)\ **Post date:** [April 18, 2012, 6:40pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/57 "2012-04-18T18:40:13Z")
> [@Irishman](#): > > The “right” answer could be any of them. The “other” animal could be dog, because it’s the only common house pet. It could be “chicken”, because it’s the only one sold in fast food restaurants. It could be “lion”, because it’s the only one from Africa. It could be “horse”, because it’s the only one you ride. Oh, good one! I was trying to think of a “right” answer for each combination of animals, and was having a hard time with the horse. The best I came up with was that a horse is the only one that doesn’t have articulated toes, but yours is much better. ---
**Author:** ![Chessic\_Sense](https://avatars.discourse-cdn.com/v4/letter/c/7c8e57/32.png) [@Chessic\_Sense](https://boards.straightdope.com/u/Chessic_Sense)\ **Post date:** [April 18, 2012, 6:41pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/58 "2012-04-18T18:41:46Z")
The thing about the regatta question is that it’s given to high schoolers. A lot of SAT questions are designed to trick you into picking an unfamiliar word because you can’t rule it out. I remember a question where the correct choice was “drunk” but the trick answer was “inebriated”. The question called for a noun, not an adjective, but test takers picked “inebriated” because they were unfamiliar with it and they were thinking of drunk-the-adjective. Or something like that. Anyway, the point is, using obscure words is fair game for the SAT. For a third grade math test, not so much. ---
**Author:** ![Ludovic](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Ludovic](https://boards.straightdope.com/u/Ludovic)\ **Post date:** [April 18, 2012, 6:43pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/59 "2012-04-18T18:43:20Z")
> [@california\_jobcase](#): > > “Three of these animals have something in common the other doesn’t have. Which is the other animal?” The doggry? ---
**Author:** ![iamthewalrus\_3](https://avatars.discourse-cdn.com/v4/letter/i/258eb7/32.png) [@iamthewalrus\_3](https://boards.straightdope.com/u/iamthewalrus_3)\ **Post date:** [April 18, 2012, 8:40pm UTC](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022/60 "2012-04-18T20:40:21Z")
> [@Sleel](#): > > I disagree. The pattern is — participant : event. Even if you have no idea what a regatta is you’d be able to guess that A, B, and E don’t fit at all. The only questionable one would be D, but it also doesn’t fit the pattern because a referee is an official, not a participant. Elimination would give you the correct answer even if you had no idea at all what a regatta was. Bias or no bias, it’s a question you could reasonably expect a person with a high-school level vocabulary to answer correctly. Your analysis doesn’t show that the question isn’t biased. Yes, if you know all of the words except regatta, you can still get the right answer. But, what if you also don’t know what an envoy is? Now you maybe can’t arrive at the correct answer, and the rich kid who doesn’t know what an envoy is, but goes to the yacht club every week has a clear advantage. Ergo, socioeconomic bias in the question. [Previous page](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022.md?page=2) [Next page](https://boards.straightdope.com/t/real-life-examples-of-biased-test-questions/619022.md?page=4)