I often hear people complaining that standardized tests are biased in some way that minorities have a tough time with them. I really don’t get this. I’m young enough to remember standardized tests I took in school. There would be sections on science, math, spelling, and reading comprehension. I don’t recall EVER reading anything on the tests that a black kid couldn’t understand just because he was black. Or a Mexican kid. Or whatever.
Can somebody give me an example of a culturally biased question?
I also remember taking standardized tests in school that had questions about football scoring. For example, “if a team makes two touchdowns and one field goal, how many points do they have?” I actually had this question on a basic skills test in junior high. They did not give the values of each, they just assumed we would know. Well, I always assumed that a touchdown scored 7 points because that’s how many the teams usually had when they had only “scored” once. I didn’t know about extra points since I don’t play or watch football. So I do know from experience that the tests can be sexually (or at least sports) biased. I would assume that they could be racially biased as well.
But mostly, claiming bias in standardized testing is a crock. There are very few questions that you wouldn’t know because of the place you lived…gimme a break. If inner-city kids score lower than suburban kids, its due to the quality of the schools, not the “biased questions”. Questions on standardized tests are rarely of the “trivia” type that people above mentioned, so I guess they haven’t taken many standardized tests.
I think it’s a little deeper than just whether or not they can do the math. Asking a question about sports puts an athletic kid at ease. Asking a question about baking gives the aspiring cook more confidence. Etc.
People are better at thinking in terms of familiar things than in the abstract. So… to a kid compltely unfamiliar with football scoring (which would have been me at that age), this question:
“A touchdown is worth 6 points. If a team scores a touchdown, it may choose to either attempt an extra kick for one point or attempt a conversion (worth 2 points). What are the possible scores if a team has scored 4 touchdowns and taken at least one kick?”
may as well be:
“A twasif lends you a tysif (worth 6 agani). You can trade it to Twooghan for either a pyrisco (usually worth 7 aganus, sometimes worth 6) or a hastick (worth 8 agani except when it’s worth 6). Assuming twasif lends you four tysives and you go for at least one pyrisco, how much agani can you make?”
Granted, anyone should be able to figure out either problem, but I bet most kids will get the top one first and struggle a little longer over the second one.
I think this is what we’re talking about. It’s not a question of calculations, it’s a question of abstraction, time-required, etc. I’m not sure what the solution is, but it probably involves making all questions somewhat abstract:
“An unladen swallow (carrying no coconuts) can travel approximately 8 kilometers in an hour. When carrying a coconut, it travels seven, and when carrying 2 coconuts, it travels six. (Swallows cannot carry more than 2 coconuts and cannot carry anything but coconuts.) Four swallows fly for one hour. What are the total distances they may have flown?”
Instead of favoring anyone (except perhaps the Monty Python fans), this question will lead every kid equally to stab a pencil through his/her eye.
So far, no one has come up with a real-life “culturally biased” question that has actually appeared on a standardized test.
As for the one that evilbeth mentioned, I wonder who compiled that test. That does go more in the direction of trivia than most standardized tests I’ve taken.
Just thought I’d jump in here with some info for those of you interested. As you may have guessed there is an organization dedicated to this question. Their web site is www.fairtest.org. The following is from the site:
SAT Bias:
The Gender Gap: The SAT consistently underpredicts the performance of females in college and overpredicts the performance of males. Although females earn higher grades in high school and college, their SAT scores were 40 points lower in 1997 (36 on the SAT-Math and 4 on SAT-Verbal). College Board research has shown that both the Verbal and Math portions of the test underpredict girls’ college performance. A 1994 ETS study found that, on average, males scored 33 points higher on the SAT-Math than females who earn the same grades in the same college math courses. Analyses of SAT gender bias cite several causes including the test’s emphasis on speed over sustained reasoning and its multiple-choice format. Mathematics tests in other countries that require solutions to long problems appeared unbiased with respect to gender.
Bilingual Students: The speeded nature of the SAT imposes an unfair burden on students for whom English is not the first language. Research suggests that the SAT does not predict Hispanic students’ first-year college grades as accurately as it does white students’ grades. One study found that even for bilingual students whose best language was English the SAT underpredicted college performance.
Impact of SAT Use on Minorities African American, Latino, new Asian immigrant and many other minority test-takers score significantly lower than white students. Rigid use of SATs for admissions will produce freshman classes with very few minorities and with no appreciable gain in academic quality. The SAT is very effective at eliminating academically promising minority (and low-income) students who apply with strong academic records but relatively low SAT scores. Colleges that have made the SAT optional report that their applicant pools are more diverse and that there has been no drop off in academic quality.
Stereotype Vulnerability: Several studies show that female and minority students who are aware of racial and gender stereotypes score lower on tests such as the SAT which purport to measure academic aptitude. One study defined this extra burden borne by some test-takers as “stereotype vulnerability,” and warned that these findings “underscore the danger of relying too heavily on standardized test results in college admissions or otherwise.”
From what I can see these folks don’t like standardized testing in general. I’m not sure I agree with their hard line approach, but they make some good points about how much income is a factor in test scores. I also haven’t been able to come up with a real-life example of a biased test question. Sorry.
Well, I can’t speak for all these groups, but my experience with Asians is that they blow past us white folks on SATs and GREs. One of my undergrad roommates was from China, and I used to hang out with him and his Chinese friends a lot.
I thought my 1200 SAT was something to be proud of. To a person, they just laughed. They couldn’t conceive of anything lower than 1400 as being acceptable. My girlfriend is Korean, and she reports similar attitudes among Asian students.
Is the American culture that blacks and women face in this country comparable to the very real East-West differences faced by Asian students taking U.S. tests? I mean, Asians take these tests having learned English as a second (or third) language, have very little practical knowledge of typical Western Culture, and yet they still tend to outperform Americans academically.
I think this says a lot more about the state of American education and diligence of students than the red herring of “culturally biased” tests.
[quote]
I think this says a lot more about the state of American education and diligence of students than the red herring of “culturally biased” tests. [\quote]
I couldn’t agree with you more. Any child who wants to do well, and works hard will succeed. As a woman, I suppose I could blame my poor math grades on gender bias, but I’d rather take personal responsiblity for my failure and admit that I hated math, and neglected to study it as hard as I could have.
By saying that a certain ethnic/racial/gender group can’t do well on a standardised test, you’re insulting the entire group’s intelligence and infantalizing that group on a whole. In a way, you’re <I> accepting </I> the <I> Bell Curve </I> assuption that certain groups are just inferior genetically and CAN’T succeed.
Might I pose a response to the supposed sexual discrimination in the SAT?
From my experience in university, I have seen most females to be harder workers than males, to a significant extent. I don’t think females are AS EASILY caught up in the debauchery and laziness of campus life, and frankly spend less time chasing the opposite sex than to males.
This is of course a generalization, but so are statistical averages. Standardized tests have little to with work ethics, or even acquired scholastic knowledge.
An example: Take the College AP Calculus AB exam. I got the lowest mark in my high school calc class, but still got 5/5 on the exam, having walked in an hour late. This is not an indication of how good a college math student I will be. In fact, my 2nd year math marks are so low, that to get into a particular honours analysis course next September, I had to email the prof, and was only admitted to the class based on math competition scores.
There are a couple of problems with this. First of all, poor inner-city kids who rely on public transportation are less likely to be familar with a “garage” than kids in the suburbs who have one attached to their house. Second, middle-class kids (read “white”)are more likely to have parents or friends who own boats than lower class (read “minority”) kids. The middle-class kids are more likely, then, to have been exposed to words like “marina” than the poorer ones.
Think of how a question like this might read to a young suburban child:
An 8-year old child who lives in a rural area might never have been exposed to the concept of a turnstile, while every 8-year old in New York City would know what one is. Hell, there are probably kids graduating from suburban or rural high schools that don’t know what a turnstile is.
So long as you accept the proposition that inner-city kids are more likely to be minority and surburban kids are more likely to be white-- not a particularly controversial assumption-- it becomes pretty clear that questions like these can favor one type of child over another, even though they don’t appear on the surface to have any particular slant.
Those who do not believe that tests can be culturally biased are mistaken. Studies have shown differences in performace based on, for example, the types of objects used in a math work question. To use a simplified example, girls would do better (accuracy & time) on “two dolls plus two dolls equals how many dolls?” than on “two tanks plus two tanks equals how many tanks?” Yes, of course everyone would get the above question… but for more difficult questions variations such as the above will make a difference, on average.
This is very simple. Americans can imagine a question on the SAT having to deal with rugby scores would take more time than the same question with football scores (assuming all pertinent information is made available)… it simply is a bit more difficult when dealing with unfamiliar subjects. You may have to read the question twice for rugby, where you could comprehend the football question in one read.
I’ll be the first to agree that the education system in this country is the pits. But this still doesn’t explain why low income students do poorly in comparison to higher income students from the same culture.
From Lissa:
I’m not sure what you mean by this. Are you saying that the students from poor families are lazier? Or possibly there is no correlation between test scores and family income? Please clarify.
Regarding the argument that inner-city kids haven’t seen snow, or rural kids a turnstile:
A test that examines the student’s knowledge of his or her own experience is pretty worthless. What would you learn from testing a bunch of New York schoolchildren on their knowledge of snow? Ask some kids in Houston or somewhere (OK, it’s probably snowed there, but you know what I mean) and you’ll find out which ones pay attention to things outside the sphere of their daily lives; in other words, which ones are open to imagination and new ideas.
One example of inner-city bias that’s been offered in the past is a question about polo. Well, I’m not from the inner-city; I’m not from the city at all. Still, I’ve never seen polo played, in person or on TV. Nor has anyone I know. But I know what it is, because I read. And public libraries are free. If a poor, inner-city kid could tell me what polo is, I would want to see that child get every opportunity to further her education, because she’ll make the most of it.
[quote]
To use a simplified example, girls would do better (accuracy & time) on “two dolls plus two dolls equals how many dolls?” than on “two tanks plus two tanks equals how many tanks?” Yes, of course everyone would get the above question… but for more difficult questions variations such as the above will make a difference, on average. [\quote]
I don’t agree. It’s the numbers that are the important part of the question, not the objects.
You know, television is a great equalizer. It exposes kids to things that aren’t necessarily part of their daily lives. Reading will also expose children to unfamiliar words. They also print books with lists of words that are on the standardized tests. Dictionaries can explain anything that is unfamiliar. I just don’t buy it. These kids don’t live in caves, for crissakes!
I didn’t say that at all. I try to avoid lumping people into categories like that. You need to take each child on an individual basis.
Each child has a choice: work hard, or don’t. It has little to do with your financial background. School is free. The library offers books for free, and my local library even offers free tutoring programs. You just don’t lower the bar for poor kids, tacitly saying “Well, don’t expect much from them. They’re poor, and being poor means you’re automatically incapable of doing as well as the wealthier children.” That’s a big steaming pile, if you ask me. I knew a lot of kids who were one step above living in a box who did extremely well in school. Why? Because they worked hard and studied. I repeat, if a child tries, he/she will succeed.
The numbers are indeed the important part, but don’t discount the time it takes to grasp the question, not to mention the power of visualization.
A very smart person will get all of the questions right no matter what. A very stupid person will get them all wrong. It’s the in-betweeners we’re worried about. If a student is not going to be able to finish a test in the time allotted, then it makes a big difference how quickly s/he can comprehend the questions.
Lissa: re work hard or don’t: it has been shown conclusively that students who took SAT (etc) test prep courses score WAY better. These courses, until recently, have only been available privately, at some cost. Poor kids can’t afford these, so weathy kids score better. Hard work is good, but not always everything.
Actually, Daniel, I’ve seen studies that say prep courses are useless as tits on a boar. I’ve seen studies that laud them tremendously, but some of those were funded by prep course companies.
Do you have any examples of studies that we can review to back up your claim?
On the main point, it should be noted that the real trouble with standardized testing is whether it makes a good proxy for estimating past success at learning or probable future success at learning. The SAT’s, ACT’s, GRE’s, LSAT’s, etc. have value only if they can accurately predict who will do well in the future. When you get a group scoring lower on the test in general, but managing to do as well as or better than other groups in later schooling, the test loses value. On the other hand, when you are testing achievement, such as with proficiency exams, the question is whether lower scores by a group truly reflect a lack of knowledge regarding the subject matter tested. If a group scores lower, but can otherwise demonstrate understanding of the subject matter, then the test loses value in advising whether a particular person has achieved to that point what is desired.
It is extremely difficult to reach significant conclusion about ‘cultural bias’ in standardized tests, because so many factors influence test results. Do the low scores reflect the effect of the way the questions are asked or the lack of resources used to teach the examinees? Do differences between boys and girls reflect gender differences in processing information or do they reflect the effect of the test methodology? Unless you have some really solid grounding in psychology and test scoring, as well as in statistics, I think it is unlikely that you will fully comprehend or be able to discern yourself what exactly is at play.
First of all, thank you Daniel. You answered Lissa better than I would have. I would add this to her - No one is talking about lowering the bar or making tests easier. We are talking about making the tests more fair. If we can make the study aids that students have traditionally had to pay for available to everyone, then I see no problem in that.
DSYoungEsq - If you want an example of a study showing the correlation between income and test scores on the ACT (a test similar to the SAT), then you can find one on the site I linked to earlier. Again, that’s www.fairtest.org .
For just these reasons, the public education system is starting to do test prep classes.
Oddly enough (& let us all rememder what the thread here is), that is one of the arguements AGAINST George W’s bragged-about increase in test scores- they are spending too much time “teaching to the test”!
