IMNSHO, the disparate scoring on SAT/ACT tests by “race” can be laid more at the door of education, not culture. It is a sad state of affairs, but schools that are predominately non-white do have a history of second-rate educational staff and facilities.
This is particularly relevant as the SAT must be completed in a fixed amount of time. “Dance company” mentioned before would be a good example. If someone was unfamiliar with a dance company being a group of artists, they might first think it meant a corporation that financed dances. If they have to stop and consider what dance company means, as opposed to those who are used to seeing dances before, this will slow them down compared to the latter group.
It’s a very poor example of what you’re discussing–“a group of geriatric billionaires in wheelchairs around a boardroom discussing financing of a dance” will STILL make sense if, and only if, they reject innovation because they prefer time-honored dances (or stock offerings, or fruit basket arrangements).
The reasoning that says we should eliminate any words that might conceivably slow down a reader who is unfamiliar with them also dictates that we should then have questions with references to…nothing? Because it’s possible that some reader, somewhere, might be unfamiliar with a horsie? A pineapple? A sidewalk? “It’s outrageous to have the analogy of pedestrian:sidewalk:
street because someone might have been raised in a rural community without sidewalks.”
To say nothing of the fact that the word “pedestrian” might be one that someone who never read a book, or spoke to anyone else, might be unfamiliar with. Any test that uses a diction level any broader than the 400 words of Basic English is potentially biased against the unread, and uneducated. Isn’t that part of what we’re testing for here?
Personally, I think the noun choice and general wording of both math problems and reading comprehension passages really makes a difference. When reading and analyzing a question, we build a model in our heads. The more familiar the context, the easier it is to remember details. This translates to increased speed and accuracy.
It’s like saying:
Kzarsky and Kzarsie went to the mart. Kzarsky got a 3-rupee gibbon. Kzarsie got a 1-rupee giblet. Kzarsie forgot his changepurse, so Kzarsky paid for both and Kzarsie paid him back once and again the next day. If both Kzarsie and Kzarsky started with 5 rupees, how many does Kzarsky have now?
vs.
John and Joe went to the store. John got a $3 pie. Joe got a $1 peach. Joe forgot his wallet, so John paid for both and Joe paid him back double the next day. If both John and Joe started with $5, how much does John have now?
A simple problem, really, but the second wording is probably easier since you have less trouble remembering the name-to-dollar mapping and don’t need to cross check as many times.
Still, you probably would have gotten both right. However, imagine if that had been a 5-paragraph reading comprehension passage and you were supposed to remember what Kzarsie had done vs. what Kzarsky had done at various points in the story!
So you’re saying that, because it’s impossible to completely eliminate all cultural bias, we shouldn’t try to remove any of it?
That would be fine, if we were living in Russia or wherever it is that Kzarsi and Kzarsky live.
I agree. The exact function of a dance company is simply irrelevant to ascertaining the correct answer. The question could be:
“The Regulan Bloodworm rejects_________, preferring to present_____
dances in a manner that underscores their traditional appeal.”
and the answer would still be obvious. I just don’t see “dance company” slowing anyone down. If they don’t know the rest of the vocabulary, sure, but not because of “dance company”.
I don’t necessarily disagree with the notion that the SAT might be biased; I just think that’s a weak example.
See, now THAT would be a good example of a bad SAT question. Unfortunately, it’s not a real one. Correct me if I’m wrong, but I don’t think anyone here has said that tests CANNOT be biased; only that the examples given thus far aren’t very convincing. I think the OP wants EXAMPLES of how the SAT is biased, not hypotheticals.
How about just presenting everything in abstractions, along the lines of the TOPS IQ test. A sample question might be:
Given:
C, therefore B
X implies Y
A implies B
M is the converse of Y
H is the negative consequent of A
Not Y
Which of the following statements, if any, are logically valid?
-
Not C And (A Or X)
-
C Or Not M Implies Not A
-
C And Not H
-
Not X
-
B implies Not A
I don’t believe a few test questions with challenging phrases like “dance company” scew the test. Even leaving aside the concept that a well-read minority student from a cultural backwash should still be able to recognize that phrase, the ability to solve a problem with "Kzarsie"s and “giblets” (or any abstract terms) should weed out those students unable to think abstractly.
It sure would have weeded me out, as I recall “solving” tricky problems by visualizing the situation and then taking a reasonable guess. But I take my hat off to students who can really solve them by rigorous logical thinking alone. Aren’t such students smarter than me, if only in this limited way?
If the test doesn’t consist wholly of such stuff, but only here and there, then it’s performing a useful service, distinguishing the students capable of abstract thought.
I gave a standardized test a few weeks ago. I can’t remember the exact wording, but one question went something like:
Rewrite the sentence, correcting the errors:
“Many people treat they’re pets like members of the family. My aunts dog sleeps in his own brass bed.”
Students were supposed to find and correct the apostrophe errors. An enormous number of my low-income students also corrected the last sentence to read ‘…sleeps in her own brass bed,’ because the concept of buying a bed for the dog was so bizarre to them that it had to be a pronoun error.
It’s my understanding (I read this here 3 or 4 years ago) that SAT doesn’t bother to try and figure out WHY a question is biased, but if it is, they drop it. The number of kids taking the SATs and ACTs is high enough that you can do a sophisticated statistical analysis. For example, if 95% of high-scoring white kids got field test question #4 correct, but only 75% of the hispanic kids *who had the same general performance level * got the question correct, they dump the question, even if they can’t figure out why it’s biased. Now, I don’t know what the actual numbers are–at what point they decide a discrepency is evidence of bias–but I don’t have any problem with the theory.
It’s unfortunate that these tests are charged with this kind of bias, because my experience leads me to believe the bias isn’t usually racial / cultural, as the Chitlin Test implies. It’s socioeconomic / cultural. The standardized tests are skewed to teach middle class cultural norms. When tests include questions flirt even beyond THAT sphere (the oarsman::regatta example) that’s when students of all non-privileged backgrounds have trouble.
Also the problem isn’t confined to the college level SATs, but to the whole slew of mandated district, state and federal standardized tests, and private brand names like the ACT and SAT administered to students from kindergarten to high school. That’s potentially dozens of kinds of tests, by as many different testing companies and the validity and quality of these tests all vary.
The big problem here is prior knowledge not being valued as an effective bridge to classroom learning, inadequate teaching and classroom management, students unprepared to take tests and not developing strong test-taking skills: pre-reading questions, quick, accurate scanning of text and reading directions being the key three. If FisherQueen’s students read directions they wouldn’t have been distracted by a pronoun question, they’d have stuck to apostrophe errors as directed.
Well, it is interesting the assumptions made by a group of middle-income testmakers* which would puzzle a lower-income testtaker, but of course that is hardly cultural bias (nor is your example illustrating cultural bias, of course, as I would assume you wouldn’t count pronoun changes at all in a test of apostophe-usage recognition.)
Which is to say, I’m very sure that an occasional unforeseen problem slips through to the test as given now and then, and is quickly corrected. That’s worlds different from “cultural bias is an inherent and undetected component of standardized testing.”
*the Valerie Perrine role in SUPERMAN: Miss Testmaker
A friend of mine wrote this regarding physics education: http://www.bridgew.edu/SoAS/jiws/May04_Special/Gender.pdf
What Lib said. Or, if you think that variables would be too confusing, we could go the route of the DLAB linguistics aptitude test and simply make up words to represent indefinite objects:
Przbl selects an umblat that costs 3 zmb. Ngvati selects a brlva that costs 1 zmb. Przbl pays for both purchases, and the next day, Ngvati pays Przbal double the cost of Ngvati’s brlva. If both Przbl and Ngvati started with 5 zmb, how many does Przbal have now?
Nobody of any cultural background has heard the names Przbl or Ngvati, or knows what an umblat, a brlva or a zmb is. Thus, everyone is on equal footing upon approaching the question.
The grlnbalt rejects _________, preferring to present _____ mzprvts in a manner that underscores their traditional appeal.
Again, nobody of any cultural background knows what a glrnbalt or mzprvt is. In context, though, it’s clear that both these words are functioning as nouns, and again, only “innovation” and “time-honored” make any sense on the grounds that the contents of the first blank are rejected due to the contents of the second blank possessing “traditional appeal”.
In both of these examples, the temporary confusion at the foreign nature of the nouns will be universal, so there can be no cultural bias.
Wouldn’t this give an advantage to illegal alians.
Please provide evidence backing up your claim.
Having actually taught in a few predominantly non-white schools, I can assure you that the staff and facilities are just as good as the staff and facilities at the predominantly white schools where I have taught. The difference between the schools (in my experience)* is in the dedication levels of the individual students.
That being said, vocabulary is vocabulary. My question about the SATs, ACTs and FCATs is more along the lines of: How does success on these tests indicate anything at all about the test-takers’ intellectual skills and readiness for graduation?
*anecdotal evidence!
You’re a little behind the times here. Last time I visited Algol-60 you could hardly find a decent umblat for less than 25 zmb. Every now and then they’d be asking 15 or 20, but the cilia would be all sticky and fibrous and sometimes they’d spontaneously divide right in front of you. Eeeeuugh!
It is very hard to eliminate all bias even if you throw out questions that certain groups do bad on statistically. I do not know if the following story is true but it illustrates the point. In a test given to young children, they are asked to cirlce teh picture of the man working. There are two pictures. In one the man is sitting in a chair reading. In the other he is digging.
The young child of a college professor who enjoyed gardening indicated that the man reading was working which of course was true for this child’s experience.