Biased SATs?

Factual Questions
21

Roksez writes:

 The argument for the silo question is that most white people (even if you grew up MILES away). Why would any inner city youth know?? Would he read the farmer's almanac? Or would he be naturally interested in farming?? I don't think you can assume that ANY child would know what a silo is. You can assume that more rural children than urban would. Ask a child from a farm what it means to tag something. He would say it means to hit. An urban child would say it means to spray paint it. Many SAT questions can be answered from personal experience, and the argument is that rural youths have an edge because they can answer more questions as a result of their life experiences. Lower income youths are also less likely to travel, and (from personal experience) attend grade schools with poorer computer and library facilities.

Buddy, I am white and i have no inclination whatsoever to read a farmers almanac or to go out and hoe the corn patch. Knowing what a silo is comes from reading books like “Burton the field mouse” and “James builds a barn” or other such books which children read at some point in their lives. (the titles are fictional, the point remains) I have as much connection to a silo as a black inner city youth. If we really are going to take this argument seriously then it would be unfair to include words like “sidewalk” or “subway”. Both things which a little-travelled rural youth has never seen and has no use for. But any rational person knows that if you’ve received an education of any merit whatsoever you would know what these things are. Again, I make my point, would it be unfair to test kids on American history because they werent there when it happened?
And you claim rural kids have an edge…They do? Are the majority of the words on the SAT related to rural life? Would you consider “regatta” a word which a farm kid from nebraska would have more access to than an inner-city kid?? There is no question that many kids are denied a good education for various reasons, and perhaps more should be focused on bringing equity to the situation, but lets not make believe that if a test shows someone to have a poor education that the test is biased.

22

Besides for this, it’s biased towards the upper class inherently.
The nature of the test is such that you can pay thousands of dollars for prep courses that will significantly improve your score…I have even heard of people having personal daily SAT tutors from when they were 13, resulting in a score of very nearly 1600.

Now, you can’t tell me that a low-income kid is going to have that opportunity…

Love,
Pippy

And now, for something completely different.

23

Besides for this, it’s biased towards the upper class inherently.
The nature of the test is such that you can pay thousands of dollars for prep courses that will significantly improve your score…I have even heard of people having personal daily SAT tutors from when they were 13, resulting in a score of very nearly 1600.

Now, you can’t tell me that a low-income kid is going to have that opportunity…

Love,
Pippy

And now, for something completely different.

24

Can we get a list of specific biased questions, and the reasons they are biased? We’ve only seen two examples here, and only one of them, “regatta” is a word that (from other reading) I know actually appeared on the SATs. I think the person who said “silo” was just giving an example of a type of word that could be biased.

Can we get some idea of what percentage of the questions on any one test are thought to be biased? How much does this affect a score?
If there are one or two “biased” vocabulary words on the test, and a student misses both, how many points difference does that translate into? Probably not much, so the amount of such bias on any test is highly relevant.

Also, the tests are clearly measuring general knowledge to some degree, but so what? It is the GENERAL knowledge that one needs in college! College is taught in colloquial American English, isn’t it? Take an extreme example - a kid fresh off the boat from… Iceland may not do too well; he does not speak English AT ALL, say, so the test is as culturally biased against him as can possibly be. He gets a zero. It certainly doesn’t measure his ability to learn, but then again it DOES reflect how well prepared he is to enter an American university: not at all, until he learns the common language of our culture.

Here’s an SAT question for you, biased in favor of the rural kids.

BODKIN : BREAST :: CAP : _________
a)FOOT
b)ASS
c)LYCANTHROPE
d)ELEEMOSYNARY
e)HAT

25

“The argument for the silo question is that most white people (even if you grew up MILES
away). Why would any inner city youth know?? Would he read the farmer’s almanac?”

Most city people know what a “silo” is, and most rural people know what a “subway” is, even if they have never left their hometown. That’s because there’s a little invention called “television”. Even before that, there were two inventions known as “radio” and “motion pictures.”

In other words, since the 1920s and the advent of the big movie studios and the nationwide radio networks, there has been a common culture in the U.S… The same movies, music, and television shows are available in the inner city, the suburbs, and the farming village.

My focus in college was on 20th Century history, and one of the big trends in this century has been the creation of a common culture. Before movies and radio, the farm dweller had only the vaguest notions of city life, while the city dweller had only the vaguest notions of life on the farm. After the 1920s, the farm boy fresh to the city would have some idea how to dress and act there from seeing movies. Not a totally accurate idea by any means, but at least a passable one, useful in a “rule of thumb” sort of way.

In other words, I don’t buy “silo” or “regatta” as some sort of sinister shibboleth to weed out the “white severely able-bodied people” (as someone put it) from everyone else.

26

Hi all. I am a white, middle class, jewish urban dweller who grew up in NYC

the only vocab question I got wrong on my SAT had to do with the definition of “Winnow” which, as I learned later has to do with separating the “wheat from the chaff”. I definately regarded this question as rurally biased at the time. I’m sorry but I had read more books than anyone else I knew at the time (everything from the Uberclassics to the complete works of Robert Heinlein) and had never come across this word.

I happened to know what chaff is because I had an obsession with horses and British publications describe chaff as something appropriate to feed a horse (never saw it recommended in a US publication).

In sum, I agree that some words on the SAT are probably be biased – to someone. On the other hand, biased questions didn’t come between me a a very high score…just me and a perfect score.

This space intentionally left blank

27

“probably be biased”

it appears that my language skills have degraded somewhat since H.S. – sorry.
-Rmariamp

28

Rmariamp: probably the primary source for separating “wheat from chaff” is not direct agricultural knowledge, but the Bible, where this symbolizes separating the good from the evil. That would give this question a distinct Christian bias.

29

Jens: Non-Christian agrarian cultures had to separate the wheat from the chaff too. Seperating wheat from chaff may infer Christianity in your experience, but it’s not a culturally specific topic.

The Bible does use the word “winnow”, but only in Isaiah and Jerremiah. And those books are a definite part of Jewish culture too.

Words in the Bible shouldn’t be excluded unless they’re not used in other contexts. An Alta-Vista search on "Winnow produces the following general subjects:

  • Gourmet wild rice
  • Efficient web searching
  • Machine language algorithms
  • A church in England
  • A UFO info site
  • An opportunity to buy a Native American agricultural tray

I’m sorry, I don’t see a bias.

-Quadell

30

Quadell, you are correct if you are using the King James version. Both the New International Version and the Revised Standard Version have “winnow” in the New Testament: Matthew 3:12 (actually winnowing").

But I have not personally read the Bible straight through (though I have read Matthew and Genesis and a few others); I did not get MY knowledge of winnowing and chaff from reading the bible. I certainly did not learn it from gourmet rice or UFO sites.

This just happens to be (in my experience) one of the most frequently used images by preachers in their sermons. For a church-going Christian, “winnow” and “chaff” are common, familiar words. For your web search, “winnow” seems to be rather esoteric.

I maintain that a CHURCH-GOING (or Bible scholar) Christian gets a bit of an advantage on that particular word (maybe a Muslim, too, my Alta Vista search on “winnow*” brought up some Koran references also) over some other groups.

Note that I am not condemning the SAT tests on the basis of that one question.

CENSURE : REPREHENSIBLE ::
(A) prize : valuable
(B) provide : supportive
© applaud : enthusiastic
(D) inquire : informed
(E) continue : initial

would probably favor those who paid close attention to the Monica Lewinski scandal.

You should be able to find SOME bias in just about every question. As long as these same biases are not repeated (if the next question were one where I had picked up the vocabulary by reading Slaughterhouse Five, and the next one came from the Odyssey I would feel far more comfortable than if the next two questions were also words seen primarily in the Bible and a few other very obscure places) that bias should cancel rather than carry through to blight the entire test.

I am not going to be impressed by people who can find one or two questions that might suggest a bias. I’d have to see what percentage of questions on a test had those questions relative to how big the testing gaps are.

31

Here’s some historical and research based info on biased testing for education and the workplace:

1st let’s look at what’s been said so far:

Omniscient states: “You may recall that over half the test is math, verbal reasoning, and science that are totally objective culturally. The only topic of argument is the reading comprehension.”
–incorrect, in fact math is 1st and verbal reasoning is 2nd with regards to discriminating among racial boundaries

smegmum V states “how can we have culturaly biased questions when the test are, by default designed to rank people within the predominant (anglo) culture?”
–incorrect, the SAT, the ACT, the GRE and its disciplinary-focused alternative forms are designed to compare the test taker with a percentile standard of performance based on historical norms of stratified samples from historical test takers. In plain English, the test taker is compared to the “normal scores” of people (taken from specific racial, gender, ethnic, socioeconomic, …categories) who have taken the test in the past. There is no ‘covert’ ranking of scores, you get a score and the colleges and institutions who use the score rank the candidates, not Educational Testing Services (ETS) or any other company or governmental agency.

quadell states: “The EEOC, the government agency that enforces Affirmative Action cases, claims that in tests for employment, a test is culturally biased if it can be shown that one race consistently performs better at it than another. In SATs, for instance, the average score among white Americans is significantly higher than the average score among African Americans. Therefore, according to the EEOC, the test is biased by definition. A number of companies have been successfully sued for using SATs for employment requirements.”
–this whole statement is incorrect. The EEOC doesn’t enforce AA cases, they do not claim discrimination based on consistent racial performance, the EEOC has not and will never address the use of the SAT or any standardized test, they focus on labor not education, and no company has ever been sued for using the SAT for employment requirements, I just looked it up. Furthermore, I have been in the employment selection and assessment business for over ten years and I have never heard of any company using the SAT for selection, and I doubt that any have for many reasons that I will not expand on at this point. However, quadell, your post is the closest thing I have read to remotely explaining culturally biased testing, by indirectly mentioning adverse impact with regards to testing.

Handy states: "Almost all tests are biased in one fashion or another. It takes some work to decide just how.
How about psychology tests like the MMPT? A question might be ‘Do people look at you when you are in a restaurant?.’ A deaf person signing in a restuarant gets a lot of that, so they would answer ‘yes’ thus bringing up their ‘paranoia’ score.
–Handy you are getting there, ALL good tests are biased. That’s what they are designed to do. Some are biased against less intelligent people, some are biased against psychopaths, bad drivers, alcoholics, …if a test doesn’t bias against something or some group of people than its a piece of shit.

I could go on, but for the sake of getting to the friggin’ point, here’s the deal. The SAT is culturally biased, the ACT is culturally biased, the GRE is culturally biased, any general g-loaded intelligence test is culturally biased, virtually any math test you have ever seen is culturally biased. That’s not my opinion, that’s fact. Why? It is because that historically, minorities (more specifically blacks) score on the average one standard deviation (remember stats class?) lower than whites on the above mentioned tests. Why? No one knows 100% of the reasons, but anyone who is breathing can see some of the factors, shitty school conditions, worse pay for teachers=worse teachers, poor role modeling,…etc. Are the tests culturally biased because of the types of questions they ask (i.e, the wheat example, the silo example) probably not, ETS about ten years ago attempted to remove all the “rural/urban, white/black questions” out of the test and the gap between scores has not significantly changed, so logic tells you nope, its not the questions it’s the subject matter. What I have read regarding the NCAA matter, is nothing that everybody involved with testing hasn’t known for years, I’m not sure why the NCAA decided to act now when this has been very common knowledge for many, many years, but then again, I have heard from contacts that the NCAA is managed by dipshits, so who knows?

The power of accurate observation is frequently called cynicism by those who don’t have it.
George Bernard Shaw

32

I wouldn’t say that Quadell’s statment is “incorrect” but rather that parts of it may not be applicable. The EEOC does not “enforce” AA cases per se- that is for the courts to do. However, “The mission of the EEOC, as set forth in its strategic plan, is to promote equal opportunity in employment through administrative and judicial enforcement of the federal civil rights laws and through education and technical assistance.” The EEOC’s def’n of “cultural bias” doesn’t apply to SATs being used for college admissions. However, I don’t see how you can say that the EEOC will NEVER address the use of standarized testing for employment reasons.

I think that Quadell was simply trying to provide a definition for “cutural bias” as there are people who assume it means “is there anything inherent in the question that it would make it easier/harder for a certain culture to properly answer?” versus those who think that it means “if people from certain cultures do worse on this question (as an aggregate) then it is culturally biased”. You won’t make any headway on this subject til y’all agree on the def’n of CB.

33

“Take an extreme example - a kid fresh off the boat from…
Iceland may not do too well; he does not speak English AT ALL, say, so the test is as
culturally biased against him as can possibly be. He gets a zero. It certainly doesn’t
measure his ability to learn, but then again it DOES reflect how well prepared he is to
enter an American university: not at all, until he learns the common language of our
culture.”

====================

This seems to me to be the entire point. What is the purpose of the SAT? Who commissioned its writing? The College Board did. Its stated purpose is for colleges to discriminate between applicants for admission. They want to know who currently has the skills to make headway at their college, not why they don’t.

It’s not the colleges’ intent or in their ability to control or speculate on what’s going on before they get ahold of these kids. That’s just not their job.

When the kids arrive at college, they will be expected to participate in literate discussions and read things written at a 12th grade + reading level. Can they? That’s all the colleges want to know. And they’re not about to throw something out of the curriculum because it mentions silos, and you’re from the heart of Manhatten. You’re just expected to either know or find out. Tough. They’re not in the business of compassion in that manner or remdiation.

After having come off hard nosed, you may notice my previous posts. I’m not saying there’s not a problem in the education system as it concerns minorities in America, I just don’t think it’s the colleges’ problem.

34

Mojo says:

“there are people who assume it means ‘is there anything inherent in the question that it would make it easier/harder for a certain culture to properly answer?’ versus those who think that it means ‘if people from certain cultures do worse on this question (as an aggregate) then it is culturally biased.’”

I agree this needs to be defined. Does anyone know which definition the NCAA used?

If I were personally accused of being biased using the second definition, I would have to admit guilt. Maybe I have, but thought I was being accused of the first definition (I have some guilt there as well, I suppose, but I eliminate when I spot it. I am not as naive as used to be, and do not profess to have completely eliminated any trace of racism in myself - but I try.). I generally have positive feelings toward people who are “well read.” I give “well read” people more respect. I have a few friends who are not “well read” but they had to work harder to gain my friendship. I don’t think this makes me racist. Elitist, perhaps. I think that colleges should be elitist in this sense but not racist. If this makes them culturally biased, then so be it.

If men had wings,
and bore black feathers,
few of them would be clever enough to be crows.

  • Rev. Henry Ward Beecher
35

Mojo, I never said the “EEOC will NEVER address the use of standarized testing for employment reasons.”

I said “EEOC has not and will never address the use of the SAT or any standardized test, they focus on labor not education”

I know this because the EEOC is one of my clients, and as long as they fall under a Congressional ‘commission’ and not a governmental ‘department’ their boundries are very clear and set in stone (that and the fact that they have about 1/100th the headcount needed to properly do their current job). That being said, I have seen the Department of Labor grow in the past 7 to 8 years into a much larger, farther reaching entity than I ever expected, so they might step into the workplace testing arena with a larger presence, but that’s really up to them and the budget dollars Congress gives them.

The power of accurate observation is frequently called cynicism by those who don’t have it.
George Bernard Shaw

36

You are correct- You did not use those words, I added the “for employment reasons” part myself. What I was trying to state (and din’t do a good job of) was that if a company used standarized testing as criteria for employment this would start to fall under EEOC juridiction. I do not know of any specific company that does this- I was being petty over your use of the word “never” and it really didn’t have anything to do with the original post.

I still believe that Quadell was simply trying to furnish a working def’n of “cultural bias”, without which we can’t decide if a test is culturally biased. Only Q knows that for sure though.

37

grumble grumble

Now you’re gonna make me go and look it up, Jahender. I’m leaving town from work today, but on Monday I’ll try to make it back with a cite.

Until then, I’ll just say there have been cases where companies used SAT scores, high school diplomas, and other similar criteria for employment, and were forced to discontinue this practice by the EEOC. That’s why companies don’t use the SAT anymore.

I’d also like to offer the following opinion.
Imagine that a company makes hiring decisions based on a test performance, and this test measures only the ability to add and subtract. Imagine further that applicants of one culture performs better on this test on average than applicants of another culture (leading to a culturally imbalanced workforce). I maintain that this is not the fault of the company in question, and that the company should not have to suffer from accusations of “culture bias” or “racism” in their hiring practices.

Am I alone in this opinion?

Your Quadell

38

quadell: I agree with reservations.

If the company has reason to believe that adding and subtracting are in some way correllated with job performance, they are free from any charge of racism in my book.

If the company studied the races carefully to find some measure that had a racial correlation (which turned out to be adding and subtracting), and then used that measure, despite the fact they were hiring people to gather litter, they are racist.

In either case, I support freedom of association, so they should be legally permitted to discriminate against anyone they don’t like. And potential employees and customers should keep that in mind (i.e., if they were truly heinous I would prefer not to shop or work there).

39

I’m a little surprised no one has commented on this in a while.

The idea of calling a test racially biased because a certain race scores lower on an average seems nonsensical to me. I really want to know if this is the definition being used.

I worked as a bartender on a boat for a couple of years. Part of my job was to carry kegs of beer up stairs on a moving vessel. We had one female bartender and, on some small cruises, when she worked alone she had to do without draft beer resulting in an occasional complaint. The captain and deck hands were not allowed to carry the kegs because the laws on alcohol and boating. Does this mean that their policy of not hiring any more bartenders who cannot carry a keg a sexist policy?

How about racist? So far, they have no black bartenders, all blacks who have applied failed the keg test, though they did have one a long time ago before the test was instituted. He could carry a keg in an emergency but the rest of us would usually cover for him.

If men had wings,
and bore black feathers,
few of them would be clever enough to be crows.

  • Rev. Henry Ward Beecher
40

You must have had some pretty wimpy black dudes applying.