Biased SATs?

Factual Questions
41

I am a fairly well-educated person, from a blue-collar white neighborhood in New York City. I went to an Ivy League college, in large measure because my SAT scores were very high. Other students from my high school, students who had better grades than I did, did not get into this same college, because their SATs were not as high as mine. To me, THAT is a far more serious issue than alleged racial bias.

I think almost everyone will agree that, all other things being equal, a student’s grades are a MUCH better predictor of how he/she will do in college than the SATs. Unfortunately, high schools in the USA are NOT equal! A student with straight A’s at some lousy city high school may be a far poorer student that a kid with a B- average at a far more demanding school.

So, since grades given by different high schools can’t really be validly compared, colleges need some OTHER measure. That’s where the SAT’s come in. Now, the SATs could save themselves a LOT of trouble if they changed their name. The “A” stands for aptitude, after all, and NOBODY believes it’s an aptitude test. It’s a vocabulary and math test- nothing more and nothing less. And, in my opinion, there’s nothing wrong with that. To succeed in college, you DO need a degree of reading/spelling/grammar and math skills.

Should the SATs be treated as gospel? Of course not. A hard-working guy with a 1200 on his SATs may do much better at Harvard than a slacker with a 1600 score. For that matter, a college dropout (Bill Gates, Michael Dell) may do much better in the business world than a guy who graduates summa cum laude from Harvard! The SATs aren’t everything, but they aren’t nothing. MOST people fall into the middle of the pack- a guy who scored 1000 one day might well have scored 900 or 1100 on a different day, even without coaching. And that swing is crucial! An 1100 will get you into almost any quality college, while a 900 makes you a borderline candidate at best. So, fair-to-middling scorers on the SAT have valid reasons to gripe. BUT… if you’re at either end of the spectrum, facts are facts. If you score a 1600, you are mighty bright. And if you can’t score even a 700… you’re a mighty dim bulb. Maybe your school system failed you, maybe you’re just a moron, but whatever the reason… you do NOT belong in college.

However, if you want to talk about stupid. pointless, worthless tests… look at the LSAT and the GMAT. These tests are of no value whatever, and I say that even though I scored extremely well on the GMAT! My ex-wife was a business major in college, and worked many years as a financial analyst and bond trader. I, on the other hand, had no background in business- I was a liberal arts major. On a whim, I took the GMAT and got a 720, which would get me into almost any good business school. She took the test, and did miserably. Never mind cultural bias- the “logic” questions on the GMAT are absurd, fit for a Mensa book of puzzles, but worthless as a way of judging whether you’re qualified to be a high-level business executive or financial wizard. The idea that I’m better qualified to succeed in business school than my ex-wife (a woman who THRIVED in the REAL business world) is insane, and for that reason, I have no respect for that test.

42

Astorian: Studies have been conducted that show that SAT scores are a better predictor of college performance than high school grades.

43

Yes, all tests are biased. You could designed a test in which only athletes would do well. How fast can you run a mile or what is your best distance in the running broad jump?

44

Astorian writes:

> An 1100 will get you into almost any
> quality college, while a 900 makes you a
> borderline candidate at best.

No, an 1100 would be considered rather mediocre at most top colleges (and a 900 would be considered hopeless for admission). That score might get you in if you were a top athlete or if you had almost straight A’s at a good school (so that the admissions people could argue that you were a good student who had a bad day when you took the SAT test). It’s not even true that a score of 1400 would guarantee your admission to a top college.

Wendell Wagner

45

Headless Cow states that the SATs are a better predictor of college success than high chool grades. That MAY be true if you’re measuring the success/failure rates of ALL students taking the test in a given year. As I stated earlier, America’s high schools vary widely in quality, and it’s hard to judge whether straight A’s at one high school are really better than straight B’s at another. There are high schools, after all, where you get an A for little more than showing up and not committing any felonies. There are also schools like the Bronx School of Science, where a truly gifted student may have a C+ average.

Given this disparity, the SAT may be a better predictor of college success than high school grades for the student population as a whole. But I GUARANTEE that, within the Bronx School of Science, the kids with high grades and mediocre SATs will far surpass the kids with low grades and high SATs.

As I stated earlier, I did quite well on the SAT- I got a 1480 overall (780 verbal, 700 math). This was far higher than many classmates with better grades achieved. I don’t criticize the SAT because I myself was wronged by it in any way. Quite the contrary! The SAT been berry berry good to me! It got me into an Ivy League college, when my grades wouldn’t have. I question its merits precisely because I KNOW there were better students than yours truly who had to settle for lesser colleges because their SATs weren’t as good as mine. That strikes me as unfair, though I admit, I don’t know of a perfect alternative.

46

One last thing: I never claimed that 110 is an outstanding score on the SAT. It certainly wouldn’t get you into any of the elite colleges (unless you had something else going for you… like a good jump shot, or a wealthy alumnus in your family). But it WOULD get you into almost any quality college. It would get you into the University of Texas, for instance. Not exactly MIT, I admit, but a quality institution nonetheless.

47

I did a term paper on this in college for a Secondary Education degree. Im not bragging, but I just want to let you know I actually studied this subjectat depth a few years back.

I found that it’s NOT the SATs that are biased . . IT’S THE COLLEGES. See, SATs only measure HOW WELL a high school graduate will suceed in college based on the COURSE MATERIAL they must master in college, NOT how intelligent the test taker is. Only an IQ type test can measure basic intelligence.

If you want to make SATs fair to minoritoies, you need to change college curriculum first. If you change the SATs without changing the curriculum, the SAT is NOT a proper measurement of future college prformance.

http://www.geocities.com/capitolhill/parliament/1685/

48

Aha! Now we’re headed for GD. ‘Change the college curriculum’ you say? To me that sounds like the first step towards altering the society itself. I agree that society could use some changing but we would have to be very careful how we change it.

The colleges will reflect SOME culture and also influence future culture. In the past, the students, faculty, and alumni have argued it out. History has shown that none of these groups is all powerful but all have varying influence.

Now, to exclude a group from this part of society, an historically powerful force in societal change, is a terrible thing. However, the current situation does not exclude. It does make it more difficult for a person from a culture other than the dominant one. I think this is inevitable. People who want to maintain their minority culture give their children a mixed blessing. The children must learn the rules of the dominant society as well as their own. This can be confusing and frustrating but those who perservere often outperform the single culture types.

I think that these problems will always exist in a multicultural environment. And, life will always be easier for the dominant culture. But, easier does not mean better. Those that embrace the additional challenges can work toward positive changes.

If men had wings,
and bore black feathers,
few of them would be clever enough to be crows.

  • Rev. Henry Ward Beecher
49

To Spot Bias in SAT Questions,
The Test Maker Tests the Test
By AMY DOCKSER MARCUS
Staff Reporter of THE WALL STREET JOURNAL
August 4, 1999
PRINCETON, N.J. – The question won’t ever appear on an SAT test, but the answer is critical to the college-entrance exam’s future: Is the SAT fair?

Educational Testing Service, which makes up the SAT, has worked for years to get rid of bias. Yet on average, black and Hispanic students continue to score below whites on the test, and women still lag behind men.

Now there’s more concern than ever. In June, the U.S. Education Department’s Office of Civil Rights began circulating draft legal guidelines outlining what it considers bias. As a result, colleges using the SAT may face legal action because of the disparate scores.

“We’re not saying throw out the SAT, but there are inherent biases in the test,” says Dennis M. Walcott, president of the New York Urban League.

“Score disparities exist, but the bias isn’t in the SAT,” responds Paul A. Ramsey, vice president of ETS.

The test maker argues that a variety of factors explain why whites and men continue to perform better on the SAT than other groups. It notes, for example, that better schools and higher parental income have an impact on test scores and that women still don’t take as much advanced math and science as men. Critics counter that the test itself is the culprit, containing language and cultural nuances to which minority kids are less exposed.

ETS says all of its tests are checked for fairness by company reviewers and outside committees appointed by ETS and the College Board, the SAT’s sponsor. Reading passages featuring women and minorities have been added to make the test more balanced, and certain topics – sports and the military, for example – are avoided.

The company also tests some questions each year on a section of the SAT that doesn’t count toward the total score and evaluates how students of various races and sexes fare in answering them. If one group answers a question significantly better than other groups do, the question is banished from future tests.

Wiped Out
Below are questions that were tried out on students who took the SAT exams in 1998. All of the questions were shown to be biased and will not appear on future SAT exams.

Verbal

DUNE: SAND:

a. beach: ocean
b. drift: snow
c. wave: tide
d. rainbow: color
e. fault: earthquake
Correct Answer: B
Results: 23% more whites than African-Americans and 26% more whites than Hispanics answered the question correctly.
Hypothesis: Regional variations. High proportions of African-Americans and Hispanics live in the south and southwest areas of the country, where there is less familiarity with terms associated with extreme winter weather.
Because Barbara McClintock’s identification of ‘jumping’ genes represented a turning point in genetics, it is considered a ____ event.

a. formulaic
b. tenuous
c. landmark
d. universal
e. surreal
Correct Answer: C
Results: 9% more men than women answered this question correctly.
Hypothesis: Although the question is about a woman scientist, the science terms may have made women more uncomfortable than men.
The actor’s bearing on stage seemed ____; her movements were natural and her technique ____.

a. unremitting … blase
b. fluid … tentative
c. unstudied … uncontrived
d. eclectic … uniform
e. grandiose … controlled
Correct Answer: C
Results: 9% more women than men answered this question correctly. 8% more African-Americans than whites answered this question correctly.
Hypothesis: Women and African-Americans tend to do better on questions dealing with the humanities or the arts.

Math

For all positive numbers a and b, [a + b] is defined by [a + b] - (a + b)(ab). If [2a + 2b]-k[a + b], what is the value of k?
Correct Answer: 8
Results: 4% more females answered this question correctly than men.
Hypothesis: Females tend to do relatively better than males on questions, like the one above, that are from the curriculum or textbooks.
If 80 percent of X is equal to 20 percent of Y, then Y is equal to what percent of X?

a. 16%
b. 25%
c. 40%
d. 250%
e. 400%
Correct Answer: E
Results: 12% more men than women answered this question correctly.
Hypothesis: Females tend to do worse than men on questions involving percentages, particularly nonrounded number percentages or percentages over 100.
If the square root of 2x is an integer, which of the following must also be an integer?

a. square root of x
b. x
c. 4x
d. x-squared
e. 2x-squared
Correct Answer: C
Results: 7% more African-Americans than whites answered this question correctly.
Hypothesis: African-Americans tend to do better on problems that come from the standard math curriculum and don’t involve applied math.
At North Industries, 1,200 employees are in the health plan and half of all company employees are in the savings plan. Of all the company savings plan members, 800 are in the health plan and 250 are not in the health plan. How many employees of North Industries are not in the health plan?
Correct Answer: 900
Results: 16% more whites than Asian-Americans answered this question correctly.
Hypothesis: Asian-Americans tend to do worse on word problems applied to real life situations than whites of the same ability.

  • O X =

The figure above is made up of 6 squares and can be folded along the dotted lines to form a cube. When the cube is formed, which of the following symbols will be on the face of the cube that is opposite the face with the symbol?

a. *
b.
c. +
d. O
e. =
Correct Answer: E
Results: 6% more whites than African Americans answered this question correctly.
Hypothesis: African-Americans tend to do worse on problems that involve figures than whites do.

Source: Educational Testing Service

Copyright © 1999 Dow Jones & Company, Inc. All Rights Reserved.

50

Wow, if those questions are biased I wonder how you can possibly write an unbiased question. I do see how the drift:snow question could be called biased, but the rest all appear fine to me as long as there is a balance of questions. If you have one applied math problem for every ‘curicular’ question, doesn’t that make the test fair?

If men had wings,
and bore black feathers,
few of them would be clever enough to be crows.

  • Rev. Henry Ward Beecher
51

Wow! What a fantastic post! Thanks, Dmitiri.

So this leads up to a question. Am I sexist against women if I ask my students questions about a woman scientist? Does the fact that women are more often intimidated by scientific terms mean that to avoid sexism, I should actively avoid using scientific terms?

I say no. And I don’t think the SAT should have to avoid using scientific terms for that reason either.

Your Quadell

52

Wow, “drift and snow” are considered to be biased terminology? I really don’t see it, especially since it is very likely to come up in some sort of reading in college, whereas “regattas” is admittedly less likely.

So I have another point to throw out, and someone can attack it:

In any random distribution, there is by definition going to be some clumping in the distribution of whatever - correct answers, or cancer cases, etc.

Deciding that clumps and high points in the distribution shows a cause is the kind of thinking that lead to the theory that power transformers on power poles were giving people leukemia.

OF COURSE someone is doing better on percentages! If that weren’t true, we would suspect everyone of having collaborated on the answers.

My cop friend once told me that when investigating a collision or crime, if the stories include all the same details and don’t slightly contradict each other, you should immediately suspect the witnesses of having agreed on what to say beforehand.

OK, now feel free to discuss and fillet.

53

"DUNE: SAND:
a. beach: ocean
b. drift: snow
c. wave: tide
d. rainbow: color
e. fault: earthquake
Correct Answer: B
Results: 23% more whites than African-Americans and 26% more whites than Hispanics answered the question correctly.

Hypothesis: Regional variations. High proportions of African-Americans and Hispanics live in the south and southwest areas of the country, where there is less familiarity with terms associated with extreme winter weather."

I’m not buying. There are lots of blacks and Hispanics in Chicago, New York, Philadelphia, Boston, etc.), and they, like anyone else living in those cities, are VERY familiar with extreme winter weather. (^:

54

The bottom line is: the verbal SAT tests reading and word comprehension. If you don’t read, you will not do well on it. The math portion tests reasoning and basic familiarity with algebra, geometry, and trigonometry. Therefore, if you don’t take these subjects, you won’t do well on it. This has nothing to do with race, culture, or background-it has EVERYTHING to do with hard work and a serious attitude toward learning. If one accepts that “ebonics” is a real language, then we must accept that such people will get low scores on the verbal SAT. I believe that when the SATs wer first instituted, a score of 500 on each was considered average-now the average is below 350.
Clearly something is wrong with our current educational system.

55

John Bredin writes:

> There are lots of blacks and Hispanics in
> Chicago, New York, Philadelphia, Boston,
> etc.

You have fallen for one of the commonest misconceptions about American blacks and Hispanics. People who’ve watched too many bad TV shows get the idea that the vast majority of American blacks live in the inner cities of Los Angeles, Chicago, and Boswash (i.e., the large cities from Boston to Washington, D.C.), plus a smaller number of blacks living in the suburbs of these cities. This isn’t true. About 55% of African-Americans live in the Southeast. When we add in those who live in the Southwest (including Southern California), it’s clear that a substantial majority of American blacks live in regions where it rarely snows.

Similarly, watching too much bad TV has probably convinced you that most Hispanics live in inner-city neighborhoods in these same cities and a smaller number live in the suburbs of these cities. Again, this isn’t the case. Texas, California, and Florida alone account for about two-thirds of all Hispanics. By an even larger margin then, most Hispanics live in regions where it rarely snows.

I’m not arguing that all blacks and Hispanics live in regions where it rarely snows or that all non-Hispanic whites live in regions where it frequently snows. I’m just saying that a much larger percentage of blacks and Hispanics have never seen a snow drift in real life than non-Hispanic whites.

gene writes:

> I believe that when the SATs wer first
> instituted, a score of 500 on each was
> considered average-now the average is
> below 350.

This also isn’t true. When the SAT tests were introduced in 1941, the averages for the students that took them at the time were set (by definition) at 500 for the verbal and math scores. About 10,000 students took the tests at that time. This was only those who were applying to the 30 or so most selective colleges in the U.S.

Over the next 40-some years, the number of colleges that required the SAT increased to the point that now nearly all colleges in the U.S. require the SAT (the only exceptions are Midwestern state universities and a few Midwestern private colleges which instead use the ACT) and the number of students that take the SAT each year increased to about 1,500,000.

What this means then is that while only the top 1% or 2% of American high school graduates took the SAT tests in 1941, now around 40% or 50% of them take the SAT. Instead of recalibrating the scores each year so that the average score would remain 500 in verbal and math, it was decided (until about five years ago) to calibrate the test so that students of the same level as took the test in 1941 would continue to score 500 on the math and verbal. The average score on the math and verbal unsurprisingly slowly dropped to about 425 (NOT 350) by 1980 and has actually increased a little since then. About five years ago it was decided to quit calibrating the scores according to the level of the 1941 students at the selective colleges. Since then the average scores are set by definition at 500 on each of the verbal and math tests.

The fact that the average score on the SAT tests was 500 on the verbal and math in 1941 and about 425 in the ‘80’s and the early ‘90’s means nothing. All that this says is that the average college applicant today (when four or five times as large a proportion of people go to college) scores about 75 points lower than the average applicant to a highly selective college in 1941 does.

Incidentally, the average student at those highly selective colleges today scores quite a bit higher than the average student at highly selective colleges in 1941. Those selective colleges were not so consistently selective back then as they are today. They had a lot of first-rate students, of course, but they also accepted a lot of rich kids with marginal grades and scores. It wasn’t until just World War II that these colleges began accepting substantial amounts of people from middle-class to poor backgrounds and began rejecting rich kids whose only recommendation was that their parents went to the same college.

56

Gene - “I believe that when the SATs were first instituted, a
score of 500 on each was considered average-now the average is below 350.
Clearly something is wrong with our current educational system.”

Wendell is indeed correct. My full time job is helping run a tutoring/SAT school, and the tests are now slaved to a 1000 point average, per test, period.

As someone who gets an 800 on every verbal section, (not that it’s so impressive at the age of 30) I can tell you that there has been a noticable creep up in the level of difficulty even in the past five years. I assume that’s because they have noticed that they are having to curve it more and more, so to avoid having a situation where only an absolute perfect score gets 800, they have increased the difficulty.

Now, isn’t anyone going to attack or support my statistical clumping theory?

57

I have lived all my life in middle Georgia. I have never seen a snow drift, but I read and I watch TV, etc. I knew the answer to the question. It takes mental reasoning.

A dune is made from sand.
So,
a. beach: ocean, a beach is not made of ocean so rule this one out.
b. drift: snow
c. wave: tide, a wave is not made of tide, rule it out.
d. rainbow: color, a rainbow is not made of color, so rule it out.
e. fault: earthquake, a fault is not made of earthquake, so rule it out.

By process of elimination, I can easily figure out, if I do not already know, that drift:snow is the correct answer.

People who go to college should be able to think, not regurgitate facts, but actually think. The SAT helps to measure the ability to think. It is not biased, the people are.

Jeffery

58

I can’t count but I can talk a little bit (780 on the old 800 pt verbal test). The only question that I remember that struck me as biased at the time was something like “Tenis is to court as lacrosse is to {blank}.”
For what it’s worth…

59

Given your reasonable first statement, it follows that the correct answer is D. Of course a rainbow is made of color! A rainbow is a spectrum of light; colors are segments of that spectrum.

On the other hand, the most common use of “drift” is as a verb! A leaf can drift in the breeze, so maybe snow, which also flows through the sky, also drifts along. Given this perspective, “type of movement: snow” is as wrong as “fault: earthquake”.

If you already know the noun “drift,” then you know that settling sand forms dunes as settling snow forms drifts. If you hadn’t commonly used “drift” as a noun, then you would follow the quoted logic and choose D.

60

Dimitri’s post showing the ETS Testing the Test indicates to me that the ETS is taking the responsible, scientific approach to eliminating any possible bias in the SAT. There need not be any reason why one group or another does better or worse on any given test, just that statistically it happens. By eliminating specific questions regarding a particular subject (not eliminating the subject - percentage calculations, for instance), it eliminates any possibility that the way the question was posed could possibly create any kind of bias. For any given subject, there can be a thousand ways of formulating questions. ETS need only use the ones that does not have any statistical biases. Of course, I am assuming that they are using valid sampling methods to model the population as a whole when testing the test.