For those of you whose browsers can’t run the test, what it does (in the case of the black/white one) is flash a series of images and words on the screen, and you are to push a key to categorize the picture as quickly as you can. The mistakes you make are supposed to show any hidden biases you have.
First it flashes pictures of faces of black and white people, and if it’s a black person you have to press a key with your right hand, if white a key under your left. Then it flashes a series of words like “Joy” or “War” and you are to press the right key for the positive ones and the left key for the negative ones. Then on the third round it does both faces and words. On the fourth round it is just faces, but this time you use the left for blacks and the right for whites. On the fifth round it mixes faces and words, with blacks and ‘bad’ going to the left and whites and ‘good’ going to the right. The sixth round reverses the faces back to blacks on the right and whites on the left.
It is supposed to catch you when you automatically associate one of the races with good or bad.
I have a couple of problems with the test. For one thing, shouldn’t they also vary the sides good and bad are on, not just the faces, to even things out more? I also think it should vary the order the combinations are in, because I think a person is going to get into a habit on the first run and make a lot of the same mistakes when it mixes it up, not out of hidden bias but simple human error. I also think that, even if this was accurately measuring how a person associated different faces with ‘good’ and ‘bad’, it may be skewed by the choices of faces used in the test. Every one of the white faces is fairly attractive and blemish free. A few are smiling, the rest have fairly neutral looks on their faces. None of the black faces are particularly attractive, the best looking one (in my opinion) is one of the males and is fairly average in looks. A couple of them have blemishes or scars on their faces, and some appear to be grimacing or almost scowling at the camera. Now, I may have some subconscious biases towards blacks, most Americans probably do due to the stereotypes pounded down their throats, but as far as appearance goes I think blacks are better looking on average than whites, so I don’t think bias has anything to do with my thinking the black faces are uglier than the white ones. This might lead to someone who subconsciously associates ‘attractive’ with ‘good’ and ‘unattractive’ with ‘bad’ to show a bias against blacks when the actual bias was strictly on appearance. I noticed on the body image bias test on the same page (fat vs. thin) that I kept clicking the wrong button on one of the ‘fat’ faces that I found attractive (well, less ugly) than the rest.
Anyway, I always get “Your data suggest a strong automatic preference for White”, which is what 48% of the people who take the test get. Anyone else going to give it a shot?
I sat the test from my normal machine, giving my ethnic background as white. It told me I had a strong preference for white people. I then resat it, and got the same result every time. I changed my ethnic background to black, and still got the same result.
I then moved to a different machine, resat the test, giving my ethnic background as black. It told me I had a strong preference for black people. I resat it a couple of time, and got the same result.
Now, the fact that I got varying results without trying in any way to change my response times to the photos is enough to make me doubt the validity of the test. The way results varied only when I moved machines is enough to make suspicious that they’re recording IP addresses or using cookies, to try to prevent different people obtaining different results. Combine that with the points you raise about manipulative use of sample photos and then add the rather dodgy connection that people would auto connect faces with good or bad (I spent most of my time trying to recall which button was which for that round) and I’d say it’s a crock.
I did the same thing as Gary and got the same result. Also, the fact that you’ve got to do the test as fast as you can increased the likelihood of unforced errors with the test would then (presumably) interpret as signs of hidden bias. It’s a neat idea but it’s definately got some flaws.
Badtz, you make a good point about the possibility that the faces they chose to use effecting the results. I didn’t notice it until you pointed it out, but you may be right.
I also took the “Asian vs. European, who is more American?” test. Now that I think about it, the sketches of Asian folks they used for that test were probably among the least attractive Asians I have ever seen. The tolerace.org people certainly seem to have a desired outcome to their tests. It can be difficult to design an unbiased test when you’re hoping for a particular outcome.
Gary, just because you got different result after retaking the test several times doesn’t necessarily suggest the test is flawed. It’s a test of subjective feelings; I’d be surprised if you did get the same result every single time. I hear what you’re saying about the cookies though. I hope there isn’t any sinister manipulation going on. Somewhere they have to show someone their test designs if they’re hoping to do anything with these figures right?
Jumping from “strong preference for white” to “strong preference for black” can seem an indication of a poor test, but it isn’t necessarily so. Hey these things happen with statistical sampling tests. It also might indicate that you “got better” at the test. That can certainly happen without trying.
I took the test from home, filling out the “background” part, and got “SP for W”.
Then I took it at work, twice, not filling out the 1st part, and got the same result as at home.
The results are not based (solely, at least) on the mistakes, but on the relative speed of your reactions to the pictures and words. Do you hesitate a little before associating “black” with “good”, or vice-versa. You need to read all the info about the methodology and how the results are scored.
The test is supposed to be for hidden (unconscious) bias. I don’t know how valid the test is, only what the site say’s about it. Any opinions from experts in this field would be of great interest to me.
Peace,
mangeorge
Did anyone read these pages? About Implicit Association Tests;] http://www.tolerance.org/hidden_bias/tutorials/05.html
and: About the science behind the tests; http://www.tolerance.org/hidden_bias/tutorials/06.html
These tests don’t test whether you’re a bigot, but simply if you tend to have some hidden bias. The tests are not conclusive, and the results can vary. Say’s so right there, in the articles. It’s just a test, guys.
Check out some of the other stuff on the site. The SPLC is a good organization, and they’ve done a lot of good work.
Of course, being a member myself, I may be a little biased.
Peace,
mangeorge
I’m not sure what the brouhaha is for. Now, it may be that the test may not operate well via internet connections, but it hardly seems that two anectdotal complaints are evidence it is not working. Of course, in general self-administered on-line tests are bound to be largely illustrative, if you want real results you gotta go do the real thing IMHO.
Ironic that you’ve jumped to that conclusion based on some spotty reports. Rather similar to another intervention I might add.
What did you guys think about the relative attractiveness of the two sample groups? Think we were looking for excuses? Perhaps so.
Also I wish they wouldn’t have told me every time I made a mistake. It was frustrating every time I put the black guy on the wrong side I just knew I was building up my hidden bias score. Doh!
I don’t know I’ve never been good at determining attractiveness. Beauty is in the eye of the beholder. Your excuse just shows you have more hidden biases!
Yeah I just retook it (for the first time) and decided that the attractiveness thing was crap. On review, I didn’t think there was any difference. I also did better this time and was able to score “no preference”, but I had to think “black is good” over and over in my head. I won’t do it anymore and I feel guilty about screwing up their data, but I just want to see if it would help.
I took the test also, but I quit when I had second thoughts about how was white or black. Now that I’ve thought about it more I believe that the only black people in the test were the very black ones. But living in Louisiana, where there are many light skinned blacks, caused me to be confused on many of the facial close ups. I kept trying to second guess the test. Thinking to myself, “is that a black person or a white person? They must be trying to trick me.”
Don’t get hung up on the validity of the online test, which is meant to demonstrate how the testing works. Everybody knows (don’t they?) that online tests, polls, etc. have zero validity. What’s interesting is the research.
Also, some technical clarifications. The test does not use cookies (the site doesn’t either), and the idea that it is basing results on your IP address is unlikely*. Your internet connection speed has no effect on the test because it’s a Java program, and once the program is downloaded, it’s running on your own computer, not across the internet.
I’m not going to touch the suggestion that they are intentionally using more unattractive faces for the African-Americans and more attractive ones for the European-Americans. If you’re trying to make a convincing argument concerning hidden bias with that statement, you can dig your own grave
The online test itself is interesting, but the fact that the averages you are compared to are from online tests and not the actual research means the results are unreliable. I’d even go so far as to say meaningless. If they bother you, ignore them and read the well-written and illuminating tutorials.
-fh
*We took the test on different computers here at my house, but we’re all on the same subnet, so as far as tolerance.org knows, we all have the same IP address. We got wildly varying results. And if they did want to fake the test/results, why bother tracking IPs when you could just write the trickery directly into the Java app?
I think it was kind of nutty by the last catagory i was so confused because every thing was completly different, not because i had any preference for certain people.
here is a completely random e then i e then i results for the gender liberal arts/science test. Note that I am a male History major.
Interpretation Percent of Total
Strong automatic association between male and science 42%
Moderate automatic association between male and science 14%
Slight automatic association between male and science 13%
Little to no automatic gender association with science or liberal arts 15%
Slight automatic association between female and science 7%
Moderate automatic association between female and science 4%
Strong automatic association between female and science 5%
