# The flaws in Randi's tests **URL:** \ **Category:** Great Debates\ **Created:** [May 2, 2003, 11:23pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746 "2003-05-02T23:23:33Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![RickJay](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@RickJay](https://boards.straightdope.com/u/RickJay)\ **Post date:** [May 5, 2003, 2:19am UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/41 "2003-05-05T02:19:16Z")
bnorton, you could certainly argue that psychics can’t tell a good test from a bad. To use the dowsing example, it would have been in their best interests to claim, say, a 65% success rate, rather than the 87% they claimed. But it has to be the way it is **because if it wasn’t, Randi would be sued every single time he held a test.** Unless you give the claimant the ability to PERSONALLY agree to the nature of the test and the success criteria, it is an absolute certainly that when they fail they’ll claim the test was loaded against them. Only by having them help set the parametes of the test can you hope to avoid that. ---
**Author:** ![Patty\_O\_Furniture](https://avatars.discourse-cdn.com/v4/letter/p/96bed5/32.png) [@Patty\_O\_Furniture](https://boards.straightdope.com/u/Patty_O_Furniture)\ **Post date:** [May 5, 2003, 11:54am UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/42 "2003-05-05T11:54:29Z")
I still maintain that it’s hardly necessary to have scientists with clip boards and stop watches hanging around because Randi only test psychics on their claims. If they would stop making such ridiculous claims (like 90% success rates in dowsing or being able to bend metal bars without touching them), then there might be more wiggle room for them to complain about protocols or what-have-you. ---
**Author:** ![Liberal](https://avatars.discourse-cdn.com/v4/letter/l/848f3c/32.png) [@Liberal](https://boards.straightdope.com/u/Liberal)\ **Post date:** [May 5, 2003, 12:40pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/43 "2003-05-05T12:40:14Z")
Peter Morris is right. His analysis is excellent, and he almost manages to make materialism seem less mystical. Not only has he pointed out the flaws in Randi’s tests, but his critics have compounded the logical errors about which Morris complains. For example, one writer actually equated scientific examinations of falsifiablity with classroom examiniations of subject mastery. Another dismissed Auther Clarke’s statements by painting him as a “dreamer”. And yet another opined that psychics ought to be subject to a different (more stringent) criteria set when their claims are examined. We have equivocation, ad hominem, and a poisoned well in just a few strokes. Hang in there, Peter Morris. You have not yet encountered any substantive refutation. ---
**Author:** ![TVAA](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TVAA](https://boards.straightdope.com/u/TVAA)\ **Post date:** [May 5, 2003, 1:01pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/44 "2003-05-05T13:01:35Z")
How would Randi’s standing in the same room as a psychic disrupt the test? Tester expectation can indeed change the way people perform, but this has more to do with communicated attitude than anything else. The psychic must know Randi is a professional skeptic and debunker, so his hostile attitude is a constant problem. If his reactions changed when the psychic examined specific objects, that would increase the chance they’d pick the important ones. The psychic agreed to the conditions of the test, including Randi’s presence. If desired, the test conditions could have stipulated that Randi step out of the room or stand behind a screen. Didn’t happen. While some of the counter-arguments in this thread are without substance, **Libertarian** is quite wrong to dismiss them all. ---
**Author:** ![Princhester](https://avatars.discourse-cdn.com/v4/letter/p/3e96dc/32.png) [@Princhester](https://boards.straightdope.com/u/Princhester)\ **Post date:** [May 5, 2003, 1:15pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/45 "2003-05-05T13:15:36Z")
Lib to understand the whole situation you have to read all the threads in which this ongoing debate with PM has been running. I can (perhaps) see how you get the impression you have just from this thread. But there have been several other threads. In those threads several posters have refuted every one of PM’s facts and arguments. And now he has switched to this thread. I’ll leave the reader to ponder why that would be. ---
**Author:** ![TVAA](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TVAA](https://boards.straightdope.com/u/TVAA)\ **Post date:** [May 5, 2003, 1:18pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/46 "2003-05-05T13:18:51Z")
And **Libertarian** knows it’s not fair to judge a person’s abilities through a very limited number of tests, doesn’t he? 😃 ---
**Author:** ![Liberal](https://avatars.discourse-cdn.com/v4/letter/l/848f3c/32.png) [@Liberal](https://boards.straightdope.com/u/Liberal)\ **Post date:** [May 5, 2003, 1:23pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/47 "2003-05-05T13:23:00Z")
> [@](#): > > \*Originally posted by Princhester \* > \*\*Lib to understand the whole situation you have to read all the threads in which this ongoing debate with PM has been running. I can (perhaps) see how you get the impression you have just from this thread. But there have been several other threads. In those threads several posters have refuted every one of PM’s facts and arguments. > > And now he has switched to this thread. I’ll leave the reader to ponder why that would be. \*\* I’ll have to take your word for that, Princhester. And based on what I consider to be your reputation, I am more than willing to do that. Thanks for filling me in. ---
**Author:** ![glee](https://avatars.discourse-cdn.com/v4/letter/g/b5a626/32.png) [@glee](https://boards.straightdope.com/u/glee)\ **Post date:** [May 5, 2003, 1:25pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/48 "2003-05-05T13:25:36Z")
> [@](#): > > \*Originally posted by bnorton \* > I’ll try. One claim that I see recurring **(from Randi and his supporters)** is that because the psychics agree to the rules of the test, then the test therefore must be legitimate. However **psychics** have no basis for knowing whether the protocol for an experiment is scientifically valid or not. So having **psychics** claim, state, announce, acknowledge, etc. that an experiment is fair doesn’t hold scientific muster. I think you are muddling up two principles. 1. If a legal contract is involved, both sides should have legal advice. (Because the lawyers have the specific training to ensure the language and intent are correct in law). 2. If someone claims to have extraordinary abilities, they should agree that a test of their abilities is fair beforehand. Otherwise they can claim afterwards that the test wasn’t fair. What Randi does is to ask people exactly what they can do. (If they claim to be able to detect water in pipes, fine.) Then Randi and the claimant agree how to test this. It seems obvious to me that both parties need to be involved. Randi checks that the test can’t be easily beaten by chance or cheating; the claimant that it is a fair test of his abilities. Given that the Randi Foundation offers $1,000,000 for a successful paranormal demonstration, I think he is to be applauded for his efforts. ---
**Author:** ![TVAA](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TVAA](https://boards.straightdope.com/u/TVAA)\ **Post date:** [May 5, 2003, 1:26pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/49 "2003-05-05T13:26:25Z")
That’s why the board allows us to search by member! 🙂 [James Randi - sincere skeptic or money-grubbing fraud?](http://boards.straightdope.com/sdmb/showthread.php?s=&threadid=178611) The originator thread: [Does dowsing for water really work?](http://boards.straightdope.com/sdmb/showthread.php?&threadid=170935) ---
**Author:** ![glee](https://avatars.discourse-cdn.com/v4/letter/g/b5a626/32.png) [@glee](https://boards.straightdope.com/u/glee)\ **Post date:** [May 5, 2003, 1:42pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/50 "2003-05-05T13:42:43Z")
> [@](#): > > \*Originally posted by Libertarian \* > Peter Morris is right. His analysis is excellent, and he almost manages to make materialism seem less mystical. Not only has he pointed out the flaws in Randi’s tests, but his critics have compounded the logical errors about which Morris complains. For example, one writer actually equated scientific examinations of falsifiablity with classroom examiniations of subject mastery. Another dismissed Auther Clarke’s statements by painting him as a “dreamer”. And yet another opined that psychics ought to be subject to a different (more stringent) criteria set when their claims are examined. We have equivocation, ad hominem, and a poisoned well in just a few strokes. > > Hang in there, Peter Morris. You have not yet encountered any substantive refutation. I agree that some counter comments in this thread have been ill-advised. I would point out that dowsers claim both high success rates and a wide range of abilities. I have corresponded with a member of the British Society of Dowsers who told me that: - “we all have it in us to learn to dowse” - “it is possible to ask the dowsing rod to point out a direction or something, whether it is True North or the direction of my sister’s house some 400 miles away” - “a list of what you can find with dowsing … water, mine shafts, oil, ore loads, old gas pipes, electricity and phone cables, cellars, capped wells, water mains, voids. Dowsing can be used for healing work … house/property healing, clearing of psychic activity/geopathic + negative energies. Dowsing can find missing people or animals. And many more.” “I will happily give (paid) instruction to anyone … but I will not take part in any form of experimentation or other tests to see if the results are repeatable over X number of experiments.” Asked why he didn’t collect Randi’s $1,000,000: “The short answer is that $1,000,000 does not attract me at all - I prefer to remain true to my craft and myself.” When you combine this amazing series of claims with a point blank refusal to be tested, even with a million dollar reward, I get very sceptical. I found the inaccuracies pointed out by **patchbunny** in **peter morris’s** recollection to be equally telling. ---
**Author:** ![Pabitel](https://avatars.discourse-cdn.com/v4/letter/p/bbce88/32.png) [@Pabitel](https://boards.straightdope.com/u/Pabitel)\ **Post date:** [May 5, 2003, 1:58pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/51 "2003-05-05T13:58:18Z")
> [@](#): > > \*Originally posted by Little Nemo \* > \*\*In every case I’ve heard of, James Randi sets up a test under controlled conditions and defines what will be considered a success. The test subject than has an opportunity to agree to the conditions, including the definition of success, or take a pass. But if they accept the test, they hardly have a basis afterwards for claiming it wasn’t what they thought it should be. \*\* A point that has not yet been brought up is that the applicant is left with a false choice. The scenario goes like this. Someone approaches Randi with a claim. Randi prepares a test. The applicant can either agree to take the test or refuse. If the applicant at this point feels the test is unfair he has two options. Option 1 : take the test and probably fail. Result 1 : Be labled a fraud by Randi Option 2 : Refuse the test. Result 2 : Be labled a fraud by Randi. So what should the applicant do? Refuse the unfair test and have 100% chance of being labled a fraud or take the test that they suspect is unfair and hope that the unfairness does not effect their ability to succeed? ---
**Author:** ![Summertime](https://avatars.discourse-cdn.com/v4/letter/s/7ab992/32.png) [@Summertime](https://boards.straightdope.com/u/Summertime)\ **Post date:** [May 5, 2003, 2:03pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/52 "2003-05-05T14:03:26Z")
If they were legit, they wouldn’t fail. Simple as that. ---
**Author:** ![Liberal](https://avatars.discourse-cdn.com/v4/letter/l/848f3c/32.png) [@Liberal](https://boards.straightdope.com/u/Liberal)\ **Post date:** [May 5, 2003, 2:22pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/53 "2003-05-05T14:22:35Z")
No, I’m afraid it’s not that simple. But at least you have pointed out what is likely the basis of the many logical fallacies raised in arguments here. There is a fundamental difference between a claim of psychic ability and a claim of being an omnipotent god. ---
**Author:** ![TVAA](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TVAA](https://boards.straightdope.com/u/TVAA)\ **Post date:** [May 5, 2003, 2:24pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/54 "2003-05-05T14:24:23Z")
I believe Randi is open to negotiation as to the exact method of the test. He _is_ quite able to screen out potential tests that are open to abuse… but I don’t think he’s refused many reasonable objections. It would actually help your position if you could offer an example of Randi being unreasonable, so: cite? ---
**Author:** ![patchbunny](https://avatars.discourse-cdn.com/v4/letter/p/dc4da7/32.png) [@patchbunny](https://boards.straightdope.com/u/patchbunny)\ **Post date:** [May 5, 2003, 2:25pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/55 "2003-05-05T14:25:57Z")
> [@](#): > > \*Originally posted by Degrance \* > \*\*A point that has not yet been brought up is that the applicant is left with a false choice. > > The scenario goes like this. > > Someone approaches Randi with a claim. Randi prepares a test. The applicant can either agree to take the test or refuse. If the applicant at this point feels the test is unfair he has two options. > > Option 1 : take the test and probably fail. > > Result 1 : Be labled a fraud by Randi > > Option 2 : Refuse the test. > > Result 2 : Be labled a fraud by Randi. > > So what should the applicant do? Refuse the unfair test and have 100% chance of being labled a fraud or take the test that they suspect is unfair and hope that the unfairness does not effect their ability to succeed? \*\* Except that the claimant is allowed to make input into the test design. From the terms of the challenge ([http://www.randi.org/research/challenge.html](http://www.randi.org/research/challenge.html)): "Applicant must state clearly in advance, and applicant and JREF will agree upon, what powers or abilities will be demonstrated, the limits of the proposed demonstration (so far as time, location and other variables are concerned) and what will constitute both a positive and a negative result. This is the primary and most important of these rules. " If Randi were simply stating “it’s my way or the highway” you’d have a better point, but he’s not. Both sides work out what will be considered a fair test. –Patch ---
**Author:** ![glee](https://avatars.discourse-cdn.com/v4/letter/g/b5a626/32.png) [@glee](https://boards.straightdope.com/u/glee)\ **Post date:** [May 5, 2003, 2:31pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/56 "2003-05-05T14:31:11Z")
> [@](#): > > \*Originally posted by Degrance \* > A point that has not yet been brought up is that the applicant is left with a false choice. > > The scenario goes like this. > > Someone approaches Randi with a claim. Randi prepares a test. The applicant can either agree to take the test or refuse. If the applicant at this point feels the test is unfair he has two options. > > Option 1 : take the test and probably fail. > > Result 1 : Be labled a fraud by Randi > > Option 2 : Refuse the test. > > Result 2 : Be labled a fraud by Randi. > > So what should the applicant do? Refuse the unfair test and have 100% chance of being labled a fraud or take the test that they suspect is unfair and hope that the unfairness does not effect their ability to succeed? The reason this has not been brought up is that **it doesn’t happen.** 1. Someone approaches Randi with a claim. 2. \*\* The claimant and Randi\*\* agree on a test. See [www.randi.org](http://www.randi.org) for full details of the million dollar challenge. And what has **always** happened thereafter: 1. The claimant takes the test. 2. He fails. He promptly claims that: - there were negative vibrations from non-believers - he wasn’t feeling well - he misunderstood the test conditions - his powers don’t work if money is involved - the sun was in his eyes - Randi isn’t fair ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [May 5, 2003, 2:33pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/57 "2003-05-05T14:33:22Z")
How about option 3? Pass the test by not being a fraud. That is, after all, what the applicants are claiming is the case. ---
**Author:** ![glee](https://avatars.discourse-cdn.com/v4/letter/g/b5a626/32.png) [@glee](https://boards.straightdope.com/u/glee)\ **Post date:** [May 5, 2003, 2:50pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/58 "2003-05-05T14:50:07Z")
> [@](#): > > \*Originally posted by Libertarian \* > But at least you have pointed out what is likely the basis of the many logical fallacies raised in arguments here. There is a fundamental difference between a claim of psychic ability and a claim of being an omnipotent god. I assume you mean that Randi sets the qualification standard too high. I don’t agree, partly because he is offering $1,000,000! If I say that I can psychically affect the results of a coin toss so that more heads come down than tails (and people have claimed this would be evidence of a psychic ability), how many coin tosses would you set in the test? And how many more heads thatn tails would you want? Remember, this is for proof of a pwer that gets you $1,000,000. Also take this real example. I’m an internationally rated chess player. I can play two games of chess simultaneously blindfold. I do make an occasional illegal move (I would estimate about 1% of the time), but I can regularly beat club players who are using a set and board. Is this a psychic power? No, but it’s damned impressive! It took me about 3 years to learn how to do it (plus the 5 years to become proficient at chess). I could teach someone with a proven aptitude for chess how to analyse mentally, though I’m not sure if they could do a full game let alone 2. My point is that I can perform this mental feat almost at will. Provided I’m not tired, nor concentrating on something else, then I will satisfy Randi’s test rules. (If there is an attractive interested member of the opposite sex present, then I will certainly do well!) Now psychics claim they have reliable powers. They offer their service for money. Why can’t they achieve the same results that I can? ---
**Author:** ![Patty\_O\_Furniture](https://avatars.discourse-cdn.com/v4/letter/p/96bed5/32.png) [@Patty\_O\_Furniture](https://boards.straightdope.com/u/Patty_O_Furniture)\ **Post date:** [May 5, 2003, 2:51pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/59 "2003-05-05T14:51:00Z")
Applicants and their apologists will probably say (afterwards) that the test was so biased or poorly constructed (even though they said beforehand that is was fair) that there was no chance of it being able to verify their “genuine” abilities. ---
**Author:** ![Patty\_O\_Furniture](https://avatars.discourse-cdn.com/v4/letter/p/96bed5/32.png) [@Patty\_O\_Furniture](https://boards.straightdope.com/u/Patty_O_Furniture)\ **Post date:** [May 5, 2003, 2:58pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/60 "2003-05-05T14:58:10Z")
That was in response to why **Little Nemo’s** option 3 was missing from **Degrance’s** post. [Previous page](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746.md?page=2) [Next page](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746.md?page=4)