# The flaws in Randi's tests **URL:** \ **Category:** Great Debates\ **Created:** [May 2, 2003, 11:23pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746 "2003-05-02T23:23:33Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![TVAA](https://avatars.discourse-cdn.com/v4/letter/t/c57346/32.png) [@TVAA](https://boards.straightdope.com/u/TVAA)\ **Post date:** [May 3, 2003, 3:57pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/21 "2003-05-03T15:57:42Z")
Randi demonstrated that the claims of accuracy of individuals meant nothing to whether astrological claims are significant or not. It seems rather conclusive evidence that general opinion is not equivalent to scientific verification. ---
**Author:** ![Little\_Nemo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/little_nemo/32/3120_2.png) [@Little\_Nemo](https://boards.straightdope.com/u/Little_Nemo)\ **Post date:** [May 3, 2003, 4:09pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/22 "2003-05-03T16:09:33Z")
In every case I’ve heard of, James Randi sets up a test under controlled conditions and defines what will be considered a success. The test subject than has an opportunity to agree to the conditions, including the definition of success, or take a pass. But if they accept the test, they hardly have a basis afterwards for claiming it wasn’t what they thought it should be. ---
**Author:** ![aramis](https://avatars.discourse-cdn.com/v4/letter/a/e68b1a/32.png) [@aramis](https://boards.straightdope.com/u/aramis)\ **Post date:** [May 3, 2003, 4:43pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/23 "2003-05-03T16:43:40Z")
I just want to add a little clarification here. **peter morris** has appealed to authority several times (and been called on it several times too) by citing Arthur C. Clarke as an “internationally reknowned scientist.” I have a lot of respect for Clarke and I’d agree that he’s a bright guy, but he hasn’t been what you’d call a working scientist for a long time. He is by far best known – even internationally renowned – as a science fiction writer. He also has some recognition as host of a short-lived TV series _Arthur C. Clarke’s Mysterious World_. ---
**Author:** ![Mr.Miskatonic](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mr.miskatonic/32/310_2.png) [@Mr.Miskatonic](https://boards.straightdope.com/u/Mr.Miskatonic)\ **Post date:** [May 3, 2003, 4:44pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/24 "2003-05-03T16:44:50Z")
> [@](#): > > \*Originally posted by bnorton \* > \*\* > One test that I recall (sorry, no cite - I’ll see if I can find something later), Randi gave a group of students an astrolgical reading telling them that each reading had been prepared for each individual student by an astrologer. After the students had read their “personal” reading they were asked how closely it came to describing their personality. For the most part they said that they were amazed how accurate the readings were. At that point Randi disclosed that all the readings were identical. > > Randi went on to claim that that test debunked astrological claims. The problem that I have with the experiment is that it really didn’t test what Randi was claiming it did. If he claimed that people are gullible when it comes to vague descriptions of who they are, then fine, but it really wasn’t putting astrology _per se_ to the test. > \*\* [That’s not what Randi was claiming.](http://www.randi.org/jr/041803.html). The event you describe was in the Nova special and he states clearly in that special and in the above link: > [@](#): > > Here, the author destroys a straw man he’s set up because he either doesn’t see, or chooses not to see, the intent behind the demonstration I’ve often conducted concerning astrology – a demo that is not, and was not designed to be, a test of whether astrology works. It simply shows that people will tend to accept almost anything, editing and selecting from otherwise meaningless data that which suits their needs and preferences. In fact, on the NOVA program that was done 10/18/93, and on which that demo was featured, I distinctly stated that the demo was not to “disprove” astrology, at all. 'nuff said. ---
**Author:** ![El\_Zagna](https://avatars.discourse-cdn.com/v4/letter/e/d2c977/32.png) [@El\_Zagna](https://boards.straightdope.com/u/El_Zagna)\ **Post date:** [May 3, 2003, 6:39pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/25 "2003-05-03T18:39:16Z")
> [@](#): > > _Originally posted by Mr. Miskatonic That’s not what Randi was claiming. The event you describe was in the Nova special and he states clearly in that special and in the above link._ It sounds like the NOVA program is what I’m talking about. If the intent of this particular experiment is only to show that people are gullible I wonder why he does it at all. No one doubts that people are gullible. At the same time there are so many easy ways to debunk astrology. For example you could give half the subjects “real” astrological readings and the other half generic ones and measure the levels of perceived accuracy between the two groups. What I would like to do here is subject some of Randi’s experiments to an SDMB “peer review”. Let’s look at his procedures on a specific experiment and see if they pass muster. One claim that I see recurring is that because the psychics agree to the rules of the test, then the test therefore must be legitimate. I don’t think that passes scientific muster. There is no reason to expect those claiming supernatural powers to be the least bit knowledgable of the scientific method. In a similar way I might sign a legal document stating that the terms of a contract are fair but since I have no legal training I wouldn’t really know if it was fair or not. ---
**Author:** ![Patty\_O\_Furniture](https://avatars.discourse-cdn.com/v4/letter/p/96bed5/32.png) [@Patty\_O\_Furniture](https://boards.straightdope.com/u/Patty_O_Furniture)\ **Post date:** [May 3, 2003, 7:29pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/26 "2003-05-03T19:29:29Z")
Then you need to hire a lawyer. That psychics claim scientific ignorance does not excuse them from passing its muster. ---
**Author:** ![El\_Zagna](https://avatars.discourse-cdn.com/v4/letter/e/d2c977/32.png) [@El\_Zagna](https://boards.straightdope.com/u/El_Zagna)\ **Post date:** [May 3, 2003, 7:37pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/27 "2003-05-03T19:37:42Z")
> [@](#): > > \*Originally posted by Attrayant \* > \*\*That psychics claim scientific ignorance does not excuse them from passing its muster. \*\* Totally unrelated to my point, which is that psychics have no basis for making that claim. ---
**Author:** ![Mr.Miskatonic](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mr.miskatonic/32/310_2.png) [@Mr.Miskatonic](https://boards.straightdope.com/u/Mr.Miskatonic)\ **Post date:** [May 3, 2003, 7:40pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/28 "2003-05-03T19:40:17Z")
> [@](#): > > \*Originally posted by bnorton \* > It sounds like the NOVA program is what I’m talking about. If the intent of this particular experiment is only to show that people are gullible I wonder why he does it at all. No one doubts that people are gullible. At the same time there are so many easy ways to debunk astrology. For example you could give half the subjects “real” astrological readings and the other half generic ones and measure the levels of perceived accuracy between the two groups. It was not to demonstrate that people are gullible. It was to demonstrate how people can fool themselves and find matches to vaugely worded statements. When I hear about the wonderful success rates of JOhn Edward and his ilk, this test inidicates just how willing people are to fit their lives to a simplistic description. > [@](#): > > What I would like to do here is subject some of Randi’s experiments to an SDMB “peer review”. Let’s look at his procedures on a specific experiment and see if they pass muster. Sure, but who would be the peers? Honestly I really doubt there’s much to object to, it is a very simple test, or the most part. Also, while Randi has no formal written review (which is really just gold plating since there’s no publications that write-up this sort of thing) he does have a scientists aroudn the world that are associates of the JREF. These scientists run the preliminary tests since they are more convenient to worldwide applicants than Randi would be. I have my doubts that these people would aid Randi if they found shortcomings in his tests. Its not peer review, I know, but it counts for something. > [@](#): > > One claim that I see recurring is that because the psychics agree to the rules of the test, then the test therefore must be legitimate. I don’t think that passes scientific muster. There is no reason to expect those claiming supernatural powers to be the least bit knowledgable of the scientific method. In a similar way I might sign a legal document stating that the terms of a contract are fair but since I have no legal training I wouldn’t really know if it was fair or not. \*\* The point of ‘agreed to’ has little to do with so much with the scientific method so much it does with the tester set his own standards of victory. An often leveled (and completely untrue) accusation thrown at Randi is that he unfairly demands applicants do things they do not claim, or set standards too high (i.e. the spoonbender has to bend the I-beam). It also prevents the unfair claim of either side andding winning test conditions after the test. Don’t laugh, one prominent parapsychologist had a spoonbender he was quite confident in, and had her take the challenge. The tester failed to bend the spoon, the agreed to condition of victory. So what does the parapsychologist do? He complains in a letter that randi didn’t test “loop density, electrical resistance, magnetic flux grain structure…” and a whole bunch of other things. (FLim Flam p. 219). That 's the point behind the “agreed to” part. Keep in mind the testee can always hvae soeone help them with testing conditions, and Randi has been remarkably flexible. Once he even used an electrical dowser’s (?!) own homemade device to test him. ---
**Author:** ![Scott\_Dickerson](https://avatars.discourse-cdn.com/v4/letter/s/f04885/32.png) [@Scott\_Dickerson](https://boards.straightdope.com/u/Scott_Dickerson)\ **Post date:** [May 3, 2003, 7:53pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/29 "2003-05-03T19:53:24Z")
Hi guys. I take it this is a continuation of the other thread on James Randi; which I would guess is a continuation of other threaded arguments that have been going on between our main actors since, what, the heyday of Eusapia Palladino? James Randi and at least some of his SICOP associates strike me as intellectually dishonest polemicists. To deconstruct that sentence: –I am not claiming to be able to “prove” my assertion, and I have no particular cite to offer. I don’t have to. Because I am discussing how these people STRIKE ME. –By “intellectually dishonest,” I do not mean, nor should I be taken to imply, that he has engaged in fraud, nor that he deliberately fudges or falsifies data, nor even that he intends to misrepresent the significance of his tests and investigations. A person can be both brilliantly intelligent and morally honest, and yet be, at the same time, “intellectually dishonest,” as it seems to me that term is commonly used. The notion is that he does not go to the trouble to set forth all the relevant premisses that underlie his approach, thus shielding that approach from critical challenge on those grounds. (For example, he evidently ignores or dismisses the presumption that the psychological “frame of mind” of any and all persons concerned with a given test–not just the designated test subject–may impact the outcome. This may indeed be “too convenient for believers;” but that fact, if it is a fact, cannot be used to justify ignoring the issue.) The flaw is of a logical nature; to proceed with the announcement of conclusions without having addressed such flaws is a form of intellectual dishonesty. –“Polemicist.” It’s hard for me to believe that anyone would disagree with this word. The whole SICOP endeavor appears to be about counteracting public sympathies toward the paranormal by means of what amounts to a propaganda campaign, dressed up (as such campaigns always are) in high-flown rhetoric about exposing fraud, teaching the public how to think logically–ultimately (given the relation of SICOP to the humanist association and the organized atheist movement by way of some key members and founders) about rolling-back public acceptance of mysticism and religiousity. (To note this latter element is not, of course, to endorse such public acceptance.) So now comes this guy Peter, making note of what seems to him to be logico-scientific shortcomings in JR’s methodology. These asserted shortcomings seem to me to fall across the normal (“bell-shaped”) distribution of shortcomings, from insignificant and inadvertent, to critical and chronic (NOT, necessarily, deliberate). They are refuted energetically; and the refutations again seem to me to fall all over the place, from “surely you jest” to “yep, you nailed that sucker.” Which returns me to my point in writing this post. James Randi and his SICOP colleagues are needed as an antidote to vast public credulity and sloppy thinking. Given the good they might do, I am all the more resentful of the “tone” of so much of their work–the wholesale belittling, blasting, ridiculing, and dismissing of “the other side” which wraps itself like a chador around such intellectually sound work as they may do. I’m afraid they’re fighting showbiz with showbiz, not delusion with logic. (Oh, and much as one might like it otherwise, it’s RENOWN, not “REKNOWN.”) ---
**Author:** ![Patty\_O\_Furniture](https://avatars.discourse-cdn.com/v4/letter/p/96bed5/32.png) [@Patty\_O\_Furniture](https://boards.straightdope.com/u/Patty_O_Furniture)\ **Post date:** [May 3, 2003, 8:09pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/30 "2003-05-03T20:09:19Z")
> [@](#): > > Totally unrelated to my point Then you need to be more clear. You said > [@](#): > > One **claim** that I see recurring is that because the psychics agree to the rules of the test, then the test therefore must be legitimate. Then you said > [@](#): > > [My point] is that psychics have no basis for making that **claim**. Psychics are not the ones making the **claim** (in your first quote). Which claim do psychics have no basis for making? I thought we were all pretty much agreed that they have no basis for most if not all of their claims, it’s just the testing protocols that seem to be the issue. That’s why I said science has to come into the picture as a fair and impartial referee. ---
**Author:** ![KidCharlemagne](https://avatars.discourse-cdn.com/v4/letter/k/ed8c4c/32.png) [@KidCharlemagne](https://boards.straightdope.com/u/KidCharlemagne)\ **Post date:** [May 3, 2003, 8:39pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/31 "2003-05-03T20:39:02Z")
I’m with ya Peter [http://boards.straightdope.com/sdmb/showthread.php?threadid=162259&highlight=randi](http://boards.straightdope.com/sdmb/showthread.php?threadid=162259&highlight=randi) ---
**Author:** ![hansel](https://avatars.discourse-cdn.com/v4/letter/h/6bbea6/32.png) [@hansel](https://boards.straightdope.com/u/hansel)\ **Post date:** [May 3, 2003, 9:47pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/32 "2003-05-03T21:47:26Z")
> [@](#): > > \*Originally posted by bnorton \* > \*\*It sounds like the NOVA program is what I’m talking about. If the intent of this particular experiment is only to show that people are gullible I wonder why he does it at all. No one doubts that people are gullible. \*\* The demonstration is an answer to the psychic’s counter-question “if astrology is bunk, then how come so many people agree with astrology’s analysis of their characters?”, which implies the argument that the correspondence between it’s perceived success and it’s actual success demonstrates the validity of the astrological method. The demonstration provides an empirical method of showing an alternative mechanism for the perceived success of astrology that doesn’t depend upon the soundness of actual astrological methods. ---
**Author:** ![Shade](https://avatars.discourse-cdn.com/v4/letter/s/2bfe46/32.png) [@Shade](https://boards.straightdope.com/u/Shade)\ **Post date:** [May 3, 2003, 9:49pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/33 "2003-05-03T21:49:07Z")
**Re: % success dowsing** Let me give an example. Bob claims he can tell which of 10 pipes has water flowing through 90% of the time. So we let him try ten times. If he is right, he will almost certainly get it right 7 or 8 times at least. The chances of him guessing this is miniscule. If he gets this many right he is probably a dowser. But say he gets 3 right. The chances of him guessing this well is about 1/3. I hope nobody pretends that something that something happening that happens one time in three is implausible. _This tells us nothing because he could easily have obtained these results by chance_ If he had told us to start with he had a 30% success rate we would have tested him 100 times, and if he had got 30 right, that might have meant something. This is a hypothetical example, but I think it illustrates the idea (and anyone who wants accurate numbers should google binomial distribution 😛 ) ---
**Author:** ![Shade](https://avatars.discourse-cdn.com/v4/letter/s/2bfe46/32.png) [@Shade](https://boards.straightdope.com/u/Shade)\ **Post date:** [May 3, 2003, 9:51pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/34 "2003-05-03T21:51:01Z")
OTOH I appreciate that if I could dowse (psychically) with accuracy only slightly greater than chance, it would be hard to persuade Randi to put in the time and effort to creating a big enough test to establish this one way or the other. I think my approach would be to find a local respected sceptic, convince them, get them to approach someone more widely known. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [May 3, 2003, 10:17pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/35 "2003-05-03T22:17:02Z")
> [@](#): > > _from [Arthur C. Clarke Unauthorized Homepage](http://www.lsi.usp.br/~rbianchi/clarke/)_ > Arthur C. Clarke is one of the most celebrated science fiction authors of our time. He is the author of more than sixty books with more than 50 million copies in print, winner of all the field’s highest honors. He was named Grand Master by the Science Fiction Writers of America in 1986. His numerous awards include the 1962 Kalinga prize for science writing, …[etc.] While I have a great deal of respect for Mr. Clarke, and I’m willing to admit he may have been the first to have thunk up the geostationary satellite idea, he is more of an intelligent dreamer than a scientist. I have been appalled to see him host and/or appear on TV “Unsolved Mysteries”-type shows that have 0% scientific basis. An authority on dowsing, I don’t think so. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [May 3, 2003, 10:25pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/36 "2003-05-03T22:25:33Z")
**Scott Dickerson,** FYI, the organization known as the **C** ommittee for the **S** cientific **I** nvestigation of **C** laims **O** f the **P** aranormal is spelled [**CSICOP.**](http://www.csicop.org/) And Mr. Randi no longer has any formal connection with that organizaion. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [May 4, 2003, 4:47am UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/37 "2003-05-04T04:47:35Z")
I am now going to prove something. I flipped a quarter 20 times (really, I did) and it came up thusly, where H = heads, T = tails: H H T H T H H H H T T H T T H T T H T H Summing up, we have 11 heads, 9 tails. It was heads 55% of the time, not far from chance. But wait! At #6 thru #10, (H H H H T) there were 4 heads out of 5 flips, or 80% heads! What a remarkable result! I have just proved that I can influence the outcome of coin flips with my mind!! Ridiculous? No more than the dowsers in the above, much-discussed Australian experiment. Note that I didn’t tell you what I was going to prove until AFTER the test and until I did some data mining, discarding the trials I didn’t want to use. The Australian dowsers did exactly the same thing: > [@](#): > > _from Mike Hutchinson’s sci.skeptic newsgroup post_ > > There were 26 attempts to dowse for brass with none successful > > There were 35 attempts to dowse for gold, with four successful > > Fifty tries at water dowsing achieved eleven successes. If we discard the brass & gold tests, and keep the wet ones, the success rate looks better (not much, it’s still abysmal, but better). **But this is not what was agreed upon in advance, but what was “mined” from the data later.** _You can always make your results look better if you discard your failures and point to your successes._ Putoff & Targ ran afoul of this in their PSI tests long ago. They discarded all the “inconclusive” runs and kept the “interesting” ones. But this is not how science is done. ALL the test results must be used for analysis, or the data is less than worthless; it is fraudulent. > [@](#): > > …The dowsers went off to the bar, already speculating about what adverse situation had stymied the tests - perhaps it was the walkie-talkie radios or the television crews… Or perhaps their claimed ability didn’t exist, they’d been fooling themselves, and this was the first _really fair test they’d ever had._ ---
**Author:** ![patchbunny](https://avatars.discourse-cdn.com/v4/letter/p/dc4da7/32.png) [@patchbunny](https://boards.straightdope.com/u/patchbunny)\ **Post date:** [May 4, 2003, 11:47pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/38 "2003-05-04T23:47:33Z")
> [@](#): > > \*Originally posted by peter morris \* > [B Failure to follow double-blind procedures: in one test Randi showed a table with a number of items on it. The items were carpentry and DIY tools. He told the audience that four of the items had been involved in a crime, then invited a psychic to divine which ones. The psychic picked four items. Then Randi smugly informed her that all her choices were wrong. He had been in her presence throughout the test. > > ``` > Failure to give the audience full information. In the above test he never bothered to tell us which items were correct, what the crime was, what the involvement of the objects, or anything. This matters. Did the psychic fail to detect the saw that Randi bought from a hardware shop on a Sunday (which was actually a crime in the UK at the time), or did she fail to detect the knife that was a murder weapon? This is a significant detail, that Randi didn’t bother telling us.** > > ``` Heck, if it’s been ten years, perhaps you should try reviewing the show again before posting. _James Randi: Psychic Investigator_ came out in book form after the series ended, where he described what occurred during the series. To correct your comments: > [@](#): > > _ **He told the audience that four of the items had been involved in a crime** _ Wrong. Randi told the audience that “any or all of [these] could have been connected with a serious crime involving loss of life.” > [@](#): > > \*\*\* The psychic picked four items.\*\*\* Wrong. The psychic, Nella Jones, picked three items. > [@](#): > > _ **In the above test he never bothered to tell us which items were correct, what the crime was, what the involvement of the objects, or anything. This matters** _ It does? Perhaps that’s why he told the audience the background of the items Nella Jones chose. Nella Jones picked the following, with the following explanations of their use: 1. a ‘Waiter’s Helper’, possibly used to pick a lock 2. a carpenter’s hammer, possibly used to break a window to gain entry 3. a hand axe, possibly used to puncture a vehicle’s tire. I note with interest that no saw what chosen, as was Peter’s claim. From the book: ``` “I asked Nella if she had anything more to say about the items she had Chosen. She had, after all, been asked to identify which, if any, of the items might have been involved in a homicide. I then opened the sealed statement and read to her and our audience the documentation on the objects she had chosen: ‘Waiter’s helper’ folding bar utensil, made in Italy, purchased in Manchester, February 1991 by James Randi. Unused for any purpose. History: Direct from factory to retailer to purchaser. ‘Carpenter’s hammer’, purchased in Manchester, February 1991 by James Randi. Unused for any purpose. History: Direct from factory to retailer to purchaser. Hand axe, of the type employed by firemen. In October of 1979, in Manchester, two men and a woman murdered Mr John monk with this axe. They were tried on May 16, 1980 at Manchester Crown Court, found guilty, and sentenced to life imprisonment." ``` > [@](#): > > \*\*He was too quick to jump to conclusions: he set the subjects just one test, and if they failed he claimed this proved they were frauds or deluded fools. > > He was inconsistent. During the series I recall only one person that passed his test. This was a map dowser, who correctly divined the location of an object on a map divided into about 100 squares. Randi congratulated him, and then he claimed that it was probably just a lucky guess, and one test proves nothing. In all the other tests, Randi was happy to say that one test proved something. \*\* Forgive me if at this point I’m somewhat skeptical about your recollections. Yes, the map dowser, Michael Cook, correctly located the map square containing the abbey, a 1-in-24 chance. As Randi also notes: 1. While Mr. Cook was at the studio, the map with the marked abbey location was located outdoors for a time, 2. While Mr. Cook was at the studio, stage rehearsals were conducted for a successful test result, and the map was visible. Randi notes that he has no proof that Mr. Cook cheated, and states that “As far as I am concerned, Mr Cook’s performance on our programme was the result of either his skill as a map dowser, or a remarkable 1-in-twenty-four coincidence.” I personally, would like to see Mr. Cook tested further to resolve the issue, but having seen enough failures, I’m rather skeptical of a positive result. [QUOTE] **He frequently changed the premises of the test, testing people for thing other than what they said they could do. If someone says they can detect an underground river, he will invite them to detect a four-inch pipe, that sort of thing.** [QUOTE] What, this argument again? 1. The testee agreed that they can detect water flowing through an underground pipe. 2. The testee signed a statement agreeing that this is, indeed, within their abilities. 3. The testee demonstrated 100% accuracy when they know exactly where the pipe is located. 4. The testee’s success rate fell to that of chance when they _didn’t_ know the location of the pipe. You see (and you obviously don’t), drilling for natural water can be expensive (don’t just drill where they say there’s water, drill where they say there is none to check). It is also fairly easy to gather groundwater characteristics for a region, which opens the door to cheating. So do something simpler, like testing pipe flow. If you want to check out what’s happening with groundwater, while the dowsers are at the site, ask them if there’s any underground rivers at that site. Remember: it’s in poor taste to giggle audibly when comparing the dowser’s maps. Please show us **one** instance where Randi has demanded that a claimant demonstrate an ability that the claimant admits they don’t have. [QUOTE] \*\*In short, I didn’t think that the tests were fair, or had any scientific validity to them. Not that I believed the claims of the psychics he tested at all, in fact most of them seemed crazy. ``` The biggest problem with Randi’s tests is that he tests intensively, rather than extensively. That is he gives a very small number of tests, and demands a perfect or near perfect score. A good example may be found here: http://www.skeptics.com.au/journal/divining.htm Note the conditions of the test. Subjects were given a choice, 5 or 10 tries and no more. This limit was set by Randi, take it or leave it. He would have liked to have done more, but was sure that his subjects wouldn’t have the patience. Setting the 10 tries was entirely for their benefit, whether they liked it or not. How considerate of him.** ``` [QUOTE] You just contradicted yourself, and made an incorrect claim. The subjects had a choice of 5 or 10 tries, and the decision was up to the _claimant_, not Randi. Try reading [http://www.skeptics.com.au/journal/divining.htm](http://www.skeptics.com.au/journal/divining.htm) once again. [QUOTE] **According to [http://skepdic.com/dowsing.html](http://skepdic.com/dowsing.html) the pass mark was 80%, i.e. subjects had to get 4 hits out of 5 tries to win the prize.** [QUOTE] No, the pass mark was that stated by the claimant. Why you’re mentioning, the Skeptic’s Dictionary article is beyond me. [QUOTE] **If they managed to get that, they had to do the test a second time in order to claim. If someone had got 4 out of 5 first, but only 3 out of 5 the second time, that would have been a failure, by Randi’s standards.** [QUOTE] Wrong again. Are you _sure_ you read the Australian article? From the Rules for Test (from [http://www.skeptics.com.au/journal/divining.htm](http://www.skeptics.com.au/journal/divining.htm)): ``` 9) The right is reserved to ask the claimant to repeat—once—the same set of tests in order to establish a more satisfactory statistical sample or to ensure security of the protocol , at the discretion of Mr Randi, There is **very little** possibility that this rule will be invoked. ``` (Emphasis Randi’s) [QUOTE] \*\*Given the nature of the test, a 1 in 10 shot, you would expect success in the region of 10%, give or take a percent. In fact, in the water test, subjects scored 22%. This fell far short of the requirements for Randi’s prize, but on the other hand is more than double the expected chance result. \*\* [QUOTE] Not if the claimant only made 5 runs (which some did), in which case the expected result is 20%. > [@](#): > > \*\*I am not the only one who sees these flaws in Randi’s testing methods and conclusions. See for example: [http://www.tricksterbook.com/ArticlesOnline/Dowsing.htm](http://www.tricksterbook.com/ArticlesOnline/Dowsing.htm) \<\< James Randi (1979), professional magician and member of the Committee for the Scientific Investigation of Claims of the Paranormal, conducted a test with four dowsers in Italy. Procedures were spelled out in detail prior to the test and agreed upon by the dowsers. The dowsers were asked to locate three buried pipes with running water and to place pegs over the route of the pipes. As stated by Chamberlin (1980), the test had several deficiencies. No meaningful statistical evaluation was possible. Even if the dowsers had been quite close, they were unlikely to fulfil the requirements for a successful test (they were required to place the pegs in a strip eight inches wide). None of them was able to claim Randi’s $10,000 reward. The test contributed little knowledge to the scientific community. \>\> Note that this is not the opinion of a dowsing believer, it is a scientific review of dowsing tests. It is equally critical of many studies that find evidence of dowsing. But this is a frequent response when real scientists take a good hard look at Randi’s tests. \*\* Gosh, to repeat myself again. 1. The claimant agreed that they can detect water flowing through an underground pipe (one of them even bragged about it) 2. The claimant signed a statement agreeing that this is, indeed, within their abilities. 3. The t claimant demonstrated 100% accuracy when they knew exactly where the pipe was located. 4. _They couldn’t find it when they didn’t_ To correct the article, to be successful the following conditions had to be met: 1. The claimant has to place 2/3 of their pegs within 10 cm of the pipe 2. The claimant each had three trials to locate a pipe. Two of the three trials had to be successful I’ve looked at the claimant’s final maps . It wouldn’t have mattered if they had to put the pegs within 3 feet of the pipeline. They still would have failed abysmally. Try again. –Patch ---
**Author:** ![patchbunny](https://avatars.discourse-cdn.com/v4/letter/p/dc4da7/32.png) [@patchbunny](https://boards.straightdope.com/u/patchbunny)\ **Post date:** [May 4, 2003, 11:49pm UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/39 "2003-05-04T23:49:05Z")
Aw, crap. It figures I blew the formatting in there somewhere. 😛 –Patch ---
**Author:** ![El\_Zagna](https://avatars.discourse-cdn.com/v4/letter/e/d2c977/32.png) [@El\_Zagna](https://boards.straightdope.com/u/El_Zagna)\ **Post date:** [May 5, 2003, 1:37am UTC](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746/40 "2003-05-05T01:37:50Z")
> [@](#): > > \*Originally posted by Attrayant \* > \*\*Then you need to be more clear. \*\* I’ll try. One claim that I see recurring **(from Randi and his supporters)** is that because the psychics agree to the rules of the test, then the test therefore must be legitimate. However **psychics** have no basis for knowing whether the protocol for an experiment is scientifically valid or not. So having **psychics** claim, state, announce, acknowledge, etc. that an experiment is fair doesn’t hold scientific muster. [Previous page](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746.md?page=1) [Next page](https://boards.straightdope.com/t/the-flaws-in-randis-tests/172746.md?page=3)