I am a sceptic, let me make that clear from the start. I do not really believe in dowsing (except- possibly- by the getting-clues-from-the –local-terrain method, and I’m even doubtful of that.) If someone wishes to claim they can dowse, its up to them to prove it. I am open minded enough to allow a fair test, but really James Randi is not the man to test their claims. Here’s why.
I first saw Randi on his show on British TV about 10 years ago. It was called “James Randi, Psychic Detective” I watched it, hoping to see some scientific tests of psychic abilities. What I actually saw was fundamentally wrong on so many levels. Here are some of the things that struck me:
The bias of the tester: Any test where the tester has made up his mind as to the result beforehand is seriously flawed.
Failure to follow double-blind procedures: in one test Randi showed a table with a number of items on it. The items were carpentry and DIY tools. He told the audience that four of the items had been involved in a crime, then invited a psychic to divine which ones. The psychic picked four items. Then Randi smugly informed her that all her choices were wrong. He had been in her presence throughout the test.
Failure to give the audience full information. In the above test he never bothered to tell us which items were correct, what the crime was, what the involvement of the objects, or anything. This matters. Did the psychic fail to detect the saw that Randi bought from a hardware shop on a Sunday (which was actually a crime in the UK at the time), or did she fail to detect the knife that was a murder weapon? This is a significant detail, that Randi didn’t bother telling us.
He was too quick to jump to conclusions: he set the subjects just one test, and if they failed he claimed this proved they were frauds or deluded fools.
He was inconsistent. During the series I recall only one person that passed his test. This was a map dowser, who correctly divined the location of an object on a map divided into about 100 squares. Randi congratulated him, and then he claimed that it was probably just a lucky guess, and one test proves nothing. In all the other tests, Randi was happy to say that one test proved something.
He frequently changed the premises of the test, testing people for thing other than what they said they could do. If someone says they can detect an underground river, he will invite them to detect a four-inch pipe, that sort of thing.
In short, I didn’t think that the tests were fair, or had any scientific validity to them. Not that I believed the claims of the psychics he tested at all, in fact most of them seemed crazy.
The biggest problem with Randi’s tests is that he tests intensively, rather than extensively. That is he gives a very small number of tests, and demands a perfect or near perfect score. A good example may be found here: http://www.skeptics.com.au/journal/divining.htm Note the conditions of the test. Subjects were given a choice, 5 or 10 tries and no more. This limit was set by Randi, take it or leave it. He would have liked to have done more, but was sure that his subjects wouldn’t have the patience. Setting the 10 tries was entirely for their benefit, whether they liked it or not. How considerate of him. According to http://skepdic.com/dowsing.html the pass mark was 80%, i.e. subjects had to get 4 hits out of 5 tries to win the prize. If they managed to get that, they had to do the test a second time in order to claim. If someone had got 4 out of 5 first, but only 3 out of 5 the second time, that would have been a failure, by Randi’s standards.
Given the nature of the test, a 1 in 10 shot, you would expect success in the region of 10%, give or take a percent. In fact, in the water test, subjects scored 22%. This fell far short of the requirements for Randi’s prize, but on the other hand is more than double the expected chance result. The probability of this happening by chance is less than 1%. In fact, this would happen only once in 107 tests. Now this is an interesting result, its high enough to be statistically significant, but low enough to be a fluke. I think it probably WAS a fluke. A person genuinely interested in testing dowsing ability would have repeated the test, several times. If the subjects had scored 20% or more on ALL the tests, then that would exclude fluke. But Randi never does that.
To get a group score of 22%, one individual must have got 30% or more. Just imagine for a moment that this is a genuine, albeit limited psychic ability. Imagine that he could score 30% every time he took the test. Could he pass Randi’s test? Could he get 4 out of 5, twice running? On his best day he would have to be very, very lucky indeed to pass.
Randi fails to give us the individual scores for the dowsers: he only gives us the group total. It might be that all subjects got 2 out of 10, and one got three out of 10, but on the other hand it might be that one got zero, while anothern got 5. I want to know what the highest score was. Randi chooses not to share the information.
In his commentary on the test Randi includes two false arguments. First of all, he lists the dowsers’ expectation of their scores, and points out that they had predicted an 86% success rate, and fell short of it. Yes, they were over-optimistic about their own level of skill, but this does not mean a thing. The test was whether they could detect water, not whether they could judge their own abilities. The fact that they did less well than they thought does not change the fact that they did a lot better than chance.
The second is worse. There is a story of the man who put one hand in the freezer, the other hand in the oven, and said that on the whole he felt fine. That man might have been Randi. Randi mentions that, oh by the way, he also ran two other tests, a test for gold dowsing, and a test for brass dowsing. Randi takes the 0% score on the brass test, adds it to the 22% score on the water test, and declares an overall result of 13% for the three tests together. A test well below chance plus a test well above chance on the whole make <<13.5%, a figure well within expectation.>> according to Randi. This is a mathematical absurdity. These were separate tests, for a different thing, taken by entirely different groups of people. Only TWO people were involved in more than one test. They have nothing to do with each other. Combining the scores gives no meaningful result. The result in the water test was 22%, not the 13% Randi implies.
Randi uses the test results to support his anti-dowsing assumptions. This is wrong. The 22% result in the water test may be a fluke, or it might be an indication of some slight dowsing ability. It certainly does not provide evidence against dowsing, in fact it gives slight evidence in its favour.
Randi’s conclusions are certainly flawed. This is not to say that he should play no part in the testing. Certainly conjurers sometimes cheat in these tests. A magician like Randi has the skill to identify the cheats. But the actual design of the test, and the conclusions from the results should be left to people who know what they’re talking about.
I am not the only one who sees these flaws in Randi’s testing methods and conclusions. See for example: http://www.tricksterbook.com/ArticlesOnline/Dowsing.htm << James Randi (1979), professional magician and member of the Committee for the Scientific Investigation of Claims of the Paranormal, conducted a test with four dowsers in Italy. Procedures were spelled out in detail prior to the test and agreed upon by the dowsers. The dowsers were asked to locate three buried pipes with running water and to place pegs over the route of the pipes. As stated by Chamberlin (1980), the test had several deficiencies. No meaningful statistical evaluation was possible. Even if the dowsers had been quite close, they were unlikely to fulfil the requirements for a successful test (they were required to place the pegs in a strip eight inches wide). None of them was able to claim Randi’s $10,000 reward. The test contributed little knowledge to the scientific community. >> Note that this is not the opinion of a dowsing believer, it is a scientific review of dowsing tests. It is equally critical of many studies that find evidence of dowsing. But this is a frequent response when real scientists take a good hard look at Randi’s tests.