I was having that discussion yet again about homeopathy with someone who swears it works. So I explained about placebo effect and double-blind studies.
Then this guy says something that I’ve heard a few times: “Well, there are some double-blind studies that show that a homeopathic medicine worked.”
I tell him I’d be (mildly) interested in seeing the studies in question. He hasn’t gotten back to me yet. But it gets me thinking:
a) What are these studies? Does anyone know of any tests that show that homeopathy worked on something?
b) If someone came to me with such a study, what would I do next? Of course, I’d want to explain it away, but that doesn’t seem very fair. On the other hand, I don’t really know how to gauge a study’s validity. Obviously, if it comes from the Moonbat Journal of Holistic Fruitcakery, I could discard it, but if not? I certainly don’t have the expertise or time to replicate the study. Does the truth of a matter come down to the kind of evidence you’re willing to accept?
If actual homeopathy (not just the efficacy of some herbal remedy, but using water with mathematically nothing left of the allegedly medicinal ingredients in it) was actually supported by a double-blind, properly conducted study, and that study was then replicated by other scientists (because once could just be some bizarre fluke–a confidence interval of 95% means that 1 time in 20 you’ll get a seemingly significant result by chance), you won’t have to explain anything, because lots and lots of scientists will be trying to figure out what the hell’s going on. Hard evidence that homeopathy works could get someone not only a medicine or physiology Nobel, or even the chemistry prize, but maybe even a physics Nobel, because it would be evidence that something seriously weird was going on that we didn’t understand at a very fundamental level.
I wouldn’t; I’d use the stuff if I had the ailment.
Of course, I’d probably cop flak from all the homeopathy people I’ve argued with over the years, but if that’s really the way the data goes, you gotta play it where it lays.
So it sounds like you’re saying that no such study exists, to your knowledge. That what I figured as well. Anyone have information to the contrary?
Okay, so if someone now hands me an article about a study, what are the give-aways that the study was poorly done? (Besides a positive result for homeopathy, that is.)
Sorry to seem dense, but scientific training is unfortunately not natural to humans, and I’m having to pick it up as best I can.
It’s not so much a matter of “explaining away” as it is pointing out the serious flaws in the studies and the importance of independent replication. In my opinion, there are still no good, repeatable studies on the efficacy of homeopathy. Showing why the studies are not good takes a lot of time and effort, but to get you started, here’s a list of references cited by an article in Health Supplement Retailer June 2003:
Kleijnen J et al. “Clinical trials of homeopathy.” Br Med J. 302, 6772:316-23, 1991.
Linde K et al. “Critical review and meta-analysis of serial agitated dilutions in experimental toxicology.” Hum Exp Toxicol. 13, 7:481-92, 1994.
Linde K et al. “Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials.” Lancet. 350, 9081:834-43, 1997.
Vallance AK. “Can biological activity be maintained at ultra-high dilution? An overview of homeopathy, evidence, and Bayesian philosophy.” J Altern Complement Med. 4, 1:49-76, 1998.]
Reilly D. “The puzzle of homeopathy.” J Altern Complement Med. 7 Suppl 1:S103-9, 2001.
Vickers A et al. “Can homeopaths detect homeopathic medicines? A pilot study for a randomized, double-blind, placebo controlled investigation of the proving hypothesis.” Br Homeopath J. 90, 3:126-30, 2001.
Shealy, CN et al. “Osteoarthritic pain: a comparison of homeopathy and acetaminophen.” Am J Pain Management. 8, 3:89-91, 1998.
Weiser M et al. “Homeopathic vs. conventional treatment of vertigo.” Arch Otolaryngol. 124, 8:879-85, 1998.
Taylor MA et al. “Randomized controlled trial of homeopathy versus placebo in perennia allergic rhinitis with overview of four trial series.” Br Med J. 321, 7259:471-6, 2000.
Reilly DT et al. “Is homeopathy a placebo response? Controlled trial of homeopathic potency, with pollen in hayfever as a model.” Lancet. 2, 8512:881-6, 1986.
Kleijnen J. “What research is needed to show the effectiveness of homeopathy.” Br Homeopath J. 89, Suppl 1:S1-2, 2000.
Jacobs J et al. “Treatment of acute childhood diarrhea with homeopathic medicine: a randomized clinical trial in Nicaragua.” Pediatrics. 93, 5:719-25, 1994.
Jacobs J et al. “Homeopathy for childhood diarrhea: combined results and metaanalysis from three randomized, controlled clinical trials.” Ped Infect Dis J. 22, 3:229-34, 2003.
Friese KH et al. “[Acute otitis media in children. Comparison between conventional and homeopathic therapy.]” (Article in German.) HNO. 44, 8:462-6, 1996.
White A et al. “Individualised homeopathy as an adjunct in the treatment of childhood asthma: a randomized placebo controlled trial.” Thorax. 58:317-21, 2003.
If reputable studies actually found homeopathic remedies to be effective beyond placebo, then the next step would be to do research as to why they are effective. My biggest problem with homeopathy is that “water has memory” stuff. If Homeopathic remedies actually worked, my guess is it would be for completely different reasons than why the theory says it works. I’d want to find those reasons.
Because homeopathy has no plausible mechanism, few reputable scientists are interested in conducting studies. Most studies are being done by people with a vested interest in the outcome. Blinding is supposed to take care of that, but it’s really easy to think you’ve got proper blinding when you haven’t.
The basic premise of homeopathy is the epitome of an extraordinary claim. It’s going to require more than an occasional P=0.05 study to get it generally accepted. Especially with so many other studies showing lack of significance.
As MEBuckner suggested, if there were really any good evidence in favor of homeopathy, we would be able to tell just by the rush of scientists from major institutions trying to get in on the ground floor of a tremendous breakthrough in our understanding of the universe. “But the dog did nothing in the nighttime.”
A few studies showing effectiveness is exactly what we would expect from an inert substance, simply due to statistical fluctuations. Watch for publication bias (announcing favorable results while ignoring the rest).
This is all you really need to counter the claims of your friends and associates. Going to the original research articles to find every hole in the evidence is way beyond the call, although it might be satisfying to find a few representative cases to illustrate your point.
I vauguely recall the editiorial of the Lancet issue thgat contained the homeopathy meta-analysis saying that homeopathy represented the blank control of meta-analysis as a scientific instrument … the fact that it was not a clear negative was an indictment against meta-analysis more than an endorsement of homeopathy. I cannot find a link to that editoria however.
What I can find is this BMJ editorial from 1996 regarding the discrepency between what meta-analysis had predicted would be the effect in a heart study (a significant effect) and what a mega-trial actually showed (none).
The only standard would be, can be, good well designed studies that are replicable.
Yes. For comparison, there is some limited evidence that the Atkins low-carb diet is somewhat more effective than average in helping the typical person lose weight, but that evidence indicates that the ketosis or whatever the Atkins cultists say is the physiological mechanism behind the diet has nothing to do with its apparent efficacy.
Having tried to look up a few cites for an essay, I can tell you there are tons of these studies, some for homeopathy, others against, some in major peer-reviewed journals, some in…un-peer-reviewed journals, I guess.
In the 1997 Lancet meta-analysis, the authors did find that the studies that fit their standards (i.e. sufficiently stringent) came down overwhelmingly in favour of homeopathy, but they concluded that “there is insufficient evidence from these studies that any single type of homoeopathic treatment is clearly effective in any one clinical condition” and they also would like “independently replicated, large-scale rigorous trials of defined homoeopathic approaches in at least a few specific disorders”. There was also a more damning report by a Homoeopathic Medicine Research Group for the European Commission in 1996 but it’s nowhere on the Web.
Like MEBuckner and Revtim say, the principles of homeopathy–that a solution gets stronger the more dilute it gets (!)–are really implausible. And I don’t think there’s been any success in proving them; Benveniste totally crashed and burned. (Nature 333:6176, Nature 334:6180, Nature 335:6193) If there’s been anyone since then, we wouldn’t have this thread!
If such a study is presented, the next step is to try to repeat the findings. If they can be reliably repeated, then further testing/tweaking is required to try to determine the nature of the mechanism. If they can’t be repeated… well…
How can a consumer of such information account for publication bias? If studies showing no effect end up unpublished, how can one find access to such studies?
The other thing I find interesting about homeopathy is how one remedy is often suggested for a wide variety of conditions. How can one remedy be effective for such a wide variety of conditions as is sometimes reported? (I’m just starting to read up on homeopathy, but it seems like nux vomica seems to be the remedy for nearly everything. How does that work?)
It should also be mentioned that homeopathy has been around for a long time so there has been plenty of time for good studies to demonstrate the method to be effictive if there is any effectiveness to demonstrate.
Mark Twain mentions it in Life on The Mississipps. In Chapter XVII (published in 1875) he describes the care with which the river steamboats were prepared for a race which was a big deal on the Mississippi at the time.
“If a boat was known to make her best speed when drawing five and a half feet forward and five feet aft, she was carefully loaded to that exact figure - she wouldn’t enter a dose of homeopathic pills* on her manifest after that.”
It isn’t as if its some whiz-bang new system that hasn’t had time to be tested yet. Notice that Twain merely wrote “a dose of homeopathic pills” without further explanation, assuming his readers would understand his meaning. In case anyone here doesn’t, see the *.
*As ME Buckner pointed out, homeopathy uses solutions so diluate as to have a very small probablility of there being even a single atom of the “medicine” in any particular dose. A “dose of homeopathic pills” would be small indeed.
How long have quality double-blind, placebo-controlled studies been around? (Long enough to study homeopathy, yes, but I get the feeling that nobody really considered studying homeopathy in such a manner until 20, 25 years ago, and not very seriously until about 10-15 years ago. Perhaps there has been more study of homeopathy in Germany.)
Also, who’s going to pay for the studies? There’s not nearly the profit in homeopathic remedies as in prescription drugs, at least in the United States.
Those factors above do seem to explain perhaps any limited number of double-blind placebo-controlled studies of homeopathy, although it appears that that’s changing.
But it gets me thinking: