People often criticize scientific studies that confirm what everyone already knows, but what are some examples of the kind of studies that everyone thought had an obvious result, but it turned out not to be true after all. I’m especially interested in fairly recent studies.
Not that recent but relevant I think:
Maybe they will do a study to see how the invasion of Ukraine increased the death rate there.
Statistical analysis reveals Mexican drug war increased homicide rates
Date:
April 2, 2015
Source:
American Statistical Association
Summary:
In the short term, the Mexican government’s war against drugs increased the average murder rate in regions subjected to military-style interventions, a new statistical analysis suggests. The Mexican government–beginning in December 2006 during former President Felipe Calderón’s term and continuing through current President Enrique Peña Nieto’s administration–has been fighting an internal war against drug traffickers. A 2013 report by Human Rights Watch estimates 60,000 people were killed between 2006 and 2012 as a result of the military interventions and drug cartels fighting each other for control of territory.
My emphasis.
I do not have any examples off the top of my head, but we can see how it might be a problem getting funding for experiments testing established results or even just replicating someone else’s published research. If you look at
https://plato.stanford.edu/entries/scientific-reproducibility/
, some scientists are even identifying a “reproducibility crisis”.
There are even some relatively recent examples cited there; e.g., in 2011 some guy published a study reporting evidence for the existence of “psi” or “ESP” in a “high-impact” journal of psychology. This naturally inspired three independent replication studies, which failed to reproduce these results. However, when the replicators went to publish their findings, they were rejected from four different journals, including the one that published the original study, on the grounds that their work was not original or novel research!
Plenty of them, I just cannot remember all of them. Here are some :
- Placebos are recommended “medicine” for diseases like IBS. Placebos without deception - i.e. both the doctor and the patient knows it is a placebo, and yet it works
Placebos without Deception: A Randomized Controlled Trial in Irritable Bowel Syndrome - PMC.
-
Barry James Marshall AC FRACP FRS FAA[1][2] (born 30 September 1951) is an Australian physician, Nobel Prize Laureate in Physiology or Medicine, Professor of Clinical Microbiology and Co-Director of the Marshall Centre[4] at the University of Western Australia.[5] Marshall and Robin Warren showed that the bacterium Helicobacter pylori ( H. pylori ) plays a major role in causing many peptic ulcers, challenging decades of medical doctrine holding that ulcers were caused primarily by stress, spicy foods, and too much acid.
-
Going back, Kola Superdeep Borehole - proved many of Geology predictions wrong. Like the type of rock at different depths and the presence of water at different depths.
-
Further back, Rutherford hitting Gold foil with alpha particles proved that the atom was mostly open space with a concentrated nucleus.
Yes, this.
There is actually considerable value in testing things that we already “know” to be true. And all of peer review is arguably a form of this. It’s a shame that much of the public sees this as a waste of time/money but it’s yet more evidence that humans aren’t instrinctively scientific and skeptical.
(Well, if we’re repeating exactly the same kind of test over and over then there is less value to that, but that’s not really a thing that’s happening anywhere)
And then there was the great Debate on Flossing, following Scientific analysis by AP that showed :
“The AP looked at the most rigorous research conducted over the past decade, focusing on 25 studies that generally compared the use of a toothbrush with the combination of toothbrushes and floss. The findings? The evidence for flossing is “weak, very unreliable,” of “very low” quality, and carries “a moderate to large potential for bias.”” null | AP News
But then there were counter arguments. And now I believe, Flossing is back to being recommended.
Here is a summary
It’s not recent, and it’s probably apocryphal, but the mother of all of these is Galileo dropping two cannon balls off the Tower of Pisa to show that there are some things that everyone knows and that do not need to be tested — in this case, that the heavier cannon ball would hit the ground first. Which is not what happened.
All experiments in psychology are not of this type, however. For example, there have been many experiments running rats through all kinds of mazes, and so on—with little clear result. But in 1937 a man named Young did a very interesting one. He had a long corridor with doors all along one side where the rats came in, and doors along the other side where the food was. He wanted to see if he could train the rats to go in at the third door down from wherever he started them off. No. The rats went immediately to the door where the food had been the time before.
The question was, how did the rats know, because the corridor was so beautifully built and so uniform, that this was the same door as before? Obviously there was something about the door that was different from the other doors. So he painted the doors very carefully, arranging the textures on the faces of the doors exactly the same. Still the rats could tell. Then he thought maybe the rats were smelling the food, so he used chemicals to change the smell after each run. Still the rats could tell. Then he realized the rats might be able to tell by seeing the lights and the arrangement in the laboratory like any commonsense person. So he covered the corridor, and, still the rats could tell.
He finally found that they could tell by the way the floor sounded when they ran over it. And he could only fix that by putting his corridor in sand. So he covered one after another of all possible clues and finally was able to fool the rats so that they had to learn to go in the third door. If he relaxed any of his conditions, the rats could tell.
Now, from a scientific standpoint, that is an A‑Number‑l experiment. That is the experiment that makes rat‑running experiments sensible, because it uncovers the clues that the rat is really using—not what you think it’s using. And that is the experiment that tells exactly what conditions you have to use in order to be careful and control everything in an experiment with rat‑running.
I looked into the subsequent history of this research. The subsequent experiment, and the one after that, never referred to Mr. Young. They never used any of his criteria of putting the corridor on sand, or being very careful. They just went right on running rats in the same old way, and paid no attention to the great discoveries of Mr. Young, and his papers are not referred to, because he didn’t discover anything about the rats. In fact, he discovered all the things you have to do to discover something about rats.
Not to be excessively pedantic, but Marshall and Warren’s work on Helicobacter is an example of non-obvious science that turned out to be right.
*An example that would fit the OP, assuming it exists, would be a study attributing all peptic ulcers to Helicobacter, when in reality some are indeed caused by hyperacidity associated with stress, smoking, certain foods etc.
IIRC, the first experiment on parity in the weak interaction was of this form: “Everyone knew” that the weak interaction would be parity-symmetric, because everything else in physics is. Except some grad students actually checked it, and found that that’s exactly wrong. And so their advisor got the Nobel Prize.
The granddaddy of all unexpected results was the Michaelson-Morley experiment that was expected to show the earth’s motion through the ether. And showed there was no such thing as ether. It led pretty directly to special relativity and eventually to modern physics.
Similarly, I recall reading about a study (though I’m having trouble finding it now) that indicated the placebo effect doesn’t even exist-- that those cases of recovery after a known ineffective treatment are really just examples of spontaneous remission, which would have occurred with or without the sugar pill and positive thinking. Blew my mind. I’ll see if I can find it.
As I understand it, Einstein wasn’t even aware of the Michelson-Morley experiment at the time he published special relativity. His inspiration was seeing that Maxwell’s equations were already relativistic.
@Esprise_Me , most medical studies compare both to a placebo and to another control with no treatment at all. If the placebo effect didn’t exist, that would have been noticed long ago.
I may be wrong, can’t find the cite, but I read that Galileo only did a Gedankenexperiment, and did not need to climb the Pisa Tower at all. The experiment allegedly goes like this: We begin with the hypothesis that heavier objects fall faster than lighter ones. Then we take two objects, a heavy one with the mass M1 and a lighter one M2. Tie them together with a rope with some slack, and throw then from a height. The heavier object will fall faster, as per the premise. Until the rope gets stretched, then both objects will behave as one with the mass M1 + M2. At this moment, both objects will suddenly accelerate, as the “new” object is heavier than any of the two original objects. But this is absurd, Galileo reasoned, so the hypothesis was disproved by reductio ad absurdum.
So whatever Galileo proved it was obvious to him already. He did not climb the tower to prove that heavier objects fall faster, but that they do not.
Risk compensation can produce counterintuitive outcomes from implementing “obvious” safety measures. There’s a list of examples here. This was one of the justified concerns surrounding mask policy in the early days of COVID.
Most medical studies I’ve seen only test the treatment against a placebo, not against a placebo group and also a no-treatment group. What would be the point of having 3 groups, unless you also wanted to test the placebo effect? Seems that for purposes of testing the efficacy of a new treatment, lumping spontaneous remission in with any placebo effect that may or may not exist would be good enough.
I was thinking of that one too
Yeah, I said it was apocryphal. But I love the thought experiment story, which I’d never heard before, so thanks!
From what I understand, Galileo did do actual experiments, except rolling spheres down a slope, not dropping them, because that way the speeds are lower and it’s easier to time them.