In Scientific Experiments, Is A Control -Always- Necessary?

Factual Questions
1

Watching an old episode of Mythbusters last night inspired this question.

The guys were testing whether or not a human could survive a fall from 1,000 feet in a giant inflatable ball. They affixed monitoring equipment to a human analog (“Buster”) and did two drops: a control, with no protective equipment, and a second, in the ball. After each drop, they noted the results (because the difference between Science and Screwing Around is writing shit down).

From a TV standpoint, I see the need for a control, because the viewers get to watch in glee as Buster falls to the ground from a thousand feet. But from a science standpoint, I don’t get it. Anyone with the most basic understanding of biology and physics will be able to tell you that it’s not survivable, and indeed in every such case a real human is going to break every bone in their body and die from such a fall. There is no ambiguity about that and it’s easily repeatable and predictable,

In a scientific experiment where the outcome of the control is going to be patently obvious and easily predictable, is it still necessary?

2

We got a workable A-bomb through experimenting without a control, so there’s that.

3

A good experiment doesn’t look just for yes/no answers. It will examine all the details and variants. The extent of the broken bones, the placement of them, the severity, the ancillary damage all should be evaluated.

Good science can be done without controls, however. Astronomy, geology, paleontology, and meteorology are all hard to create controls for. “Science” is not a single thing, nor a unitary one.

And the A-bomb was not at all a scientific experiment. It was a test of engineering, something very different.

4

In your example, you wouldn’t. You want to know if a human would survive being dropped inside a giant ball. Put a person in the ball, drop it and they’ll either survive or they won’t.
Where you’d need a control is if you want to compare it to something. If you want to see how much less mangled they’d be inside the ball, you’d need a control.

WRT the a-bomb, it’s kinda the same thing. Either the bomb worked or it didn’t, that seems more along the lines of prototyping/inventing, even if they called it experimenting. However, if you want to know how much more damage the a-bomb does than TNT or how much energy it releases compared to TNT, you’d need to use TNT as a control.

That’s where the control comes in. If you’re going to make a statement like that as part of your experiment, you need proof (and there’s a number of people that survived freefalling from far higher than 1100 feet).
If you were to present your results to the scientific community with the unsubstantiated (and easily disproven) statement that freefalling 1100 feet would break every bone in your body, not survive and suggest that this is, unambiguous, repeatable and predictable, you’d risk getting your paper tossed out.

ETA, what I think you’re doing is conflating “experimenting” with the scientific method.

5

To me, the control in that experiment sounds absolutely sensible. I have no idea how the balloon trial came out, but it’s useful to have something to compare to? Was it as ineffective as an unprotected body falling 1000 feet? Was it only a slightly better outcome? Was it significantly better? If the dummy in the balloon survives fine then, yeah, I guess it’s not absolutely necessary to have a control, but if there’s any degree of damage, it’s useful to have a comparison to quantify the differences.

6

In the OP’s example, it looks like they were specifically studying the difference the inflatable ball makes. If you want to investigate or demonstrate the difference something makes, you compare what happens with vs. without that thing.

But not all scientific experiments fall into this category.

7

They actually did detonate a big pile of TNT a couple of months before the Trinity test as a smaller scale rehearsal, in order to test procedures and calibrate instruments and so forth.

8

You almost always want to be comparing two or more sets of conditions, but it’s not always possible to label one of those sets of conditions as the control, and sometimes there’s more than one set of conditions that could be called that.

And then there are situations where, no matter what you’d like, you just can’t get it. Like, suppose that Betelgeuse does go supernova. We’ll obviously observe it with everything we’ve got, and get a lot of science from it. But what exactly would we be comparing it to? All the stars that aren’t exploding? The other stars that have exploded, but which were so far away that we got only minimal data on them, and none at all from before they blew? I mean, we’d love to set up 20 identical copies of Betelgeuse and see what they all do, but that’s just not possible.

9

One reason that it’s necessary to do a control for this experiment is that you don’t actually know by pure intuition what the effect of dropping an object of a given size for a given length of drop will be. There’s a famous quotation from “On Being the Right Size” by J. B. S. Haldane about this. He wrote, "You can drop a mouse down a thousand-yard mine shaft; and, on arriving at the bottom, it gets a slight shock and walks away, provided that the ground is fairly soft. A rat is killed, a man is broken, a horse splashes. For the resistance presented to movement by the air is proportional to the surface of the moving object. Divide an animal’s length, breadth, and height each by ten; its weight is reduced to a thousandth, but its surface only to a hundredth. So the resistance to falling in the case of the small animal is relatively ten times greater than the driving force.’

https://irl.cs.ucla.edu/papers/right-size.html

10

And they tested the gadget without plutonium first, a test that failed and gave Oppenheimer fits until they figured out that they had botched some calculations. But again, this was nothing like a control experiment. Making sure the wiring is done correctly is engineering. This isn’t a knock at engineering, which is obviously crucial. No engineering, no blast. It simply isn’t science.

11

Part of the problem here is that we have many different definitions of “experiment.” As Exapno points out, there are what are called controlled experiments. And, by most definitions, they rely on comparisons of test conditions - variables, typically holding all parameters the same except for the one being examined. Hard to know if Mythbusters did that. But there are also the common use definitions of experiments that are more in the realm of “Let’s do this and see what happens.” Think of the guys who put things in a microwave or a clothes dryer and watch what happens. And as a former science teacher, I can tell you that the general public - kids included - consider *anything *you do in a classroom with materials and equipment to be an experiment. Many classroom activities are designed to illustrate some phenomenon or concept, but no matter how carefully a teacher watches his use of language, i.e. only calling the event an activity or an investigation, at the end of the day, kids, other teachers, parents, etc. will report that everyone really enjoyed the experiment we did.

12

However, did you know that parachutes are ineffective against preventing broken bones when jumping out of aircraft? As this completely serious article actually published in the BMJ will tell you.

It helps in appreciating the linked article if you’re an active researcher, particularly in a medical field, because their perfectly idiomatic use of medical research terminology in what’s definitely not a standard medical research paper is part of the point.

But their serious point is actually completely relevant to the Mythbusters experiment, because the point is that ethics, and the imperative to not hurt your fellow human beings, can often bugger up ‘controlled’ experiments to the point where you think you’ve learned something, but actually haven’t.

Buster is probably pretty good because I imagine he’s been tested in a number of car crashes that testers know (through observation of events) will cause damage in humans, and observed to suffer damage. But if there were some way they hadn’t considered in which the ball-drop was quite different from a car-crash (for instance, the fact that you don’t tend to stay in the same orientation), you could still be misled by a “Buster is ok with protection” result to think that a human would be more ok than is actually true (note - I haven’t watched the ep. Do they end up doing the ball drop with a person?)

13

If they are testing some drugs they compare the old drug vs. the one being tested. That’s what happens for cancer drugs for obvious reasons. If it turns out the new drug works well they might stop the study and everyone in the study gets the new drug.

14

Assuming the obvious relies on your ability to correctly assume.

Some scientist out there would drop the dummy from 30’, saying that it’s a 100% mortality rate at that height because ** obviously**.

If he then gets a 50% survival rate with his non-control experiment, has he actually proved anything?

And the thing is, you might be that guy. It’s that guy who will be most certain that he’s not that guy.

And, of course, one might note that assumptions have prevented science from advancing before. “Obviously, more people would die if we didn’t provide medical treatment.”

If you can have a control, having it prove valueless in the majority of cases is, to be sure, a bit of a loss in time and money. But, in theory, you are always better to have one than not.

15

One other thing: Aside from the fact that “proof” is a very elusive concept, just doing one event, even with a control, hardly proves anything. Maybe the dummy that the Mythbusters used was defective. Only multiple trials can lead to a reasonable degree of validation.

16

A common style of experiment consists of testing a great many subjects (which can be people, animals, or things) to see if some experimental treatment has an effect. That is the case where a control is called for.

It is also the case that a meaningful experiment typically requires many subjects so that some statistics can be collected. The experimental treatment may fail for some subjects, and some control subjects may reacts as if they were treated. That’s why you need a large body of data from which conclusions may be drawn.

Dropping one or two bodies in a ball and one or two bodies without won’t tell you much that you didn’t already know. (Note the above remark that some people do fall 1000 feet and survive.)

Dropping 100 people in balls and 100 people without, however, will tell you a lot about the probability of surviving such a drop with and without a protective ball.

17

Consider those studies (“experiments”?), some well-known and some less so, in which various animals (mostly chimpanzees) were taught some rudimentary aspects of human language or sign language.

What would be a “control” for those? Is a “control” needed?

I worked on a project that was teaching some rudimentary language skillz to dolphins. (Herman, Richards and Wolz. Comprehension of sentences by bottle nosed dolphins. Cognition, Vol. 16, No. 2, March 1984, pp 129-219. Abstract. Includes link to full text, which requires subscription or payment I think.)

We fussed a lot over whether a “control” was needed and if so, what it the world it would look like. The study entailed extensive training over several years, and we observed the dolphins responses to the simple “sentences” we presented to them.

What would a control look like? Take two other dolphins, and not train them anything at all, then present them with sentences to see how they react? That obviously isn’t going to work.

Or take two other dolphins and train them a lot but omitting some critical aspect of the training? There were whole bunches of possibilities for what to omit, none of which made a whole lot of sense.

Besides, dolphins are expensive.

We tried to develop some statistical model by which we could compute a “p-score”, that is, a probability that the desired result could have happened by chance. (Having a good control makes it possible to compute this.) A typical “good” p-score in many experiments is p<0.10 or p<0.05

We came up with something like p<0.00000000001 which was totally ridiculous. We knew it was ridiculous. But we wrote that into the draft of the research report because we just knew that some reviewer somewhere was going to demand it.

Not unexpectedly, some reviewers commented that the statistic was ridiculous. I don’t remember if we included that in the final published report. I kind of think we didn’t.

18

For many people in human history it was patently obvious and easily predictable that a flat earth would sit there and a sun would rise in the east and chug along to set in the west; that a heavy object would fall faster than a light one; that it was turtles all the way down.; that the world had always looked like it did that morning.

Using controls where they are available and appropriate in experimental design helps to free yourself from the sometimes mistaken assumptions about what actually is reality, not assumption, before you begin to see what happens if you do something different.

19

An important scientific practice is to run an experiment two ways (when possible). Then compare the differences in results, if any.

It’s important that the two experiments are conducted exactly the same,* except for only one difference.* Then, if the outcome is different, you know that the one difference was the most likely factor.

If you have more differences at the start, you do not know which was the cause for the different outcome.

Yes, controls are vital to science.

20

I hope I live to see it.

(Yes, I know that whatever I see actually happened centuries ago, but you get my point.)