The Post Hoc Fallacy Fallacy

[december]

No doubt all posters here are familiar with the Post Hoc Fallacy:The post hoc ergo propter hoc (after this therefore because of this) fallacy is based upon the mistaken notion that simply because one thing happens after another, the first event was a cause of the second event. Post hoc reasoning is the basis for many superstitions and erroneous beliefs. http://skepdic.com/posthoc.html

However, what I'm calling the Post Hoc Fallacy fallacy" is the idea that post hoc reasoning is automatically wrong. I've seen posters on the SDMB try to refute opposing evidence by simply writing the words, "Post hoc." Judging Science by Kenneth R. Foster and Peter W. Huber says Post hoc theorizing is not necessarily wrong, of course. This process is benign and generally useful for science; quite often the theories prove to be useful.This book later addresses circumstances where post hoc reasoning is likely to provide a wrong answer. It is easy to fish through the health records of an entire country--Sweden, for example--in search of an association between occupation and disease. According to a statistical test typically employed in such studies, a comparison has one chance in twenty of showing a "statistically significant" difference, even thought there may be no real difference between the groups of individuals being compared. ...

The same phenomenon accounts for the hundreds of "clusters" of cases of disease that are reported in the United States every year....

One final example of the dangers of post hoc theorizing is found in the many reports of biological effects of electromagnetic fields. Many of the studies were simply "fishing expeditions"... So, one ought to consider evidence, even if its not conclusive. Particularly when it' based on facts naturally relevant to a debate, rather than the result of a "fishing expedition."

In particular, I object to supporting a theory that has no evidence at all, because the evidence on the other side is less than perfect.

[edwino]

When I am trying to plan my science, I formulate a hypothesis and then perform experiments to try and support or disprove the hypothesis. I have been taught that the most important thing to do is to consider all outcomes of the experiment equally, and thus to consider all models equally. My hypothesis gets no particular favor just because it is mine.

This is all well and good in basic science. But in the clinics, there is less room for manuevering if one is only going to analyze population data. This sticks you with only this one tool in the arsenal. To do a good population study is extremely complicated, as it requires unbiased ascertainment and correct analysis. Most people don't know how to do this correctly. The data from good population studies can be very enlightening, for instance showing that smokers get lung cancer. The tobacco industry will have you believe that that is an example of post-hoc reasoning on a large scale, and I remember quotes out of their 'scientists' about "no one has ever seen cigarette smoke causing cancer."

People have analyzed compounds from cigarette smoke and found carcinogens among them, for instance benzo-a-pyrenes. Cell biology has provided a crucial datum which can support the observation made, and therefore the hypothesis that smoking causes cancer. Also, at some point the number of smokers getting cancer becomes too great to ignore.

This is not true with many other popular medical conclusions. The numbers in the studies are not great enough, the statistical confidence is not high enough, and the causative action is foggy at best. I can say this about EM causing cancer, vaccines causing autism or brain damage, breast implants causing lupus or connective tissue disease, and a host of other problems. While on small scales, stories may be convincing, these effects vanish on a population scale.

In short, we rely on p-values to tell us if our data are significant. A gold standard is a p<0.05, which means that there is less than a 5% chance that a particular correlation is due to chance. If you think about it, though, this is quite bad. I can take a large set of data (for instance the Framingham survey or something) and perform 20 random correlations on the data, with no backing hypothesis. Probability will show one nice correlation. These observations mean nothing until fit into a broader hypothesis and have correlative experiments performed to support them.

[DPWhite]

Just because you are paranoid doesn't mean that they are not out to get you.

Just because you are not paranoid doesn't mean that they are not out to get you.

Just because you are paranoid doesn't mean that they are out to get you.

Just because you are not paranoid doesn't mean that they are out to get you.


Have we now covered all the possibilities without explaining any of the reasons why?

[FortMarcy]

So we have:
A. Correlation does not imply causation.
B. Correlation does not exclude causation.

Both are correct statements. But since B is a true, A should be disregarded or given less weight? No, A is still true.

[december]

Originally posted by FortMarcy
So we have:
A. Correlation does not imply causation.
B. Correlation does not exclude causation.

Both are correct statements. But since B is a true, A should be disregarded or given less weight? No, A is still true. FortMarcy's post could be read to say that correlation is irrelevant to causation. This conclusion would be silly. In science correlation is usually a key to proving causation, but one also needs the right conditions: falsifiable thesis, adequate sample size, random sample, etc.

im·ply (m-pl)
tr.v. im·plied, im·ply·ing, im·plies

1.To involve by logical necessity; entail: Life implies growth and death.
2.To express or indicate indirectly: His tone implied disapproval.FortMarcy's point A is true under definition 1. Correlation doesn't prove causation as a logical necessity. That's important in science, but much less so here on SDMB, where we debate politics, religion, sociology, economics, morals, etc. Nothing could ever prove our theses as a logical necessity. So, it's not saying much that correlation also doesn't meet that standard.

We need to add another principle:

C. Correlation makes causation more likely to some degree. The degree depends upon various aspects of the correlation.

[kabbes]

Oooh, I'd never guess that you were a GI actuary, december.

Have you had to do an exercise with some shitty data recently or something? Sometimes I think that a pricing exercise is composed of sitting around chanting "There is no such thing as a post hoc fallacy. There is no such thing as a post hoc fallacy".

pan

[december]

Originally posted by kabbes
Oooh, I'd never guess that you were a GI actuary, december.

Have you had to do an exercise with some shitty data recently or something?Is there another kind? :confused:

[FortMarcy]

Originally posted by december
We need to add another principle:

C. Correlation makes causation more likely to some degree. The degree depends upon various aspects of the correlation.

But therein lies the problem. "Some degree" can be anywhere from 0% to 100% and it is unique for each instance. Once enough information is gathered to determine the degree for a specific instance, the correlation is not needed to as evidence.

[kabbes]

Hah!

Actually - musn't grumble. I've got a job on at the moment with complete and comprehensive information about every claim going back for years. And exposure data almost as good.

So apparently the prophets weren't lying - the golden data really does exist.

Mind you - I haven't tried reconciling it yet...[/quote]For the non-actuaries out there, the point I'm trying to make is that if there were absolutely nothing to tie together correlation and causation, your financial experts would be in biiig trouble. The past can be a guide to the future, so long as you approach it right. And it is only a guide because cause-and-effect, whether direct or indirect shows some consistancy.

First thing you learn when studying statistics: correlation does not imply causation.

First thing you learn when trying to use statistics: there bloody well better be some causation going on somewhere or you're screwed.

pan

[Phlosphr]

December Check out Stuart Vyse's "Believing in Magic: The Psychology of Superstition"

Great book it it explains wondrous amounts of info on 'The Gamblers Fallacy' and your post hoc as well as ad hoc inconsistencies.

If I flip a coin and get ten heads in a row. Whats the Probability I'd get a tail next? Would you bet on it. Am I due for a tail??? Is there any such thing a being 'due' to get some thing?

Also the book is currently utilized in Grad applications for probabilities and stat classes. Granted it is mostly used here in New England but I have seen it at ASU as well.

[kabbes]

Probability of getting a tail after ten heads is a classic example of Bayesian statistics.

Start with a prior distribution of P(head) = 0.5

Assume a parameter distribution for p.

Then posterior is proportional to parameter x prior.

Renormalise and bob's your aunty's uncle. You have a new estimation for the probability of a head.

Not sure what that's got to do with post hoc fallacies though.

pan

[december]

Originally posted by Phlosphr
December Check out Stuart Vyse's "Believing in Magic: The Psychology of Superstition"Thanks, I will.

FortMarcy But therein lies the problem. "Some degree" can be anywhere from 0% to 100% and it is unique for each instance.Yes, I agree this is a problem. IMHO it is a problem which ought to receive the lion's share of attention in Great Debates.Once enough information is gathered to determine the degree for a specific instance, the correlation is not needed to as evidence. FortMarcy, I'm unclear on the meaning of this sentence. Should the third word from the end be too, rather than to?

Assuming that's what you meant, I would disagree. We seldom if ever are able to agree on the degree to which corellation implies causation -- at least not for the topics debated here. Can you clarify your point with examples or further explanations?

[tracer]

Phlosphr wrote:

If I flip a coin and get ten heads in a row.
... I'd want to check your coin for proper balance to make sure you weren't cheating.

[kabbes]

Oops. The prior is a distribution for p, and the posterior for p is proportional to the prior x the distribution for heads.

As the alert reader would have spotted.

pan

[december]

Originally posted by kabbes
Start with a prior distribution of P(head) = 0.5

Assume a parameter distribution for p...

Renormalise and bob's your aunty's uncle. You have a new estimation for the probability of a head.

Not sure what that's got to do with post hoc fallacies though.

pan I think it's related. Suppose we're debating whether God exists. If a religious person has a prior that gives, say 99% chance that God exists and 1% not, then evidence might lead to a shift in the probabilities.

However, if the prior is 100% that God exists, then the posterior will be 100% that God exists. In other words, if someone starts a debate totally certain of their position, then no evidence will sway them.

[Guinastasia]

Then it's not really a debate, then, but a rant.


Is this related to some of your other debates in the past month or so, december.

[december]

Originally posted by Guinastasia
Then it's not really a debate, then, but a rant.Guin, what does your word then refer to? Why isn't this a debate?
Is this related to some of your other debates in the past month or so, december. IIRC two posters on different threads deigned to respond to an example, and simply mentoned the fallacy of post hoc ergo propter hoc. Unfortunately, I can't recall where they were, or I'd provide a link.

I thought this topic might be a debate. Those who (mis)use the post hoc fallacy this way might want to dispute my point and justify their practice. I think FortMarcy was disagreeing with my POV.

Also, the deep questions of epistomology are by no means settled and agreed by philosphers of science, let alone by SDMB posters. There seems to be room for debate here.

[kierk]

We know intuitavely that correlation and causation are linked. If we see a strong correlation between two factions, and its a good bet there is a causative relationship, so it makes sense to research just how that underlying causation might work. e.g stat analysis shows a link between em radation and cancer, biologists get interested and take a look at what kind of effects em fields have on living things. they find out that only at ridiculously high levels do the fields have any observable effect, thus the causative link hypothesised is drastically weakened.

In a debate, however, the underlying info is generally known, the debate is about the meaning of the data, to continue in the example above, a great debate now rages over em radation and cancer, some arguing the process of the study was flawed, bias, etc. nice lively discussion. now someone comes in and says 'what about that link between em fields and cancer, they MUST cause cancer' to which the response is 'post hoc' shorthand for 'look, we all know the data, and there is a correlation but all data so far shows no known method for em field to cause or even influence cancer. It's up to you to prove your point, not me to disprove it.'

So basically, to sum it up in a short pithy statement, I'd offer a grant based on post hoc reasoning but I wouldn't publish a paper based on post hoc reasoning.

As far as the last statement of the OP, could you be more specific as to which argument(s) you're referring?

[december]

Originally posted by kierk
As far as the last statement of the OP, could you be more specific as to which argument(s) you're referring? I presume kierk is asking about my statement, "In particular, I object to supporting a theory that has no evidence at all, because the evidence on the other side iscless than perfect." Fair question. Here are some examples:
[list] Religion (based on faith, rather than evidence) Some economic arguments, such as the idea that minimum wage laws don't cause job loss. Foreign policy ideas that are merely based on general principles, such as unilateral disarmament.

[DrDeth]

It is kinda related to the old "arguement from authority fallacy". Yes, indeed, if we are debating, say the death penalty, and some nimrod quotes a "respected authority" that claims the "death penalty is immoral"- then, yes, that same dude has committed the "authority fallacy". However, if another dude quotes an authority that shows that the "death penalty is not a deterrant"- and argues from that cite that, then; the "death penalty is wrong"- then they are NOT using the "authority fallacy". You can argue that his conclusion is invalid- or- you can cite another source that shows the death penalty IS a deterrant. But saying that he is "argueing from authority" is wrong. (Note, also wrong is "attacking his source". If you think his source is not kosher- come up with a better source, then point out why you feel your source is better.)

So, yes- argueing that "since a>b, then a causes b" may well be incorrect- however, as long as you are aware that the two might not be related, you're OK. I know some dudes that claim that the 9/11 WTC did not lead to the bombing of the Taliban ("It is all about the Oil"), but it seems to me that they are ...... what's the technical term?....... Oh yes.... wrong. :)

[Triskadecamus]

The cigarette studies were touted at proof long before they had sufficient real evidence that cigarette smoking did indeed cause cancer. Dozens of additional studies all showed the same correlation, even after removing other factors. But the mechanism of causation was not known, and still remains incompletely revealed. The evidence was sufficient to convince a lot of people, but it falls short of proof. The reason is that it is insufficient that the correlation exists, the mechanism of causation must be revealed. That takes a different level of examination than simple statistics.

Statistical analysis is a slippery devil. Cancer is caused by exposure to graduate students. You can prove it, statistically. Control populations in laboratory cancer studies have a higher incidence of cancer than do captured members of the same species. The lab animals are deliberately not exposed to carcinogens in the test. Yet they have a higher rate of cancers found in autopsy than are found in animals taken from the wild. One factor that laboratories have that wild places do not have is higher levels of population of graduate students.

The problem is that laboratories themselves are a factor. So too is the limited genetic diversity of lab animals. So, you don't rely on the statistical data to prove anything, you use that data to indicate what you should study. If you really are worried about the graduate students, you higher illiterate migrants to care for your animals, and keep the grad students out of the building. If your results indicate a drop in cancer rates, you may be on to something. But probably, it won't change. Not graduate students. Maybe fluorescent lights? Gary Larson cartoons?

If you base your entire proof on statistics, you have proven nothing but you own ability to assemble statistics. Post hoc ergo propter hoc is not true, even when the antecedent did cause the result. The contention is that 'therefore' the antecedent is causal. Simple correlation does not even examine the mechanism of causation.

Statistics are a good tool, for determining a direction for investigation. But there are other areas than science in which they are valuable. Insurance companies don't care why male drivers between 16 and 19 are more likely to cause expensive damages with automobiles. They know it happens. They charge rates based on that knowledge. They are not wrong.

Red cars go faster than brown cars. It certainly is ridiculous to think that color affects the speed of an automobile. But the fact is that if you stand on the edge of the highway with a radar gun for a week you will find out that red cars go faster than brown cars. Why? So far, every thing I have read on the subject is speculation on psychology. But they do go faster, even so. Now, is it wrong to say that the car goes faster because it is red? Yes. Implying that I understand the mechanism of causation is incorrect. I know that they go faster. I don't know why.

The error is in the implication of understanding cause.

Tris
----------------------------
"Criticism comes easier than craftsmanship." ~ Zeuxis ~ (400 BC)

[Dryga_Yes]

Originally posted by december
IIRC two posters on different threads deigned to respond to an example, and simply mentoned the fallacy of post hoc ergo propter hoc. Unfortunately, I can't recall where they were, or I'd provide a link.


I think I was one of them... I don't know if it was you I was replying to, but it was post-hoc-related, alright. We were talking about religion in government and to what extent we should have a separation of church and state.

[phartizan]

Originally posted by Guinastasia
Then it's not really a debate, then, but a rant.

More of a meta-debate, I think. And where do those belong?

I've been staying away from Great Debates because they don't seem to go anywhere, but I looked at this one because someone I know loves to use post hoc ergo propter hoc to dismiss arguments he disagrees with...which is how people usually use it. So I'm delighted with the Post Hoc Fallacy fallacy; it's something I'll use to hit back with, even if I'm just using it to dismiss an argument I disagree with. He started it.:p

[DPWhite]

Its probably the most irritating fallacy because it is so common, there is no good English translation of it and when you try to explain why one isn't the cause of another they suddenly pretend that logic is too fancy for what everybody knows. Frankly ad hominems are less annoying.

[december]

Originally posted by Dryga_Yes I think I was one of them... I don't know if it was you I was replying to, but it was post-hoc-related, alright. We were talking about religion in government and to what extent we should have a separation of church and state. [/B] Thanks, Dryga_yes. I check that thread and found that MEBuckner had mentioned Post Hoc:

I had written, "I find this argument quite logical, but it hasn't worked in the real world. E.g., Nazi Germany specificially disavowed Christianity. Lenin and Stalin had no religion in their government. OTOH, the US, England, etc., who had a more-or-less Christian base, wound up with excellent civil liberties." MEBuckner responded, "Post hoc, ergo propter hoc. There are also plenty of examples of Christian establishmentarianism leading to tyranny and cruelty--often of one set of Christians oppressing another; Christian societies have become more free and more enlightened as their governments have become more secular and as they have guaranteed religious liberty for all. http://boards.straightdope.com/sdmb/showthread.php?threadid=99723

phartizan , thanks for you kind words.

[MEBuckner]

The predominant language of both Great Britain and the United States is English, and both countries have excellent records of civil liberties. Conversely, in the primary languages of both Nazi Germany and Soviet Russia the word America is spelled with a "k", which everyone knows is one of the telltale signs of Evil Totalitarianism.

[jmonster]

Originally posted by DPWhite
Frankly ad hominems are less annoying.
You're stupid. They are not.

[FortMarcy]

Originally posted by december
Once enough information is gathered to determine the degree for a specific instance, the correlation is not needed to as evidence. FortMarcy, I'm unclear on the meaning of this sentence. Should the third word from the end be too, rather than to?

Assuming that's what you meant, I would disagree. We seldom if ever are able to agree on the degree to which corellation implies causation -- at least not for the topics debated here. Can you clarify your point with examples or further explanations?

Sorry, the word "to" shouldn't have been in the sentence. Triskadecamus summed up my view on the subject more elegantly than I could have. A good correlation does not pinpoint causation. Once the true fundamentals of the correlation have been discovered, there is no need to rely on the mere existence of the correlation for support.

In a debate, it might be nice if your opponent helped you discover the true nature of the correlation. But if he would rather use post hoc ergo propter hoc, then you must do the dirty work yourself.

[kabbes]

Tris - what you say is all very true in theory. But the fact remains that unless you assume that there is some causation going on somewhere, you can't extrapolate. Every time we extrapolate some statistics in an insurance pricing exercise, we are in effect assuming a causation - a causation that is not justified by an underlying mechanism.

Note that the causation needn't be direct. And you're right - we don't necessarily care what the cause is. But in your red and brown car example we could argue a cause all right: that the kind of person who likes to drive fast is the kind of person who buys a red car. This is still a causal link. It isn't coincidence that the red and brown cars have different speeds. This is the crucial point I think.

pan

[xenophon41]

kabbes, I don't think you've disagreed with Tris at all. As he says, statistics are useful in determining where to look for causation (which I took to mean the same thing you're driving at when you speak of extrapolation). Insurance companies are not in the business of extrapolation (although they certainly benefit from robust scientific studies), so, where a strong correlation can be shown (e.g. 16 - 19 y.o. drivers and accidents) they simply don't need to establish specific causation.

Unfortunately, this type of approach, while sufficient for the estimation of probable risks and benefits, is inadequate as a basis for preventive or corrective actions. Thus, if we assert that there's a probable psychological link between aggressive driving and chassis color preference, we've not committed any breaches of logic. But if we suggest that this information "proves" a prohibition against red paint on cars would be likely to reduce the incidence of speeding, we've run afoul of the post hoc ergo propter hoc fallacy.

[Algernon]

Originally posted by kabbes
First thing you learn when studying statistics: correlation does not imply causation.

First thing you learn when trying to use statistics: there bloody well better be some causation going on somewhere or you're screwed.

There is, in my experience, a very low correlation between statistics and laughter.

However, there is a very direct causation between kabbes closing line above and the spewed coffee on my monitor.

...love this thread...

[erislover]

Originally posted by jmonster
Originally posted by DPWhite
Frankly ad hominems are less annoying.
You're stupid. They are not.
No, no... not frank ad hominems! :p :D

I agree that a simple dismissal of causality is not sufficient, in my mind, to counter implied causality (insert relative Monty Python "argument" sketch here). post hoc counters should not be used, IMO, to decimate an argument. Though it should be up to the poster suggesting the causality to support it, it is very difficult to fully explain causal relationships when the other party is wont to disagree, and, frankly, the party who is disagreeing would have a ahrd time explaining why—exactly—they think one doesn't follow from the other. Anyone interested in testing this with me, I have two fun examples.

[xenophon41]

originally posted by erislover:
Anyone interested in testing this with me, I have two fun examples.
Let's have some fun, then. :)

While we wait, though, I submit that a simple dismissal of causation is, in fact, entirely sufficient to refute an argument where a) broader inferences are drawn from the supposed causative link and b) no plausible mechanism is theorized to explain the causation. ("Plausible" in this case would mean a mechanism for which ample evidence exists in the form of established principles and observable phenomena.)

For example, if I were to assert that, based on the fact that all nations with high standards of living also have highly developed ground transportation infrastructures, road and rail systems are a necessary component of any transitioning third-world country, then my opponent could legitimately request that I show more than that basic correlation. My assertion could very well be quite valid, but until I demonstrate a mechanism I haven't offered a strong argument in support, and my opponent need only focus on that lack.

[erislover]

So, xen, I was driving down the highway the other day when I noticed that as I passed a light (and other cars were passing other lights) they came on. Hence, there must be some mechanism whereby cars turn on streetlights.

[kabbes]

post hoc.



heheheh

[kabbes]

Xeno - I'd disagree. Insurance companies are in the business of extrapolation.

If there is no causation, then there is nothing to suggest that the observed correlation will occur again in the future.

The very fact that we do use data to price and reserve risks indicates that although we don't know or care what the underlying causes are, we believe that the causes are there. We trust in those causes enough to assume that they will carry on causing the same claims in the same pattern as they have before.

That's extrapolation. It is extrapolating the past to the future. Only if you believe that what happened before will happen again can you do this.

Without causation, all you have is coincidental order in the random patterns. You couldn't do diddly with that.

pan

[xenophon41]

erislover, that's an interesting thing. Did you stop to observe the phenomenon of lights turning on to see if they turned off after a set time? I don't doubt that the lights came on as you describe, but I wonder what the precise mechanism is. Certainly, it could be that the street on which you drove has light actuating switches set to trip on the passage of vehicles. It's one of many possibilities. I can suggest a few methods by which you could make a more certain determination...

Of course, since you haven't proposed any particular light-avoidance/preventive/stimulative action, or based any social theory on your supposition regarding the lights, it really doesn't matter whether I agree that the cars caused them to turn on. I trust that you observed the correlation, and I agree that there could be a causative link.

;)

[xenophon41]

kabbes: If all you mean is that insurance companies assume that strongly correlated phenomena will continue to exhibit correlation in similar circumstances, then I agree that they are in the business of extrapolation, and I withdraw my assertion to the contrary.

I continue to believe however that, while risk assessment doesn't require one to identify causation, creating any theory regarding the significance of correlated phenomena does.

[kabbes]

Okiedokie Xeno. We are in agreement. But do you know, for the life of me I can't remember what my original point was?

I think it was merely in support of december's observation that just because correlation doesn't imply causation, one shouldn't dismiss the correlation out of hand. Correlation strongly implies that there is some causative event at some level. In the context of a debate, this is worth further investigation.

pan