The Post Hoc Fallacy Fallacy

No doubt all posters here are familiar with the Post Hoc Fallacy:

http://skepdic.com/posthoc.html

However, what I’m calling the Post Hoc Fallacy fallacy" is the idea that post hoc reasoning is automatically wrong. I’ve seen posters on the SDMB try to refute opposing evidence by simply writing the words, “Post hoc.” Judging Science by Kenneth R. Foster and Peter W. Huber says

This book later addresses circumstances where post hoc reasoning is likely to provide a wrong answer.

So, one ought to consider evidence, even if its not conclusive. Particularly when it’ based on facts naturally relevant to a debate, rather than the result of a “fishing expedition.”

In particular, I object to supporting a theory that has no evidence at all, because the evidence on the other side is less than perfect.

When I am trying to plan my science, I formulate a hypothesis and then perform experiments to try and support or disprove the hypothesis. I have been taught that the most important thing to do is to consider all outcomes of the experiment equally, and thus to consider all models equally. My hypothesis gets no particular favor just because it is mine.

This is all well and good in basic science. But in the clinics, there is less room for manuevering if one is only going to analyze population data. This sticks you with only this one tool in the arsenal. To do a good population study is extremely complicated, as it requires unbiased ascertainment and correct analysis. Most people don’t know how to do this correctly. The data from good population studies can be very enlightening, for instance showing that smokers get lung cancer. The tobacco industry will have you believe that that is an example of post-hoc reasoning on a large scale, and I remember quotes out of their ‘scientists’ about “no one has ever seen cigarette smoke causing cancer.”

People have analyzed compounds from cigarette smoke and found carcinogens among them, for instance benzo-a-pyrenes. Cell biology has provided a crucial datum which can support the observation made, and therefore the hypothesis that smoking causes cancer. Also, at some point the number of smokers getting cancer becomes too great to ignore.

This is not true with many other popular medical conclusions. The numbers in the studies are not great enough, the statistical confidence is not high enough, and the causative action is foggy at best. I can say this about EM causing cancer, vaccines causing autism or brain damage, breast implants causing lupus or connective tissue disease, and a host of other problems. While on small scales, stories may be convincing, these effects vanish on a population scale.

In short, we rely on p-values to tell us if our data are significant. A gold standard is a p<0.05, which means that there is less than a 5% chance that a particular correlation is due to chance. If you think about it, though, this is quite bad. I can take a large set of data (for instance the Framingham survey or something) and perform 20 random correlations on the data, with no backing hypothesis. Probability will show one nice correlation. These observations mean nothing until fit into a broader hypothesis and have correlative experiments performed to support them.

Just because you are paranoid doesn’t mean that they are not out to get you.

Just because you are not paranoid doesn’t mean that they are not out to get you.

Just because you are paranoid doesn’t mean that they are out to get you.

Just because you are not paranoid doesn’t mean that they are out to get you.
Have we now covered all the possibilities without explaining any of the reasons why?

So we have:
A. Correlation does not imply causation.
B. Correlation does not exclude causation.

Both are correct statements. But since B is a true, A should be disregarded or given less weight? No, A is still true.

** FortMarcy’s** post could be read to say that correlation is irrelevant to causation. This conclusion would be silly. In science correlation is usually a key to proving causation, but one also needs the right conditions: falsifiable thesis, adequate sample size, random sample, etc.

FortMarcy’s point A is true under definition 1. Correlation doesn’t prove causation as a logical necessity. That’s important in science, but much less so here on SDMB, where we debate politics, religion, sociology, economics, morals, etc. Nothing could ever prove our theses as a logical necessity. So, it’s not saying much that correlation also doesn’t meet that standard.

We need to add another principle:

C. Correlation makes causation more likely to some degree. The degree depends upon various aspects of the correlation.

Oooh, I’d never guess that you were a GI actuary, december.

Have you had to do an exercise with some shitty data recently or something? Sometimes I think that a pricing exercise is composed of sitting around chanting “There is no such thing as a post hoc fallacy. There is no such thing as a post hoc fallacy”.

pan

Is there another kind? :confused:

But therein lies the problem. “Some degree” can be anywhere from 0% to 100% and it is unique for each instance. Once enough information is gathered to determine the degree for a specific instance, the correlation is not needed to as evidence.

Hah!

Actually - musn’t grumble. I’ve got a job on at the moment with complete and comprehensive information about every claim going back for years. And exposure data almost as good.

So apparently the prophets weren’t lying - the golden data really does exist.

Mind you - I haven’t tried reconciling it yet…
[/quote]
For the non-actuaries out there, the point I’m trying to make is that if there were absolutely nothing to tie together correlation and causation, your financial experts would be in biiig trouble. The past can be a guide to the future, so long as you approach it right. And it is only a guide because cause-and-effect, whether direct or indirect shows some consistancy.

First thing you learn when studying statistics: correlation does not imply causation.

First thing you learn when trying to use statistics: there bloody well better be some causation going on somewhere or you’re screwed.

pan

December Check out Stuart Vyse’s “Believing in Magic: The Psychology of Superstition”

Great book it it explains wondrous amounts of info on ‘The Gamblers Fallacy’ and your post hoc as well as ad hoc inconsistencies.

If I flip a coin and get ten heads in a row. Whats the Probability I’d get a tail next? Would you bet on it. Am I due for a tail??? Is there any such thing a being ‘due’ to get some thing?

Also the book is currently utilized in Grad applications for probabilities and stat classes. Granted it is mostly used here in New England but I have seen it at ASU as well.

Probability of getting a tail after ten heads is a classic example of Bayesian statistics.

Start with a prior distribution of P(head) = 0.5

Assume a parameter distribution for p.

Then posterior is proportional to parameter x prior.

Renormalise and bob’s your aunty’s uncle. You have a new estimation for the probability of a head.

Not sure what that’s got to do with post hoc fallacies though.

pan

Thanks, I will.

Yes, I agree this is a problem. IMHO it is a problem which ought to receive the lion’s share of attention in Great Debates.

FortMarcy, I’m unclear on the meaning of this sentence. Should the third word from the end be too, rather than to?

Assuming that’s what you meant, I would disagree. We seldom if ever are able to agree on the degree to which corellation implies causation – at least not for the topics debated here. Can you clarify your point with examples or further explanations?

Phlosphr wrote:

… I’d want to check your coin for proper balance to make sure you weren’t cheating.

Oops. The prior is a distribution for p, and the posterior for p is proportional to the prior x the distribution for heads.

As the alert reader would have spotted.

pan

I think it’s related. Suppose we’re debating whether God exists. If a religious person has a prior that gives, say 99% chance that God exists and 1% not, then evidence might lead to a shift in the probabilities.

However, if the prior is 100% that God exists, then the posterior will be 100% that God exists. In other words, if someone starts a debate totally certain of their position, then no evidence will sway them.

Then it’s not really a debate, then, but a rant.
Is this related to some of your other debates in the past month or so, december.

Guin, what does your word then refer to? Why isn’t this a debate?

IIRC two posters on different threads deigned to respond to an example, and simply mentoned the fallacy of post hoc ergo propter hoc. Unfortunately, I can’t recall where they were, or I’d provide a link.

I thought this topic might be a debate. Those who (mis)use the post hoc fallacy this way might want to dispute my point and justify their practice. I think FortMarcy was disagreeing with my POV.

Also, the deep questions of epistomology are by no means settled and agreed by philosphers of science, let alone by SDMB posters. There seems to be room for debate here.

We know intuitavely that correlation and causation are linked. If we see a strong correlation between two factions, and its a good bet there is a causative relationship, so it makes sense to research just how that underlying causation might work. e.g stat analysis shows a link between em radation and cancer, biologists get interested and take a look at what kind of effects em fields have on living things. they find out that only at ridiculously high levels do the fields have any observable effect, thus the causative link hypothesised is drastically weakened.

 In a debate, however, the underlying info is generally known, the debate is about the meaning of the data, to continue in the example above, a great debate now rages over em radation and cancer, some arguing the process of the study was flawed, bias, etc. nice lively discussion. now someone comes in and says 'what about that link between em fields and cancer, they MUST cause cancer' to which the response is 'post hoc'  shorthand for 'look, we all know the data, and there is a correlation but all data so far shows no known method for em field to cause or even influence cancer. It's up to you to prove your point, not me to disprove it.'

So basically, to sum it up in a short pithy statement, I’d offer a grant based on post hoc reasoning but I wouldn’t publish a paper based on post hoc reasoning.

As far as the last statement of the OP, could you be more specific as to which argument(s) you’re referring?

I presume kierk is asking about my statement, “In particular, I object to supporting a theory that has no evidence at all, because the evidence on the other side iscless than perfect.” Fair question. Here are some examples:
[list][]Religion (based on faith, rather than evidence) []Some economic arguments, such as the idea that minimum wage laws don’t cause job loss. Foreign policy ideas that are merely based on general principles, such as unilateral disarmament.

It is kinda related to the old “arguement from authority fallacy”. Yes, indeed, if we are debating, say the death penalty, and some nimrod quotes a “respected authority” that claims the “death penalty is immoral”- then, yes, that same dude has committed the “authority fallacy”. However, if another dude quotes an authority that shows that the “death penalty is not a deterrant”- and argues from that cite that, then; the “death penalty is wrong”- then they are NOT using the “authority fallacy”. You can argue that his conclusion is invalid- or- you can cite another source that shows the death penalty IS a deterrant. But saying that he is “argueing from authority” is wrong. (Note, also wrong is “attacking his source”. If you think his source is not kosher- come up with a better source, then point out why you feel your source is better.)

So, yes- argueing that “since a>b, then a causes b” may well be incorrect- however, as long as you are aware that the two might not be related, you’re OK. I know some dudes that claim that the 9/11 WTC did not lead to the bombing of the Taliban (“It is all about the Oil”), but it seems to me that they are … what’s the technical term?.. Oh yes… wrong. :slight_smile: