Lets discuss Bayesian Statistics

[Kimmy_Gibbler]

@Left_Hand_of_Dorkness

Definitely doesn’t help me! For one thing, you’re trying to solve the probability of B|C, but since you already know the probability of B=1 in your example, that’s not necessary, I don’t think. For another, introducing a whole different set of numbers from the OP’s makes it much more confusing. But maybe it’ll help someone else.

B is the probability that the first native you encounter would have (not “does have”) a bone in his nose. There is some philosophical hay to be made over the counterfactual nature of the proposition, but in the real world I don’t think it should detain us.

If someone joined your club and you were the first person he met at his first meeting he attended and it happened to come up that you share a birthday and the new member said “Wow! The first person I met here has the same birthday as me! What are the chances of that!?” You would not be bewildered and say “Well, obviously, it is 1.”

So I think you’re being a little disingenous here because I came up with a better example than you did.

[Left_Hand_of_Dorkness]

@Kimmy_Gibbler

So I think you’re being a little disingenous here because I came up with a better example than you did.

:rolleyes: Oh, for pity’s sake, you caught me. I’m soooo jealous of your superior example.

I’ll leave it to the readers to discuss.

[Maeglin]

@Evil_Economist

Now, that’s not to say that we can’t apply Bayesian statistics in this case, we just have to use the right data. We need to know the percentage of black strippers who are raped while alone in a house with drunk frat boys, and we need to know the percentage of women who claim they were raped during a drunken frat party who are lying, and then maybe we can start making some estimates. But if we start applying population statistics to the wrong population, then we will get the wrong answer. And it won’t just be wrong, it’ll probably be more wrong that if we just ignored the statistics entirely.

This error isn’t due to Bayesian inference but to the mistaken assumption that human behavior is ergodic. You rightly point out that it’s not. The behavior of individual people over time does not tend to converge to the population average when incentives and circumstances differ. Statistical inference, whether Bayesian or frequentist, is very limited when behavior is really non-ergodic.

[Maeglin]

@Left_Hand_of_Dorkness

Definitely doesn’t help me! For one thing, you’re trying to solve the probability of B|C, but since you already know the probability of B=1 in your example, that’s not necessary, I don’t think. For another, introducing a whole different set of numbers from the OP’s makes it much more confusing. But maybe it’ll help someone else.

No, Kimmy is right. There are two states of the world. You can observe one directly (bone in the nose) and one you can’t (which island you are on). But these states of the world are somehow connected: you can learn something about the unobservable state from the observable one. Bayes’ Rule is a way to update your beliefs about what you can’t observe directly based on something you can. The numbers often produce some counterintuitive results, but it is a remarkably good model for how people actually think.

[Left_Hand_of_Dorkness]

@Maeglin

No, Kimmy is right. There are two states of the world. You can observe one directly (bone in the nose) and one you can’t (which island you are on). But these states of the world are somehow connected: you can learn something about the unobservable state from the observable one. Bayes’ Rule is a way to update your beliefs about what you can’t observe directly based on something you can. The numbers often produce some counterintuitive results, but it is a remarkably good model for how people actually think.

Yes, I get this. My question is whether, given what we now know about B, we need to figure out B|C. Maybe we do, but I don’t think so.

edit: wait, I may be misunderstanding notation. Does “|” mean “and” or does it mean “given the first proposition, the probability of the second”? If it means the latter, then I withdraw my question in shame.

[Maeglin]

@Left_Hand_of_Dorkness

Yes, I get this. My question is whether, given what we now know about B, we need to figure out B|C. Maybe we do, but I don’t think so.

edit: wait, I may be misunderstanding notation. Does “|” mean “and” or does it mean “given the first proposition, the probability of the second”? If it means the latter, then I withdraw my question in shame.

Yes. Prob(B|C) is “the probability of B given C”. The rest is apparently self-explanatory.

ETA: For anyone interested, Conditional probability - Wikipedia

[you_with_the_face]

The problem with applying Bayesian inferences to the real-world situations is that how we slice-and-dice human beings is largely arbitrary, and you’ll draw difference inferences depending on the variable you’re fixated on.

What’s the odds that a black woman will lie about being raped by a white man, if we have no other information? Let’s say the probability is 3%.

What’s the odds that a woman whose favorite color is purple will lie about being raped by a white man, if we have no other information? Let’s say the probability is 2%.

What’s the odds that an overweight woman will lie about being raped by a white man? Let’s saying the probabilty is 7%.

If you’re assessing a specific situation, and all you know is the race of the accuser and the accused, her favorite color (purple), and her BMI, how will you know which set of probabilities to hang your hat on? Whatever variable you fixate on, it’s going to be purely abitrary, and so in turn so will be your estimate for truthfulness. Which means it’s a completely useless way of assessing credibility.

[monstro]

Bricker, I kind of asked this question in some unholy incarnation of the original trainwreak thread, and you never addressed it. Perhaps now that things are calmer, you will please oblige me a minute of your time. (I’ll be nice as long as you are!)

You are an attorney for someone accused of embezzling money.

Anderson Cooper 360 devotes twenty minutes to a segment focused on findings from a Harvard study of white collar crime. The most surprising finding: 3% of convicted white collar criminals have “Hennessy” as a surname. This is two orders of magnitude greater than this name’s representation in the general population.

Your client’s last name? Hennessy!

It’s time to select the jury.

Would you screen out people who admit to watching Anderson Cooper?

Why or why not?

Are you more skeptical of your guy’s claim of innocence than you would be if his last name was something else?

Why or why not?

[Maeglin]

@you_with_the_face

The problem with applying Bayesian inferences to the real-world situations is that how we slice-and-dice human beings is largely arbitrary, and you’ll draw difference inferences depending on the variable you’re fixated on.

[/QUOTE]

This is partly true but I have a different interpretation.

As a technical aside, you don’t need to fixate on only one variable. You can update beliefs about the likelihood of an unobserved event with as many variables you like. Even though the calculations change slightly, the intuition is exactly the same.

But you make a fair point. What you get out of inference is what you put into it. It is perhaps more helpful to think of Bayesian inference as a way of updating your beliefs rationally rather than drawing ironclad conclusions about the real world. For example, the assertion that extraordinary claims require extraordinary evidence is just an implication of Bayes’ Rule. If you have a strong prior belief that something is unlikely, then it takes a very large amount of evidence to move your belief such that you accept that the event is likely. Anyone can play with the formula by plugging in numbers to see this for him/herself.

But what is an extraordinary claim to one person is trivial to another depending on his/her prior beliefs about the likelihood of the phenomenon. Different people react completely differently when confronted with the same evidence even if they all update their beliefs rationally via Bayes’ Rule. What happens when beliefs collide with evidence depends on the beliefs you start out with. Bayes’ Rule just articulates the relationship between prior and posterior probabilities. It does not fully determine these posterior probabilities. There is a lot of room to make arguments.

It is very easy to calculate right and wrong beliefs when the prior probability of an event happening is generally agreed on. But when we don’t know the prior probability, we have to guess. Opinion, experience, and bias all influence this choices. This doesn’t invalidate the rule, but it does highlight the fact that we are fallible creatures who really aren’t very good at making probabilistic guesses about unobserved things. This doesn’t mean that our inferences are invalid, it just means that we run a big risk of being wrong and should adjust our decision-making to account for this.

[Bricker]

@monstro

Would you screen out people who admit to watching Anderson Cooper?

I would certai ly try, because I would be worried that those people, knowing nothing about my client, would enter the jury box predisposed to infer guilt.

Are you more skeptical of your guy’s claim of innocence than you would be if his last name was something else?

Neither more or less skeptical, because my opinions would be formed from my preparation of the case.

[monstro]

@Bricker

I would certai ly try, because I would be worried that those people, knowing nothing about my client, would enter the jury box predisposed to infer guilt.
Neither more or less skeptical, because my opinions would be formed from my preparation of the case.

This makes me think you understand why using statistics to make judgments about individual cases is not a good idea.

Because I could replace “Hennesy” with any variable, right? Gender. Race. Underwear preference. Favorite color. Cholesterol levels. We can compile statistics for an infinite number of parameters, but only a small handful would be related to a person’s credibility. Without knowing which of those variables are related to credibility, it’s foolhardy to just go with the one that happens to be documented in someone’s database (such as race).

If someone were to tell me, “Well, of course I know the race of the accuser/accused shouldn’t be the sole factor in determining credibility. But I think it’s deserving of attention until I get more information”, they are admitting to me that they are allowing themselves to be prejudiced by irrelevant information. A jury member will not wipe their memory of the Anderson 360 segment once they hear your client’s side of the story. They will evaluate your client’s story against that Anderson 360 story. That is wrong. And therefore it is wrong to entertain that type of thought process even if it’s just “in theory”.

Unless a person can demonstrate a causal relationship between a variable (gender, race, ethnicity, BMI, etc.) and credibility, picking one variable out of all the possibilities is just not a good thing to do. At all.

Credibility is also not the same as likelihood. Red cars are a lot more common than purple cars. But if a trusted friend were to tell you that a purple car slammed into him, you would probably believe him. Because even though red cars are more common, this is irrevelant to the likelihood of Person X slamming his purple car into your friend, Person Y–where X and Y have their idiosyncratic risk factors for reckless driving. To be more explicit, even though white women are more likely to be raped by white guys, this is irrevelant to the likelihood of a specific group of white men, with their own proclivities and risks of violence and prospensities for dishonesty, attacking Woman Z, who has her own level of credibility and risk factors for craziness.

[Maeglin]

@monstro

This makes me think you understand why using statistics to make judgments about individual cases is not a good idea.

This is the ergodicity problem. The likelihood that an individual does something does not converge over time with the base rate of the underlying population. The odds of you committing a crime do not, over the course of your life, converge with the incidence of black female criminality.

There is nothing wrong with the inferential process, though. It’s just that anyone making the inferences just has to be aware that he is making a heroic assumption if he holds as his prior belief that the odds of you committing a crime are the same as those of the population from which, in some (possibly meaningless) way, you are drawn.

[ultrafilter]

@Acsenray

What about the more general issue? I’m not interested in arguing about what Bricker did or didn’t say, but I think there’s something here that’s more broadly relevant to public discourse.

Isn’t there some basic logical fallacy regarding taking generalized statistics and using them to determine the facts of a particular case?

For example. I have flipped a quarter 100 times and 75 times it has come up heads and 25 times it has come up tails. So then I say, “According to the statistics, there is a 75 percent chance that the next flip will be heads.”

Now, we know that this is wrong, because we know the probabilities regarding a flipped coin. But what if we didn’t? Isn’t it still some kind of fallacy to take a generalized statistic and apply it to a specific case when we have not shown that the statistic grows out of an inherent characteristic of the coin?

If you’re flipping a fair coin 100 times, you’d expect to see 75 or more heads about once every 350,000 attempts. I’d be skeptical that your coin is fair.

[cerberus]

An Application of Conditional Probability

D
D*
Total
T
Disease Present, Screens +
Disease Absent, Screens +
Screens +
T*
Disease Present, Screens -
Disease Absent, Screens -
Screens -
Total
Disease Present
Disease Absent
Total

There are four possible combinations of Disease and Test:

DandT ~ Disease Present and Screen Shows Positive
DandT* ~ Disease Present and Screen Shows Negative

DandT ~ Disease Absent and Screen Shows Positive
DandT* ~ Disease Absent and Screen Shows Negative

Pr{D|T} ~ Probability that a subject has the disease, given a positive test
Pr{D*|T} ~ Probability that a subject lacks the disease, given a positive test

Pr{D*|T*} ~ Probability that a subject lacks the disease, given a negative test
Pr{D|T*} ~ Probability that a subject has the disease, given a negative test

What is Pr{D|T}, the probability that disease is present given a positive screen?

Pr{D|T} =
Pr{DandT}/Pr{T} =
Pr{T|D}*Pr{D}/Pr{T} =
Pr{T|D}Pr{D}/(Pr{TandD} + Pr{TandD}) =
Pr{T|D}*Pr{D}/(Pr{T|D}Pr{D} + Pr{T|D}Pr{D})

That is,

Pr{D|T} = ( Pr{T|D}*Pr{D} ) / ( Pr{T|D}Pr{D} + Pr{T|D}Pr{D} )

So we need the prevalence of disease (Pr{D}), Pr{T|D} = 1 - Pr{T*|D}, where Pr{T*|D} is the false-negative rate, and Pr{T|D*}, the false positive rate.

Suppose that we are given that Pr{T|D*} = “False Positive” = .05, Pr{T*|D} = “False Negative” = .01 and Pr{D} = .001. Then Pr{T|D*} = “False Positive” = .05, so then Pr{T*|D*} = .95 and Pr{T*|D} = “False Negative” = .01, so then Pr{T|D} = .99

Plugging in what we know:

Pr{D|T} =(.99)(.001)/(.99(.001) + .05*(.999)) = .00099/(.00099+.04995) = .019 (<2%). So in this case, approximately 2% of the positive tests actually indicate disease, which leaves the other 98% with a false finding.

Let’s repeat the calculation, but with Pr{T|D*} = “False Positive” = .005, Pr{T*|D} = “False Negative” = .005 and Pr{D} = .001. Then Pr{T|D*} = “False Positive” = .05, so then Pr{T*|D*} = 1 - Pr{T|D*} = 1 - .005 = .995 and then Pr{T|D} = 1 - Pr{T*|D} = 1 - .005 = .995.

Pr{D|T} =(.995)(.001)/(.995(.001) + .005*(.999)) = .00099/(.00099+.04995) = .16611 (16%). So in this case, approximately 16% of the positive tests actually indicate disease, which leaves the other 84% with a false finding.

What is Pr{D*|T*}, the probability that disease is absent given a negative screen?

Pr{D*|T*} =
Pr{DandT}/Pr{T*} =
Pr{T*|D*}Pr{D}/Pr{T*} =
Pr{T*|D*}Pr{D}/(Pr{TandD} + Pr{TandD)) =
Pr{T|D*}Pr{D}/(Pr{T*|D*}Pr{D} + Pr{T*|D}*Pr{D})

That is,

Pr{D*|T*} = Pr{T*|D*}Pr{D}/(Pr{T*|D*}Pr{D} + Pr{T*|D}*Pr{D})

We need the prevalence of disease (Pr{D}), Pr{T*|D*} = 1 - Pr{T|D*}, where Pr{T|D*} is the false-positive rate, and Pr{T*|D}, the false negative rate. Recall that Pr{D*) = 1 - Pr{D}.

Suppose that we are given that Pr{T|D*} = “False Positive” = .05, Pr{T*|D} = “False Negative” = .01 and Pr{D} = .001. Then Pr{T|D*} = “False Positive” = .05, so then Pr{T*|D*} = .95 and Pr{T*|D} = “False Negative” = .01, so then Pr{T|D} = .99

Plugging in what we know:

Pr{D*|T*} = (.95)(.999)/( (.95)(.999) + (.01)*(.001)) = .9999+. So in this case, approximately 99.99% of the negative tests actually indicate absence of disease.

Let’s repeat the calculation, but with Pr{T|D*} = “False Positive” = .005, Pr{T*|D} = “False Negative” = .005 and Pr{D} = .001. Then Pr{T|D*} = “False Positive” = .05, so then Pr{T*|D*} = 1 - Pr{T|D*} = 1 - .005 = .995 and then Pr{T|D} = 1 - Pr{T*|D} = 1 - .005 = .995.

Pr{D*|T*} = .995*.999/(.995*.999 + .005*.001) = .9999+. So in this case, approximately 99.99% of the negative tests actually indicate absence of disease.

What this application suggests is that the likely problem with a diagnostic test is the proper interpretation of positive tests.

[monstro]

@Maeglin

There is nothing wrong with the inferential process, though.

I don’t mean to imply that you can never infer anything from anything. You can. I do it all the time. But if I’m trying to explain why, say, dissolved oxygen is decreasing over time in a particular waterbody, I don’t seek an explanation by studying all the variables in my database. I will look at only those that I know are related closely enough to indicate cause-and-effect. I’ll look at chlorophyll and nutrient levels. I won’t look at salinity or arsenic concentration, even if those are the only parameters in the database.

If I’m judging the credibility of a person, I will look at their criminal background and the trustworthiness of their past claims. But I won’t consider their favorite color or the length of their mustache. It’s not that I know for a fact that these things aren’t related to credibility. It’s just that I can’t surmise how they would be.

I know I’m not discussing Bayesian deduction and apologize for hijacking.

[Acsenray]

@ultrafilter

If you’re flipping a fair coin 100 times, you’d expect to see 75 or more heads about once every 350,000 attempts. I’d be skeptical that your coin is fair.

That’s the issue though. 1:350,000 is not an astronomical figure. If you are performing that test just once, there’s not sufficient basis for suspecting a wonky coin. It’s going to happen once in a while.

[ultrafilter]

@Acsenray

That’s the issue though. 1:350,000 is not an astronomical figure. If you are performing that test just once, there’s not sufficient basis for suspecting a wonky coin. It’s going to happen once in a while.

Two questions:
[ol]
[li]What would cause you to suspect that the coin is not fair?[/li][li]If I’m willing to bet $100 that the coin is not fair, how much would you be willing to bet against me?[/li][/ol]

[Autolycus]

@Kimmy_Gibbler

You also know this: On Cannibal Island, 90% of the inhabitants wear bones in their noses. This is not the fashion on Vegetarian Island, where only 8% of the inhabitants do.

That goes without saying. The fashion on Vegetarian Island is to tie an onion to your belt.

[Acsenray]

@ultrafilter

Two questions:
What would cause you to suspect that the coin is not fair?

Having a suspicion and having a scientific basis for a conclusion are two different things.

If I’m willing to bet $100 that the coin is not fair, how much would you be willing to bet against me?

Bet? Are you serious? This is how you approach a statistical question? Betting is not a scientific exercise. You still might be wrong, because in 350,000 cases, you would expect this to happen once. So why couldn’t this be that once?

But you’re missing the entire point of my example. It’s about taking a isolated statistic and assuming that it tells you something about an individual case. We are talking about a situation in which the observer, for whatever reason, doesn’t know what the actual probabilities are.

[Bricker]

@monstro

That is wrong. And therefore it is wrong to entertain that type of thought process even if it’s just “in theory”.

I don’t understand this logical leap.

Why is it wrong to point out that, in theory, a theory that represents an unobtainable zero-level “if we knew nothing else,” state that exists only in rarified theory-land… Why can’t this be mentioned? It’s true. It’s meaningless in any real world situation, but it’s true. I feel like I’m defending frictionless surfaces or something.

If the statement is accurate, let’s say it’s accurate. If it’s also utterly meaningless in the real world, let’s say that too.