december
Probability and odds have completely different meanings. If kesagiri had meant odds, he/she should have said odds. Compare these two statements: “Swinburn concluded that the probability was 97%” vs. “Swinburn concluded that he would give odds of 97%”. The first is a statement about the world, the second about Swinburn.
Odds are just another way of expressing probability. E.g., if a coin has a 70% chance of coming up heads, then one could equally well say that the odds of coming up heads are 7 to 3.
The relation of probabilities to odds would be the same, whether we’re addressing a frequentist or Bayesian POV.
No, odds represent what people think the probability is.
December is right. Odds are just another way of expressing probability, and can be useful for some purposes.
For example, the odds of picking an Ace out of a freshly shuffled deck of cards are 12 to 1, which is the same as saying 1 in 13, which is the same as saying 7.7%. That is not an estimate, or a judgement call. It is a precise statement that accurately reflects the probability.
Odds are used a lot in gambling because they better represent the situation presented to the gamber. The odds of rolling a 4 on a six-sided dice are 5 to 1. If you are only offering me 4 to 1 on my bet, you have an advantage. If you offer 6 to 1, I have an advantage. That’s a lot easier for a player to get his head around than saying you have a 16.7% chance of rolling a 4, and I’ll pay you three dollars if you win and charge you a dollar if you lose.
What’s tripping you up is that because gambling uses odds a lot in describing probability, you’ve come to believe that it’s a judgement call. (“I think there’s a 4 to 1 chance that my horse will win”).
Kesagirl, I think you misunderstood my criticism. My criticism was not of Bayesian analysis at all, but of the use of it in trying to assign hard probabililities to biblical events. My point was that the terms of the equation are largely matters of faith, so any result you get out of that equation will also be a matter of faith. For the terms to be valid (at least, the terms that they used, such as counting miracles), you have to start with the a priori assumption that God exists. But if you’re going to do that, the whole exercise is meaningless. Garbage in, Garbage out.
Now, if you could construct a much more complex Bayesian analysis and do hard scholarship to assign real probabilities to every possible explanation of the events of the bible, then maybe something interesting would come out the other end. The trick would be even coming close to a semblance of accuracy when analyzing the historicity of a 2000+ year old document.
Basically, the problem is not of Bayesian analysis, but of simplistic models. The analysis is only as good as the assumptions fed into it.
Just to join in with the others in the pig-pile. No, when you have subjectivity probabilities they also depend on people’s beliefs.
But this is true even of more mundane uses of Bayesian analsysis. Granted it is not quite as…eyebrow raising, but nonetheless the assignment of priors in many instances do not depend solely on data.
Are we debating the utility of the Bayesian approach or are we debating whether one particular application of it was skillful? (I thought it was the latter).
Correct. The Bayesian method with a “noninformative prior” gives identical results to the Frequentist method.
Some argue that within most scholarly contexts, a non-informative prior is appropriate. Possible exceptions include literature reviews and meta-analyses, although I understand that they can be evaluated with frequentist methodology as well.
Decision makers, OTOH, may want to consider Bayesian techniques.
Yeah, that’s what the Bayesians say. I reply, “Terrific, show me the software and give me some rules of thumb for constructing prior distributions and I’ll think about it”.
(Possibly useful reference for the latter: Morgan and Henrion, Uncertainty.)
Q: Am I wrong or don’t Bayesians make implicit model selection choices? In practice. I don’t mean to be dismissive of Bayesian techniques, it’s just that I’ve never seen the 2 approaches contrasted in a nonhypothetical way. OTOH, I haven’t really looked for such an exercise either.
Just because some odds are accurate, that doesn’t ean that all odds are accurate.
First, telling people what’s tripping the up is somewhat presumptious. Second, the fact is that it is a judgement call, at least in some cases. For instance, deciding which horse is more likely to win is a judgement call. Third, there clearly are two different concepts here. Given that there are two different concepts, and that it is accepted practice among many people in this field to restrict the term “probability” to random variables satisfying the definition set out by probability theory, doesn’t it make sense to follow their lead?
I realize that what I am saying is at odds, if you’ll excuse the pun, with how many people use these terms, but it is completely consistent with how mathematics view the ters. It seems to me that if we’re going to use mathematical techniques, we should use mathematical nomclamature as well.
BTW, it’s kesagiri, not kesagirl.
kesagiri
From a mathematical point of view, “subjective probability” is an oxymoron.
From Data in Doubt by John D. Hey.
Subjective Probability:
p. 5: “Central to this book is the idea that probability is subjective; that is, that probability statements are expressions of subjective belief.”
Objective Probability, p. 129
“The crucial difference is that Classical statistics regards probability as an objective concept, and thus not a statement of subjective belief. This implies inter alia that probability statements can not be made about fixed numbers - such as the actual proportion with some characteristic in the population. Thus, a Classical statistician would regard the statement ‘the probability is .95 that P lies between .168 and .232’ as meaningless: to him or her, P either lies in that interval or it does not.”
Emphasis in original.
(Instead, Classical statisticians talk of P lying in an x% confidence interval. The idea is that of all 95% confidence intervals for P, 5% will not contain P.)
Hay continues parenthetically: “(My own view is that the Classical notion of a confidence interval is highly contorted, and that most students, even when ostensibly operating within the Classical school, implicitly employ the more intuitive Bayesian interpretation.)”
I would add that there is a distinction between the machinery of stochastic theory and the interpretation of it. It is the interpretation of it that makes up one interesting part of the Bayesian / Classical debate. (Indeed, I seriously doubt whether many Classical statisticians would deny Bayes Theorem. Not that anybody has asserted that they would.)
TheRyan: You are making a distinction where none exists. You are trying to claim a different definition for ‘odds’ as opposed to ‘probability’, claiming that one is typically used in a more subjective way.
That’s just not the case. They are just two ways of stating the same thing. Even in the gambling world, the term ‘odds’ is usually used in a very precise manner. Roulette, craps, blackjack, poker, you name it - these games are all typically described in terms of odds of certain outcomes, but none of this is subjective - those odds are mathematically provable.
But hey, thanks for the heads up on Kesagiri/Kesagirl. You know, it’s funny - I’ve been reading that particular poster’s messages for a long time, and I ALWAYS read the name as ‘Kesagirl’. I guess I just fit the name into a pattern that I inferred, and never looked at it closely again.
Kesagiri: My apologies for confusing your name. Especially if you are not, in fact, a girl. Or maybe especially if you are. It was certainly unintentional.
Aren’t both topics for “great debates”? Sorry…stupid joke. I’ll flagellate myself with a noodle. Besides I think most here agree the usage above is…a bit silly.
Is the use of noninformative priors necessary for this? It was my impression that this was not so.
But with the Classical analysis you are also selecting a prior (at least so the Bayesian would say). If it is a noninformative prior why select that if you have (non-data based) reasons to believe something specific about whatever you are analyzing.
Here is a specific example. You have a series on the number of customers for a given electricity rate. You find out the rate is going to be closed to new customers. You also know that whenever a customer on that rate moves they have to go off the rate (rates apply to addresses not individuals). So you have a damn good idea the number of customers are going to be decreasing from here on out. However, you have no customer data to reflect this belief. Now it is a reasonable belief, no? With the Classical approach you are implicitly choosing a prior that ignores this information.
Yeah, selecting the appropriate prior is tough, but there ins’t anything stopping you from selecting several priors that would allow you to evaluate which one performs the best.
As for software, well SAS has something called, IIRC, timsac or some such that uses their ILM (interactive matrix language) module for doing Bayesian analysis.
Also there is a program called BATS. And software such as Gauss and O-matrix are more than sophisticated enough to allow for Bayesian analysis.
Sam,
No problems on the name thing. Heck I didn’t even catch it.
No, there most definitely is a distinction. You may not agree with the terms used to describe the different concepts, but there is a clear difference between “I have a 90% confidence that this will happen” and “This happens 90% of the time”.
As I stated before, the fact that “odds” is used in a precise manner in no way proves that it means a precise mathematical probability. Just because some A are B, that doesn’t mean that all A are B.
Observation #1) Being able to merge statistical and impressionistic information is certainly very cool. IMHO.
Certain problems are amenable to a Bayesian approach. Sam gave a good example, methinks. A superior gambler may have beliefs about the sort of opponent she is facing. In this case, Bayes theorem could allow her to combine strictly probabilistic information as well as her assessment about the behaviors of her fellow players.
If we are evaluating what a specific dataset can tell us about a given problem, I would assert that assuming a noninformative prior is likely to be the most illuminating. So the frequentist approach is (generally) best for investigative purposes.
3b) When a decision or point-estimate is desired, a Bayesian approach might be considered.
Furthermore, December has shown one danger of a Bayesian approach: the priors can be cooked to given any conclusion you want. (Similarly for the sensitivity analysis). I’m not claiming that this problem can’t be overcome, however.
Within scientific investigation, protocols are established to address the bias of the investigator. (eg double-blind studies). I would assert that Bayesian methods are more vulnerable to this sort of bias than Classical methods. It is true that the Frequentist makes certain arbitrary specification assumptions. But constructing a prior -and just as important evaluating a prior- is a trickier and more demanding endeavor, one that is prone to intentional or inadvertent investigator bias. IMHO.
This can all happen in frequentist analysis as well. Here is a counter example (real world)
So by ignoring prior information you are, in a way, biasing your results.
kesagiri: Very nice example. Thanks.
I’ve reached the edge of my knowledge on this topic, so I’m afraid I can only muse.
It is my understanding that there are frequentist methods of meta-analysis, none of which I am familiar with. I would like to see a contrast between Bayesian and frequentist methods of meta-analysis.
Assuming I understand it, your example appears to use a Bayesian approach to input results from previous studies into a more recent study. This is often done implicitly in the literature review, but adding some rigor to it appears useful to me.
That said, taking (prior) distributions from previous studies is a different exercise than constructing a prior basis on the investigator’s beliefs. The latter introduces certain problems that need addressing. I do not claim that these problems are insurmountable.
My summary of Kesagiri’s example
4) Frequentists may very well have a tendency to place too much weight on the particular dataset that they have worked on. A Bayesian approach could potentially offset this bias.
I believe one traditional disadvantage of Bayesian methods is that the calculations were more difficult. Now that the computer has solved that problem, Bayesian analysis is becoming more popular.
There is an interesting article on the future of statistics by, IIRC, James Berger. Let me see if I can find the link.
Berger points out that both the Frequentist and the Bayesian methods have some good elements to them. For example, he points to nonparametric analysis.
For those of you who want more info on this here is the link to Duke University’s Institute for Statistics and Decision Sciences.
You’ll need something to read postscript files (I recommend ghostview–which is free).
This is an intriguing thread - I find myself coming to the aid of The Ryan and december.
Firstly december’s point - I can’t get at the first linked article, but I assume that the “resurrection” analysis only performs one Bayesian iteration? In which case the priors are critically important and what we are really seeing is a reflection of the assumptions - the data only has a limited chance to pull us back.
As december says, Bayesian statistics plays a key role in insurance. But one always bears in mind the importance of the prior assumption. After all if the data is that good to make the prior irrelevant then there is no need to use a Bayesian approach in the first place.
Secondly The Ryan’s (and ultrafilter’s) point what does “97% chance that Jesus was resurrected” actually mean? Because - and this is they key issue - the issue of Jesus’ resurrection was not a probabilistic event. One cannot say:
P(Jesus resurrected) = 97%
because then you’d have to be able to rewrite it as
P(J = res) = 97% where J is a random variable and there is no random variable here.
What we are really saying is (assuming the analysis is worthwhile, which I dispute): given what we know (and what we believe are reasonable starting points), the statement “Jesus was resurrected” has a 97% chance of being true.
This, as The Ryan has said, is a statement that reflects on the prior knowledge and data - i.e. the analyser - not on the event itself.
One of the biggest mistakes you see amongst those who have had elementary training in statistics is the mixing up of random variables and variables, leading to meaningless equations and misunderstood concepts. “Jesus was resurrected with probability 97%” has such a misunderstanding written all over it.
pan
kabbes: From a classical perspective the statement, “There is a 97% chance that x occurred” is indeed meaningless: the probability is either zero or one. You are correct that this is a common misunderstanding of conventional statistical techniques. (Heck, I make this verbal error all too often.)
However. The Bayesian approach allows for subjective probability, something that the Classical model explicitly forecloses. That is, Bayesians maintain it is meaningful to use a probabilistic framework to describe one’s views about an hypothesis. And since all Frequentist statistical machinery can be nested within a Bayesian approach (assuming noninformative priors), those elementary misunderstandings alluded to above can be transformed into meaningful (Bayesian) statements.
Specifically, Prob(J resurrected)=97%
can be a valid description of the intensity of somebody’s belief.
To quote John Hey in Data in Doubt: “…the crucial difference between Classical and Bayesian statistics is that in the former probability statements about (fixed) parameters are not allowed, while in the latter they are not only permitted but they are mandatory.”
Quite true Flowbark, I have no issue with any of that. But by your own admission, we are still talking about intensity of belief rather than the event itself, meaning that the result in this case in particular (where the priors are so subjective and fairly meaningless and data so nebulous) says more about the believer than it does about the believed.
pan