But you are taking the position that the probability is something that manifests in a particular sequence of coin flip results, rather than in a distribution across many possible sequences. This is something I find hard to make tenable.
Suppose I flip the coin infinitely often and it comes up with a limiting heads/everything ratio of x. Then I flip it infinitely often again and it comes up with a limiting heads/everything ratio of y. Is its heads probability x or is it y?
On the account of “probability” in which “probability” just means “the value assigned by a probability distribution”, and “probability distribution” in turn just means “A way of assigning numbers >= 0 to every possibility such that the values assigned to any exclusive and exhaustive series of events adds up to 1”, there’s certainly no difficulty in each flip independently having probability 1/2 of coming up heads, and yet the sequence of all heads remaining in the sample space of possibilities. Indeed, this is, I would think, the simplest model… Of course, you seem to be interpreting the word “probability” more strongly than that. But in a way which I think is hard to make strict coherence of.
Mind you, by the way, this is not the same objection as saying “probability zero events never happen”. If I had a hypothetical coin which always strictly alternated between heads and tails, then the ratio of the number of heads to the total number of flips would approach 1/2.
If the coin actually behaves in that way, then I would say that it does not have a well-defined probability of coming up heads. Why, how would you define probability? Better yet, if someone handed you a coin and asked you if it were fair, how would you determine the answer?
As an ordinary language, real-world question? Not by making infinitely many flips, of course. No one would or could. As an ordinary language, real-world question, to ask whether a coin is fair is not a question about the result of flipping it infinitely often, I say.
As a mathematical question, with the definition you gave of “fairness” as “each flip is independent and has heads and tails equiprobable”? I would say “Fairness is not a probability of any particular outcome; it’s a property of the probability distribution as a whole.”
Suppose you had a coin which you flipped infinitely often, and got a limiting heads/tails ratio of 1/2 with. Then I say “Aha! But look: on these infinitely many trials, it came up all heads”, pointing to the infinitely many times it came up heads. “So, from this particular subset, we observe a probability 1 of heads.” “And from this subset, a probability 0.73 of heads”. And so on. And this is in some sense precisely what you just called a situation where a coin does not have a well-defined probability of coming up heads; I can’t possibly arrange for every subsequence of infinite trials to have a limiting ratio of 1/2.
Every infinite series of coin flips with well-defined limiting ratios for whatever properties does give rise to a probability distribution (in the mathematical sense) for those properties, of course, just by taking the probabilities to be those ratios. But I don’t take this to mean that whenever we speak of a probability, we therefore must mean precisely the values determined in this way by some distinguished infinite series of trials. (As a technical term, I don’t take “probability” to mean anything except “the value assigned by a probability distribution”, and as an ordinary language term, I take it to mean a number of different pre-formal concepts, not all entirely coherent.)
That’s a perfectly reasonable statistical line of thinking, but from a mathematical standpoint, it doesn’t fly. It’s perfectly reasonable to define probability on the space of infinite sequences of coin flips in such a way that the probability of heads at any fixed element of the sequence is one half without denying that one possible sequence of coin flips is all heads.
But then again, because I am also amenable to making the idea work (the more ideas to explore, the better), I could arrange to have every “priorly definable” subsequence have a limiting ratio of 1/2, in some sense (the same sense as I mentioned before for contriving to identify “probability 1” with “necessarily true”), and so I could make things coherent that way. Which is fine; that was the point of my bringing up that way of thinking about things.
So, there’s that ambivalence again over whether this is a good or bad way to think formally about the pre-formal concept of “probability”. But, regardless of the fact that one can perhaps tame it into some kind of coherence, I think there are more thorns in your frequentist account of “probability” than you may realize or acknowledge.
Let me ask you this, Chronos: when you say each flip of the coin is independent from all the other flips, what does that mean? How would you define that in terms of a limiting ratio? If I asked “Is flip 7 independent from flip 18”, would that be a meaningful question, and if so, what would make the answer “Yes”?
(I could ask the same sort of question more simply about “Is flip 7, specifically, a flip with heads probability 1/2?”, but I think the above instance may better highlight the issue I am getting at with this query)
@Shagnasty: In the technical sense of “expected”, 10 + 1/2 * 990 = 505 heads. You are of course referring to the gambler’s fallacy, the belief that some force will cause the coin to act to balance out in the long run, when in reality, the coin is memoryless.
@Chronos: I told you how a mathematician would. As a mathematical definition, I would say probability is only defined relative to a probability distribution. A probability distribution is just a way of assigning values >= 0 to events in such a way as that exhaustive exclusive sets of events have values that sum up to 1. The connection between a probability distribution, in this sense, and the real world depends on what one intends to model about the real world; it’s up to you to pick a probability distribution that models whatever it is that you want to model. If you want to model limiting ratios of infinite series of trials, then, go ahead; as I said, that will produce a probability distribution in this sense, sure. But there are precious few infinite series of trials in the real world, so perhaps, when one sits down to think about it, that’s often not the sort of thing one is really modelling, even when one thinks it is…
In case anyone wants to nitpick me on this, the Kolmogorov axioms only impose this for countable series of events. I’ll put some wiggle room on the exact mathematical formalization of “probability distribution” to take; anything in the spirit of what I’ve quoted works for me as worthy of living under the umbrella of the term. The point being, a probability distribution, in this sense, is just any way of assigning weights to things so as to be able to take weighted averages; nothing’s forcing those weights to have any particular interpretation except your own particular goals in taking the weighted averages.
Isn’t a “fair coin” a sort of Platonic ideal, like a straight line or a perfect circle? You start with that as an assumption and see what theorems and deductions follow. Then those theorems apply to real-world lines or circles or coins as an approximation, to whatever extent they conform to the ideal. At least, isn’t that one way to look at it?
It means that the sequence 1/1, 1/2, 1/3, 1/4, 1/5, 1/6, … approaches 0 in some sense. What sense specifically? Well, the standard interpretation is that it approaches 0 in the sense that, for any natural number d, that sequence eventually gets and forevermore stays smaller than 1/d (thus, its distance from 0 gets and stays arbitrarily small, even though it never actually hits 0).
The particular expression you linked to says “the limit, as n approaches infinity, of 1/n, is 0.” Roughly, this means that, as n gets arbitrarily large, 1/n gets arbitrarily close to 0. A little more precisely, it means that you can ensure that values of 1/n are as close to 0 as you want (without actually being 0) provided n is large enough. The mathematicians’ definition would involve stating things like “as close to 0 as you want” and “large enough” more precisely.
As it relates to the topic of this thread, the probability of getting a finite number n of flips to come up all heads isn’t 0, but you can make that probability come as close to 0 as you want by making the number of flips (n) large enough. If you want the probability to be less than, say, 1/1000000000, I can tell you how many flips are necessary to make it that small.
I was heading there next, and you’ve answered one of the follow up questions. But it also states In fact, the probability of tails never being flipped in an infinite series is zero. I’ve heard that numerous times, but still don’t see an explanation for it (excepting qualifiers like ‘almost surely’). Yet it does seem like there could be an infinite set of coin flips that are all heads, and so the probability of a tails would be 0 for that set. Or am I just further confusing myself here?
