I feel like I should know this. Indeed I feel like I have known it. In fact I probably will know it again soon. But right now I find myself perplexed.
Often I think that a measure of how surprising something should be is its probability. Getting 99 heads in a row has an extremely low probability, hence is extremely surprising.
But of course every single possible sequence of flips has exactly the same probability.
So the probabilities don’t differentiate these flip-sequences, yet some are more surprising than others.
What is it that separates out some as more surprising than others?
I don’t mean this as a psychological question, I mean it as a question of logic or critical thinking. Some of these sequences rightly should be thought of as more surprising than others (given suitable filling in of details in the context surrounding the flips) and that’s what I’m blanking on. Why should they be found more or less surprising?
There’s a 1 in 2 chance of flipping heads on this coin toss and a 1 in 2 chance on getting heads on the second coin toss. The thing is, there’s only a 1 in 4 chance of getting heads on both of them. The chance of flipping heads 100 times in a row is 1/2[sup]100[/sup].
Look at it this way. With one coin toss, there’s two possible outcomes, heads or tales. With 2 coin tosses, you can think of 4 outcomes. heads/heads, heads/tales, tales/tales, tales/heads. Think of all the possible outcomes for 100 coin tosses (1267650600228229401496703205376)…you’re aiming for just one of those outcomes.
Getting heads on any specific flip isn’t that big of a deal, it’s getting them all in a row that is. Or really, just getting any specific pattern. Heads 50 times in a row, then alternating heads and tales for the last 50 would be the same odds as all Tales, for example.
ETA, pretend like you wanted 100 in a row instead of 99.
Of course, some random flip could be surprising–if, for example, it matched exactly some prediction made beforehand. So as I sort of gestured at in the OP, it seems like it’s not the probability, but something about the context of flip sequence event, that makes for proper surprise.
But what?
And does probability have nothing to do with it then? I’m surprised if that’s so!
You’re right, it’s just that all heads in a row would make people take notice (as you approached it). Just like bowling your 20th perfect game in a row is just an arbitrary number and all your friends might as well have come out and watched you bowl your 17th perfect game.
Beacuse there are many, many more ways those possible sequences of 100 tosses come up ~50 heads and ~50 tails vs. 100 heads (just start with 2 tosses - you’re twice as likely to come up one heads and one tails vs. 2 heads.)
To echo what other people have said, it’s because 99 heads in a row is more noticeable than an even number of heads and tails. For example, these patterns are all equally likely but which one stands out?
To put it another way - it’s kind of like asking why it would be surprising for a monkey to type out the entire works of Shakespeare, when any other sequence of letters would be equally probable.
I wonder if it would be enlightening to look at the conditional probability the other way around. That is, not “What is the probability that you flip 99 heads in a row, given a fair coin?” but “What is the probability of a fair coin, given that you flip 99 heads in a row?”
If I see you flip 99 heads in a row, I’m going to consider other possibilities (like a two-headed coin, a tricky way of flipping, or something like that) as perhaps more likely than a series of fair flips each of which has an equal chance of coming up heads or tails.
My statistics professor assured us that, “Statistically impossible events occur all the time.” By that he meant that on a day that it rained exactly 1.24533246536345 inches, wind speed also peaked at 7.345324534534 miles per hour, while the temperature was 14.69884598752223 degrees Celsius.
Inconceivable!
Similarly, when a probability distribution is plotted, probability is only meaningfully defined over a range of outcomes. For a given point the probability is essentially zero before the fact. IIRC: it’s been a while.
One approach to help make sense is Kolmogorov complexity – the Wikipedi article starts with example simple and complex sequences. A random sequence is unlikely to be simple; it’s surprising when it is.
The sequence of all heads is surprising because it can be described with few words: “All heads.” Similarly “Heads and tails alternate” (HTHTHTHTHTHTHTH…) is easy to describe, and thus surprising; as is “counts(*) out the digits of pi”:
HHH T HHHH T HHHHH TTTTTTTTT HH TTTTTT HHHHH TTT HHHHH TTTTTTTT HHHHHHHHH TTTTTTT HHHHHHHHH TTT HH TTT HHHHHHHH TTTT HHHHHH
Perhaps another way of thinking about it - rather than a predicted sequence - toss two fair coins together 100 times. What is that chance that they will come up the same way (HH or TT) 100 times in a row?
I don’t know if I understand your question correctly, but what you are looking at are joint occurrences. You’re not just looking at and evaluating one coin toss, but 99 coin tosses (or 100, or 107, or 3). You can either imagine 99 people tossing a fair coin at the same time or one person doing this 99 times in a row. The way to compute the probability for this compound event is to multiply the probabilities for the single events:
Probability and its related field of Expectation are pretty fuzzy areas of mathematics in which reality and theory seldom collide in an intuitive way.
Flip a fair coin 100 times, and maths tells you to expect 50 heads and 50 tails, but what is the probability that this actually occurs? In practice you are far more likely to get anything other than the expected result.
Yes, any given sequence of 99 tosses is equally unlikely. It’s just that we’re mentally more inclined to group all the sequences with 49 heads and 50 tails together, and maybe even lump the ones with 50 heads and 49 tails in with them. And so on.
Where you go wrong is in not examining it as a psychological question, because that’s what it is. Surprise is an emotion.
Simplifying to 10 flips to save space here, but
HHHHHHHHHH
TTTTTTTTTTTTT
HTHHTHTTHHH
HTHTHTHTHTH
THTTTTTHHTT
are all equally likely. If you gamble on particular sequence beforehand, you have the same chance of winning with any of them, but if you just look at the sequences afterwards some appear to you to be more meaningful than others.
I think this question has already been thoroughly answered, but I’ll give an analogous example to flesh it out:
When rolling a pair of dice, scoring a double six seems quite special. Why? It has exactly the same chance as any other roll… BUT
A 3 and a 4 is effectively the same thing as a 4 and a 3. This class of rolls that contains one 3 and one 4 is twice as likely as the class that contains 2 6s.
There is another class of rolls that adds up to 7. This class is twelve times as likely as the class that adds up to 12.
So in terms of exact rolls, a double six isn’t unlikely. However, in terms of equivalence classes, double six is an unlikely result.
There is both psychology and probability involved. Psychology, in particular pattern recognition, is what creates the equivalence classes - in the OP example, all “jumbled” combinations of heads and tails are in the same equivalence class, and so far more likely than patterned combinations. Some patterns (e.g. 5 heads alternating with 5 tails versus 6 heads alternating with 6 tails) are going to seem equivalent, but still a much smaller class than the jumbled combinations. All heads is a very small equivalence class (only one member) so very unlikely.