Probability question (easy to ask, hard to answer)

Factual Questions
1

Over in this thread, we were discussing the counterintuitive nature of probability when the sample space is infinite. I came up with what I thought was a very interesting question, deserving of its own thread. Here goes.

Let’s say you’ve got a uniform distribution on [0, 1]. IOW, if B is a subset of [0, 1], the probability of picking an element of B is just the Lebesgue measure of B (call that m(B)).

As we all know, m(Q) = 0. So the probability of picking a rational number is zero. Doesn’t mean it’ll never happen (see the earlier referenced thread for an explanation), but it does have probability zero.

Here’s the question: what’s the probability of picking a rational number less than 1/2, given that you’ve picked a rational number? Intuitively, it’s 1/2, but we all know what intuition’s worth in higher math.

There are two ways to look at it that I can think of.[ol]
[li]Let L denote the event that we pick a number less than 1/2, and Q denote the event that we pick a rational number. In this case, we want P(LQ|Q). By the standard formula for conditional probabilities and the idempotence of set intersection, P(LQ|Q) = P(LQ)/P(Q). If picking a rational number is independent from picking a number less than 1/2, then P(LQ|Q) = P(L)P(Q)/P(Q), and we might be able to cancel, to get P(LQ|Q) = P(L). But that cancellation is a little fishy, as P(Q) = 0.[/li][li]Let L denote the event that we pick a rational number less than 1/2, and let Q denote the event that we pick a rational number. Now we want P(L|Q) (note that LQ = L here, as L is a subset of Q). Again, by the formula for conditional probability, P(L|Q) = P(LQ)/P(Q). Since LQ = L, P(L|Q) = P(L)/P(Q). No hope here, I think.[/ol]So there’s the quandary. Is there any hope of resolving this? Is there any theoretical reason to prefer the first analysis over the second?[/li]
On preview, I’m pretty sure that in the first case, picking a rational number is independent from picking a number less than 1/2. LQ is a subset of Q, and a subset of a set of measure zero must have measure zero. Since we’ve got a uniform distribution, P(LQ) = m(LQ) = 0. P(L)P(Q) = 0, as P(Q) = 0. So the question remains, is P(L)P(Q)/P(Q) = L?

2

Of course I’ll be no help with this problem, but I wanted to ask something that may make it easier for me to understand. Consider the question: If we pick a random rational number p on the interval [0, 1], what is the probability that p < 1/2?

Now, I seem to remember someone saying that it’s impossible to construct a uniform distribution over a countable set, like the rationals on [0, 1]. Whether this means that no such distribution exists, is the above question unanswerable, because the given conditions are ill-defined?

3

No, because our distribution is on all of [0, 1]. It’s just a conditional probability question dealing with a variable with that distribution.

4

I’m sorry. I was unclear. Is it possible, given that you picked a random rational number (let’s call it q instead of p) on [0, 1], what’s the probability that q < 0.5? I know that this isn’t the same as your question in the OP, but maybe it’s related.

Also I’m thinking that in the problem in the OP, P(p < 0.5) = P(p > 0.5), by some kind of symmetry or something… That’s my intuition, anyway, which is of course worth nothing.

5

They look like the same question to me. Whether they are the same question is still open.

6

The way I see it, once we start talking about the conditional probability given that a rational is picked, we’re talking about trying to define a uniform probability function on the countable set of rationals, which is impossible.

Take the positive integers for a simple example. One of the properties of a probability function is that it must be countably additive. So we have to have:

Sum P(n) = 1

where n runs through the positive integers.

Since it’s uniform, we also need P(m)=P(n) for all pos. integers m and n.

Together, these two conditions are impossible to meet. Is P(m) > 0 for all positive integers m? No, since then the above sum would diverge. Is P(m) = 0 for all m? No, since then the above sum would converge to zero, not one.

So I don’t think your question can be answered strictly within the probability axioms. The only alternative I see is to weaken the axioms somewhat.

Here’s one way I’ve often thought of defining a uniform probability function on the positive integers (though, of course, it doesn’t satisfy the axioms):

Given a subset A of positive integers, define a(n) (n a pos. integer) to be the number of positive integers less than n and contained in A. Define the probability that a randomly picked integer m is in A to be

P(m in A) = lim a(n)/n

(limit as n goes to infinity).

Of course this isn’t countably additive, so it doesn’t satisfy the axioms, but I think it’s still interesting to look at.

Some problems with it, however:

  1. It’s not defined for all subsets of positive integers. I don’t consider that much of a problem, though; Lebesgue measure isn’t defined for all subsets of reals, either. Even the axioms don’t require the prob. function to be defined on all subsets of the sample space, only a sigma algebra of subsets.

  2. It depends on the order of the pos. integers. But lots of prob. functions on infinite sample spaces have similar problems. In Lebesgue measure, for example, sets with the same cardinality can have vastly different measures; a set with the cardinality of the reals can have measure zero, measure one, one thousand, or even infinite. Lebesgue measure depends on the natural structure of the reals, just as this function depends on the natural structure of the integers.

  3. And this is the real problem for this example, I think. The rationals aren’t well ordered, but the function I defined above required the positive integers to be well ordered. How do you choose a well ordering for the rationals? You’ll certainly want it to match certain intuitions, like the probability of a rational being < 1/2 should be 1/2. There may be a natural way of well ordering the rationals and defining a prob. function on the rationals in a way similar to the one above, but I don’t see one offhand.

This, of course, is one approach; I’m sure there are others.

7

Of course, you can’t cancel zero from the numerator and denominator, so that’s out.

I don’t think this amounts to attempting to construct a uniform distribution over Q–it’s just asking what the probability of B is given A.

8

It’s times like this that I really wish I could remember that day in class we talked about countability & uncountability in probability. I remember some of it, but I seem to have lost my notes, and my text doesn’t really cover it.

Not much to add, just that it seems to me #2 doesn’t seem right. L in this case is the event you’re looking for ( rational number less than 1/2 ) so doesn’t P(L|Q) = P(L)? Maybe not. But I just like #1 better.

Of course it’s not quite satisfactory either. I think we can agree that the probability of picking a rational number given that we picked a number less than 1/2 must be zero, which may or may not be relevant at all.

One thing I do note is that if you use infinitesimal conditioning on a continuous distribution, you assume that X is in some small interval near x. Then the answer is 0 ( I think) since the result that X is a rational number less than 1/2 is 0 for any interval about x in Q. This may be one way to restate the question so that it’s different from Achernar’s, but then again you may not want to restate it. (By the way, my text recommends Probability and Measure by P. Billingsley for “a rigorous treatment of conditioning on a continuously distributed variable.”)

9

I can’t see why it isn’t constructing a uniform distribution over Q. We can ask what is the probability of B given A, what is the probability of C given A, what is the probability of an arbitrary subset given A. The probabilities of those aribitrary subsets (given A) will form a new probability function, mapping the arbitrary subsets to their conditional probabilities. You’re simply asking for a particular value of that function.

It’ll be uniform since, given that a rational is picked, no rational is preferred over another.

10

I can only imagine two possible answers to this question. Either (i) the answer is 1/2; or (ii) the question is meaningless. At present I am inclining towards the latter. For one thing the conditional probability P(A|B) is only defined when P(B)>0.

11

Why not just answer it logically without math? I know that it sounds bad to say that, especially since I love stats so much (why, I’ll never know)… but look at it this way, one step at a time:

The odds of picking a rational number between [0,1] approaches 0 but never quite is 0.

The odds that if a rational is picked is <0.5 approaches 0.5 but never quite is 0.5.

So then in laymans terms, you would have

~0.5(~0) which is still almost 0 probablility, but not quite. And just not quite half as probable as picking any rational at all.

That isn’t that hard, is it?

12

Except that

isn’t true ( at least I don’t believe it and you haven’t proved it).

13

Jabba’s right. There are no limits here–the probability of picking a rational number is zero. Not very close to zero, not even infinitesimally small, but actually zero.

I’m starting to agree with Cabbage, though. It’s obvious at this point that it’s undefined, which is counterintuitive but far from paradoxical.

14

it is not actually 0. It is a limit that approaches 0. The reason it is given as 0 is because of those silly infinities. But think about it logically. If there is any possibility at all of being selected, then the probability can not be zero. but really really close to it…

I know the arguements that you are going to make regarding the infinities involved and how it is undefined. But an “undefinable” amount more of irrational numbers still will not correspond to exactly 0. Infinities are used to explain things for ease of calculation within an allowable margin of error which is so small that it makes almost no difference whatsoever. But that does not mean that it truly is zero in this case. Merely really close.

15

Suppose that the probability of picking 1/2 is 0.0000000001 ( the same argument will apply to any other probability you might use instead). Since this is a uniform distribution, the probability of picking any other rational number is then also 0.0000000001. However there are more than 1000000000 rational numbers in [0,1] ( there are infinitely many) and so the total probability ( i.e. the probability that some number is chosen) is greater than 1 ( it will be infinite, in fact) which is impossible.

16

if you are reducing the probablility to 0.00000000001 and then taking a greater amount of numbers than the probability you have selected, of course you are going to come out with a number greater than one.

Using your same argument, take a random selection of 1000 numbers between 0 and 1. Chances are that you will not get a rational number. Keep taking samples in the same fashion. Eventually maybe after say 1000 such examples (or even 1,000,000,000 or higher) you will get a rational number. say after 1000 samples for ease of math here, you get 1 lonely rational. So out of 1,000,000 total numbers you have ONE. or a 0.000001 probability. Now you may take 1,000,000,000,000,000,000 more samples and never get another rational. But the probability is not 0, just getting increasingly smaller. Infinesimally small (or undefinably small according to our current ability to measure) but NOT 0. Merely approaching it.

17

Sorry, but this is incorrect. The probability is zero, and there is some possibility that it will happen. Please read the thread referenced in the OP for an explanation of why this is.

18

This does not follow. Suppose I roll an ordinary die and get a ‘6’ on my second throw. Does this mean that the probability of a ‘6’ was 1/2.

Since you did not like my previous attempt can you give me some idea of the order of magnitude you have in mind for this non-zero probability?

19

No, it doesn’t mean that your probability was 1/2. As for order of magnitude? It does not matter… the higher orders we go to the probability merely gets smaller so small it may as well be zero, but never miraculously changes to 0.

I agree with you that for all intents and purposes it may as well be 0, but to argue that because the numbers get so small on the march to the infinite that the probabilities become exactly 0 is not correct. Infinity is a mathematical tool used for ease of calculation (and sometimes to hinder it as well) but it is a tool only. When infinites are used in equations there is always a small but normally allowable margin of error. Which is where the probability lies. Within that margin of error.

20

CuriousCanuck, the OP was a mathematical question but I’m afraid you’ve left the realm of mathematics. Arguments about the margin of error don’t have any weight in a mathematical discussion. (In physics perhaps, but not in mathematics.)

In the real world there are non-zero quantities which are too small for us to measure, but in mathematics there’s no such thing as an infinitesimally small, non-zero real number. Similarly, it doesn’t make a lot of real-world sense that an event which can occur can have a probability of 0, but mathematically it’s necessary to avoid contradictions in cases where there are an infinite number of outcomes.