Irrational numbers and information content

Irrational numbers are represented by infinite decimal expansions, which would seem to mean that they would require an infinite amount of information to specify. But irrationals can sometimes be exactly defined by often very terse polynomial functions, and others can be specified by iterative processes that require fairly short definitions coupled with the instruction “repeat forever”. So how do irrationals relate to information content?

Rational numbers, too, have an infinite decimal representation:

For example, 2 can be represented as 2.000000000000…
and as 1.9999999999999999…

And if we had a numbering system using base pi instead of base 10, pi would be represented as 10 (or 10.0000000…).

A rational number is just a point on the number line, as is an irrational number. Both contain exactly the same amount of information.

The problem is that information content depends on your coding mechanism, which needs to be explicitly defined before we start talking about information content. Without that you end up with contradictions like the following.

All positive integers can be represented by a string of less than 100 characters and spaces (ie an infinite set can be represented by 27^100 different elements).

Proof:
If there were positive integers that could not be so represented, than there would be a least one, but this least one can be represented by the string
“the smallest positive integer not represented by a hundred characters”
which is itself a string of less than 100 characters, leading to a contradiction.
Ergo: no such number exists.

There are uncountably infinitely many irrational numbers, and so no matter what coding mechanism you use, you can’t represent them all with any set of finite strings. However a relatively small number can be represented easily, for example when I say pi, or e, or sqrt(2) you know exactly what I’m talking about, but these are by far the exception rather than the rule.

Yes. A thing which may have a short description in one notation system may have a long description in another notation system (or even no description!). There’s no point talking about the information content of a number in itself. What matters is the relationship between that number and a particular notation system. And there’s nothing magic about the positional value notation system we use so often, such that we shouldn’t consider any others.

The general topic is part of Kolmogorov Complexity. I.e., how many bits are needed to describe a number (or string).

As far as infinite strings go, the overwhelming majority cannot be encoded with a finite number of bits, of course. They are effectively random in some sense and contain no information. The “easiest” to encode are of course rational numbers. But many irrational numbers have small encodings, for example algebraic numbers.

One of the more interesting numbers is Ω. It is algorithmically random but it so densely encodes information that it could be used to solve all halting-problem equivalent problems. If we could only compute it. Which we can’t.

That sounds like a bit of a cheat, because the definition isn’t algebraic or geometric, like pi, or e. But yeah, I think I see the point about information content being relative to encoding. I understand that coming up with a truly objective definition of randomness has similar problems.

I’m not sure if you meant this to be taken seriously or not, but in case you did, this is not a proof. The string
“the smallest positive integer not represented by a hundred characters”
might already have been used to represent 10,627 so it would be unavailable for your smallest non-represented number.

In fact if there are k different characters allowed (including the null character), then you can have at most k[sup]100[/sup] different 100 character strings. So all possible strings will have been used up by then including
“the smallest positive integer not represented by a hundred characters”

Of course, if you have an infinite number of characters, then you can represent an infinite number of integers, and you can do so with one character.

Irrationals vary in complexity from nearly zero all the way to infinity depending on how easy it is to create a generating algorithm. Of course, this is related closely to the problem of data compression. Any system of information can be compressed but the information itself doesn’t have it’s own inherent amount of compressibility - it also depends on the complexity of the compression system. A more complex compressor can compress the same data more, but at the cost of increasing it’s own size. I suppose there must be some theoretical point at which these two things balance each other out.

Just a reminder. “The general topic is part of Kolmogorov Complexity.”

I.e., the basics for defining information content algorithmically was figured out years ago. In fact, Alan Turing’s famous 1937 paper On Computable Numbers With an Application to the Entscheidungsproblem used information theory as a jumping off point. There is no need to guess how it is defined.

Of course I didn’t mean it seriously. The point was to illustrate the importance of having a clearly unambiguous fixed mapping, rather than allowing for ambiguous common sense understanding. For a given injection from the subset of the 27^100 character strings onto a subset of integers, there will indeed be a lowest positive integer X that is not covered. But in mapping X to the string “the smallest positive integer not represented by a hundred characters”, you are altering your mapping mid-stream, and then the ambiguous common sense understanding of the phrase no longer applies to this new mapping.