One ‘big question’ that information theoretic approaches may shed new light on is the question of why there is anything at all. There’s an interesting book by Russell K. Standish, called ‘Theory of Nothing’, in which he claims that ‘something is just the inside view of nothing’. That sounds like something more fit for eastern philosophy than western science at first brush, but it can be given a rigorous meaning, in terms of information content.
First of all, the information content of nothing is, rather trivially, zero. The information content of something can be measured by how much it can be compressed, using a universal Turing machine (I reckon your reading has introduced you to the concept; if not, just think of it as a sort of formalized definition for a computer, because that’s what it is). In other words, the information content of something – say, a picture, or a bit string – is measured by the smallest amount of data needed to reproduce it faithfully. A highly redundant bit string, such as ‘01010101010101…’ can be very highly compressed, to something like ‘n times 01’, and so can a ‘boring’ picture, like, say, one which just displays a perfectly uniform, red wall.
One thus obtains a measure for complexity (commonly called algorithmic or Kolmogorov complexity). Now, one might think that in some everyday sense more complicated objects also always have a higher complexity in the algorithmic sense, and that, in particular, ‘everything’, formalized in a suitable way, should have the highest complexity possible, as everything includes, well, everything, including every thing that has a high complexity, and should thus have a higher complexity than all of those things.
However, it turns out that in fact, everything has exactly the same complexity, and hence information content, as nothing: zero. You could give me nothing, and from this, I could generate a description of everything. It’s simple: you can create a Turing machine (i.e. write a program) that, requiring no input, outputs all possible binary strings in sequence – 0, 1, 00, 01, 10, 11, 001, 010… – and hence, all information.
Standish uses Borges’ infinite ‘library of Babel’ as an illustrative example: in it, all possible books – in fact, all possible permutations of a certain amount of letters – are stored. So you’ve got everything in there, from the complete works of Shakespeare to a volume containing just the letter q, 150,000 times.
This library, too, has zero information content. But it also contains the work of Shakespeare, so in this toy world, Shakespeare came out of nothing – just as, if our universe is really just information at bottom, it ‘came out of nothing’, in the sense that it is part of all possible ‘bit-strings’ – representations of information – that are derivable from ‘nothing’ in the sense of ‘containing no information at all’.
So, from an information theoretic viewpoint, that something could come from nothing is not as paradoxical as it is commonly taken to be – as in fact, nothing contains (the collection of) all possible somethings, in the same sense that a coded message contains the plain-English message.
This may be taken to be rather more philosophy than science, and I wouldn’t quibble with that characterization; however, once you gun for the big questions, the boundaries always get a little fuzzy.
As for more ‘mundane’ physical approaches, one might point out that essentially all of special relativity can be derived from the postulate that ‘information can’t be transferred faster than light’, and that quantum theory, or at least large patches of it, can be derived from assuming some variation of a ‘principle of finite information’, i.e. that every quantum system only contains a finite amount of information (this is for instance the starting point of Rovelli’s ‘relational interpretation’ and reconstruction of quantum mechanics). The viewpoint has certainly been expressed that both theories are really theories about information underneath.
Regarding the validity of this approach to physics, there is an issue here: conventional physics relies on continuum quantities, which aren’t computable, and many information-based approaches require computable physics. In fact, this is probably the greatest obstacle of ‘digital physics’. A simple example is the apparent loss of symmetry: anything information based will typically have something like a shortest length scale, or a shortest time step. However, in conventional physics, these quantities are typically assumed to be continuous, which produces some nice features. There’s a famous result, called Noether’s theorem, which associates a conserved quantity with every continuous symmetry – concretely, the fact that physics ‘stays the same’ whether or not it’s observed now or any amount of time (in particular, any arbitrarily small amount of time) later leads to the conservation of energy; similarly, spatial translation symmetry leads to the conservation of momentum. A shortest time step, or a smallest length, breaks these symmetries, which may seem too high a price.
There are ways around this: some parts of Noether’s theorem can be salvaged even in a discrete world (there’s been some recent work on this – here’s a blog link), and there’s also a sense in which information can be continuous – take sampling: what you do is take a finite amount of discrete points for which you note the amplitudes, i.e. you turn the analog signal in a digital, discrete one. However, providing certain conditions are met, the analog signal can be arbitrarily well re-created (given enough processing power) thanks to the Shannon-Nyquist theorem. Spacetime may be ‘both discrete and continuous’ in a similar sense.
The good thing about these speculations is that, contrary to many other ‘new physics’ proposals (string theory, I’m looking in your direction), they may not be very far from being testable, or may be testable already – the folks at Fermilab are currently busy building the Holometer, which may actually have a shot at detecting the underlying discreteness of the universe.
). Einstein’s made him a popular icon precisely because of incomprehensibility (it also gives us GPS). Whoever came up with quantum theory does not even register in popular culture, although the insights achieved proved essential in electronics. String theory seems to be not just hard to understand but also at present non-verifiable, so no gizmos are coming out of there.