Heh, I had a look at this a while back. I really do appreciate the work Schiller put in his ‘Motion Mountain’ free textbook, and what little of it I’ve seen contained some quite clear illustrations and surprising examples, but the strands are, well, a bit of a tangled mess…
I’m a bit of a fan of the whole entropic gravity thing, I must confess – it does seem to yield some simple solutions to a couple of long-standing puzzles, dark energy most notably. There’s also the possibility that one should in principle be able to extend the reasoning in a Kaluza-Klein-like fashion in order to have all the gauge fields emerge in a similar way. Of course, there is a lot of arbitrariness – like, for instance, the holographic principle and simply the dimension of spacetime – going into this approach, which seems to cry out for a more fundamental justification.
As for Tegmark’s mathematical universe, this is something that is in principle very close to my own thinking, but I’m not too sure if he isn’t too ad hoc in banishing undecidable structures from existing; after all, there do exist physical structures (computers, most notably) whose evolution contains undecidable questions. (And just as a side note, there does seem to be some strange connection between undecidability and quantum mechanics: Paterek and collaborators noticed that when you encode a set of axioms into a quantum state, and a proposition into your measurement (basis), then you get a random answer exactly if the proposition is logically independent from the axioms. Here’s the paper if you’re interested.)
In the end, however, I think the only thing that would really clear up the mud would be some a priori deduction of physics – at least, its fundamental principles. Whether or not something like that is indeed possible, however, is anybody’s guess, of course; however, if it weren’t, then we’d live in the strange situation of there not being ‘sufficient reason’ for us to live in the situation we do. I think there’s a fascinating line of research, which is, unfortunately, largely abandoned today, that has some plausible connection to these questions, originated by Weizsäcker in his ‘theory of elemental alternatives’, called ‘ur theory’ (after the German prefix Ur-, meaning something like primordial). Basically, it amounts to taking the qubit (the ur in his terminology) as the fundamental building block of the quantum world, and manages to deduce some quite remarkable consequences from that, such as a plausible justification of a three dimensional position space, which even naturally obeys a holographic paradigm.
Unfortunately, googling yields a string of Lolcats informing me that ur theory sucks (though I’m not sure I should trust an internet meme on questions of fundamental physics), but if you’re interested (and didn’t already know about this line of research), this work of Holger Lyre provides a good self-contained intro.
Anyway, I’ve gotten into rambling a bit, and the OP’s topic can’t even really be seen from where I ended up, so I’d better leave it at that…