I know almost nothing about Neural Nets except what I have picked up in a pop-science book or article here and there.
It’s my understanding that the way they work basically goes like this. Each “neuron” recieves several input signals from other neurons, and sends several output signals to other neurons. Neurons send their output signal just when the sum (or some function anyway) of their input signals reaches some threshold or other.
Meanwhile, in the Game of Life, each pixel activates when there are enough activated pixels nearby, and deactivates otherwise. (That’s on the sort of “standard” rules anyway.)
So it’s just occured to me that the Game of Life is set up just a bit like a very simple neural net. Pixels are like neurons, each sends an output signal to the eight pixels bordering it, and recieves input from the same, and the “threshold” is just that number of pixels that need to be activated borering a pixel to cause it itself to be activated.
If this is a valid analogy, I know the analogy is discussed somewhere by smart people who know what they’re talking about.
And my question is: is this a valid analogy, and if so, who has discussed this analogy, and where?
The game of life doesn’t have a learning mechanism.
Programatic neural nets, as I understand it, work randomly at first and then receive a feedback which tells them when they got the answer right. When that happens, the odds that the signal will be passed down the pathway that worked correctly become higher. Eventually, if there was enough neurons for it to be feasible to program in the solution to the problem (like addition), it will always give the correct answer.
You just need the right keywords. The game of life is more formally a cellular automaton. Now the question is how smart would you like your smart person to be?
link
My advisor has a cellular automaton model of fundamental physics found here. You can install it and play with it or just watch videos of black holes and whatnot. Scroll back to his home page and you’ll notice he’s a neural network expert.
Conway’s Life can be used to construct a Turing machine. It can therefore be set up to store and retreive information, make decisions, execute programmed algorithms. It would be a dog of a thing to program, but I guess it would be possible in theory to write any algorithm you wanted and run it on the Game of Life.
That might answer your question. (Or it might not. I don’t know anything about neural networks.)
Neural nets are Turing complete as well, so there’s another similarity. You really ought to be comparing nets and general cellular automata, though, as the structure of a net can vary greatly, but Conway’s Game of Life is fixed.
I think, though, that the fundamental similarity is that they’re both models of computation. I’m not sure that there’s any deeper connection.
The one quoted above says CAs and Neural Nets are able to solve the halting problem, and so are more powerful than a universal turing machine. (Right?)
As one example, it says they can take an infinite decimal expansion and determine whether it is a representation of an integer or not. It says a Turing machine can’t do this because it can never finish reading the input. But I wonder how a CA or NN could finish reading the input? And it also says the CA and NN can’t be guaranteed to always be right when carrying out this task. But if that’s the case, what’s the significance? Couldn’t a Turing machine also be programmed to come to a stop at some point while reading the input and give an answer–which may or may not be right?
I think that’s quite deciptive – it says that INFINITE neural nets can solve the halting problem; but that’s sort of like saying “if a turing machine were allowed to run for an infinitely long period of time, it could solve the halting problem.” True (google “infinite time turing machine” for more) but not very informative. The only difference between an infinite CA/NN and an infinite time turing machine seems to be that an infinite CA/NN can do the calculation in parallel, and so can deliver an answer in a finite period of time, while a turing machine does all its calculations serially.
In terms of the larger question, my thought is that indeed NNs and CAs are essentially equivalent – but Conway’s Life would be a VERY simple feedforward NN. And in turn, a NN that does back-propagation could be implemented as a CA, but it would look nothing like Conway’s Life; it would have lots of states and (probably) a complex topology. The main way they are similar is that they do more of their computation in parallel, while turing machines compute strictly in series.
I’m not seeing how this could be possible. Don’t cellular automata have an inherent maximum speed for information propagation (one cell per tick, for nearest-neighbor schemes)? That means that if your automaton runs for a finite length of time, then there’s going to be a horizon beyond which information couldn’t have affected the result, so you might as well have just made your automaton the size of the horizon to begin with.