Current the AI boom is powered by GPU chips but there are efficiency logjams having them transfer information between themselves in bigger arrays.
There are multiple plays trying to develop new architectures for AI chips from ground up to do the job better and break into Nvidia’s GPU market dominance.
The brain solved the problem with its complex three dimensional convoluted organization. I’ve heard of some huge chips being proposed and even some stacked ones l, but any that are based on imitating the brain’s solution?
Neuromorphic computing is currently being researched by both established players (Intel, IBM) and startups (Brainchip was the first to offer a commercially available implementation in 2021) seeking to find applications of brain-like designs in AI and computing more generally. Although to the best of my knowledge, there’s nobody working on a 3D-solution.
There is a lot of work on 3D chips, but most I’ve seen have been memories stacked on top of another chip. I’d think the heat generated by an AI chip would be too great to make such things very practical. I’d guess that 2.5D would be more practical, where the chips are put on a substrate that allows signals to move between them without buffering which slows things down.
Another problem with 3D is that if anything goes wrong in the stack, including in the interconnect, you have to toss the entire stack which can get expensive.
This isn’t an answer to your question, but I feel like it’s a point that needs to be mentioned so it can be put aside. Quite a few recent processors and SoCs have included an NPU - neural processing unit, as distinct from CPU or GPU, but these aren’t themselves neural in architecture - they are (as I understand it), just specialised collections of cores that are very efficient at massive parallel processing of data handling tasks - these are also present in GPUs (which is what made GPUs useful for AI stuff), but NPUs are like a distillation of GPUs, containing just the bits that are most useful for AI computation.
That computation is usually still part where the ‘neural’ stuff actually happens, in the software.
Heat management is likely to be an issue with any kind of electronic processor that extends significantly into the third dimension - at present, more-or-less 2D architectures are basically immune to the square-cube law - this might be mitigated somewhat by using diamond as a substrate - diamond is an almost absurdly good conductor of heat, as demonstrated here: https://youtube.com/shorts/4tNtPWvk7Ro?
As it happens, I’m currently reading Neuromorphic Engineering, which is a basic primer on methods of neuromorphic computing and programming. Although the brain is a 3D structure, a lot of components are ‘built’ evolutionarily onto the brain in relatively flat layers (which is unsurprising as the braincase has to expand and blood vessels have to extend out to support more tissue) but the real thing that distinguishes ‘processing’ as done by the brain from that in a digital computer simulating a ‘neural network’ as a logical matrix with backpropagation is the a combination of massive interconnection, the complex potentiation of the synapse, and the adaptability of Hebbian (and Hebbian-derived) learning, which makes organic brains somehow much faster at learning and dealing with novel sensory inputs despite the fact that in terms of a ‘clock speed’ the brain is magnitudes of order slower and receives much lower bandwidth of data than a computer being trained on a vast pool of digitized images and electronic text.
Yes the human brain has a large surface area essentially covered with parallel processing units. The convoluted surface of the cortex not only allows many of those units to be packed into the volume but also minimizes the distance and thus the communication time between them, allowing that massive interconnection to work efficiently.
Emulating that functionality seems to me to an engineering challenge more than a software issue?
If many processing units are idle waiting on input to arrive that is a major decrease in efficiency and potential, yes?
I recently attended a lecture on AI in medicine. After a brief history and discussion of new chips, Chat GPT, hallucinations, and a basic discussion of problems and potential, they showed this humourous video of a squirrel getting to a bird feeder by outsmarting a savage Slinky. I hadn’t seen this video, but seemingly even animal brains have a lot more processing power than is sometimes credited.
That’s an understatement; we understand almost nothing about the brain at anything but a superficial level. Even our understanding and ability to model single neurons, much less interconnected networks of neurons, is manifestly incomplete.
We make a lot of unvalidated assumptions about the lack of thinking and problem solving ability in non-primate animals, but just observing creatures like squirrels, raccoons, corvids, and many other mammals and aves shows clear indications of complex problem solving behavior and likely at least some ability to produce complex mental models of the real world that are generalized enough to develop learned skills that are well beyond any ‘instinct’.
The brain is dramatically more complex than is dreamt of in our current philosophies.
From the Guardian link (above): The clump of brain [analyzed using complex computing techniques] amounted to a mere cubic millimetre of tissue, but working out the wiring still presented a huge task for the team. Electron microscope images of more than 5,000 slices of the sample revealed 57,000 individual cells, 150m neural connections and 23cm of blood vessels…The generated images comprised 1.4 petabytes of data, equivalent to 14,000 full length, 4k resolution movies.
The lecturer pointed out that the squirrel seemed to solve the problem, which it presumably had never encountered, within a few seconds. And that AI might reasonably look at many complicated pressing global concerns and conclude “humans are the problem”.
Somebody oughta smack that lecturer on the nose with a rolled-up grant application. That’s a wholly unsupported and almost certainly incorrect assertion.
Meh. Most of those I have known researching brain function appreciate how much is unknown and the complexity of it.
But that fact, and fears of AI deciding we are the problem, are outside the intended scope of this thread. I am asking about the architecture of the machine machinery and if a fresh look at chip architecture and interchip physical structures using biological intelligences as inspiration may be productive.
And maybe not limited to mammalian models. Avian structure maybe a good one to look at? It gets a lot done in a small volume and with relatively little mass, with very different structure but apparently similar interconnections.
You can only appreciate known unknowns. Like the incredible energy efficiency of the brain. The adult blood supply is so tightly regulated that it only contains about a teaspoon of glucose. It will take AI a long time to get similar results from a soupçon of sugar water.
Of course printing a useful artificial heart or organ often requires some sort of 3D scaffold. AFAIK the more common AI approach is to stack chips to dissipate heat. However, the paper below says laws may require AI to become “explainable” (at the expense of efficiency) and copying the brain is a useful model for this. One presumes many AI companies prefer a black box approach if it is more efficient and less litigious.
The lecturer (>= four smacks applied) stressed the importance of asking “where the training data came from” when using scribes or programs to evaluate pathology. Companies may not always wish to give you the honest answer. And what happens when AI trains on data generated by other AI after running low on training sources? Better or worse than YouTube or X?
Nah. As you’ve pointed out, we are very far from explaining the brain!
The point is just to consider if the solutions evolution has created for dealing with hardware/architectural constraints in intelligence processing may have lessons for reimagining how we build our machines.
On the software side, yes quality of training sets matters more than quantity. I just listen to a podcast the other day about how an AI is being utilized to help improve mammography screening outcomes (called MIA). The comment there relevant is this (from the transcript):
From Babbage from The Economist: AI and health part one: DrGPT will see you now, May 22, 2024
Sure. But your question is about what is “in progress”. Of course programmers would like to duplicate the brain. And of course doing so is impossible at this time with any fidelity. A computer model stringing together a stream of speech based on what next word seems most likely is not exactly replicating neurological processes, since no one does that^ (though perhaps the process of thinking in a language where you only know basic words is slightly closer). Printing 3D boards is much more expensive and technically difficult. Doubtless it is a better long-term solution, but many problems must be solved to make it possible, nevermind economically viable. The current chips designed to better process graphics are a long way from biomimicry.
^The four-smack lecturer pointed out Microsoft does significant ChatGPT training in English speaking Nigeria to lower costs. This, allegedly Chat frequently uses distinctive features of local speech, such as “let’s delve into this” which are uncommon in, say, Canada. Is this true? Dunno.
Do we understand the behavior of individual neurons and synapses well enough to accurately simulate (e.g. The Human Brain Project) and the main challenges are the scope and interconnected behaviors?
Or are we still finding challenges in just simulating individual components?
The examples section of the neuromorphic computing wiki page gives me the impression it is the sheer scope, but I’m curious if there are tons of different synapses and we haven’t modeled them all.
There are certainly standardized models of neurons, although they are largely focused on potentiation and synaptic strength, or more generally, treating them as control systems. However, neurons are highly dynamic and complex electrochemical systems which experience a lot of interactions beyond just receiving and transmitting electrical signals, and there are hundreds of different types of neurons in the brain that are actively functioning and in constant flux in terms of their internal states. There is a lot that occurs within the brain neuron which is involved in processing signals that is not well understood, and processing that occurs within non-brain sensory and motor, and interneurons.
The focus of neuromorphic engineering is to use principles from naturally occurring neural systems to make fundamental improvements in computing, but we are very far from making (or simulating in software) a synthetic neural system that functions with comparable fidelity to even a simple real brain.
How would we even make a truly 3-dimensional computer chip? The methods we use now are based on shining light on a surface. That wouldn’t work in the core of a lump of silicon.
