# AI chip architecture: any biomimetic work in progress? **URL:** \ **Category:** Factual Questions\ **Tags:** ai\ **Created:** [May 24, 2024, 4:51am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010 "2024-05-24T04:51:13Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 24, 2024, 4:51am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/1 "2024-05-24T04:51:13Z")
Current the AI boom is powered by GPU chips but there are efficiency logjams having them transfer information between themselves in bigger arrays. There are multiple plays trying to develop new architectures for AI chips from ground up to do the job better and break into Nvidia’s GPU market dominance. The brain solved the problem with its complex three dimensional convoluted organization. I’ve heard of some huge chips being proposed and even some stacked ones l, but any that are based on imitating the brain’s solution? ---
**Author:** ![Half\_Man\_Half\_Wit](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/half_man_half_wit/32/21766_2.png) [@Half\_Man\_Half\_Wit](https://boards.straightdope.com/u/Half_Man_Half_Wit)\ **Post date:** [May 24, 2024, 5:40am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/2 "2024-05-24T05:40:06Z")
[Neuromorphic computing](https://medium.com/tecnolog%C3%ADa/a-primer-on-neuromorphic-computing-9d63b4f35302) is currently being researched by both established players ([Intel](https://www.intel.com/content/www/us/en/research/neuromorphic-computing.html), [IBM](https://research.ibm.com/projects/neuromorphic-computing)) and startups ([Brainchip](https://www.electronics-lab.com/first-mini-pciexpress-board-with-spiking-neural-network-chip/) was the first to offer a commercially available implementation in 2021) seeking to find applications of brain-like designs in AI and computing more generally. Although to the best of my knowledge, there’s nobody working on a 3D-solution. ---
**Author:** ![Voyager](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/voyager/32/133_2.png) [@Voyager](https://boards.straightdope.com/u/Voyager)\ **Post date:** [May 24, 2024, 6:32am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/3 "2024-05-24T06:32:07Z")
There is a lot of work on 3D chips, but most I’ve seen have been memories stacked on top of another chip. I’d think the heat generated by an AI chip would be too great to make such things very practical. I’d guess that 2.5D would be more practical, where the chips are put on a substrate that allows signals to move between them without buffering which slows things down. Another problem with 3D is that if anything goes wrong in the stack, including in the interconnect, you have to toss the entire stack which can get expensive. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [May 24, 2024, 6:45am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/4 "2024-05-24T06:45:58Z")
This isn’t an answer to your question, but I feel like it’s a point that needs to be mentioned so it can be put aside. Quite a few recent processors and SoCs have included an NPU - neural processing unit, as distinct from CPU or GPU, but these aren’t themselves neural in architecture - they are (as I understand it), just specialised collections of cores that are very efficient at massive parallel processing of data handling tasks - these are also present in GPUs (which is what made GPUs useful for AI stuff), but NPUs are like a distillation of GPUs, containing just the bits that are most useful for AI computation. That computation is usually still part where the ‘neural’ stuff actually happens, in the software. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [May 24, 2024, 6:52am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/5 "2024-05-24T06:52:16Z")
> [@DSeid](#): > > The brain solved the problem with its complex three dimensional convoluted organization. I’ve heard of some huge chips being proposed and even some stacked ones l, but any that are based on imitating the brain’s solution? Heat management is likely to be an issue with any kind of electronic processor that extends significantly into the third dimension - at present, more-or-less 2D architectures are basically immune to the square-cube law - this might be mitigated somewhat by using diamond as a substrate - diamond is an almost absurdly good conductor of heat, as demonstrated here: [https://youtube.com/shorts/4tNtPWvk7Ro](https://youtube.com/shorts/4tNtPWvk7Ro)? ---
**Author:** ![Stranger\_On\_A\_Train](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Stranger\_On\_A\_Train](https://boards.straightdope.com/u/Stranger_On_A_Train)\ **Post date:** [May 24, 2024, 10:39pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/6 "2024-05-24T22:39:44Z")
> [@Half\_Man\_Half\_Wit](#): > > [Neuromorphic computing](https://medium.com/tecnolog%C3%ADa/a-primer-on-neuromorphic-computing-9d63b4f35302) is currently being researched by both established players ([Intel](https://www.intel.com/content/www/us/en/research/neuromorphic-computing.html), [IBM](https://research.ibm.com/projects/neuromorphic-computing)) and startups ([Brainchip](https://www.electronics-lab.com/first-mini-pciexpress-board-with-spiking-neural-network-chip/) was the first to offer a commercially available implementation in 2021) seeking to find applications of brain-like designs in AI and computing more generally. Although to the best of my knowledge, there’s nobody working on a 3D-solution. As it happens, I’m currently reading [_Neuromorphic Engineering_](https://www.taylorfrancis.com/books/mono/10.1201/9781003143499/neuromorphic-engineering-elishai-ezra-tsur), which is a basic primer on methods of neuromorphic computing and programming. Although the brain is a 3D structure, a lot of components are ‘built’ evolutionarily onto the brain in relatively flat layers (which is unsurprising as the braincase has to expand and blood vessels have to extend out to support more tissue) but the real thing that distinguishes ‘processing’ as done by the brain from that in a digital computer simulating a ‘neural network’ as a logical matrix with backpropagation is the a combination of massive interconnection, the complex potentiation of the synapse, and the adaptability of Hebbian (and Hebbian-derived) learning, which makes organic brains somehow much faster at learning and dealing with novel sensory inputs despite the fact that in terms of a ‘clock speed’ the brain is magnitudes of order slower and receives much lower bandwidth of data than a computer being trained on a vast pool of digitized images and electronic text. Stranger ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 24, 2024, 11:54pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/7 "2024-05-24T23:54:03Z")
Yes the human brain has a large surface area essentially covered with parallel processing units. The convoluted surface of the cortex not only allows many of those units to be packed into the volume but also minimizes the distance and thus the communication time between them, allowing that massive interconnection to work efficiently. Emulating that functionality seems to me to an engineering challenge more than a software issue? If many processing units are idle waiting on input to arrive that is a major decrease in efficiency and potential, yes? ---
**Author:** ![Dr\_Paprika](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr_paprika/32/3042_2.png) [@Dr\_Paprika](https://boards.straightdope.com/u/Dr_Paprika)\ **Post date:** [May 25, 2024, 12:18am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/8 "2024-05-25T00:18:01Z")
We don’t much understand the human brain. > **[Scientists find 57,000 cells and 150m neural connections in tiny sample of...](https://www.theguardian.com/science/article/2024/may/09/scientists-find-57000-cells-and-150m-neural-connections-in-tiny-sample-of-human-brain)** > > Harvard researchers teamed up with Google to analyse the makeup of the brain, much of which is not yet understood I recently attended a lecture on AI in medicine. After a brief history and discussion of new chips, Chat GPT, hallucinations, and a basic discussion of problems and potential, they showed this humourous video of a squirrel getting to a bird feeder by outsmarting a savage Slinky. I hadn’t seen this video, but seemingly even animal brains have a lot more processing power than is sometimes credited. [![](https://img.youtube.com/vi/9AfVSIdqBVo/maxresdefault.jpg "Clever squirrel outsmarts slinky at bird feeder") ](https://www.youtube.com/watch?v=9AfVSIdqBVo) ---
**Author:** ![Stranger\_On\_A\_Train](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Stranger\_On\_A\_Train](https://boards.straightdope.com/u/Stranger_On_A_Train)\ **Post date:** [May 25, 2024, 12:39am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/9 "2024-05-25T00:39:29Z")
> [@Dr\_Paprika](#): > > We don’t much understand the human brain. That’s an understatement; we understand almost nothing about the brain at anything but a superficial level. Even our understanding and ability to model single neurons, much less interconnected networks of neurons, is manifestly incomplete. > [@Dr\_Paprika](#): > > I recently attended a lecture on AI in medicine. After a brief history and discussion of new chips, Chat GPT, hallucinations, and a basic discussion of problems and potential, they showed this humourous video of a squirrel getting to a bird feeder by outsmarting a savage Slinky. I hadn’t seen this video, but seemingly even animal brains have a lot more processing power than is sometimes credited. We make a lot of unvalidated assumptions about the lack of thinking and problem solving ability in non-primate animals, but just observing creatures like squirrels, raccoons, corvids, and many other mammals and aves shows clear indications of complex problem solving behavior and likely at least some ability to produce complex mental models of the real world that are generalized enough to develop learned skills that are well beyond any ‘instinct’. Stranger ---
**Author:** ![Dr\_Paprika](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr_paprika/32/3042_2.png) [@Dr\_Paprika](https://boards.straightdope.com/u/Dr_Paprika)\ **Post date:** [May 25, 2024, 12:44am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/10 "2024-05-25T00:44:40Z")
The brain is dramatically more complex than is dreamt of in our current philosophies. From the Guardian link (above): _The clump of brain [analyzed using complex computing techniques] amounted to a **mere cubic millimetre of tissue** , but working out the wiring still presented a huge task for the team. Electron microscope images of more than 5,000 slices of the sample revealed 57,000 individual cells, 150m neural connections and 23cm of blood vessels…The generated images comprised 1.4 petabytes of data, equivalent to 14,000 full length, 4k resolution movies._ The lecturer pointed out that the squirrel seemed to solve the problem, which it presumably had never encountered, within a few seconds. And that AI might reasonably look at many complicated pressing global concerns and conclude “humans _are_ the problem”. ---
**Author:** ![Stranger\_On\_A\_Train](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Stranger\_On\_A\_Train](https://boards.straightdope.com/u/Stranger_On_A_Train)\ **Post date:** [May 25, 2024, 2:38am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/11 "2024-05-25T02:38:24Z")
> [@Dr\_Paprika](#): > > And that AI might reasonably look at many complicated pressing global concerns and conclude “humans _are_ the problem” Indeed: [![](https://img.youtube.com/vi/lOxE8EEBwjQ/hqdefault.jpg "Our Future Under AI | Colossus: The Forbin Project") ](https://www.youtube.com/watch?v=lOxE8EEBwjQ) Stranger ---
**Author:** ![Johnny\_Bravo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/johnny_bravo/32/493_2.png) [@Johnny\_Bravo](https://boards.straightdope.com/u/Johnny_Bravo)\ **Post date:** [May 25, 2024, 8:07am UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/12 "2024-05-25T08:07:39Z")
> [@Dr\_Paprika](#): > > The lecturer pointed out that the squirrel seemed to solve the problem, which it presumably had never encountered, Somebody oughta smack that lecturer on the nose with a rolled-up grant application. That’s a wholly unsupported and almost certainly incorrect assertion. > [@Dr\_Paprika](#): > > And that AI might reasonably look at many complicated pressing global concerns and conclude “humans _are_ the problem”. That’s worth at least three more smacks. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 25, 2024, 2:05pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/13 "2024-05-25T14:05:51Z")
> [@Dr\_Paprika](#): > > The brain is dramatically more complex than is dreamt of in our current philosophies. Meh. Most of those I have known researching brain function appreciate how much is unknown and the complexity of it. But that fact, and fears of AI deciding we are the problem, are outside the intended scope of this thread. I am asking about the architecture of the machine machinery and if a fresh look at chip architecture and interchip physical structures using biological intelligences as inspiration may be productive. And maybe not limited to mammalian models. Avian structure maybe a good one to look at? It gets a lot done in a small volume and with relatively little mass, with very different structure but apparently similar interconnections. [https://www.science.org/content/article/newfound-brain-structure-explains-why-some-birds-are-so-smart-and-maybe-even-self-aware](https://www.science.org/content/article/newfound-brain-structure-explains-why-some-birds-are-so-smart-and-maybe-even-self-aware) ---
**Author:** ![Dr\_Paprika](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr_paprika/32/3042_2.png) [@Dr\_Paprika](https://boards.straightdope.com/u/Dr_Paprika)\ **Post date:** [May 25, 2024, 3:40pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/14 "2024-05-25T15:40:34Z")
You can only appreciate known unknowns. Like the incredible energy efficiency of the brain. The adult blood supply is so tightly regulated that it only contains about a teaspoon of glucose. It will take AI a long time to get similar results from a soupçon of sugar water. Of course printing a useful artificial heart or organ often requires some sort of 3D scaffold. AFAIK the more common AI approach is to stack chips to dissipate heat. However, the paper below says laws may require AI to become “explainable” (at the expense of efficiency) and copying the brain is a useful model for this. One presumes many AI companies prefer a black box approach if it is more efficient and less litigious. The lecturer (\>= four smacks applied) stressed the importance of asking “where the training data came from” when using scribes or programs to evaluate pathology. Companies may not always wish to give you the honest answer. And what happens when AI trains on data generated by other AI after running low on training sources? Better or worse than YouTube or X? > **[Explainable AI: A Neurally-Inspired Decision Stack Framework](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9496620/)** > > European law now requires AI to be explainable in the context of adverse decisions affecting the European Union (EU) citizens. At the same time, we expect increasing instances of AI failure as it operates on imperfect data. This paper puts forward a... ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 25, 2024, 6:47pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/15 "2024-05-25T18:47:27Z")
> [@Dr\_Paprika](#): > > copying the brain is a useful model for this Nah. As you’ve pointed out, we are very far from explaining the brain! The point is just to consider if the solutions evolution has created for dealing with hardware/architectural constraints in intelligence processing may have lessons for reimagining how we build our machines. On the software side, yes quality of training sets matters more than quantity. I just listen to a podcast the other day about how an AI is being utilized to help improve mammography screening outcomes (called MIA). The comment there relevant is this (from the transcript): > [@](#): > > “And typically how much data is going into it? How many scans, how many biopsies, what sort of order of magnitude are we talking about? > The order of magnitude is a couple of million. It’s important for AI that it’s not necessarily the number of cases that you need to look at, but you need to optimize for the maximum amount of information that it is trained on. So for instance, if I have an algorithm that is trained on one particular country in Europe, and if you get more of those cases, that’s much less valuable than you get different ethnicities from across the globe, from Asia, from South America, Africa. > I see, so you want as diverse a set as possible around the world. > Diverse in a specific way that it gives you the highest information so that you can reduce confounders.” From Babbage from The Economist: AI and health part one: DrGPT will see you now, May 22, 2024 > **[‎Babbage from The Economist: AI and health part one: DrGPT will see you now...](https://podcasts.apple.com/us/podcast/babbage-from-economist-radio/id508376907?i=1000656422070)** > > ‎Show Babbage from The Economist, Ep AI and health part one: DrGPT will see you now - May 22, 2024 Not a chip issue though. ---
**Author:** ![Dr\_Paprika](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dr_paprika/32/3042_2.png) [@Dr\_Paprika](https://boards.straightdope.com/u/Dr_Paprika)\ **Post date:** [May 25, 2024, 7:01pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/16 "2024-05-25T19:01:11Z")
Sure. But your question is about what is “in progress”. Of course programmers would like to duplicate the brain. And of course doing so is impossible at this time with any fidelity. A computer model stringing together a stream of speech based on what next word seems most likely is not exactly replicating neurological processes, since no one does that^ (though perhaps the process of thinking in a language where you only know basic words is slightly closer). Printing 3D boards is much more expensive and technically difficult. Doubtless it is a better long-term solution, but many problems must be solved to make it possible, nevermind economically viable. The current chips designed to better process graphics are a long way from biomimicry. ^The four-smack lecturer pointed out Microsoft does significant ChatGPT training in English speaking Nigeria to lower costs. This, allegedly Chat frequently uses distinctive features of local speech, such as “let’s delve into this” which are uncommon in, say, Canada. Is this true? Dunno. ---
**Author:** ![CaveMike](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/cavemike/32/16379_2.png) [@CaveMike](https://boards.straightdope.com/u/CaveMike)\ **Post date:** [May 25, 2024, 8:28pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/17 "2024-05-25T20:28:47Z")
Do we understand the behavior of individual neurons and synapses well enough to accurately simulate (e.g. The Human Brain Project) and the main challenges are the scope and interconnected behaviors? Or are we still finding challenges in just simulating individual components? The examples section of the neuromorphic computing wiki page gives me the impression it is the sheer scope, but I’m curious if there are tons of different synapses and we haven’t modeled them all. ---
**Author:** ![Stranger\_On\_A\_Train](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@Stranger\_On\_A\_Train](https://boards.straightdope.com/u/Stranger_On_A_Train)\ **Post date:** [May 25, 2024, 10:38pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/18 "2024-05-25T22:38:45Z")
> [@CaveMike](#): > > Do we understand the behavior of individual neurons and synapses well enough to accurately simulate (e.g. The Human Brain Project) and the main challenges are the scope and interconnected behaviors? > > Or are we still finding challenges in just simulating individual components? There are certainly standardized models of neurons, although they are largely focused on potentiation and synaptic strength, or more generally, treating them as control systems. However, neurons are highly dynamic and complex electrochemical systems which experience a lot of interactions beyond just receiving and transmitting electrical signals, and there are hundreds of different types of neurons in the brain that are actively functioning and in constant flux in terms of their internal states. There is a lot that occurs within the brain neuron which is involved in processing signals that is not well understood, and processing that occurs within non-brain sensory and motor, and interneurons. The focus of neuromorphic engineering is to use principles from naturally occurring neural systems to make fundamental improvements in computing, but we are very far from making (or simulating in software) a synthetic neural system that functions with comparable fidelity to even a simple real brain. Stranger ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [May 25, 2024, 10:45pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/19 "2024-05-25T22:45:50Z")
How would we even make a truly 3-dimensional computer chip? The methods we use now are based on shining light on a surface. That wouldn’t work in the core of a lump of silicon. ---
**Author:** ![DSeid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/dseid/32/20194_2.png) [@DSeid](https://boards.straightdope.com/u/DSeid)\ **Post date:** [May 25, 2024, 11:02pm UTC](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010/20 "2024-05-25T23:02:55Z")
> **[3 Ways 3D Chip Tech is Upending Computing](https://spectrum.ieee.org/amp/amd-3d-stacking-intel-graphcore-2656931957)** > > Our ability to shrink transistors and the logic and memory blocks they make up is slowing down. Plus, chips have reached their size limits. Photolithography tools can only pattern an area of about 850 square millimeters, about the size of a top... > [@](#): > > to sling truly huge volumes of data around as if it were all on the same chip, you need even shorter and denser connections, and that can be done only by stacking one chip atop another. Connecting two chips face-to-face can mean making thousands of connections per square millimeter. > > It’s taken a lot of innovation to get it to work. Engineers had to figure out how to keep heat from one chip in the stack from killing the other, decide what functions should go where and how they should be manufactured, keep the occasional bad chiplet from leading to a lot of costly dud systems, and deal with the resulting added complexities of figuring all that out at once. > > Here are three examples, ranging from the reasonably straightforward to the confoundingly complicated, that show where 3D stacking is now: [Next page](https://boards.straightdope.com/t/ai-chip-architecture-any-biomimetic-work-in-progress/1002010.md?page=2)