Electronic neural networks, one of the key concepts in artificial intelligence research, have drawn inspiration from biological neurons since their inception—as their name attests. New research now reveals that the influential AI transformer architecture also shares unexpected similarities with human neurobiology.
In a collaborative studyscientists have suggested that the biological astrocyte-neuron networks can simulate the core computations of transformers. Or vice versa. The findings—jointly reported by MIT, the MIT-IBM Watson AI Lab, and Harvard Medical School—were published this week in the journal Proceedings of the National Academy of Sciences.
Astrocyte-neuron networks are networks of brain cells that consist of two types of cells: astrocytes and neurons. Astrocytes are cells that support and control neurons, which are brain cells that send and receive electrical impulses. Their activity is basic thinking. Astrocytes and neurons communicate with each other using chemicals, electricity, and touch.
On the other hand, the AI transformers—first introduced in 2017—is one of the base technologies behind generative systems like ChatGPT. –in fact, that’s where the “T” in GPT comes from. Unlike neural networks that process inputs sequentially, transformers can access all inputs directly through a mechanism called self-attention. This allows them to learn complex dependencies in data such as text.
The researchers focused on tripartite synapses, which are junctions where astrocytes form connections between a neuron that sends signals (presynaptic neuron) and a neuron that receives signals (postsynaptic neuron).
Using mathematical modeling, they show how the integration of signals in astrocytes over time can provide the necessary spatial and temporal memory for self-attention. Their models also show that a biological transformer can be built using calcium signaling between astrocytes and neurons. TL;DR, this study explains how to make an organic transformer.
“Remaining electrically silent for more than a century in brain recordings, astrocytes are one of the most abundant, yet least understood, brain cells,” said Konstantinos Michmizos, associate professor of computer science at Rutgers University. told MIT. “The potential to unleash enormous computational power in the other half of our brain.”
The hypothesis uses emerging evidence that astrocytes play an active role in information processing, unlike their previously thought housekeeping tasks. It also outlines a biological basis for transformers, which can outperform traditional neural networks in facilitating tasks such as generating coherent text.
The proposed biological transformers could provide new insights into human cognition if experimentally verified. However, significant gaps remain between people and data-hungry transformer models. While transformers require large training sets, the human brain transforms the experience of organic language on a modest energy budget.
Although the links between neuroscience and artificial intelligence offer insight, understanding the sheer complexity of our minds remains a huge challenge. Biological connections represent only one piece of the puzzle—unraveling the intricacies of human intelligence requires a sustained effort across disciplines. How neural biology came to be so close to magic continues to be science’s deepest mystery.
