The wiring and rewiring of the brain never ends. Neural pathways are constantly being reshaped as we interact with the world and learn new things. At York University and MIT's McGovern Institute, scientists are combining detailed analysis of brain activity with computational modelling to better understand that change.
York Assistant Professor Kohitij Kar, McGovern Institute Postdoctoral Fellow Lynn Sörensen, an affiliate member of Kar's lab and a member of the Centre for Integrative and Applied Neuroscience at York, and Investigator James DiCarlo worked together to compare what happened when monkeys and an artificial neural network with brain-like architecture were trained to visually identify the same objects. As the model's performance improved, it reorganized itself in ways that closely paralleled changes the team detected in the brains of monkeys.
Their research, reported today (July 8) in the journal Nature Communications, shows how changes in visual processing support animals' ability to learn to discriminate new kinds of objects. By modelling these changes, the researchers hope to better predict how training reshapes perception, which could one day inform educational strategies for a wide range of learners.
"Our visual brain does not undergo a drastic reconfiguration when we learn a new object. Instead, it subtly reshapes how visual information is represented, making the distinctions that matter easier for the rest of the brain to read out. What is powerful about this study is that artificial neural networks could predict these subtle brain changes, giving us a concrete bridge between neuroscience and AI," says Kar. "By translating inferences from animal neuroscience into computational models that can generate human-testable predictions, this framework makes translation much more feasible. It allows us to move from understanding how learning changes the macaque visual system to perhaps asking in the near future: why some children may struggle to form stable, generalizable representations - such as recognizing letters across fonts, linking words to objects, distinguishing faces, or applying knowledge in new situations."
Subtle changes
Learning about a new object calls on many parts of the brain. Visual-processing areas work together to make sense of information taken in through the eyes, then communicate with other brain areas to give the visual information meaning and guide behavior. Multiple parts of this system likely change during learning, and the research team wanted a clearer understanding of how that change is distributed.
Neuroscientists have debated how much change occurs in the brain's visual-processing areas when an animal learns to recognize new objects. Some suspected that visual-processing pathways remain largely unchanged during learning to avoid broadly disrupting visual perception, but others have reported changes in activity within dedicated visual-processing areas with this kind of learning in humans and other primates.
To take a closer look, the team focused on neural activity in a key component of the brain's visual object-processing network, the inferior temporal (IT) cortex. By the time visual information reaches the IT cortex, key object features are clearly represented - so much so that it's possible to "decode" what object a monkey is seeing and even predict what errors it's likely to make in identifying it, simply by analyzing patterns of neural activity there.
The team recorded neural activity in the IT cortex from two groups of monkeys as the animals looked at and identified images of objects. Some of the monkeys were untrained, so the images they saw had little meaning to them. Others had already learned to identify similar objects, so they could usually discriminate between elephants, chairs and other select objects, even when those objects were presented at different sizes, from different angles, or against different backgrounds than the ones they had seen before.
The broad pattern of activity in the IT cortex was largely similar in trained and untrained monkeys, suggesting that learning had not dramatically rewritten this high-level visual representation. Still, the group found subtle but reliable differences in the way neurons in the IT cortex responded to images in monkeys that had learned to recognize the kinds of objects they were shown, compared to the untrained monkeys.
Modelling learning
The group turned to computational models to investigate how those modest changes might contribute to learning. Sörensen trained a suite of artificial neural networks whose internal components had been mapped to monkey IT cortex to identify the same categories of objects the monkeys had seen. The models were designed to learn using gradient descent, meaning they continually improved their accuracy by adjusting their parameters in response to errors.
Only some of the primate-like models showed learning behavior that matched that of the monkeys. In those that did, the IT-like stage changed in ways that resembled the learning-related changes the researchers had observed in the IT cortex of trained monkeys.
While gradient descent is commonly used to train artificial intelligence, it is generally considered biologically implausible as a direct model of how the brain learns. The researchers say the strong match in learning effects between the monkeys and their model demonstrates that these kinds of artificial neural networks can offer insights into biological learning at a useful level of abstraction, even if the brain does not learn in the same way.
"This shows that you can actually build in silico versions of future experiments," Sörensen says. "I think that gives us this playground of asking what if' questions - and potentially predicting new things that go beyond the experimenter's intuition."
Most of the changes that allowed for learning in the model occurred outside of the IT cortex. "This tells us that there is a lot between the area we recorded from and the final behavioral readout that needs to change during this process," Kar says. He adds that the team's model will be useful as researchers look more deeply into how downstream brain areas contribute to learning.
The researchers stress that their study allowed more granular measurements of brain activity than would be possible in humans, and because monkeys' brains are organized similarly to our own, their experiments have direct relevance to human learning. They say understanding the impact of plasticity in monkeys' IT cortex could help researchers design new learning strategies for humans.
"Our prior conceptual working model of you - or a monkey - learning new objects was that your brain makes changes to synaptic connections that are largely downstream of your visual system, so you don't destroy your visual system," says DiCarlo, who is also the Peter de Florez Professor of Brain and Cognitive Sciences and director of the MIT Siegel Family Quest for Intelligence. "You wouldn't want your whole visual system to become an elephant detector [just because you've learned to identify an elephant]. But this study went beyond that to say actually, when you learn elephant,' your IT does change a little bit to make it a little more relevant to elephants."
That likely has consequences for recognizing other visual features, too. Subtle changes in the IT cortex that support elephant recognition might also make you better at identifying things other than elephants, DiCarlo says. Likewise, the same changes might make it a little harder to identify something else.
These kinds of consequences may be difficult to predict intuitively, but become obvious with computational modelling. For instance, the team's models revealed that after learning to recognize new objects, the IT cortex contained more information about objects' locations. By providing insights like these, models could aid the design of more effective training strategies for visual tasks, including for people with altered sensory processing, who may learn from visual information in atypical ways.
With files from MIT's McGovern Institute
