New insights into the way the brain combines memories to solve problems

Results from a team of neuroscientists and artificial intelligence researchers at DeepMind, Otto von Guericke University Magdeburg and the German Center for Neurodegenerative Diseases (DZNE), publishing in the journal Neuron on September 19, provide a window into the way the human brain connects individual episodic memories to solve problems.

For example, imagine you see a woman driving a car on your street. The next day, you see a man driving the exact same car on your street. This might trigger the memory of the woman you saw the day before, and you might reason that the pair live together, given that they share a car.

The researchers propose a novel brain mechanism that would allow retrieved memories to trigger the retrieval of further, related memories in this way. This mechanism allows the retrieval of multiple linked memories, which then enable the brain to create new kinds of insights like these.

In common with standard theories of episodic memory, the authors posit that individual memories are stored as separate memory traces in a brain region called the hippocampus.

"Episodic memories can tell you whether you have met someone before, or where you parked your car," says Raphael Koster (@raphael_koster), a researcher at DeepMind (@DeepMindAI). "The hippocampal system supports this type of memory, which is crucial for rapid learning."

Unlike standard theories, the new theory explores a neglected anatomical connection that loops out of the hippocampus to the neighboring entorhinal cortex but then immediately passes back in. It is this recurrent connection, the researchers thought, that allows memories retrieved from the hippocampus to trigger the retrieval of further, related memories.

The researchers devised a way of testing this theory by taking high-resolution 7-Tesla functional MRI scans from 26 young men and women as they performed a task that required them to draw insights across separate events.

The volunteers were shown pairs of photographs: one of a face and one of an object or a place. Each individual object and place appeared in two separate photo pairs, each of which included a different face. This meant that every photo pair was linked with another pair through the shared object or place image.

In a second phase of the experiment, the researchers tested whether the participants could infer the indirect connection between these linked pairs of photos by showing one face and asking the participants to choose between two other faces. One of the choices -- the correct one -- had been paired with the same object or place image, and one had not.

The researchers guessed that the presented face would trigger the retrieval of the paired object or place and thus spark brain activity that would pass out of the hippocampus into the entorhinal cortex. Crucially, the researchers also expected to see evidence that this activity would then pass back into the hippocampus to trigger the retrieval of the correct linked face.

"Using specialized techniques developed in our lab in Magdeburg, we were able to separate out the parts of the entorhinal cortex that provide the input to the hippocampus," says Yi Chen, researcher at Otto von Guericke University. "This allowed us to precisely measure the patterns of activation in the hippocampus input and output separately."

The researchers trained a computer algorithm to distinguish between activation for scenes and objects within these input and output regions. The algorithm was then applied when only faces were displayed on the screen. If the algorithm indicated the presence of scene or object information on these trials, it could only be driven by retrieved memories of the linked scene or object photos.

"Our data showed that when the hippocampus retrieves a memory, it doesn't just pass it to the rest of the brain," says DeepMind's Dharshan Kumaran (@dharshsky). "Instead, it recirculates the activation back into the hippocampus, triggering the retrieval of other related memories."

The researchers think of the algorithm's results as a synthesis of new and old theories.

"The results could be thought of as the best of both worlds: you preserve the ability to remember individual experiences by keeping them separate, while at the same time allowing related memories to be combined on the fly at the point of retrieval," says Kumaran. "This ability is useful for understanding how the different parts of a story fit together, for example -- something not possible if you just retrieve a single memory."

The authors believe that their results could help AI learn faster in the future.

"While there are many domains where AI is superior, humans still have an advantage when tasks depend on the flexible use of episodic memory," says Martin Chadwick (@MartinJChadwick), another researcher at DeepMind. "If we can understand the mechanisms that allow people to do this, the hope is that we can replicate them within our AI systems, providing them with a much greater capacity for rapidly solving novel problems."

This research was funded by DeepMind and the German Research foundation.