How our brains know when something's different

"Our results suggest that every experience we store into memory can be used to set our expectations and predictions for the future," Kareem Zaghloul, M.D., Ph.D., principal investigator at the NIH's National Institute of Neurological Disorders and Stroke (NINDS), and senior author of the study published in Nature Communications. "This study shows how the brain uses certain neural activity patterns to compare our expectations with the present. Ultimately, we hope that these results will help us better understand how the brain portrays reality under healthy and disease conditions."

The study was led by Rafi Haque, an M.D., Ph.D. student at Emory University School of Medicine, Atlanta, who was completing his dissertation work with Dr. Zaghloul. His primary research goal was to test out whether a theory called predictive coding can be applied to how our brains remember past experiences, known as episodic memories.

"Predictive coding basically states that the brain optimizes neural activity for processing information. In other words, the theory forecasts that the brain uses more neural activity to process new information than it does for things that we are familiar with," said Dr. Haque. "Years of research has shown that over time this is how we learn to expect what common sights, like green grass, looks like or everyday noises, such as certain bird chirps, sound like. We wanted to know whether the brain uses a similar process to manage our experiences."

To test this idea, the team worked with 14 patients with drug-resistant types of epilepsy whose brains had been surgically implanted with grids of electrodes as part of an NIH Clinical Center trial aimed at diagnosing and treating their seizures.

The experiment began when the patients were shown and asked to memorize a series of four natural scenes displayed on a computer screen. For example, one of the scenes was of a brown bicycle leaning upright on a kickstand in front of a green bush. A few seconds later they were shown a new set of images and asked whether they recognized the scene or noticed something different. Some images were the same as before while others were slightly modified by adding or removing something, such as a red bird, from the scene.

On average, the patients successfully recognized 88% of the repeat scenes, 68% of scenes that were missing something, and 65% of the ones in which something was added. In each case, it took them about two and a half seconds to notice.

Further analysis of a subset of the patients showed that they successfully located 82% of additions and 70% of removals. Curiously, their eyes fixated often (83%) on additions but barely at all (34%) on areas in the scene where something was removed.

"Overall, these results suggest it takes just one moment to not only remember a new experience but also to use memories of that experience to set future expectations," said Dr. Zaghloul.

Meanwhile, electrical recordings uncovered differences in brain wave activity between the times the patients successfully remembered repeat scenes and the times they spotted changes to a scene.

In both situations, the appearance of a scene on the computer screen triggered a rise in the strength of high frequency waves of neural activity in the lateral occipital cortex, a visual processing center in the back of the brain. The surge flowed forward arriving a few milliseconds later at a memory center called the medial temporal lobe.

Also, in both situations, the patients' brains appeared to replay neural activity patterns observed when they first witnessed the scenes.

"These results support the idea that memories of visual experiences follow a certain pathway in the brain," said Dr. Haque.

The difference though was that the surge in activity was stronger when the patients recognized a change to a scene.

In addition, during these moments, a second, lower frequency wave appeared to synchronously rumble through the lateral occipital cortex and the medial temporal lobe.

"Our data supports the idea that our expectations of visual experiences are controlled by a feedback loop between the visual cortex and the medial temporal lobe," said Dr. Zaghloul. "High frequency waves of neural activity appear to carry an error message when we see something that does not match our expectations, while the lower frequency waves may be updating our memories."