'Sherlock' and the case of narrative perception

"We're interested in examining how we take the kind of continuous environment that we all live in and break it into pieces that we can understand and remember," says Chris Baldassano (@ChrisBaldassano), a postdoctoral research associate in the Princeton Neuroscience Institute and the study's first author. "The main finding of this paper is that we can actually observe what's going on in the brain during this chunking behavior and see that it's happening on a lot of different temporal scales, from seconds up to minutes."

The study made use of two different fMRI datasets, both of which were collected for other research. In the first experiment, participants watched part of the first episode of the BBC show "Sherlock" while being scanned with fMRI. A second group of participants listened to an audio description of the same episode while being scanned. (The research team, led by Uri Hasson and Kenneth Norman, professors in the Department of Psychology at Princeton, frequently makes use of movies and television shows to study brain activity; "Sherlock" is a particular favorite of the lab members.)

Using their new method, the researchers found that brain activity in both cases looked like a sequence of stable activity states punctuated by quick jumps. In some brain regions, these chunks of stable activity lasted for minutes at a time, and the same sequence of activity patterns occurred whether people were watching the show or listening to the audio description. "That suggests these regions are representing something more than just the visual and auditory inputs that are coming in, but a kind of higher-level processing," Baldassano says.

The primary set of regions representing these high-level chunks is called the default mode network. It was originally identified about two decades ago as a collection of brain regions that's active "by default" -- when people are not doing anything that's focused on a particular external goal. It lights up during thought activities that are completely internal, like remembering the past or thinking about the future, as well as during daydreaming and mind-wandering.

"This paper, as well as other recent studies from Hasson's group, challenges the idea that this region of the brain is only about internal activity," Baldassano says. "We're arguing that these regions also represent what's going on in the outside world but on a longer time scale. They are building up representations of what's happening in the world around you."

In the first dataset, some of the participants listening to the audio description of the "Sherlock" episode had previously watched it, and some had not. The researchers tracked the brain activity of the two groups and found that it was different in the people who had seen the show and could anticipate what was going to happen next. "They tend to start these chunks a little sooner than people who haven't seen it," Baldassano explains.

In the second dataset, participants watched the same episode of "Sherlock" and were then asked to retell the story from memory. The researchers found that chunks in high-level areas reappeared during recall in the same order they had appeared during perception, replaying the same sequence of activity patterns. Furthermore, the hippocampus -- a key structure involved in memory -- activated at the end of high-level chunks during movie viewing, and the strength of this hippocampal activity during perception predicted how strongly that chunk was later retrieved. These findings suggest that the hippocampus takes "snapshots" of chunks right before they disappear in a manner that supports long-term memory for those chunks.

Baldassano believes that these findings provide vital clues about how these chunks relate to long-term memory. "Almost all the research that's been done in the field of memory uses a paradigm where people memorize lists of words or photos. This work provides a bridge to be able to understand how we break a continuous, ongoing experience into chunks that can be stored and remembered," he concludes.