How the brain understands one voice in a noisy crowd

Recently, his lab found a new clue into how the brain is able to unpack this information and intentionally hear one speaker, while weaning out or ignoring a different speaker. The brain is actually taking an extra step to understand the words coming from the speaker being listened to, and not taking that step with the other words swirling around the conversation. "Our findings suggest that the acoustics of both the attended story and the unattended or ignored story are processed similarly," said Lalor. "But we found there was a clear distinction between what happened next in the brain."

For this study, recently published in The Journal of Neuroscience, participants simultaneously listened to two stories, but were asked to focus their attention on only one. Using EEG brainwave recordings, the researchers found the story that participants were instructed to pay attention to was converted into linguistic units known as phonemes -- these are units of sound that can distinguish one word from another -- while the other story was not. "That conversion is the first step towards understanding the attended story," Lalor said. "Sounds need to be recognized as corresponding to specific linguistic categories like phonemes and syllables, so that we can ultimately determine what words are being spoken -- even if they sound different -- for example, spoken by people with different accents or different voice pitches." Co-authors on this paper include Rochester graduate student Farhin Ahmed, and Emily Teoh of Trinity College, University of Dublin. The research was funded by the Science Foundation of Ireland, Irish Research Council Government of Ireland, Del Monte Institute for Neuroscience Pilot Program, and the National Institute on Deafness and Other Communication Disorders.

This work is a continuation of a 2015 study lead by Lalor that was published in the journal Cerebral Cortex. That research was recently awarded the 2021 Misha Mahowald Prize for Neuromorphic Engineering for its impact on technology aimed at helping disabled humans improve sensory and motor interaction with the world, like developing better wearable devices, e.g. hearing aids. The research originated at the 2012 Telluride Neuromorphic Engineering Cognition Workshop and led to the multi-partner institution Cognitively Controlled Hearing Aid project funded by the European Union, which successfully demonstrated a real-time Auditory Attention Decoding system.

"Receiving this prize is a great honor in two ways. First, it is generally very nice to be recognized by one's peers for having produced valuable and impactful work. This community is made up of neuroscientists and engineers, so to be recognized by them is very gratifying," Lalor said. "And, second, it is a great honor to be connected to Misha Mashowald -- who was a pioneer in the field of neuromorphic engineering and who passed away far too young."

John Foxe, Ph.D., director of the Del Monte Institute for Neuroscience was a co-author on this study that showed it was possible to use EEG brainwave signals to determine who a person was paying attention to in a multi-speaker environment. It was novel work in that it went beyond the standard approach of looking at effects on average brain signals. "Our research showed that -- almost in real time -- we could decode signals to accurately figure out who you were paying attention to," said Lalor.

Additional authors were James O'Sullivan and Alan Power of Trinity College, Nima Mesgarani of University of California, San Francisco, Siddharth Rajaram and Barbara Shinn-Cunningham of Boston University, Malcolm Slaney of Google Research, and Shihab Shamma of the University of Maryland.