To identify a voice, brains rely on sight

The new study, published last week in the Journal of Neurophysiology, suggests that voice and face recognition are linked even more intimately than previously thought. It offers an intriguing possibility that visual and auditory information relevant to identifying someone feeds into a common brain center, allowing for more robust, well-rounded recognition by integrating separate modes of sensation.

"From behavioral research, we know that people can identify a familiar voice faster and more accurately when they can associate it with the speaker's face, but we never had a good explanation of why that happens," said senior author Taylor Abel, M.D., associate professor of neurological surgery at the University of Pittsburgh School of Medicine. "In the visual cortex, specifically in the part that typically processes faces, we also see electrical activity in response to famous people's voices, highlighting how deeply the two systems are interlinked."

Even though the interplay between the auditory and the visual brain processing systems has been widely acknowledged and investigated by various teams of neuroscientists all over the world, those systems were traditionally thought to be structurally and spatially distinct.

Until recently, few studies attempted to directly measure activity from the brain center -- the primary role of which is to consolidate and process visual information -- to determine whether this center is also engaged when participants are exposed to famous voice stimuli.

Researchers at Pitt had a unique opportunity to study that interaction in patients with epilepsy who, as part of their medical care, were temporarily implanted with electrodes measuring brain activity to determine the source of their seizures.

Five adult patients consented to participate in the study, where Abel and his team showed participants photographs of three U.S. presidents -- Bill Clinton, George W. Bush and Barack Obama -- or played short recordings of their voices, and asked participants to identify them.

Recordings of the electrical activity from the region of the brain responsible for processing visual cues -- called the fusiform gyri, or FG -- showed that the same region became active when participants heard familiar voices, though that response was lower in magnitude and slightly delayed.

"This is important because it shows that auditory and visual areas interact very early when we identify people, and that they don't work in isolation," said Abel. "In addition to enriching our understanding of the basic functioning of the brain, our study explains the mechanisms behind disorders where voice or face recognition is compromised, such as in some dementias or related disorders."

Ariane Rhone, Ph.D., of the University of Iowa, and Kyle Rupp, Ph.D., of Pitt, are co-first authors. Additional authors of the study are Dan Tranel, Ph.D., and Matthew Howard, III, Ph.D., both of the University of Iowa; and Jasmine Hect, Ph.D., and Emily Harford, Ph.D., both of Pitt.

This work was supported by National Institutes of Health grants R01 DC004290 and R21 DC019217.