Feeling Your Words: Hearing With Your Face

"How your own face is moving makes a difference in how you 'hear' what you hear," said first author Takayuki Ito, a senior scientist at Haskins.

When, Ito and his colleagues used a robotic device to stretch the facial skin of "listeners" in a way that would normally accompany speech production they found it affected the way the subjects heard the speech sounds.

The subjects listened to words one at a time that were taken from a computer-produced continuum between the words "head" and "had." When the robot stretched the listener's facial skin upward, words sounded more like "head." With downward stretch, words sounded more like "had." A backward stretch had no perceptual effect.

And, timing of the skin stretch was critical — perceptual changes were only observed when the stretch was similar to what occurs during speech production.

These effects of facial skin stretch indicate the involvement of the somatosensory system in the neural processing of speech sounds. This finding contributes in an important way to our understanding of the relationship between speech perception and production. It shows that there is a broad, non-auditory basis for "hearing" and that speech perception has important neural links to the mechanisms of speech production.

Details of this study are reported in an article titled "Somatosensory function in speech perception" by Ito, Mark Tiede, and David J. Ostry which appeared in the Proceedings of the National Academy of Sciences (PNAS). All of these researchers are scientists affiliated with Haskins Laboratories. Tiede is also at the Research Laboratory of Electronics at the Massachusetts Institute of Technology and Ostry is a Professor in the Department of Psychology at McGill University.

Haskins Laboratories was founded in 1935 by the late Dr. Caryl P. Haskins. This independent research institute has been in New Haven, Connecticut since 1970 when it formalized affiliations with Yale University and the University of Connecticut. The Laboratories' primary research focus is on the science of the spoken and written word.