Study Shows Combination Of Sight And Sound Helps Adults Learn Basic Visual Tasks More Rapidly

The study, published in a recent issue of Current Biology, demonstrates that using stimuli that involve both vision and hearing can be combined to produce speedier learning of visual information and suggest that multi-sensory training programs may be more effective for adults learning new skills -- such as discriminating differences between highly similar objects, or finding an item in cluttered scene.

According to Aaron Seitz, a research assistant professor of psychology at BU and lead investigator of the study, the traditional belief among neuroscientists is that the five senses operate largely as independent systems. However, mounting data suggests that interactions between vision, hearing, smell, touch and taste are the rule rather than the exception when it comes to how the human brain processes sensory information and thus perceives things.

Using this as their basis, Seitz and his colleagues, Robyn Kym and Ladan Shams of UCLA, set out to determine if engaging both the eyes and ears can help people learn to identify patterns of motion more rapidly.

Using a specially-designed computer program, the team tested how accurately subjects could recognize whether or not dots were moving across the screen in a coherent pattern. One group of participants saw just the dots while a second group also heard a sound moving in coordination with the motion of the dots.

"We showed subjects a series of screens -- each for about half of a second. In half of the screens, the dots were moving randomly. The other half contained a few dots moving in a particular direction, in this case either left or right, hidden among a bunch of randomly moving dots," explained Seitz. "We then asked the subjects to report which screens in the series they thought contained dots that were moving coherently as well as in which direction they were going."

Participants were trained on the same series of screens for 10 days and the data indicate that individuals who saw the dots accompanied by a sound learned more quickly to correctly decipher the coherent movement and course of the dots and reached near peak performance on their third day of training. Individuals trained in silence failed to reach that same level of performance even after 10 days of training.

"While all subjects did show improvement, the audiovisual group learned much faster and showed better rates of retention from session to session compared to the group that received visual signals alone," said Seitz.

While it may sound obvious that both seeing and hearing cues resulted in faster learning, the benefits of the multi-sensory training surprised the team.

"Learning how to perceive motion is thought to be controlled by lower-level functions of vision believed to be largely "fixed" or unchangeable after critical periods of development during the first few years of life," explained Seitz. "And the fact that hearing benefits this low-level visual learning is rather surprising. What our results demonstrate is that with the right training paradigms, you can actually achieve alterations in adult perceptual systems."

The team believes this new information could eventually enhance the ways in which adults and even children learn new skills and that multi-sensory training strategies could someday be incorporated into rehabilitative methods for people with sensory impairments.