Touch Helps Make The Connection Between Sight And Hearing

These results, published March 16th in the journal PloS One, should improve learning methods, both for children learning to read and adults learning foreign languages.

To read words that are new to us, we have to learn to associate a visual stimulus (a letter, or grapheme) with its corresponding auditory stimulus (the sound, or phoneme). When visual stimuli can be explored both visually and by touch, adults learn arbitrary associations between auditory and visual stimuli more efficiently. The researchers reached this conclusion from an experiment on thirty French-speaking adults. They first compared two learning methods with which the adults had to learn 15 new visual stimuli, inspired by Japanese characters, and their 15 corresponding sounds (new auditory stimuli with no associated meaning). The two learning methods differed in the senses used to explore the visual stimuli. The first, "classic", method used only vision. The second, "multisensory", method used touch as well as vision for the perception of the visual stimuli. After the learning phase, the researchers measured the performances of each adult using different tests (1). They found that all the participants had acquired an above-chance ability to recognize the visual and auditory stimuli using the two methods.

The researchers then went on to test the participants by two other methods (2), this time to measure the capacity to learn associations between visual and auditory stimuli. The results showed that the subjects were capable of learning the associations with both learning methods, but that their performances were much better using the "multisensory" learning method. When the subjects were given the same tests a week after the learning phase, the results were the same.

These results support those already found by the same team, in work done with young children. The explication lies in the specific properties of the haptic sense (3) in the hands, which plays a "cementing" role between sight and hearing, favoring the connection between the senses. What goes on in the brain remains to be explored, as does the neuronal mechanism: the researchers plan to develop a protocol that will let them use fMRI (4) to identify the areas of the cortex that are activated during the "multisensory" learning process.

(1) The first two tests respectively measured the learning capacity for visual and auditory stimuli using recognition tests. In a visual test, a visual stimulus had to be recognized among 5 new visual stimuli. In an auditory test, the target had to be recognized among 5 new sounds.

(2) In the "visual-auditory" test, the subject was presented with a visual stimulus and had to recognize its corresponding sound among 5 other sounds. In the "auditory-visual" test, the opposite was done.

(3) Or tactile-kinesthetic. "Haptic" corresponds to the sense of touch, used to feel the letters.

(4) Functional magnetic resonance imaging: the application of magnetic resonance imagery to study the function of the brain.