Technology To Improve Learning For Visually-impaired Children

Partners in the IST programme-funded MICOLE project, the teams responsible are working in close contact with national and local associations and organisations of visually-disabled persons, as well as schools. Their main task is to design the system itself. However, project coordinator Roope Raisamo, University of Tampere, Finland, describes several supporting activities emphasising users and their real needs.

“We are experimenting with how to use different senses to partially replace missing visual capabilities, especially in tasks that are central in the construction of the system,” he says. “Empirical research of collaborative and cross-modal haptic interfaces for visually-impaired children is one of the most important research activities.”

Haptic technology interfaces with the user through the sense of touch. This emerging technology adds the sense of touch to previously visual-only solutions. MICOLE’s software architecture and applications are multimodal, that is, they use hearing and touch to complement different levels of visual disability.

Their work extends beyond developing an assistive tool. “In addition to MICOLE’s immediate value as a tool, the system will have societal implications by improving the inclusion of the visually disabled in education, work and society in general,” explains Raisamo.

Collaborative learning brings benefits
Initial field studies involved interviews with teachers, children and related user organisations as well as observations of actual group work in schools. The objective was to determine how visually-impaired children collaborate in school with peers and teachers, and to understand to what extent they engage in group work.

“The interaction among the pupils, with teachers and with their peers is very important for learning,” says Raisamo. “We know that collaborative learning has benefits because the pupils learn through a dialogue with their peers and construct their own knowledge by doing tasks together with others.”

Field study results from Austria, Finland, France, Greece, Ireland, Lithuania, Sweden and the UK showed major differences in the education of visually-impaired children, however, they revealed many similarities regarding aspects of collaboration. Based on these results, a prototyping workshop was held in Stockholm where the school situation for such pupils was addressed. Various hapitc and auditory applications developed within MICOLE were assessed and new designs formulated.

He notes there are no specific requirements for the users of the system. “The system adapts to the users. It is aimed at visually-impaired children, but because it facilitates collaboration among sighted and visually-impaired children, it also supports sighted children.”

A multimodal system with visual, audio and haptic feedback can support many kinds of users with disabilities because missing one of the modalities does not make the system unusable, Raisamo adds.

Prototypes being tested
Project partners have developed or tested 16 different interfaces and application prototypes, such as explorative learning of the earth’s internal layers, rhythm reproduction, a tactile maze game, virtual maracas (percussion instruments), post-its with a haptic barcode, an electric circuit browser, a haptic simon game, memory games, a haptic turtle and a haptic game of the classic first video game, pong.

For example, to better teach natural phenomenon, such as seasons, gravity and the solar system, project partners constructed a system using proactive agents that offer the pupil help when necessary. The user decides whether to accept help comprised of visual, auditory and haptic feedback to present content.

King Pong is a fully accessible remake of pong, supporting a spatially localised 3D audio environment, force feedback (translating sound into haptic feedback), recording and playback. It also offers a high level of configurability concerning the auditory grid.

Their MAWEN prototype software demonstrates how blind and visually-impaired children can be helped in mathematical exercises. Geometry is a difficult subject to teach to visually-impaired pupils, yet one of the most useful as it is necessary for the construction of their mental space representation. It is also essential for general education as well as for everyday tasks. Classic geometry teaching is based on visual modality: drawings, graphs, lines and curves - all unavailable to visually impaired students.

The underlying technology of their SALOME system is a haptic force feedback device. A software application uses this device like a pen, writing with friction on a virtual horizontal plane, much like a notebook page. Geometric figures are coded as haptic magnetised grooves that attract the pen toward the different elements of the diagram.

Each element has an audio description (voice synthesiser) that enhances the perception of the element. This sequential audio-haptic scheme shapes a spatial representation of the figure.

Software architecture is being developed in parallel with the prototypes. The next phase will be defining and programming the agents. More than 100 visually-impaired test users in the partner countries are part of this research process.

Building on Europe’s strengths
Project partners include European and world leaders in the area of haptics and multimodal-human-computer interaction. For example, Reachin Technologies AB is a world leader in haptic technology; France Telecom has experience in developing applications for the blind.

“MICOLE offers an outstanding opportunity and the critical mass for the consortium to integrate and realise results of their earlier work and to test the most novel ideas to meet the needs of the visually impaired,” says Raisamo. “The results are expected to make a valuable European contribution to the development of the information society and real-world equality for visually-disabled children, empowering them as future citizens.”

The multimodal software architecture to create new applications is under construction. Scientific results from multimodal navigation and cross-modal presentation of information are being fed in to the team’s work. The three-year project is scheduled to end in August 2007.