Computer Helps Deaf Children To Learn Sign Language

Automatic recognition

The insights obtained into human perception can subsequently be used for developing the technology for automatic sign language recognition by cameras and computers. In the TU Delft study, this led to a remarkable application of automatic sign language recognition: an Electronic Learning Environment (ELo) for children who are deaf or hard of hearing.

Pictures

In conjunction with the NSDSK (Dutch foundation for children who are deaf or hard of hearing), the TU Delft researchers developed a computer which can recognise sign language in real time and can therefore be used to teach children who are deaf and hard of hearing new signs more quickly.

When shown a picture, the children aged three to six were asked to make the correct sign. The computer then had to assess the sign. Arendsen: “And that is quite difficult, because a computer is more easily confused than people by irrelevant fidgeting.”

Comprehensible

The task of fellow PhD candidate Jeroen Lichtenauer (EEMCS faculty) was to convert the signs into information which is comprehensible to a computer. As an Industrial Design Engineering student, Arendsen was more involved in the human aspects. Gineke Ten Holt was the third PhD candidate, who worked as a bridge between the two disciplines.

Sign language vocabulary

Further research showed that the Electronic Learning Environment really does work. The sign language vocabulary of those children who had practised with it increased compared to that of a control group. This only applied to the slightly older children however.

Whether the ELo will indeed be applied in practice is still uncertain, Arendsen says. “It depends on aspects such as finance and organisational issues.”