Room With 85 Microphones And 8 Cameras Recognizes Speakers And Focuses On Them

The research at the Polytechnic University of Catalonia (UPC) that has led to the design of this room seeks to change, first and foremost, the way in which we interact with computers. According to Josep Ramón Casas, a research involved in the project, "the classic computer interface is comprised of a monitor, a keyboard and a mouse. In one way or another, the keyboard and the mouse are the methods used to enter information in the system; the monitor and the speakers, on the other hand, are how the system returns the information we are looking for".

In this room, that interface is generated by a system that more closely resembles what we as humans use. "We don't interact with another human with a monitor, keyboard and mouse, rather we talk to a person, who sees us and reacts accordingly. This room works in the same way", says Casas. The cameras and the microphones in the "smart room" act as sensors, so the room can see and hear what is going on inside and, according to the researcher, "when something is asked of it, the room reacts".

The output in this case comes from a talking head that addresses the person who asked the question and responds precisely with the information requested. "It is a talking bust projected with a synthesizer that moves in synch with its speech; it is like a secretary. The talking head moves to look at and address the person who asked the question", Josep R. Casas tells SINC.

Despite its promising outlook, the room is not a "science fiction house". Or at least that is not the project carried out by UPC. Through this interface, the room "can do the same thing that any other computer can do; that is, it can not do anything physical, but it can provide information. The room", says Josep Ramón Casas, "does not act as a servant, rather it provides answers similar to those we would obtain from a computer".

Another of the unique features offered by this project is that the response from the "head" changes depending on the number of people in the room. The talking head also has a certain amount of good manners: if there is just one person in the room, it assumes that the information projected through the speakers will only be heard by that person and, therefore, it will not be a nuisance. However, if there are several people in the room, explains Casas, "it shows good manners and does not interrupt when the people are talking, and when someone asks it for information, it provides that information without bothering the others".

According to the researchers in the work published this month in Personal and Ubiquitous Computers, the smart room applied to an educational environment works in the following way: when people enter the room, it uses "multi-mode detection" to start to perceive and analyze the individuals. During the class, the room intervenes when a student raises their hand, alerting the teacher; when the teacher leaves the area, a pre-recorded message is played automatically; when the students take too long to finish a task, the room timer takes notice and plays another pre-recorded message. Finally, when the students leave, the room plays a good-bye message or complementary instructions (on how to exit the building or with the exercises that they must complete at home for that subject).

This application, called Memory Jog, was first developed in 2004 with the "meeting latecomer" version (the room provides information when someone enters). From there, the "teaching environment" was developed (the one described in the article). Finally, there is a demo called "journalists scenario", based on Memory Jog, that emulates an editorial office instead of a classroom.

Despite these efforts, the objective of these researchers is not so much to create a concept of a futuristic home, rather to evaluate the mechanisms used by the software to understand what it is seeing. Josep Ramón Casas, speaking for his colleagues, said "we are not specialists in artificial intelligences, rather we use the room as an analysis environment; that is, how the room understands what happens inside, which is what are work is based on".