Which speaker are you listening to? Hearing aid of the future uses brainwaves to find out

Having a casual conversation at a cocktail party is a challenge for someone with a hearing aid, says Professor Tom Francart from the Department of Neurosciences at KU Leuven: "A hearing aid may select the loudest speaker in the room, for instance, but that is not necessarily the person you're listening to. Alternatively, the system may take into account your viewing direction, but when you're driving a car, you can't look at the passenger sitting next to you."

Researchers have been working on solutions that take into account what the listener wants. "An electroencephalogram (EEG) can measure brainwaves that develop in response to sounds. This technique allows us to determine which speaker someone wants to listen to. The system separates the sound signals produced by different speakers and links them to the brainwaves. The downside is that you have to take into account a delay of ten to twenty seconds to get it right with reasonable certainty."

Artificial intelligence to speed up the process

A new technique makes it possible to step up the pace, Professor Alexander Bertrand from the Department of Electrical Engineering at KU Leuven continues: "Using artificial intelligence, we found that it is possible to directly decode the listening direction from the brainwaves alone, without having to link them to the actual sounds."

"We trained our system to determine whether someone is listening to a speaker on their left or their right. Once the system has identified the direction, the acoustic camera redirects its aim, and the background noise is suppressed. On average, this can now be done within less than one second. That's a big leap forward, as one second constitutes a realistic timespan to switch from one speaker to the other."

From lab to real life

However, it will take at least another five years before we have smart hearing aids that work with brainwaves, Professor Francart continues. "To measure someone's brainwaves in the lab, we make them wear a cap with electrodes. This method is obviously not feasible in real life. But research is already being done into hearing aids with built-in electrodes."

The new technique will be further improved as well, PhD student Simon Geirnaert adds. "We're already conducting further research, for instance into the problem of combining multiple speaker directions at once. The current system simply chooses between two directions. While first experiments show that we can expand that to other possible directions, we need to refine our artificial intelligence system by feeding the system with more brainwave data from users who are also listening to speakers from other directions."