New AI Headphones Let Users Listen to One Person in Crowd by Filtering Only Unwanted Noise, Looking at Them Just Once

A team from the University of Washington is providing a remarkable leap in personal audio technology through AI-powered headphones that filter only unwanted noise.

These headphones, part of a system dubbed "Target Speech Hearing" (TSH), enable users to isolate the voice of a single speaker in a noisy environment by merely looking at them.

While not yet commercially available, this innovative tech was recently showcased at the ACM CHI Conference on Human Factors in Computing Systems in Honolulu.

The TSH System on Headphones

The TSH system allows a user to focus on a person speaking for three to five seconds, effectively "enrolling" them into the system. Once enrolled, the system cancels out all other ambient sounds and plays back only the voice of the targeted speaker, even as the listener moves around and no longer faces the speaker.

To activate the system, the user wears standard headphones equipped with microphones. When the user directs their head toward a speaker and taps a button on the headphones, the sound waves from the speaker's voice reach the microphones simultaneously, with a margin of error of 16 degrees.

These sound signals are then sent to an embedded computer on the headphones, where machine learning software identifies and learns the vocal patterns of the desired speaker.

Senior author Shyam Gollakota, a professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, explained that while many view AI as web-based chatbots, this project demonstrates AI's potential to alter auditory perception based on user preferences.

The team conducted tests with 21 subjects who evaluated the clarity of the enrolled speaker's voice compared to unfiltered audio. On average, the subjects rated the clarity of the enrolled speaker's voice almost twice as high, according to the research team.

The current system can enroll only one speaker at a time and requires a relatively quiet environment to ensure accuracy when registering a new speaker. If the user is dissatisfied with the sound quality, they can re-enroll the speaker to improve clarity.

The technology is expected to evolve, with plans to adapt it to be used in earbuds and hearing aids, expanding its accessibility and functionality.

Semantic Hearing

This AI system builds on previous research by the same team, which developed "semantic hearing." This earlier project allowed users to choose specific sound classes to focus on while canceling sounds within their environment.

The new TSH system takes its concept further by enabling real-time focus on a particular speaker in a dynamic and noisy setting.

"We tend to think of AI now as web-based chatbots that answer questions," Senior author Shyam Gollakota said in a press statement. "But in this project, we develop AI to modify the auditory perception of anyone wearing headphones, given their preferences. With our devices you can now hear a single speaker clearly even if you are in a noisy environment with lots of other people talking."