RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices

summary

The gist

Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication; this paper introduces RT-Tango…

In short

RT-Tango is a real-time distributed system for enhancing speech in hearing aids that have limited power and processing capability. It uses a two-stage approach involving feature compression, lightweight mask estimation, and temporal sparsification to achieve low latency and reduced computational cost. This allows for high-quality speech enhancement on resource-constrained platforms like hearing aids.

Key concepts

ERBscaled filterbank
This technique compresses the input audio features by scaling them based on the Equivalent Received Bandwidth (ERB). Instead of processing every frequency band, it aligns the features with perceptual frequency scaling. This reduces the number of parameters needed for mask estimation, making it much lighter and faster for devices with limited power.
Lightweight grouped recurrent mask estimation
This method estimates speech masks by partitioning the feature space into groups. By processing these groups in parallel, the total computational complexity is significantly reduced to O(H^2/G), where G is the group size. This allows for efficient, partitioned processing suitable for distributed binaural setups on hearing aids.
Temporal sparsification (Fixed-Rate Skipping)
This strategy reduces the frequency of mask updates during streaming by reusing previously calculated masks. By skipping updates, the system performs inference less frequently—for example, one inference every four frames. This drastically lowers the computational load while maintaining real-time performance.
Asymmetric STFT configuration
The system uses a long analysis window (aW) to keep good frequency resolution for accurate sound separation while using a shorter synthesis window (sW) to minimize reconstruction latency. This decoupling allows the system to operate at ultralow latencies, as low as 8 ms, which is critical for real-time hearing aid use.

Terminology used across episodes

This episode discusses

The paper

RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices · Read on arXiv

Zahra Benslimane, Pierre Chouteau, Martyna Poreba, Fabrice Auzanneau, Michal Szczepanski, Fabian Chersi

Universite Paris-Saclay · Universite de Lorraine

Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication, yet existing efficient solutions predominantly address single-channel settings. In this paper, we introduce RT-Tango, a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids. RT-Tango relies on a two-stage distributed architecture combining perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost. Stringent latency constraints are addressed by decoupling spectral resolution from algorithmic delay using an asymmetric STFT, together with causal recurrent inference and online estimation of spatial statistics. Experimental results show that RT-Tango achieves competitive speech enhancement while significantly reducing MACs operations and functioning at ultra-low latencies as low as 8 ms.

DOI: 10.21437/Interspeech.2026-3301

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices".

Jane: Here's the gist of the paper. Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication; this paper introduces RT-Tango, a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids <ref:2607.01834#pg2> Framework Overview RT-Tango is a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids <ref:2607.01834#pg2>.

Tom: It relies on a two-stage distributed architecture combining perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost <ref:2607.01834#pg2>.

Jane: Stringent latency constraints are addressed by decoupling spectral resolution from algorithmic delay using an asymmetric STFT, together with causal recurrent inference and online estimation of spatial statistics <ref:2607.01834#pg2>.

Tom: Efficiency Design Choices The framework employs several efficiency-driven design choices to meet the strict operational requirements of hearing devices <ref:2607.01834#pg2>.

Title and authors: Tom: We’ve covered the mechanics of RT-Tango, but let's get back to what the authors actually summarized about their work and why they felt this redesign was necessary.

Jane: They start by stating that real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication, which is why existing efficient solutions predominantly address single-channel settings.

Lu: So they set the stage by pointing out the limitations of prior work when applied to binaural scenarios.

Tom: They then introduce RT-Tango as their solution: a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids.

Meng: That description is very direct about the target audience; it’s not just a theoretical model; it’s aimed squarely at devices with strict operational limits.

Jane: The core of their approach is a two-stage distributed architecture that combines perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost.

Tom: So the summary boils down to three main mechanisms: feature compression, grouped recurrent estimation, and temporal sparsification for efficiency.

Lu: They also address latency by decoupling spectral resolution from algorithmic delay using an asymmetric STFT configuration together with causal recurrent inference.

Jane: And they handle the streaming aspect by using online estimation of spatial statistics via a recursive exponential moving average to maintain performance in a causal manner.

Tom: So, they’re addressing all three major pain points: cost, latency, and communication complexity in a distributed setting.

Meng: That sounds like a comprehensive attack on the problem space because they aren't just optimizing one part of the system in isolation.

Jane: It really is about creating a cohesive solution that works across different stages of processing while respecting those hardware limitations.

The paper's summary: Tom: Now that we know what they are doing, let’s look at the specific improvements they suggested over the baseline Tango framework to see where the real gains come from.

Jane: They revisit the baseline Tango framework which uses a Single-Node DNN and Multi-Node DNN to estimate speech and noise masks independently before refinement, and RT-Tango improves efficiency by replacing recurrent layers in Tango-RNN with GRNNs to reduce computational cost.

Lu: They also detail several efficiency design choices: first, they use perceptually motivated feature compression using an ERB scaled filterbank which aligns the features with perceptual frequency scaling.

Tom: So that means the input data itself is pre-processed to be less complex before it even hits the main model. What does that do for the mask estimators?

Lu: It allows mask estimators to operate with fewer parameters and operations, lowering computational cost for on-device deployment because of this feature compression.

Meng: Fewer operations mean lower power consumption, which is a huge win when you're talking about battery life in hearing aids.

Jane: They also have lightweight grouped recurrent mask estimation that partitions the feature space into groups of size H/G, making the total recurrent complexity become G times O((H/G) squared), which makes it significantly lower compared to a full-band RNN.

Tom: That partitioning sounds like a clever way to manage complexity in a distributed setting; does that help with parallel processing?

Lu: It enables parallel and partitioned processing within each group, which is key for distributed binaural settings. They also introduced temporal sparsification using fixed-rate skipping to reduce the mask’s update frequency by reusing previously estimated masks.

Meng: Reusing old masks sounds like a practical trick for reducing the load during streaming inference; how much of a load reduction are we talking about?

Jane: This strategy allows for one inference every four frames in one case, significantly reducing load during real-time streaming. And they also use an asymmetric STFT configuration to decouple spectral resolution from algorithmic delay.

Tom: So to summarize the key improvements: feature compression for input dimensionality, grouped recurrent estimation for complexity reduction, temporal sparsification for update frequency control, and an asymmetric STFT for latency management.

Lu: They also included online estimation of spatial statistics via a recursive exponential moving average update to maintain the spatial covariance matrices, which is crucial because it allows the system to operate effectively in a causal manner while keeping speech intelligibility competitive with baselines.

The paper's improvements: Jane: So we’re wrapping up with this paper, "RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices."

Tom: We’ve looked at how they addressed latency, computational cost, and communication complexity through their methods.

Lu: The overall implication is that by revisiting the Tango architecture from a systemlevel perspective, they adopted complementary architectural and algorithmic design principles to meet the stringent constraints of hearing aids.

Meng: For us in production, this means we have a model that’s demonstrably faster and smaller than some prior approaches when compared to the baseline Tango, which is what matters.

Jane: The work demonstrates that RT-Tango achieves competitive speech enhancement while significantly reducing MACs operations and functioning at ultralow latencies as low as eight milliseconds <ref:2607.01834#pg3>.

Tom: The streaming variant, RT-Tango-OS, operates with an algorithmic latency of eight milliseconds through the asymmetric STFT and online SDW-MWF updates <ref:2607.01834#pg3>.

Lu: They also highlighted that RTTango-OS is particularly well-suited for practical deployment in modern binaural hearing aids because it satisfies the stringent real-time, latency, and computational constraints of those devices.

Meng: The cost of thirty-five point one four MMACs/s for RT-Tango is a number we can actually work with when compared to the gains in speed and power efficiency for that specific use case <ref:2607.01834#pg3>.

Jane: So, the paper introduces RT-Tango as a distributed binaural SE framework designed to satisfy the stringent real-time, latency, and computational constraints of modern hearing devices.

Tom: It’s about taking a complex system and applying targeted optimizations to make it fit the hardware reality of wearables.

Meng: This work gives us a concrete model that’s demonstrably faster and smaller than some prior approaches when compared to the baseline Tango.

Jane: It shows that distributed binaural speech enhancement is becoming feasible on resource-constrained platforms, which has big implications for assistive listening technology everywhere.

Conclusion: Tom: So we’ve been looking at RT-Tango, the Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices paper, and it really shows how much work goes into making things run on tiny hardware.

Jane: Exactly, and what stuck with me was how they tackled those three big problems: latency, computational cost, and communication overhead in a distributed setup.

Lu: The idea of replacing recurrent layers in the Tango-RNN with GRNNs is smart because it directly attacks that computational complexity you get when you scale things up across multiple nodes.

Meng: From my side, I’m looking at the efficiency gains—the temporal sparsification and that fixed-rate frame reuse—that makes it actually viable for a battery-powered device instead of just running on a powerful server.

Lalam: The authors are pretty sure this framework successfully maintains competitive enhancement performance while achieving ultralow latencies, down to eight milliseconds for the streaming variant.

Tom: Eight milliseconds is low enough for real-time interaction, which is what makes this work immediately interesting for anyone building something wearable right now.

Jane: And they manage that latency by using an asymmetric STFT configuration, decoupling the frequency resolution from the algorithmic delay so you can get a good sound quality without waiting too long.

Lu: They also introduced perceptual feature compression using an ERB scaled filterbank, which means they’re not feeding the model as much raw data as they are needing to for speech cues.

Meng: Fewer operations mean less power draw, and that's what really matters when you're designing something that needs to sit in someone's ear all day.

Lalam: So, if you’re building a system where the constraints are this tight, RT-Tango is positioned as a strong contender because it balances the need for high quality with the necessity of low complexity.

Tom: It seems like they didn't just tweak an existing model; they completely rebuilt how those recurrent layers were handled to suit this specific distributed binaural streaming requirement.

Jane: That’s right, it’s a complete revision of the Tango architecture through these architectural and signal-processing optimizations to keep the interaural balance intact while cutting the computational load.

Lu: The online estimation of spatial statistics using that recursive exponential moving average is crucial because it ensures the system stays causal even when you’re running across different nodes.

Meng: I just hope they didn't leave too much to chance in those spatial updates, because if the estimates drift, the whole thing breaks at scale.

Tom: Well, for now, that’s where we land with RT-Tango and its real-time streaming capabilities.

Jane: It’s a solid piece of work that proves you can design for the real-world limitations of assistive listening devices without totally sacrificing performance.

Lu: It gives us a strong blueprint for how to apply these distributed processing techniques to other low-resource audio tasks.

Lalam: The implications are clear: we can start thinking about complex speech enhancement models that actually fit on a chip and run continuously in the field.

More episodes

← Home