RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices".
Jane: Here's the gist of the paper. Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication; this paper introduces RT-Tango, a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids <ref:2607.01834#pg2> Framework Overview RT-Tango is a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids <ref:2607.01834#pg2>.
Tom: It relies on a two-stage distributed architecture combining perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost <ref:2607.01834#pg2>.
Jane: Stringent latency constraints are addressed by decoupling spectral resolution from algorithmic delay using an asymmetric STFT, together with causal recurrent inference and online estimation of spatial statistics <ref:2607.01834#pg2>.
Tom: Efficiency Design Choices The framework employs several efficiency-driven design choices to meet the strict operational requirements of hearing devices <ref:2607.01834#pg2>.
Title and authors: Tom: We’ve covered the mechanics of RT-Tango, but let's get back to what the authors actually summarized about their work and why they felt this redesign was necessary.
Jane: They start by stating that real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication, which is why existing efficient solutions predominantly address single-channel settings.
Lu: So they set the stage by pointing out the limitations of prior work when applied to binaural scenarios.
Tom: They then introduce RT-Tango as their solution: a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids.
Meng: That description is very direct about the target audience; it’s not just a theoretical model; it’s aimed squarely at devices with strict operational limits.
Jane: The core of their approach is a two-stage distributed architecture that combines perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost.
Tom: So the summary boils down to three main mechanisms: feature compression, grouped recurrent estimation, and temporal sparsification for efficiency.
Lu: They also address latency by decoupling spectral resolution from algorithmic delay using an asymmetric STFT configuration together with causal recurrent inference.
Jane: And they handle the streaming aspect by using online estimation of spatial statistics via a recursive exponential moving average to maintain performance in a causal manner.
Tom: So, they’re addressing all three major pain points: cost, latency, and communication complexity in a distributed setting.
Meng: That sounds like a comprehensive attack on the problem space because they aren't just optimizing one part of the system in isolation.
Jane: It really is about creating a cohesive solution that works across different stages of processing while respecting those hardware limitations.
The paper's summary: Tom: Now that we know what they are doing, let’s look at the specific improvements they suggested over the baseline Tango framework to see where the real gains come from.
Jane: They revisit the baseline Tango framework which uses a Single-Node DNN and Multi-Node DNN to estimate speech and noise masks independently before refinement, and RT-Tango improves efficiency by replacing recurrent layers in Tango-RNN with GRNNs to reduce computational cost.
Lu: They also detail several efficiency design choices: first, they use perceptually motivated feature compression using an ERB scaled filterbank which aligns the features with perceptual frequency scaling.
Tom: So that means the input data itself is pre-processed to be less complex before it even hits the main model. What does that do for the mask estimators?
Lu: It allows mask estimators to operate with fewer parameters and operations, lowering computational cost for on-device deployment because of this feature compression.
Meng: Fewer operations mean lower power consumption, which is a huge win when you're talking about battery life in hearing aids.
Jane: They also have lightweight grouped recurrent mask estimation that partitions the feature space into groups of size H/G, making the total recurrent complexity become G times O((H/G) squared), which makes it significantly lower compared to a full-band RNN.
Tom: That partitioning sounds like a clever way to manage complexity in a distributed setting; does that help with parallel processing?
Lu: It enables parallel and partitioned processing within each group, which is key for distributed binaural settings. They also introduced temporal sparsification using fixed-rate skipping to reduce the mask’s update frequency by reusing previously estimated masks.
Meng: Reusing old masks sounds like a practical trick for reducing the load during streaming inference; how much of a load reduction are we talking about?
Jane: This strategy allows for one inference every four frames in one case, significantly reducing load during real-time streaming. And they also use an asymmetric STFT configuration to decouple spectral resolution from algorithmic delay.
Tom: So to summarize the key improvements: feature compression for input dimensionality, grouped recurrent estimation for complexity reduction, temporal sparsification for update frequency control, and an asymmetric STFT for latency management.
Lu: They also included online estimation of spatial statistics via a recursive exponential moving average update to maintain the spatial covariance matrices, which is crucial because it allows the system to operate effectively in a causal manner while keeping speech intelligibility competitive with baselines.
The paper's improvements: Jane: So we’re wrapping up with this paper, "RT-Tango: Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices."
Tom: We’ve looked at how they addressed latency, computational cost, and communication complexity through their methods.
Lu: The overall implication is that by revisiting the Tango architecture from a systemlevel perspective, they adopted complementary architectural and algorithmic design principles to meet the stringent constraints of hearing aids.
Meng: For us in production, this means we have a model that’s demonstrably faster and smaller than some prior approaches when compared to the baseline Tango, which is what matters.
Jane: The work demonstrates that RT-Tango achieves competitive speech enhancement while significantly reducing MACs operations and functioning at ultralow latencies as low as eight milliseconds <ref:2607.01834#pg3>.
Tom: The streaming variant, RT-Tango-OS, operates with an algorithmic latency of eight milliseconds through the asymmetric STFT and online SDW-MWF updates <ref:2607.01834#pg3>.
Lu: They also highlighted that RTTango-OS is particularly well-suited for practical deployment in modern binaural hearing aids because it satisfies the stringent real-time, latency, and computational constraints of those devices.
Meng: The cost of thirty-five point one four MMACs/s for RT-Tango is a number we can actually work with when compared to the gains in speed and power efficiency for that specific use case <ref:2607.01834#pg3>.
Jane: So, the paper introduces RT-Tango as a distributed binaural SE framework designed to satisfy the stringent real-time, latency, and computational constraints of modern hearing devices.
Tom: It’s about taking a complex system and applying targeted optimizations to make it fit the hardware reality of wearables.
Meng: This work gives us a concrete model that’s demonstrably faster and smaller than some prior approaches when compared to the baseline Tango.
Jane: It shows that distributed binaural speech enhancement is becoming feasible on resource-constrained platforms, which has big implications for assistive listening technology everywhere.
Conclusion: Tom: So we’ve been looking at RT-Tango, the Real-Time Distributed Binaural Speech Enhancement for Low-Power Hearing Aid Devices paper, and it really shows how much work goes into making things run on tiny hardware.
Jane: Exactly, and what stuck with me was how they tackled those three big problems: latency, computational cost, and communication overhead in a distributed setup.
Lu: The idea of replacing recurrent layers in the Tango-RNN with GRNNs is smart because it directly attacks that computational complexity you get when you scale things up across multiple nodes.
Meng: From my side, I’m looking at the efficiency gains—the temporal sparsification and that fixed-rate frame reuse—that makes it actually viable for a battery-powered device instead of just running on a powerful server.
Lalam: The authors are pretty sure this framework successfully maintains competitive enhancement performance while achieving ultralow latencies, down to eight milliseconds for the streaming variant.
Tom: Eight milliseconds is low enough for real-time interaction, which is what makes this work immediately interesting for anyone building something wearable right now.
Jane: And they manage that latency by using an asymmetric STFT configuration, decoupling the frequency resolution from the algorithmic delay so you can get a good sound quality without waiting too long.
Lu: They also introduced perceptual feature compression using an ERB scaled filterbank, which means they’re not feeding the model as much raw data as they are needing to for speech cues.
Meng: Fewer operations mean less power draw, and that's what really matters when you're designing something that needs to sit in someone's ear all day.
Lalam: So, if you’re building a system where the constraints are this tight, RT-Tango is positioned as a strong contender because it balances the need for high quality with the necessity of low complexity.
Tom: It seems like they didn't just tweak an existing model; they completely rebuilt how those recurrent layers were handled to suit this specific distributed binaural streaming requirement.
Jane: That’s right, it’s a complete revision of the Tango architecture through these architectural and signal-processing optimizations to keep the interaural balance intact while cutting the computational load.
Lu: The online estimation of spatial statistics using that recursive exponential moving average is crucial because it ensures the system stays causal even when you’re running across different nodes.
Meng: I just hope they didn't leave too much to chance in those spatial updates, because if the estimates drift, the whole thing breaks at scale.
Tom: Well, for now, that’s where we land with RT-Tango and its real-time streaming capabilities.
Jane: It’s a solid piece of work that proves you can design for the real-world limitations of assistive listening devices without totally sacrificing performance.
Lu: It gives us a strong blueprint for how to apply these distributed processing techniques to other low-resource audio tasks.
Lalam: The implications are clear: we can start thinking about complex speech enhancement models that actually fit on a chip and run continuously in the field.
Zahra Benslimane, Pierre Chouteau, Martyna Poreba, Fabrice Auzanneau, Michal Szczepanski, Fabian Chersi
Universite Paris-Saclay · Universite de Lorraine
cs.SD
Submitted: 2026-07-02
Updated: 2026-07-02
Comments: Accepted at Interspeech 2026
DOI: 10.21437/Interspeech.2026-3301
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 88/100
The gist: Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication; this paper introduces RT-Tango…
Key concepts
- ERBscaled filterbank
- This technique compresses the input audio features by scaling them based on the Equivalent Received Bandwidth (ERB). Instead of processing every frequency band, it aligns the features with perceptual frequency scaling. This reduces the number of parameters needed for mask estimation, making it much lighter and faster for devices with limited power.
- Lightweight grouped recurrent mask estimation
- This method estimates speech masks by partitioning the feature space into groups. By processing these groups in parallel, the total computational complexity is significantly reduced to O(H^2/G), where G is the group size. This allows for efficient, partitioned processing suitable for distributed binaural setups on hearing aids.
- Temporal sparsification (Fixed-Rate Skipping)
- This strategy reduces the frequency of mask updates during streaming by reusing previously calculated masks. By skipping updates, the system performs inference less frequently—for example, one inference every four frames. This drastically lowers the computational load while maintaining real-time performance.
- Asymmetric STFT configuration
- The system uses a long analysis window (aW) to keep good frequency resolution for accurate sound separation while using a shorter synthesis window (sW) to minimize reconstruction latency. This decoupling allows the system to operate at ultralow latencies, as low as 8 ms, which is critical for real-time hearing aid use.
Terminology
Summary
Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication; this paper introduces RT-Tango, a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids <ref:2607. p. 0 fr
Abstract
Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication,
yet existing efficient solutions predominantly address singlechannel settings. In this paper, we introduce RT-Tango, a
real-time distributed binaural speech enhancement framework
designed for streaming on resource-constrained platforms and
specifically for hearing aids.Where the paper says it
Framework Overview
RT-Tango is a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids <ref:2607.01834#pg2>. It relies on a two-stage distributed architecture combining perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost <ref:2607. p. 0 In this paper, we introduce RT-Tango, a
real-time distributed binaural speech enhancement framework
designed for streaming on resource-constrained platforms and
specifically for hearing aids. RT-Tango relies on a two-stage
distributed architecture combining perceptually motivated ERB
feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational
cost. RT-Tango relies on a two-stage
distributed architecture combining perceptually motivated ERB
feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational
cost. Stringent latency constraints are addressed by decoupling
spectral resolution from algorithmic delay using an asymmetric STFT, together with causal recurrent inference and online
estimation of spatial statistics.Where the paper says it
Where the paper says it
Efficiency Design Choices
The framework employs several efficiency-driven design choices to meet the strict operational requirements of hearing devices <ref:2607.01834#pg2>. These include:
-
Perceptually motivated feature compression using an ERBscaled filterbank, which reduces input dimensionality while preserving speech-relevant cues <ref:2607. p. 0
01834#pg2>.Where the paper says it
This paper addresses this gap by introducing RT-Tango, a binaural SE framework specifically designed for real-time deployment on resource-constrained hearing devices. Concretely, our framework relies on the following efficiency-driven design choices: • Perceptually motivated feature compression using an ERBscaled filterbank, reducing input dimensionality while preserving speech-relevant cues.
-
Lightweight grouped recurrent mask estimation, enabling parallel and partitioned processing to reduce computational cost in distributed binaural settings <ref:2607. p. 0
01834#pg2>.Where the paper says it
Concretely, our framework relies on the following efficiency-driven design choices: • Perceptually motivated feature compression using an ERBscaled filterbank, reducing input dimensionality while preserving speech-relevant cues. • Lightweight grouped recurrent mask estimation, enabling parallel and partitioned processing to reduce computational cost in distributed binaural settings.
-
Decoupled frequency resolution and algorithmic latency reduction through an asymmetric STFT configuration tailored for streaming operations <ref:2607. p. 0
01834#pg2>.Where the paper says it
• Lightweight grouped recurrent mask estimation, enabling parallel and partitioned processing to reduce computational cost in distributed binaural settings. • Decoupled frequency resolution and algorithmic latency reduction through an asymmetric STFT configuration tailored for streaming operations.
-
Temporal sparsification for real-time inference, including both learned skip-RNN gating and fixed-rate frame reuse to reduce recurrent updates <ref:2607. p. 0
01834#pg2>.Where the paper says it
• Temporal sparsification for real-time inference, including both learned skip-RNN gating and fixed-rate frame reuse to reduce recurrent updates. arXiv:2607.
Architectural Enhancements
RT-Tango revisits the Tango framework from a systemlevel perspective, adopting complementary architectural and algorithmic design principles detailed in subsequent sections <ref:2607.01834#pg2>. The baseline Tango uses a Single-Node DNN (SN-DNN) and Multi-Node DNN (MN-DNN) to estimate speech and noise masks independently before refinement <ref:2607. p. 1 Baseline
Tango serves as our baseline due to its two-stage distributed architecture (illustrated in Figure 1) and proven effectiveness in
spatially unconstrained microphone configurations [16]. Each
ear-node independently estimates speech and noise masks using
a Single-Node DNN (SN-DNN) and computes a Speech Distortion Weighted Multichannel Wiener Filter (SDW-MWF) [17]. RT-Tango replaces the recurrent layers in Tango-RNN with
1Online: https://zenodo. org/records/7256984
GRNNs to reduce its computational cost.Where the paper says it
Where the paper says it
Performance Analysis
Experimental results demonstrate that RT-Tango achieves competitive speech enhancement while significantly reducing MACs operations and functioning at ultralow latencies as low as 8 ms <ref:2607.01834#pg2>. Specifically, RT-Tango (ours) operates at 4 ms with a total DNN cost of 33. p. 1 Since the computational
cost of recurrent layers scales approximately quadratically as
O(H2
) with the hidden state dimension H, partitioning the
feature space into G groups of size H/G, the total recurrent
complexity becomes G · O((H/G)
2
) = O(H2
/G), making
it significantly lower compared to a full-band RNN. Although
Out R
SDW-MWF
ERB Features
SN-DNN
GRNN+FC
Inv ERB
STFT
STFT
SDW-MWF
SN-DNN
ERB Features
GRNN+FC
Inv ERB
STFT
STFT
MN-DNN
ERB Features
GRNN+FC
Inv ERB
MN-DNN
ERB Features
GRNN+FC
Inv ERB
Stage 1
Out L Stage 2
Rear R
Front R
Rear L
Front L
SDW-MWF iSTFT iSTFT SDW-MWF
Figure 1: Block diagram of RT-Tango. • Temporal sparsification for real-time inference, including
both learned skip-RNN gating and fixed-rate frame reuse to
reduce recurrent updates. arXiv:2607. 8
ture, where neural networks guide the spatial filter rather than
directly reconstructing the signal. RT-Tango-OS enables strictly causal streaming with an algorithmic latency of 8 ms, through the asymmetric STFT and
online SDW-MWF updates, resulting in a slight but expected
performance degradation relative to offline RT-Tango.Where the paper says it
Where the paper says it
Where the paper says it
Conclusion
This paper introduced RT-Tango, a distributed binaural SE framework designed to satisfy the stringent real-time, latency, and computational constraints of modern hearing devices <ref:2607.01834#pg2>. By revisiting the Tango architecture through complementary architectural and signal-processing optimizations, RT-Tango significantly reduces computational complexity while preserving enhancement performance and interaural balance <ref:2607. p. 3 Conclusions
This paper introduced RT-Tango, a distributed binaural SE
framework designed to satisfy the stringent real-time, latency,
and computational constraints of modern hearing devices. By
revisiting the Tango architecture through complementary architectural and signal-processing optimizations, RT-Tango significantly reduces computational complexity while preserving enhancement performance and interaural balance. By
revisiting the Tango architecture through complementary architectural and signal-processing optimizations, RT-Tango significantly reduces computational complexity while preserving enhancement performance and interaural balance. In addition, RTTango-OS, its strictly causal streaming variant, operates with an
algorithmic latency of 8 ms, makes RT-Tango particularly well
suited for practical deployment in modern binaural hearing aids.Where the paper says it
Where the paper says it
Generative AI Use Disclosure
The authors used AI tools solely for language editing and clarity improvement <ref:2607.01834#pg2>. All scientific ideas, content, and results were developed and verified by the authors <ref:2607. p. 4 Generative AI Use Disclosure
The authors used AI tools solely for language editing and clarity improvement. All scientific ideas, content, and results were
developed and verified by the authors.Where the paper says it
References
[1] K. Tan and D. Wang, “Towards Model Compression for Deep Learning Based Speech Enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. p. 4 References
[1] K. Wang, “Towards Model Compression for Deep
Learning Based Speech Enhancement,” IEEE/ACM Transactions
on Audio, Speech, and Language Processing, vol.Where the paper says it
[2] J.-Y. Wu, C. Yu, S.-W. Fu, C.-T. Liu, S.-Y. Chien, and Y. p. 4 1785–
1794, 2021. Chien, and Y. Chien, and Y. Tsao,
“Increasing Compactness of Deep Learning Based Speech Enhancement Models With Parameter Pruning and Quantization
Techniques,” IEEE Signal Processing Letters, vol.Where the paper says it
Where the paper says it
[3] I. Fedorov, M. p. 4 1887–1891, 2019. Fedorov, M. Fedorov, M. Stamenovic, C. Stamenovic, C. Mandell,
Y. Mandell,
Y. Mattina, and P.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[4] M. Stamenovic, N. p. 4 Fedorov, M. Stamenovic, C. Stamenovic, N. Westhausen, L. Westhausen, L. Jensen, and
A. Jensen, and
A. Pawlicki, “Weight, Block or Unit?Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[5] E. Cohen, H. V. Habi, and A. p. 4 Exploring Sparsity Tradeoffs for Speech Enhancement on Tiny Neural Accelerators,” in
NeurIPS Workshop on Efficient Natural Language and Speech
Processing (NeurIPS ENLSP), 2021. Habi, and A.Where the paper says it
[6] M. Rusci, M. Fariselli, M. p. 4 Netzer, “Towards Fully Quantized Neural Networks For Speech Enhancement,” in Interspeech,
2023, pp. Fariselli, M. Fariselli, M. Paci, and E.Where the paper says it
Where the paper says it
[7] E. Cohen, H. V. Habi, R. Peretz, and A. p. 4 Flamand,
“Accelerating RNN-based Speech Enhancement on a Multi-Core
MCU with Mixed FP16-INT8 Post-Training Quantization,” in
European Conference on Machine Learning and Principles and
Practice of Knowledge Discovery in Databases (ECML PKDD),
2022, pp. Peretz, and A. Peretz, and A. Netzer, “Fully Quantized
Neural Networks for Audio Source Separation,” IEEE Open Journal of Signal Processing, vol.Where the paper says it
Where the paper says it
[8] K. Tan and D. Wang, “Learning Complex Spectral Mapping With Gated Convolutional Recurrent Networks for Monaural Speech Enhancement,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. p. 4 926–933, 2024. Wang, “Learning Complex Spectral Mapping With
Gated Convolutional Recurrent Networks for Monaural Speech
Enhancement,” IEEE/ACM Transactions on Audio, Speech, and
Language Processing, vol.Where the paper says it
[9] Y. Luo, C. Han, and N. p. 4 380–390, 2020. Han, and N.Where the paper says it
[10] Z.-Q. Wang, S. Cornell, S. p. 4 Mesgarani, “Ultra-Lightweight Speech
Separation Via Group Communication,” in IEEE International
Conference on Audio, Speech and Signal Processing (ICASSP),
2021, pp. Cornell, S. Cornell, S. Kim, and S.Where the paper says it
Where the paper says it
[11] X. Rong, T. Sun, X. Zhang, Y. Hu, C. Zhu, and J. p. 4 Watanabe, “FNeural Speech Enhancement with Very Low Algorithmic
Latency and Complexity via Integrated full- and sub-band Modeling,” in IEEE International Conference on Audio, Speech and
Signal Processing (ICASSP), 2023, pp. Zhu, and J.Where the paper says it
[12] A. Pandey and J. p. 4 Lu, “GTCRN: A
Speech Enhancement Model Requiring Ultralow Computational
Resources,” in IEEE International Conference on Audio, Speech
and Signal Processing (ICASSP), 2024, pp. Pandey and J.Where the paper says it
[13] Y. Zhang, H. Pei, W. Wang, and G. p. 4 References
[1] K. Wang, “Towards Model Compression for Deep
Learning Based Speech Enhancement,” IEEE/ACM Transactions
on Audio, Speech, and Language Processing, vol.Where the paper says it
[14] A. Bertrand and M. p. 4 Huang, “LMFCA-Net: A
Lightweight Model for Multi-Channel Speech Enhancement with
Efficient Narrow-Band and Cross-Band Attention,” in IEEE International Conference on Audio, Speech and Signal Processing
(ICASSP), 2025, pp. Bertrand and M. Bertrand and M. Moonen, “Distributed Adaptive NodeSpecific Signal Estimation in Fully Connected Sensor Networks—Part II: Simultaneous and Asynchronous Node Updating,” IEEE Transactions on Signal Processing, vol.Where the paper says it
Where the paper says it
[15] S. Doclo, M. Moonen, T. Van den Bogaert, and J. p. 4 5292–5306, 2010. Van den Bogaert, and J. Van den Bogaert, and J. Wouters,
“Reduced-Bandwidth and Distributed MWF-Based Noise Reduction Algorithms for Binaural Hearing Aids,” IEEE Transactions
on Audio, Speech, and Language Processing, vol.Where the paper says it
Where the paper says it
[16] N. Furnon, R. Serizel, S. p. 0 RT-Tango: Real-Time Distributed Binaural Speech Enhancement for
Low-Power Hearing Aid Devices
Zahra Benslimane ID 1,2
, Pierre Chouteau1
, Martyna Poreba ID 1
, Fabrice Auzanneau ID 1
, Michal
Szczepanski ID 1
, Fabian Chersi ID 1
, Romain Serizel ID 2
1 Universite Paris-Saclay, CEA, List, F-91120 Palaiseau, France ´
2 Universite de Lorraine, CNRS, Inria, LORIA, F-54000 Nancy, France ´
zahra-hafida. benslimane@cea. Serizel, S. Essid, and I. Essid, and I. Illina, “DNN-Based Mask
Estimation for Distributed Speech Enhancement in Spatially Unconstrained Microphone Arrays,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol.Where the paper says it
Where the paper says it
Where the paper says it
[17] S. Doclo, A. Spriet, J. Wouters, and M. p. 4 Van den Bogaert, and J. Wouters,
“Reduced-Bandwidth and Distributed MWF-Based Noise Reduction Algorithms for Binaural Hearing Aids,” IEEE Transactions
on Audio, Speech, and Language Processing, vol. Wouters, and M. Moonen, “Frequencydomain criterion for the speech distortion weighted multichannel
wiener filter for robust noise reduction,” Speech Communication,
vol. Moonen, “Frequencydomain criterion for the speech distortion weighted multichannel
wiener filter for robust noise reduction,” Speech Communication,
vol. 636–656, Jul.Where the paper says it
Where the paper says it
Where the paper says it
[18] F. Gao, L. Wu, L. Zhao, T.-Y. Liu, X.-Cheng and T. p. 4 636–656, Jul. Cheng, and T.Where the paper says it
[19] V. Campos, B. Jou, X. G. i Nieto, J. p. 4 Stent, Eds. i Nieto, J. i Nieto, J. Torres, and S.Where the paper says it
Where the paper says it
[20] Z.-Q. Wang, G. Wichern, S. p. 4 Chang,
“Skip RNN: Learning to Skip State Updates in Recurrent Neural
Networks,” in International Conference on Learning Representations, 2018. Wichern, S. Kim, and S. Watanabe, “FNeural Speech Enhancement with Very Low Algorithmic
Latency and Complexity via Integrated full- and sub-band Modeling,” in IEEE International Conference on Audio, Speech and
Signal Processing (ICASSP), 2023, pp. Watanabe, and J. Le Roux, “Stftdomain neural speech enhancement with very low algorithmic latency,” IEEE/ACM Transactions on Audio, Speech, and Language
Processing, vol.Where the paper says it
Where the paper says it
Where the paper says it
[21] S. U. N. Wood and J. p. 4 397–410, 2023. Wood and J. Wood and J. Rouat, “Unsupervised Low Latency Speech
Enhancement With RT-GCC-NMF,” IEEE Journal of Selected
Topics in Signal Processing, vol.Where the paper says it
Where the paper says it
[22] N.-E. Monir, P. Magron, and R. p. 4 332–346, May 2019. Magron, and R.Where the paper says it
[23] V. Panayotov, G. p. 4 Serizel, “Frequency-Weighted
Training Losses for Phoneme-Level DNN-based Speech Enhancement,” in IEEE International Workshop on Multimedia Signal Processing (MMSP), 2025, pp. Panayotov, G. Panayotov, G. Povey, and S.Where the paper says it
Where the paper says it
[24] L. Delebecque and R. p. 4 5206–5210. Delebecque and R.Where the paper says it
[25] C. Pavlovic, R. p. 4 Serizel, “BinauRec: A dataset to test the influence of the use of room impulse responses on binaural speech
enhancement,” in European Signal Processing Conference (EUSIPCO), 2023, pp. Pavlovic, R. Pavlovic, R. Kassayan, S. Kassayan, S. Prakash, H. Prakash, H. Hohmann,
and A. Hohmann,
and A. Atamaniuk, “A high-fidelity multi-channel portable platform for development of novel algorithms for assistive listening
wearables,” The Journal of the Acoustical Society of America, vol.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
[26] J. L. Roux, S. Wisdom, H. Erdogan, and J. p. 4 2878–2878, 2019. Erdogan, and J. Erdogan, and J. Hershey, “SDR
– Half-baked or Well Done?Where the paper says it
Where the paper says it
[27] A. Rix, J. Beerends, M. p. 4 ” in IEEE International Conference
on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. Beerends, M. Beerends, M. Hollier, and A. Hollier, and A. Hekstra, “Perceptual evaluation of speech quality (PESQ)-a new method for speech quality
assessment of telephone networks and codecs,” in IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP), vol.Where the paper says it
Where the paper says it
Where the paper says it
[28] C. H. Taal, R. C. Hendriks, R. p. 4 2, 2001, pp. Hendriks, R. Hendriks, R. Heusdens, and J.Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Perceptually motivated feature compression using an ERBscaled filterbank reduces input dimensionality while preserving speech-relevant cues by
aligning the features with perceptual frequency scaling.
This allows mask estimators to operate with fewer parameters and operations, lowering computational cost for on-device deployment. -
Lightweight grouped recurrent mask estimation reduces computational complexity by partitioning the feature space into groups of size H/G, making
the total recurrent complexity become G · O((H/G)2) = O(H2 / G), making it significantly lower compared to a full-band RNN.
This enables parallel and partitioned processing within each group for distributed binaural settings. -
Temporal sparsification using Fixed-Rate Skipping (FRS) reduces the mask's update frequency by reusing previously estimated masks, which
reduces the mask’s update frequency and thus the overall computational cost during streaming inference.
This strategy allows forone inference every four frames
in one case, significantly reducing load during real-time streaming. -
Asymmetric STFT configuration decouples spectral resolution from algorithmic delay by using a long analysis window (aW) to preserve frequency resolution while using a shorter synthesis window (sW) to
reduce the reconstruction latency.
This enables operation atultralow latencies as low as 8 ms
for real-time streaming. -
Online estimation of spatial statistics via a recursive exponential moving average (EMA) updates the spatial covariance matrices, which is crucial for maintaining performance under streaming constraints. This allows the system to operate effectively in a causal manner while keeping speech intelligibility competitive with baselines.
Abstract
Real-time binaural speech enhancement is constrained by latency, computational cost, and inter-device communication, yet existing efficient solutions predominantly address single-channel settings. In this paper, we introduce RT-Tango, a real-time distributed binaural speech enhancement framework designed for streaming on resource-constrained platforms and specifically for hearing aids. RT-Tango relies on a two-stage distributed architecture combining perceptually motivated ERB feature compression, lightweight grouped recurrent mask estimation, and temporal sparsification to reduce computational cost. Stringent latency constraints are addressed by decoupling spectral resolution from algorithmic delay using an asymmetric STFT, together with causal recurrent inference and online estimation of spatial statistics. Experimental results show that RT-Tango achieves competitive speech enhancement while significantly reducing MACs operations and functioning at ultra-low latencies as low as 8 ms.
Related papers
- Speech-DRAME: A Framework for Human-Aligned Benchmarks in Speech Role-Play
- MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control
- Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems