SLT 2026 REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational Recordings

summary

The gist

Real-world Target Speaker Extraction from Conversational Recordings introduces the REAL-TSE Challenge to evaluate systems on authentic conversational data under real-world conditions.

In short

The REAL-TSE Challenge introduced real-world Target Speaker Extraction from Conversational Recordings to test systems on authentic Mandarin and English audio with natural overlaps, noise, and dynamics. It features two tracks: online for low latency and offline for full context. Evaluation uses four metrics—TER, SpkSim, DNSMOS, and activity F1—revealing that success requires multi-objective optimization across data preparation and model design.

Key concepts

Online Track: This track is designed for applications needing fast results, demanding an end-to-end algorithmic latency under 100 milliseconds. It focuses on low-latency streaming extraction of the target speaker's speech in real-time scenarios.
Offline Track: This track is for batch processing where systems can utilize the entire utterance and global context without strict constraints on inference cost. It allows for more comprehensive, full-context processing of the audio.
Token Error Rate (TER): TER measures intelligibility by counting errors. For English, it uses word error rate, while for Mandarin, it uses character error rate to assess how accurately the extracted speech is represented.
Speaker Similarity (SpkSim): This metric quantifies speaker consistency by calculating the cosine similarity between the embeddings of the recovered target speech ($Ψs$) and the original enrollment utterance ($e$). It checks if the extracted voice sounds like the target speaker.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 26 episodes
Flow Matching — explained in 23 episodes
Quantum Fisher Information (QFI) — explained in 22 episodes
Retrieval-Augmented Generation (RAG) — explained in 21 episodes
Sample Complexity — explained in 20 episodes
Federated Learning — explained in 19 episodes

This episode discusses

The paper

SLT 2026 REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational Recordings · Read on arXiv

Shuai Wang, Zihan Qian, Ke Zhang, Jiangyu Han, Zikai Liu, Xiaoyang Yu, Haoyu Li, Marc Delcroix, Kai Yu, Lei Xie, 一大 Li

Nanjing University, China · Chinese University of Hong Kong (Shenzhen), China · Brno University of Technology, Czechia · Northwestern Polytechnical University, China · NTT, Inc., Japan · Shanghai Jiao Tong University

We introduce the REAL-TSE Challenge, an IEEE SLT 2026 satellite challenge on target speaker extraction (TSE) from real conversational recordings. Given a multi-speaker mixture and one or more enrollment utterances from a target speaker, participating systems must recover only the target speech. Unlike simulated read-speech benchmarks, REAL-TSE evaluates Mandarin and English recordings that contain natural overlap, reverberation, noise, channel mismatch, and conversational dynamics. The challenge defines two complementary tracks: an Online track for low-latency streaming extraction and an Offline track for full-context processing. Systems are evaluated with Token Error Rate (TER), Speaker Similarity (SpkSim), DNSMOS, and target-speaker activity F1. This overview paper describes the task definition, datasets, baselines, evaluation protocol, submitted systems, condition-wise findings, and lessons for future real-world TSE benchmarks.

Transcript

Introduction to the show: ident: Electrical Engineering. The day in electrical engineering: signals, systems and the hardware that carries them.

Rosa: I'm Rosa.

Dev: And I'm Dev, here with Taro.

Rosa: We're looking at the paper: "SLT 2026 REAL-TSE Challenge".

Dev: Here's the gist of the paper. Real-world Target Speaker Extraction from Conversational Recordings introduces the REAL-TSE Challenge to evaluate systems on authentic conversational data under real-world conditions.

Rosa: The challenge addresses a gap in existing benchmarks by evaluating Mandarin and English recordings containing natural overlap, reverberation, noise, channel mismatch, and conversational dynamics.

Dev: Challenge Design and Tracks The REAL-TSE Challenge defines two complementary tracks to assess system performance under different constraints.

Paper summary: Dev: Looking at this paper called "SLT two thousand twenty-six REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational Recordings," the challenge itself is significant because it moves away from simulated read-speech data and puts systems in a much harder, real world situation <ref:2607.15198#pg1,SLT 2026 REAL-TSE Challenge: Real-world Target Speaker Extraction from Conversational>.

Rosa: The authors are trying to bridge that gap by evaluating performance on authentic Mandarin and English recordings that have all the typical problems we hear in life: overlap, reverberation, noise, channel mismatch, and conversational dynamics.

Taro: They set up those two tracks—the online track for low-latency streaming and the offline track for full context processing—which shows they are thinking about how these tools will actually be used in different applications.

Dev: The challenge is structured around those four metrics we talked about earlier: Token Error Rate, Speaker Similarity, DNSMOS, and Target-Speaker Activity F1. This multi-objective design is what makes the evaluation comprehensive, but as they found, it also shows where things can get tricky with metric overoptimization.

Rosa: The title itself points to the goal: real world target speaker extraction. It’s about building a system that can reliably pull out just one person's speech from a chaotic environment.

Taro: The implication here is that for future work, we need to look at the difficulty at the sample, interaction, and channel level instead of just relying on broad scenario labels to understand why things fail.

Dev: And they specifically pointed out that transparent reporting of metric usage and post-processing is necessary because you can over-optimize a metric like DNSMOS without getting corresponding gains in the actual perceptual score it's supposed to measure.

Rosa: So what this means for us is that we need systems to simultaneously preserve the linguistic content, accurately detect when the target speaker is active, maintain that speaker identity, and produce speech that sounds natural despite all the real-world mess.

Taro: The open-training policy they have also encourages creative solutions because it pushes participants to find practical ways to adapt models for these complex scenarios.

Dev: So in short, this paper establishes a new standard by grounding evaluation in authentic data and forcing systems to be robust enough for actual conversational use cases.

Rosa: It’s a step toward making TSE systems that are not just good in the lab but actually work reliably when you're driving or cooking or walking.

Conclusion: Rosa: So, we just finished looking at the REAL-TSE Challenge paper on extracting target speakers from real conversations.

Dev: Yeah, it’s about testing these systems under messy conditions that aren't just clean lab data anymore.

Rosa: The authors are really pushing for this to be a test of how robust a system is when things get complicated in the real world.

Taro: They're using Mandarin and English recordings that have all the usual junk—overlap, noise, bad channels—which is way more realistic than stuff you see in textbooks.

Dev: And they set up these two tracks, online for low latency streaming and offline for full context processing to see how systems handle different demands.

Rosa: The whole point seems to be moving past simulated read-speech benchmarks and seeing if the tech actually holds up when it hits the actual world.

Taro: It’s about seeing if a system can still do its job well even when the recording quality is really poor or there's a lot of crosstalk.

Dev: The challenge collects twenty-four systems from eighteen teams, which gives us a good spread to see where the current state of the art sits across all these metrics.

Rosa: What this means for us is that we need to start thinking about how these extraction tools will actually perform when you’re not in a perfect studio.

Taro: It pushes the community to design models that can handle those real-world failure modes gracefully, instead of just working on clean data.

Dev: We'll be looking at the specific results and where systems are excelling or failing across those four main metrics we talked about.

More episodes

← Home

Watch the video for this episode