CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment

arXiv:2602.19574 · eess.AS, cs.AI, cs.SD · Submitted 2026-02-23 · Read on arXiv

eess.AS, cs.AI, cs.SD

Submitted: 2026-02-23

Updated: 2026-09-07

Comments: Fix two typos in Figure 2 in INTERSPEECH 2026 version

Project page: https://ctctts.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Alignment — 51× in this paper · explained in 1 episode
• Embeddings — 6× in this paper · explained in 2 episodes
• Transformer — 5× in this paper · explained in 2 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Association for Computational Linguistics — 2× in this paper
• Connectionist Temporal Classification — 2× in this paper
• Generative AI — 2× in this paper · explained in 2 episodes
• International Conference on Learning Representations — 2× in this paper
• International Conference on Machine Learning — 2× in this paper
• Ablation Studies — 1× in this paper · explained in 1 episode
• Character Error Rate — 1× in this paper
• Core — 1× in this paper · explained in 2 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Large Language Model (LLM) — 1× in this paper · explained in 3 episodes
• Mean Opinion Score — 1× in this paper
• Regularization — 1× in this paper · explained in 3 episodes
• Word Error Rate — 1× in this paper

Sources

• SyncSpeech: Efficient and Low-Latency Text-to-Speech based on Temporal Masked Transformer
• Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
• CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens
• CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models
• Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming
• Interleaved Speech-Text Language Models for Simple Streaming Text-to-Speech Synthesis
• SpeakStream: Streaming Text-to-Speech with Interleaved Data

Related papers

• Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
• X-VC: Zero-shot Streaming Voice Conversion in Codec Space
• Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers
• VM-ARRAYDPS: Virtual Microphone Augmented Diffusion Posterior Sampling for Unsupervised Blind Speech Separation
• Conditional Flow Matching for Visually-Guided Acoustic Highlighting
• Anonymization, Not Elimination: Utility-Preserved Speech Anonymization