Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning".
Jane: Here's the gist of the paper. In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective This paper proposes ClariCodec, a neural speech codec operating at 300 bits per second (bps) that reformulates quantisation as a stochastic policy, enabling reinforcement learning (RL)-based optimisation of intelligibility.
Tom: Core Problem and Motivation The objective of speech coding shifts from preserving waveform fidelity to ensuring speech intelligibility in environments where transmission capacity is restricted to only a few hundred bits per second (bps).
Jane: This constraint motivates the development of ultra-low-bitrate coding strategies that prioritise semantic clarity while reducing the need to reconstruct fine-grained acoustic details.
Tom: Conventional reconstruction objectives, including mel-spectrogram L1 losses and adversarial waveform losses, do not enforce this property because they prioritise acoustic similarity over the information most critical for linguistic decoding.
Paper summary: Tom: So, to recap, this paper "Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning" proposes ClariCodec <ref:2605.19541#pg1,Optimising Neural Speech Codecs for>. The core thesis is that in super low bitrates, speech coding should stop focusing on perfect acoustic fidelity and start focusing squarely on intelligibility.
Jane: They claim they created a neural speech codec operating at three hundred bits per second, and the key mechanism they use is reformulating the quantization process as a stochastic policy so they can use reinforcement learning to optimize for intelligibility directly.
Lu: The paper outlines a two-stage training strategy: Stage one uses reconstruction losses, like L1 mel loss and adversarial loss, with an improved FSQ to get a stable discrete representation. Then, in Stage two, the encoder is fine-tuned using reinforcement learning where the reward comes from a pretrained ASR model defined by negative word error rate.
Meng: So they’re essentially freezing the acoustic generation part and only tuning how those bits are quantized based on what an ASR model predicts will be useful for understanding speech. That sounds like a very targeted approach to compression.
Tom: Exactly, they are explicitly optimizing against the WER reward within that frozen pipeline using group relative policy optimisation, or GRPO. They anchor this process with an L1 mel reconstruction loss during the RL stage to keep the audio quality from collapsing too much.
Jane: That’s interesting because it addresses a major problem: conventional methods prioritize acoustic similarity, which leads to bit allocation to perceptual details that don't help decoding, and this paper argues we should prioritize what is needed for linguistic decoding.
Lu: The methodology also tackles the streaming constraint by proposing a variant using causal modules and limited latent look-ahead for low latency operation at three hundred bits per second.
Meng: That streaming variant still managed to achieve a word error rate of four point five three percent, which shows that this RL approach holds up even when you have those tight temporal constraints are applied.
Tom: And they confirmed that the reward signal is robust, meaning it works well whether you use different pretrained recognizers or if you switch the objective to character error rate or phoneme error rate.
Jane: It sounds like they’ve built a system that balances competing goals: achieving ultra-low bitrates while ensuring high intelligibility, and they’ve done this by using reinforcement learning as the optimization engine to navigate that trade-off.
Lu: The paper is essentially showing how to use RL not just for one task, but as a general tool to find representations that are maximally informative for speech recovery under extreme compression.
Meng: For deployment, I’m still wondering about the operational cost—how much does this whole training process actually take compared to what we could achieve with simpler methods in terms of time and resources.
Conclusion: Tom: So, wrapping up "Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning", the authors are Wang, Zhang, Qian, Luo, Wang, Jin, and Zhang from Tsinghua University and Huawei Technologies <ref:2605.19541#pg1,Optimising Neural Speech Codecs for>. The core finding is that ClariCodec achieves a word error rate of three point five five percent on LibriSpeech test-clean.
Jane: That result is significant because it’s not just about the number; it’s about demonstrating a way to get much clearer speech delivered over those extremely limited bandwidth channels without having to drastically increase the transmission rate.
Lu: It shows that this approach works by reformulating quantization as a stochastic policy, allowing reinforcement learning to directly optimize for what matters linguistically under extreme compression.
Meng: It gives us a concrete tool for handling the trade-off between audio quality and intelligibility when we're talking about bandwidth limitations in real-time systems.
Tom: Ultimately, this work suggests that we can maintain competitive acoustic quality while maintaining high intelligibility even when the communication channel is severely restricted, which is what matters for real-world applications.
Tsinghua University · Huawei Technologies Co. Ltd
cs.SD
Submitted: 2026-05-19
Updated: 2026-11-12
Comments: Accepted at the 2026 IEEE Spoken Language Technology Workshop (SLT 2026). Code and pretrained checkpoints: https://github.com/demo941/ClariCodec
Code: https://github.com/demo941/ClariCodec
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 96/100
The gist: In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective This paper…
Key concepts
- Information Bottleneck Principle
- This principle suggests that an effective compression system must learn representations that remove acoustically redundant details while keeping only the minimal information required for speech to be understood. The codec's goal is to find this crucial, non-redundant information for intelligibility.
- Stochastic Policy Quantization
- Instead of using fixed quantization rules, ClariCodec treats the process of quantizing speech representations as a stochastic policy that can be optimized using reinforcement learning. This allows the system to learn the best way to quantize data based on an external reward signal, directly targeting intelligibility.
- Group Relative Policy Optimisation (GRPO)
- GRPO is the specific reinforcement learning framework used to fine-tune the stochastic quantizer in ClariCodec. It calculates an advantage for each sample by comparing its reward to the average reward across a group of samples, helping the policy learn which quantization actions lead to better intelligibility.
- Word Error Rate (WER) Reward
- The primary objective function used during RL training is the negative Word Error Rate (WER) calculated by a pretrained Automatic Speech Recognition (ASR) model. By minimizing this reward, the codec learns representations that are most useful for accurate linguistic decoding, ensuring the transmitted speech is intelligible.
Terminology
Summary
In bandwidth-constrained communication such as satellite and underwater channels, speech must often be transmitted at ultra-low bitrates where intelligibility is the primary objective This paper proposes ClariCodec, a neural speech codec operating at 300 bits per second (bps) that reformulates quantisation as a stochastic policy, enabling reinforcement learning (RL)-based optimisation of intelligibility. p. 0
Core Problem and Motivation
The objective of speech coding shifts from preserving waveform fidelity to ensuring speech intelligibility in environments where transmission capacity is restricted to only a few hundred bits per second (bps). This constraint motivates the development of ultra-low-bitrate coding strategies that prioritise semantic clarity while reducing the need to reconstruct fine-grained acoustic details. p. 0 Conventional reconstruction objectives, including mel-spectrogram L1 losses and adversarial waveform losses, do not enforce this property because they prioritise acoustic similarity over the information most critical for linguistic decoding. p. 0 The challenge can be interpreted through the lens of the information bottleneck principle where an effective codec must learn representations that remove acoustically redundant detail while retaining the minimal information necessary for intelligible speech recovery. p. 0
Model Architecture and Training Strategy
ClariCodec utilizes a two-stage training strategy to achieve semantic alignment under extreme compression.
-
Stage 1 involves an initial reconstruction-based pre-training phase employing improved FSQ to establish a stable discrete representation. The model operates on log-mel spectrograms extracted with a hop size of 160 samples (10 ms) using a ConvNeXt V2-based encoder. p. 1
-
Stage 2 introduces RL fine-tuning where the encoder is modelled as a stochastic policy πθ over quantisation actions, reformulating discrete quantisation as stochastic sampling via Gumbel-Softmax. This formulation enables direct optimisation against non-differentiable WER reward within a frozen acoustic pipeline. p. 1
RL Optimisation Mechanism
The RL fine-tuning stage uses the group relative policy optimisation (GRPO) framework to optimise the stochastic quantizer. The reward signal is derived from a pretrained ASR model, where the reward is defined as negative Word Error Rate (WER). p. 2 The group-relative advantage for the i-th sample is computed as Aˆi = Ri − mean(Ri) stddev(Ri). p. 3 To anchor the policy to original acoustic characteristics, a mel spectrogram reconstruction loss is incorporated into the total loss function. p. 3
Performance and Generalisation Gains
ClariCodec achieves 3.55% WER on LibriSpeech test-clean after RL fine-tuning, representing a 23. p. 15% relative reduction compared to its non-RL counterpart. p. 4 The streaming variant achieves a WER of 4. p. 6 p. 653% with a theoretical latency of 374 ms, demonstrating effectiveness under restricted temporal context. p. 1 Furthermore, RL optimisation reduces WER from 13. p. 53% to 10.4% on LibriSpeech test-other, showing that the benefits of WER-guided optimisation generalise beyond clean speech. p. 5 The results indicate that the proposed representation generalises effectively across domains and that the benefits of WER-guided optimisation extend beyond the training distribution. p. 5
Acoustic Quality Preservation
The combination of RL fine-tuning with a mel spectrogram reconstruction constraint proves beneficial for maintaining acoustic quality. Applying RL fine-tuning without any acoustic constraint yields a substantial reduction in WER—from 4. p. 464 to 3.54 (23.7% relative)—confirming that the optimisation effectively steers the model towards more intelligible outputs. p. 4 Introducing the mel reconstruction loss as an acoustic regulariser largely recovers this degradation, raising PESQ from 1. p. 483 to 1.87, while preserving nearly all of the intelligibility benefit. p. 4 The final results show that subjectively perceived quality improves from 3. p. 498 to 4.06 after RL fine-tuning, indicating that this gain does not come at the cost of perceived quality. p. 3
Streaming Operation Analysis
The streaming variant is designed for low-latency operation, using causal convolutions and a finite-look-ahead convolution to manage context. The theoretical latency for this configuration is 374 ms. p. 0 After Stage 2 RL fine-tuning, the streaming variant reduces WER from 6. p. 163% to 4.53%, corresponding to a 31. p. 07% relative reduction, while STOI, PESQ, and SIM remain unchanged. p. 4 These results indicate that the proposed RL-based optimisation strategy is not limited to the offline setting; it also remains effective under streaming constraints where the codec must operate with restricted temporal context and low latency. p. 4
Reward Robustness Assessment
The study assesses the robustness of the Stage 2 reward design from two perspectives: the recogniser used to compute the reward and the granularity of the error-rate objective. When WER is used as the reward, replacing it with different pretrained recognisers still yields consistent WER reductions over the Stage 1 baseline, indicating that the learned policy is driven by general intelligibility cues rather than by overfitting to a single ASR model. p. 5 Comparing WER setting with character error rate (CER) and phoneme error rate (PER) rewards shows that all three formulations improve intelligibility over the Stage 1 codec, with PER yielding the lowest value in this experiment. p. 5 These results suggest that the proposed RL framework is not highly sensitive to either the reward recogniser or the granularity of the error-rate objective. p. 5
Conclusion and Future Work
ClariCodec achieves a WER of 3.55% on LibriSpeech test-clean, outperforming baseline models operating at substantially higher bitrates. p. 6 The streaming variant achieves a WER of 4.53% with a theoretical latency of 374 ms, demonstrating that the proposed optimisation remains effective under streaming constraints. p. 6 Future work will further evaluate the effects of operating at such low bitrates on downstream generative tasks, including speech synthesis and codec-based speech large language models. p. 6 The paper concludes that ClariCodec maintains competitive acoustic quality and intelligibility under extreme compression. p. 1
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Reinforcement learning based optimisation of intelligibility: The system can be fine-tuned using
WER-driven rewards
during Stage 2 training to directly optimise for speech intelligibility, achieving a23.5% relative reduction while preserving perceptual quality.
-
Stochastic policy reformulation: Quantisation is reformulated as a stochastic policy where the quantiser acts as a trainable policy pi theta, enabling direct optimisation against non-differentiable WER reward within a frozen acoustic pipeline.
-
Streaming operation capability: The codec can be adapted to streaming with causal convolutions and limited latent look-ahead, achieving
4.53% WER on test-clean with a theoretical latency of 374 ms.
-
Acoustic constraint preservation: The combination of RL fine-tuning and a mel spectrogram reconstruction loss ensures that the optimisation
does not yield a substantial reduction in WER—from 4.64 to 3.54 (23.7% relative)—confirming that the optimisation effectively steers the model towards more intelligible outputs.
-
Robust reward design: The RL framework is shown to be
not tightly coupled to a particular recogniser, and that the learned policy is driven by general intelligibility cues rather than by overfitting to the idiosyncrasies of a single ASR model,
as it yields consistent WER reductions across different recognition models. -
Out-of-domain generalisation: The system demonstrates effective cross-domain performance on LJSpeech, where
RL optimisation reduces WER from 4.44% to 3.82% (14.0% relative) while maintaining nearly unchanged acoustic quality metrics.
Sources
- Low-Resource Audio Codec (LRAC): 2025 Challenge Description
- HiFi-Codec: Group-residual Vector quantization for High Fidelity Audio Codec
- BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec
- PSCodec: A Series of High-Fidelity Low-bitrate Neural Speech Codecs Leveraging Prompt Encoders
- Moshi: a speech-text foundation model for real-time dialogue
- High-Fidelity Generative Audio Compression at 0.275kbps
- Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis
- SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization
- The information bottleneck method
- iFSQ: Improving FSQ for Image Generation with 1 Line of Code
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Related papers
- Speech-DRAME: A Framework for Human-Aligned Benchmarks in Speech Role-Play
- MuLaCover: Flexible Cover Song Generation via Symbolic Tonal Control
- Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement
- Spectral Masking and Interpolation Attack (SMIA): A Black-box Adversarial Attack against Voice Authentication and Anti-Spoofing Systems
- Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems
- Harmonic Representations in Symbolic Music Transformers are Localized and Controllable