The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA
cs.LG
Submitted: 2026-09-02
Updated: 2026-09-02
Comments: Accepted at the ICML 2026 Mechanistic Interpretability Workshop, https://openreview.net/forum?id=Spof4PusVI. 9 pages. Corrects a data-entry error in the workshop version: the seed-1337 accuracy in the three-seed replication was reported as 80.47% (a control run); the archived value is 78.91%, so the three-seed mean is 81.0 +/- 2.0pp (was 81.5 +/- 1.2pp). All other results are unchanged
License: http://creativecommons.org/licenses/by/4.0/
The gist: Continuous chain-of-thought models compress reasoning into latent tokens.
Terminology
Abstract
Continuous chain-of-thought models compress reasoning into latent tokens. Matrix-valued variants, which route each latent token through a d x d matrix bottleneck, introduce rank as a single-sample structural observable on the latent matrix Z. If matrix latents carry parallel reasoning paths via superposition, rank should track them, and truncating Z to low rank should hurt accuracy on tasks whose solutions plausibly require multiple components. Across four training regimes of a matrix-CODI model (three on ProsQA, one on GSM8K-Aug below the learning threshold), the rank-k projection ablation curve is flat to within 0.6 percentage points. A three-seed replication yields 81.0 +/- 2.0 percentage points accuracy while the final effective rank of Z spans 4, 12, 13; the loss does not reward any particular rank. To test whether rank-blindness arises from the flatten-then-project readout alone, we trained four readouts: a bilinear reparametrization, a bilinear-plus-GELU readout nonlinear in Z, an SVD-augmented readout feeding singular values through an MLP, and a quadratic readout in Z Z T. All four rank-k curves remain flat (Spearman p-values 0.63, 0.14, 0.82, 0.46). The flat curves persist for readouts nonlinear in Z. A linear probe on Z underperforms a raw pretrained hidden state at target prediction (AUC 0.673 vs. 0.846). A negative control on vanilla GPT-2 SFT (no matrix bottleneck, no Z, three seeds, n=500) reproduces a flat rank-k curve under the same intervention paradigm with pooled-mean range 0.20pp, and a random-h sensitivity floor lands at the same accuracy: the rank-k ablation alone conflates rank-blindness with position-irrelevance.
Sources
- Implicit Regularization in Deep Learning May Not Be Explainable by Norms
- The Illusion of Superposition? A Principled Analysis of Latent Thinking in Language Models
- Implicit Regularization in Deep Matrix Factorization
- CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation
- LLM Latent Reasoning as Chain of Superposition
- SIM-CoT: Supervised Implicit Chain-of-Thought
- Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
- State Rank Dynamics in Linear Attention LLMs
- Continuous Chain of Thought Enables Parallel Exploration and Reasoning
- Addressing Token Uniformity in Transformers via Singular Value Transformation
- Implicit Regularization in Matrix Factorization
- Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
- Training Large Language Models to Reason in a Continuous Latent Space
- Weight decay induces low-rank attention layers
- Optimal ablation for interpretability
- Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure
- Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer
- On State Reduction in Linear Attention
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks