Separators Make Carry Propagation Learnable:The Geometry of Latent Carry in a Multiplication Transformer
cs.LG
Submitted: 2026-10-05
Updated: 2026-10-05
Terminology
Sources
- Refusal in Language Models Is Mediated by a Single Direction
- Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls
- Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers
- Not All Language Model Features Are One-Dimensionally Linear
- Think before you speak: Training Language Models With Pause Tokens
- Language Models Represent Space and Time
- Language Models Use Trigonometry to Do Addition
- Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers
- The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
- Progress measures for grokking via mechanistic interpretability
- Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics
- The Geometry of Categorical and Hierarchical Concepts in Large Language Models
- Let's Think Dot by Dot: Hidden Computation in Transformer Language Models
- Dissecting Multiplication in Transformers: Insights into LLMs
- Understanding Addition in Transformers
- Understanding Addition and Subtraction in Transformers
- Analysing Mathematical Reasoning Abilities of Neural Models
- Transformers represent belief state geometry in their residual stream
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks