Linear Probes to Bayesian Manifolds: Geometry of Arithmetic in Large Language Models
cs.LG, cs.CL
Submitted: 2026-11-08
Updated: 2026-11-08
Code: https://github.com/kingoflolz/mesh-transformer-jax
Terminology
Sources
- Understanding intermediate layers using linear classifier probes
- Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls
- Probing Classifiers: Promises, Shortcomings, and Advances
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- LEACE: Perfect linear concept erasure in closed form
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Shrinkage Algorithms for MMSE Covariance Estimation
- Towards Automated Circuit Discovery for Mechanistic Interpretability
- Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
- Toy Models of Superposition
- Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts
- Convergent Evolution: How Different Language Models Learn Similar Number Representations
- Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability
- The Llama 3 Herd of Models
- Language Models Represent Space and Time
- Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)
- How to use and interpret activation patching
- Designing and Interpreting Probes with Control Tasks
- Causality is Key for Interpretability Claims to Generalise
- Language Models Use Trigonometry to Do Addition
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis