A model of errors in transformers
cs.LG, cs.AI, cs.CL, hep-th
Submitted: 2026-01-20
Updated: 2026-11-12
Terminology
Sources
- Layer Normalization
- Evaluating Large Language Models Trained on Code
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It
- Theoretical limitations of multi-layer Transformer
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Training Large Language Models to Reason in a Continuous Latent Space
- Constant Bit-size Transformers Are Turing Complete
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Softmax Transformers are Turing-Complete
- A Survey on Large Language Models for Code Generation
- A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers
- Efficient Estimation of Word Representations in Vector Space
- Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics
- Effective Field Theory and the Fermi Surface
- Positional Description Matters for Transformers Arithmetic
- Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers
- If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents
- On Limitations of the Transformer Architecture
- Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis