Limit Analysis for Symbolic Multi-step Reasoning Tasks with Information Propagation Rules Based on Transformers
cs.AI
Submitted: 2025-09-27
Updated: 2026-11-06
Terminology
Sources
- Round and Round We Go! What makes Rotary Positional Encodings useful?
- Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
- What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks
- The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains
- Localizing Model Behavior with Path Patching
- II-MMR: Identifying and Improving Multi-modal Multi-hop Reasoning in Visual Question Answering
- Making Long-Context Language Models Better Multi-Hop Reasoners
- Understanding and Patching Compositional Reasoning in LLMs
- Generating Wikipedia by Summarizing Long Sequences
- Transformers Can Do Bayesian Inference
- How Transformers Learn Causal Structure with Gradient Descent
- In-context Learning and Induction Heads
- Transformers, parallel computation, and logarithmic depth
- Distributional reasoning in LLMs: Parallel reasoning processes in multi-hop reasoning
- Function Vectors in Large Language Models
- LLaMA: Open and Efficient Foundation Language Models
- Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
- Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism
- GPT-4 Technical Report
- Learning Compositional Functions with Transformers from Easy-to-Hard Data
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents