Why Knowing Both Hops Is Not Enough: Understanding Two-Hop Generalization in Language Models
cs.CL
Submitted: 2026-08-07
Updated: 2026-08-28
Comments: 24 pages, 20 figures
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
- Looped Transformers for Length Generalization
- Evolution of SAE Features Across Layers in LLMs
- Taken out of context: On measuring situational awareness in LLMs
- Toy Models of Superposition
- Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
- How Do LLMs Perform Two-Hop Reasoning in Context?
- Towards Reasoning in Large Language Models: A Survey
- VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers
- How Do Language Models Compose Functions?
- Improve Mathematical Reasoning in Language Models by Automated Process Supervision
- Looped Transformers are Better at Learning Learning Algorithms
- Polysemanticity and Capacity in Neural Networks
- Do Large Language Models Latently Perform Multi-Hop Reasoning?
- Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoning
- How do Transformers Learn Implicit Reasoning?
- Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Models
- A Survey of Large Language Models
- Scaling Latent Reasoning via Looped Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering