ARLoop: Fully-Open Pre-trained Looped Transformers with Attention Residuals
cs.CL
Submitted: 2026-11-03
Updated: 2026-11-03
Code: https://github.com/OSU-NLP-Group/ARLoop
Terminology
Sources
- OpenCodeReasoning: Advancing Data Distillation for Competitive Coding
- GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- Program Synthesis with Large Language Models
- Evaluating Large Language Models Trained on Code
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Training Verifiers to Solve Math Word Problems
- Do Language Models Use Their Depth Efficiently?
- Loop the Loopies!
- Gemma 3 Technical Report
- Measuring Mathematical Problem Solving With the MATH Dataset
- Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
- Attention Residuals
- Adam: A Method for Stochastic Optimization
- Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it Deserves
- Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
- Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model
- Solving Quantitative Reasoning Problems with Language Models
- DeepLoop: Depth Scaling for Looped Transformers
- Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data