CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

arXiv:2607.05378 · cs.LG · Submitted 2026-07-06 · Read on arXiv

cs.LG

Submitted: 2026-07-06

Updated: 2026-10-01

Code: https://github.com/krafton-ai/kira

Terminology

• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Agentic Coding — 1× in this paper · explained in 1 episode

Sources

• Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs
• Longformer: The Long-Document Transformer
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
• TreeRL: LLM Reinforcement Learning with On-Policy Tree Search
• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling
• Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
• RULER: What's the Real Context Size of Your Long-Context Language Models?
• LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
• LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
• Scaling LLM Multi-turn RL with End-to-end Summarization-based Context Management
• WebGPT: Browser-assisted question-answering with human feedback
• MemGPT: Towards LLMs as Operating Systems
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• Proximal Policy Optimization Algorithms
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Scaling Long-Horizon LLM Agent via Context-Folding
• Solving math word problems with process- and outcome-based feedback
• SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks