Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI
cs.AI, cs.CL
Submitted: 2026-08-26
Updated: 2026-08-26
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks.
Terminology
Abstract
Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks. However, many existing approaches rely on fixed or preallocated reasoning controls, such as fixed token budgets, pre-execution difficulty estimates, or activation-space interventions, and are often evaluated on standalone reasoning benchmarks rather than full agentic workflows. These assumptions may not hold in agentic AI systems, where reasoning requirements evolve dynamically through planning, tool use, memory retrieval, and agent-to-agent interactions. Consequently, reasoning can become either excessive or insufficient, resulting in unnecessary computation, increased latency, planning drift, excessive tool use, or incomplete solutions. We argue that a major challenge for next-generation agentic AI is not merely how much reasoning a language model should perform, but how it should allocate reasoning according to evolving task demands. We characterize over-reasoning and under-reasoning as recurring failure modes of misallocated reasoning and evaluate them on MATH-500 and the GAIA public validation benchmark. Using tool-decision latency, token consumption, token-limit exhaustion, and answer correctness, our results suggest that cases classified as over-reasoning are associated with higher computational cost without proportional accuracy gains, whereas cases classified as under-reasoning are consistently associated with incorrect or incomplete solutions. These findings motivate future research on adaptive reasoning mechanisms for agentic AI.
Sources
- Between Underthinking and Overthinking: An Empirical Study of Reasoning Length and correctness in LLMs
- Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
- Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
- Efficient Agents: Building Effective Agents While Reducing Cost
- ReAct: Synergizing Reasoning and Acting in Language Models
- When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
- Exploring the Necessity of Reasoning in LLM-based Agent Scenarios
- Phi-4-reasoning Technical Report
- OptimalThinkingBench: Evaluating Over and Underthinking in LLMs
- Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directions
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems