Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving
cs.AI
Submitted: 2026-10-05
Updated: 2026-10-05
Comments: Jiaqi Zhao, Haodong Chen, and Jitai Hao contributed equally
License: http://creativecommons.org/licenses/by/4.0/
The gist: LLM agents increasingly execute complex workflows involving multi-turn reasoning, tool use, and parallel agents.
Abstract
LLM agents increasingly execute complex workflows involving multi-turn reasoning, tool use, and parallel agents. Efficient serving requires decisions that span two layers with complementary information: the agent harness understands workflow dependencies, context lifecycles, and execution objectives, whereas the inference engine observes request queues, KV-cache state, resource pressure, and execution capabilities. Existing interfaces do not systematically connect these views, limiting workflow-aware execution. HEAR, a bidirectional Harness--Engine Pairing protocol for agentic LLM serving. HEAR standardizes how the harness communicates workflow intent and execution requirements and how the engine returns runtime state, capabilities, and outcomes. By separating protocol semantics from optimization policies, HEAR supports diverse coordination strategies without changing workflow or model semantics. We instantiate HEAR for online cache-aware runtime coordination and workload-aware execution-mode selection for agent roles. Across four conversational and research-agent benchmarks under memory-constrained, concurrent serving, HEAR achieves a 1.61 times batch speedup and reduces median time-to-first-token by 2.23 times on SCBench. Mooncake shows that workflow intent and live engine state provide complementary benefits across load regimes. On BrowseComp-Plus and DeepResearchBench, workload-specific configurations yield 1.23 times and 2.45 times end-to-end speedups, respectively, without observed task-quality degradation. These results establish HEAR as a reusable coordination substrate for efficient agentic LLM serving.
Sources
- CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
- AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applications
- Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- Qwen3 Technical Report
- Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving
- Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems