Coordinated Mixed-Precision KV Cache Quantization for Efficient LLM-based Multi-agent Systems
cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Terminology
Sources
- TokenCake: A KV-Cache-centric Serving Framework for LLM-based Multi-Agent Applications
- A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application
- QAQ: Quality Adaptive Quantization for LLM KV Cache
- Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks
- AgentScope: A Flexible yet Robust Multi-Agent Platform
- QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMs
- GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
- A Survey on Large Language Model Acceleration based on KV Cache Management
- The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey
- PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
- TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
- Gemma 4 Technical Report
- Qwen3 Technical Report
- LLM-based Multi-Agent Systems: Techniques and Business Perspectives
- Verified Multi-Agent Orchestration: A Plan-Execute-Verify-Replan Framework for Complex Query Resolution
- Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Use
- RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis