Beyond Query Similarity: Safety-Conditioned Cache Routing for LLM Response Reuse
cs.AI
Submitted: 2026-11-07
Updated: 2026-11-07
Terminology
Sources
- Continuous Semantic Caching for Low-Cost LLM Serving
- Closing the Operational Gap in Semantic Caching
- From Exact Hits to Close Enough: Semantic Caching for LLM Embeddings
- Prompt Cache: Modular Attention Reuse for Low-Latency Inference
- RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
- Hydragen: High-Throughput LLM Inference with Shared Prefixes
- LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
- TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text
- StepCache: Step-Level Reuse with Lightweight Verification and Selective Patching for LLM Serving
- Marconi: Prefix Caching for the Era of Hybrid LLMs
- KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows
- GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Caching
- Cortex: Achieving Low-Latency, Cost-Efficient Remote Data Access For LLM via Semantic-Aware Knowledge Caching
- vCache: Verified Semantic Prompt Caching
- Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?
- Asynchronous Verified Semantic Caching for Tiered LLM Architectures
- ContextCache: Context-Aware Semantic Cache for Multi-Turn Queries in Large Language Models
- CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion
- $\tau$-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
- From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents