Semantic Caches for LLM Web Services Leave Their Error Budget Unspent
cs.LG
Submitted: 2026-11-02
Updated: 2026-11-02
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- From Exact Hits to Close Enough: Semantic Caching for LLM Embeddings
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- An Ensemble Embedding Approach for Improving Semantic Caching Performance in LLM-based Systems
- Rethinking Caching for LLM Serving Systems: Beyond Traditional Heuristics
- Towards General Text Embeddings with Multi-stage Contrastive Learning
- vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models
- The Llama 3 Herd of Models
- Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs
- GPT-4o System Card
- Qwen3 Technical Report
- GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Caching
- Category-Aware Semantic Caching for Heterogeneous LLM Workloads
- Text Embeddings by Weakly-Supervised Contrastive Pre-training
- Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
- Similarity Is Not Validity: Defending LLM Semantic Caches Against Poisoning
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning