The Optimization Landscape of Learning Compacted Context Models
cs.LG
Submitted: 2026-10-05
Updated: 2026-10-05
Sources
- LoCoCo: Dropping In Convolutions for Long Context Compression
- Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
- KV-Distill: Nearly Lossless Learnable Context Compression for LLMs
- Cartridges: Lightweight and general-purpose long context representations via self-study
- In-context Autoencoder for Context Compression in a Large Language Model
- KVSculpt: KV Cache Compression as Distillation
- delta-mem: Efficient Online Memory for Large Language Models
- End-to-End Context Compression at Scale
- Still: Amortized KV Cache Compaction in a Single Forward Pass
- Kwai Summary Attention Technical Report
- Efficient Streaming Language Models with Attention Sinks
- Fast KV Compaction via Attention Matching
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks