KVPath: Connecting Key-Value Representations Across Transformer Layers
cs.LG
Submitted: 2026-11-11
Updated: 2026-11-11
Code: https://github.com/luoxuan-cs/KVPathhttps:
Terminology
Sources
- DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
- Reconstructing KV Caches with Cross-layer Fusion For Enhanced Transformers
- Low-Rank Key Value Attention
- Fast Transformer Decoding: One Write-Head is All You Need
- XQuant: Breaking the Memory Wall for LLM Inference with KV Cache Rematerialization
- CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing
- Dual Dimensionality for Local and Global Attention
- YOCO++: Enhancing YOCO with KV Residual Connections for Efficient LLM Inference
- Lossless KV Cache Compression to 2%
- MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis