LatentPress: Context Compression Beyond Text and Vision
cs.LG, cs.AI
Submitted: 2026-09-01
Updated: 2026-10-10
Code: https://github.com/HJSang/LatentPress
License: http://creativecommons.org/licenses/by/4.0/
The gist: Compressed context is usually carried as human-readable text or as rendered images that must be decoded, even when its consumer is a language model.
Terminology
Abstract
Compressed context is usually carried as human-readable text or as rendered images that must be decoded, even when its consumer is a language model. We introduce LatentPress, which writes conversational histories and long documents into a third representation: continuous memory tokens that a frozen decoder reads directly through its input-embedding interface, with no text reconstruction at inference. A small reader-matched writer compresses 4 - 16 times while training only an adapter (4.2M-26.2M parameters, about!0.1% of the decoder). On LongMemEval, LatentPress reaches 0.504 accuracy at 7.70 times compression versus 0.490 for uncompressed evidence, outperforming text summaries (0.184) and OCR-based compression (0.426 to 0.312). On LongBench-QA, in-domain writers match or exceed raw-context reading at 4 - 8 times compression, while 16 times trails raw. Writing takes 43ms per conversation, roughly an order of magnitude faster than text summarization or OCR reconstruction, and reading is 5 - 9 times faster than raw context or cached OCR. We validate the interface under two transfer settings, zero-shot from UltraChat to LongMemEval memory QA and from LongMemEval-derived QA to unseen LongBench document domains, establishing direct soft tokens as a practical machine-facing context interface beyond text and vision. The implementation of the experiments could be found at: https://github.com/HJSang/LatentPress.
Sources
- Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search
- Glyph: Scaling Context Windows via Visual-Text Compression
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering
- 500xCompressor: Generalized Prompt Compression for Large Language Models
- MemGPT: Towards LLMs as Operating Systems
- The Llama 3 Herd of Models
- AgentOCR: Reimagining Agent History via Optical Self-Compression
- CRISP: Compressed Reasoning via Iterative Self-Policy Distillation
- Highway Networks
- DeepSeek-OCR: Contexts Optical Compression
- TIP: Token Importance in On-Policy Distillation
- Qwen3 Technical Report
- Qwen2.5 Technical Report
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks