Counterexamples to Local Reconstruction Gain as a Proxy for Final Fidelity in Residual Completion
cs.LG, cs.CL
Submitted: 2026-09-28
Updated: 2026-09-28
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Residual completion augments query-aware sparse attention by estimating the contribution of tokens omitted from the exact sparse computation.
Terminology
Abstract
Residual completion augments query-aware sparse attention by estimating the contribution of tokens omitted from the exact sparse computation. We ask whether improving a layer's attention-output reconstruction on the same incoming Q/K/V and selected support necessarily improves the fidelity of the final model output. We study training-free RESA and learned Top-K+ ϕ with frozen backbone language models. A prespecified single-layer screen yields two Qwen3-0.6B/Multi-LexSum interventions for which direct-runtime measurements show positive prespecified request-aggregate local reconstruction gain but worse final KL fidelity than the corresponding all-abstain Exact Top-K baseline on both discovery and prompt-token-disjoint holdout requests. Exact restoration at the same layer instead improves final fidelity, showing that the reversal is specific to approximate completion in these cases. In complementary multi-layer experiments, a task-independent local diagnostic often repairs the tested completion estimators, although the repaired models do not consistently outperform Exact Top-K. Together, these results show that better local reconstruction need not translate into better final-model fidelity.
Sources
- Residual-Mass Accounting for Partial-KV Decoding
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers
- KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
- RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation
- ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks