Right Reset: Chunking by Prefix Removal
Mike Vegeto
cs.CL, cs.LG
Submitted: 2026-08-05
Comments: 12 pages, 2 figures, 4 tables. Code, data, and reproduction materials: https://github.com/ZECTBynmo/right-reset-paper
Code: https://github.com/ZECTBynmo/right-reset-paper
License: http://creativecommons.org/licenses/by/4.0/
The gist: Removing the left context from a causal language model reveals a useful kind of boundary: an edge where the model processes the same right-hand tokens with little change.
Terminology
Abstract
Removing the left context from a causal language model reveals a useful kind of boundary: an edge where the model processes the same right-hand tokens with little change. We turn this observation into prefix-removal probing and introduce Right Reset (RR), which measures preservation of the right-hand hidden-state trajectory. A dynamic program converts RR edge scores into variable-length chunks. On flattened text formed by concatenating topically similar records after deleting their separators and layout, RR recovers 47.7% of the original records as clean units, versus 25.9% for a BGE embedding boundary, the strongest tested conventional baseline without task-specific model training. The gain persists after rendering and OCR. Passive scores from the same Qwen3-4B layer and direct prompting of a same-scale instruction model perform substantially worse on flattened records. Across six language models, RR-selected cuts also undergo consistently less local output disruption than unselected candidate edges. An observed-token likelihood-ratio readout is competitive in some architectures, indicating that the central contribution is the intervention: context dependence itself can provide a boundary signal when surface structure is weak.
Sources
- LumberChunker: Long-Form Narrative Document Segmentation
- Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models
- Text Embeddings by Weakly-Supervised Contrastive Pre-training
- Qwen2.5 Technical Report
- Qwen3 Technical Report
- Meta-Chunking: Learning Text Segmentation and Semantic Completion via Logical Perception
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering