Knowing When to Stop: Efficient Context Processing via Latent Sufficiency Signals
Researchers developed dynamic context cutoff to allow LLMs to stop processing input when sufficient information is acquired. By analyzing attention heads, they found latent 'sufficiency signals' that predict when a task is complete. This method improves accuracy by 3.4% and reduces token usage by 1.33x on average, showing efficiency emerges from internal model understanding.