The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices

arXiv:2603.18482 · cs.CL, cs.LG, stat.ML · Submitted 2026-03-19 · Read on arXiv

cs.CL, cs.LG, stat.ML

Submitted: 2026-03-19

Updated: 2026-09-23

Comments: Accepted at INLG 2026

Code: https://github.com/EstebanGarces/human_vs_machine

License: http://creativecommons.org/licenses/by/4.0/

The gist: Why does machine-generated text remain detectable? We investigate a mechanistic explanation at the decoding stage: standard strategies such as top- k and nucleus sampling restrict generation to

Terminology

Abstract

Why does machine-generated text remain detectable? We investigate a mechanistic explanation at the decoding stage: standard strategies such as top- k and nucleus sampling restrict generation to high-probability tokens, while human writers routinely choose contextually appropriate words from deeper in the model's probability distribution. Truncation makes a measurable share of these choices unreachable; we call this the truncation blind spot. Across five open models and three domains, 8--18% of human-selected tokens fall outside common truncation boundaries. Linguistic analysis further reveals disproportionate exclusion of content-word tokens. In a benchmark comprising 1.8 million machine generations, classifiers using only predictability and lexical diversity achieve mean AUC-ROC near 0.97, with substantial variation across decoding settings and strong transfer across generators. Probability-floor samplers substantially narrow the blind spot, demonstrating that the choice of truncation criterion matters for retaining human-used tokens. Together, these findings characterize a source of human--machine distributional mismatch and motivate decoding methods that preserve contextually appropriate low-probability choices while maintaining generation quality. Code and data are available at https://github.com/EstebanGarces/human vs machine.

Sources

Related papers