Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability

arXiv:2607.04061 · cs.CL, cs.AI, cs.LG, stat.ML · Submitted 2026-07-05 · Read on arXiv

Christopher Nassif, Josh F. Cooper

cs.CL, cs.AI, cs.LG, stat.ML

Submitted: 2026-07-05

Comments: 50 pages, ICML, 20 figures, Equal contribution

Code: https://github.com/ChrisNassif/Telescope

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Distinguishing Large Language Model (LLM) generated text from human writing is a critical and difficult challenge.

Terminology

Abstract

Distinguishing Large Language Model (LLM) generated text from human writing is a critical and difficult challenge. While LLMs are trained to write like humans, we hypothesize that this training leaves an indelible mark. LLMs develop a particularly strong aversion to token repetition very early in training. This bias persists as a ''Vestigial Heuristic'' (a developmental artifact) that is activated in LLM-generated text, separating LLM from human writing. To probe this phenomenon, we introduce Telescope Perplexity, a metric that evaluates the token repetition of the model, P(s i s 1:i). Our empirical investigation reveals that the Telescope Perplexity signature emerges early in pre-training, and Telescope Perplexity empirically enables highly effective zero-shot LLM detection. We show state-of-the-art or competitive performance across diverse datasets (including modern evaluation sets we introduce), reference models, and perturbation schemes with greater efficiency than other methods.

Sources

Related papers