How Much Were You Told? Measuring External Information in Peer Reviews

arXiv:2609.28041 · cs.CL · Submitted 2026-09-23 · Read on arXiv

cs.CL

Submitted: 2026-09-23

Updated: 2026-09-23

Code: https://github.com/BaggerOfWords/Measuring-External-Information

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: Conference policies distinguish using Large Language Models (LLMs) to polish one's own review from delegating the critique, but current Artificial Text Detection (ATD) methods largely measure surface

Terminology

Abstract

Conference policies distinguish using Large Language Models (LLMs) to polish one's own review from delegating the critique, but current Artificial Text Detection (ATD) methods largely measure surface form rather than the origin of its content. We instead measure the external information carried by a review: information not explained by the reviewed paper and a generic reviewing instruction. We propose Self-Conditioning, an unsupervised information-theoretic estimator that compares the likelihood of a review under its production context with its likelihood when that context is augmented with hints extracted from the review itself. On the IntelLabs peer-review benchmark, Self-Conditioning separates fully-delegated from machine-polished reviews with AUC up to 1.0 while remaining largely insensitive to surface rewriting. Moreover, as generators receive increasing amounts of externally-provided information, their scores move monotonically towards the human regime, unlike standard ATD baselines. High-temperature sampling can evade the estimator, but at the cost of output quality.

Sources

Related papers