Ontological Instability and Statistical Amplification: The Paradox of "Humanizing" LLM-Generated Text
cs.CL
Submitted: 2026-10-02
Updated: 2026-10-02
License: http://creativecommons.org/licenses/by/4.0/
The gist: Supervised AI-text detectors report high benchmark accuracy, but it is not clear what their decisions are based on.
Abstract
Supervised AI-text detectors report high benchmark accuracy, but it is not clear what their decisions are based on. We analyze a RoBERTa-based detector under semantic, structural, and tokenizer-level perturbations, using the M4 dataset (N = 10,000) and controlled generations (N = 300). When Mistral-7B-Instruct was asked to make machine text sound more human, Verb Diversity rose from 0.77 to 0.92 and the outputs became easier to detect. Detection scores appear to track statistical complexity, which also leads to a 76.3% false-positive rate on formal human writing. As a control, we evaluate event-based Latent Space detection. Paraphrasing changed 87% of its event sequences (Jaccard = 0.067), and homoglyphs altered 70% of the extracted verbs even though extraction still ran (Jaccard = 0.30). Its best domain AUC was 0.577. RoBERTa's robustness seems specific to the features it uses, and structural abstraction did not make detection more robust.
Sources
- Scaling Instruction-Finetuned Language Models
- Are AI Detectors Good Enough? A Survey on Quality of Datasets With Machine-Generated Texts
- Mistral 7B
- RoBERTa: A Robustly Optimized BERT Pretraining Approach
- Can AI-Generated Text be Reliably Detected?
- Release Strategies and the Social Impacts of Language Models
- Attacking Neural Text Detectors
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering