Beyond Heuristic Tuning: Power-Calibrated LLM Watermarking
Xiaopu Wang, Zelin He, Chengyuan Liu, Runze Li
stat.ML, cs.LG
Submitted: 2026-07-06
Comments: Accepted ICML 2026
Code: https://github.com/shooof/wm
License: http://creativecommons.org/licenses/by/4.0/
The gist: Logit-based watermarking is a widely used mechanism for identifying LLM generated content, yet its effectiveness is governed by a fundamental trade-off between detectability and semantic distortion.
Terminology
Abstract
Logit-based watermarking is a widely used mechanism for identifying LLM generated content, yet its effectiveness is governed by a fundamental trade-off between detectability and semantic distortion. Existing analyses provide limited guidance for principled hyperparameter selection, leaving practical deployments reliant on heuristic tuning. In this work, we develop a power-calibrated statistical framework that establishes explicit quantitative relationships between watermark hyperparameters, detection power, and distortion. This characterization transforms watermark design into a guided optimization problem. Building on these results, we derive practical parameter selection procedures that achieve optimal tradeoffs under constraints. Extensive experiments across multiple language models and datasets validate the theory and demonstrate that the proposed framework consistently identifies Pareto-optimal points.
Sources
- GPT-4 Technical Report
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Basic Properties of Strong Mixing Conditions. A Survey and Some Open Questions
- Towards Better Statistical Understanding of Watermarking LLMs
- Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- A Watermark for Large Language Models
- On the Reliability of Watermarks for Large Language Models
- Robust Distortion-free Watermarks for Language Models
- DeepSeek-V3 Technical Report
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Can AI-Generated Text be Reliably Detected?
- The Curse of Recursion: Training on Generated Data Makes Models Forget
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
- A Critical Evaluation of Evaluations for Long-form Question Answering
- OPT: Open Pre-trained Transformer Language Models
- BERTScore: Evaluating Text Generation with BERT
- Provable Robust Watermarking for AI-Generated Text
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Efficient quadratic entropy with distance sketches
- Embedding-Bias in Conditional Independence Testing
- From Log-Odds to Shapley Values: An Explanatory Geometry for the Weighted Naive Bayes Classifier