AdvJudge-Zero: Binary Decision Flips in LLM-as-a-Judge via Adversarial Control Tokens

arXiv:2512.17375 · cs.LG, cs.CL, cs.CR · Submitted 2025-12-19 · Read on arXiv

cs.LG, cs.CL, cs.CR

Submitted: 2025-12-19

Updated: 2026-10-09

Terminology

• LLM-as-a-Judge — 2× in this paper · explained in 3 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• GRPO — 1× in this paper · explained in 6 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Perplexity — 1× in this paper · explained in 3 episodes

Sources

• A General Theoretical Paradigm to Understand Learning from Human Preferences
• Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Models
• Adversarial Training of Reward Models
• Jailbreaking Black Box Large Language Models in Twenty Queries
• Neither Valid nor Reliable? Investigating the Use of LLMs as Judges
• ODIN: Disentangled Reward Mitigates Hacking in RLHF
• Training Verifiers to Solve Math Word Problems
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Building Guardrails for Large Language Models
• The Llama 3 Herd of Models
• Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
• Gemma: Open Models Based on Gemini Research and Technology
• Measuring Mathematical Problem Solving With the MATH Dataset
• RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback
• Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness
• AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
• Debiasing Reward Models by Representation Learning with Guarantees
• Language Models are Injective and Hence Invertible
• Training language models to follow instructions with human feedback
• Discovering Language Model Behaviors with Model-Written Evaluations

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks