Efficient Exploration for Iterative Nash Preference Optimization
cs.LG, cs.AI
Submitted: 2026-05-31
Updated: 2026-10-02
Terminology
Sources
- GPT-4 Technical Report
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Reward-free Alignment for Conflicting Objectives
- How Sampling Shapes LLM Alignment: From One-Shot Optima to Iterative Dynamics
- Foundations of Reinforcement Learning and Interactive Decision Making
- The Statistical Complexity of Interactive Decision Making
- Direct Language Model Alignment from Online AI Feedback
- Model-Based RL for Mean-Field Games is not Statistically Harder than Single-Agent RL
- KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
- Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences
- Statistical Impossibility and Possibility of Aligning LLMs with Human Preferences: From Condorcet Paradox to Nash Equilibrium
- Jackpot! Alignment as a Maximal Lottery
- AI Alignment and Social Choice: Fundamental Limitations and Policy Implications
- Achieving Logarithmic Regret in KL-Regularized Zero-Sum Markov Games
- A unified view of entropy-regularized Markov decision processes
- Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
- Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences
- Fundamental Limits of Game-Theoretic LLM Alignment: Smith Consistency and Preference Matching
- Proximal Point Nash Learning from Human Feedback
- LLaMA: Open and Efficient Foundation Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks