Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function
cs.LG, cs.CR
Submitted: 2026-11-12
Updated: 2026-11-12
Code: https://github.com/git-disl/Bait
Terminology
Sources
- Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases
- OpenAI Gym
- Fight Fire with Fire: Defending Against Malicious RL Fine-Tuning via Reward Neutralization
- Reinforcement Learning for Long-Horizon Interactive LLM Agents
- Token Buncher: Shielding LLMs from Harmful Reinforcement Learning Fine-Tuning
- BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B
- LoRA: Low-Rank Adaptation of Large Language Models
- LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
- Exploiting Novel GPT-4 APIs
- Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents
- GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Prompt
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning
- Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models
- Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models
- Unleashing the Unseen: Harnessing Benign Datasets for Jailbreaking Large Language Models
- Universal and Transferable Adversarial Attacks on Aligned Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis