Restoring General and Improving Domain-Specific Safety in Fine-Tuned LLMs Without Retraining
cs.AI, cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Code: https://github.com/google-research-datasets/boolean-questions
Terminology
Sources
- Alignment with Preference Optimization Is All You Need for LLM Safety
- Refusal in Language Models Is Mediated by a Single Direction
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Adapting Large Language Models to Domains via Reading Comprehension
- Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Sparse Autoencoders Find Highly Interpretable Features in Language Models
- KTO: Model Alignment as Prospect Theoretic Optimization
- Fine-Tuning Lowers Safety and Disrupts Evaluation Consistency
- The Llama 3 Herd of Models
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models
- Lisa: Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Attack
- FinanceBench: A New Benchmark for Financial Question Answering
- Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks
- Multitask Mayhem: Unveiling and Mitigating Safety Gaps in LLMs Fine-tuning
- PubMedQA: A Dataset for Biomedical Research Question Answering
- Programming Refusal with Conditional Activation Steering
- LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents