PsychoSafe: Eliciting Psychologically-Informed Refusals in Large Language Models
cs.CL
Submitted: 2026-06-08
Updated: 2026-10-28
Code: https://github.com/aisilab/psychological-safety
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models (LLMs) routinely face requests that should be refused, creating a trade-off between helpfulness and harm prevention.
Terminology
Abstract
Large language models (LLMs) routinely face requests that should be refused, creating a trade-off between helpfulness and harm prevention. However, refusals themselves can be helpful. In high-risk interactions involving crisis, coercion, or escalating intent, blunt non-compliance may prevent direct harm while still failing to support the needs of the person behind the request. We present PsychoSafe, a psychologically-informed refusal framework that reframes refusal as structured supportive communication grounded in evidence-based intervention strategies. To develop PsychoSafe, we construct a corpus of 8,019 prompt-response pairs spanning five psychologically salient risk domains and apply prompting and parameter-efficient fine-tuning to Qwen 3.5 27B. On a balanced validation set of 500 prompts, evaluated with an LLM judge and validated through human ratings, PsychoSafe prompting improves overall refusal quality by 28.1% over a generic baseline, with particularly strong gains in external resource referral (+46.8%) and psychological grounding (+34.8%), while preserving downstream performance on non-refusal tasks. Fine-tuning achieves near-perfect refusal and resource-referral rates but reduces response relevance. Additional evaluations on SORRY-Bench and XSTest show strong in-domain robustness but limited out-of-domain generalization, suggesting that future work should diversify fine-tuning data to help models apply interventions selectively rather than schematically.
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- OR-Bench: An Over-Refusal Benchmark for Large Language Models
- AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Trace Gadgets: Minimizing Code Context for Machine Learning-Based Vulnerability Prediction
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
- gpt-oss-120b & gpt-oss-20b Model Card
- LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
- ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
- Introducing v0.5 of the AI Safety Benchmark from MLCommons
- SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models
- Self-Preference Bias in LLM-as-a-Judge
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data