Fair on the Surface? Benchmarking Hidden-Output Fairness Gaps in LLM Recommenders

arXiv:2608.08284 · cs.AI · Submitted 2026-08-08 · Read on arXiv

Chan Aristella Lu, Arya Fayyazi, Junhao Zhang, Saeid Shokoufa, Yue Xing, Zhen Xiang, Kyu Hyung Lee, Mehdi Kamal, Massoud Pedram

cs.AI

Submitted: 2026-08-08

Updated: 2026-08-11

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 43× in this paper · explained in 2 episodes
• Steering — 39× in this paper · explained in 2 episodes
• Decoupling — 15× in this paper · explained in 2 episodes
• Robustness — 14× in this paper · explained in 6 episodes
• Activation Steering — 7× in this paper · explained in 7 episodes
• Consistency — 7× in this paper · explained in 3 episodes
• Ablation — 5× in this paper · explained in 1 episode
• Transformer — 5× in this paper · explained in 2 episodes
• Benchmarking — 3× in this paper · explained in 4 episodes
• AUC — 2× in this paper · explained in 2 episodes
• Interpretability — 2× in this paper · explained in 15 episodes
• Linear Probe — 2× in this paper · explained in 3 episodes
• Mechanistic Interpretability — 2× in this paper · explained in 4 episodes
• Activation Patching — 1× in this paper · explained in 2 episodes
• Coherence — 1× in this paper · explained in 2 episodes
• Decoder — 1× in this paper · explained in 2 episodes
• Evaluation Protocol — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Internal Bias Score — 1× in this paper
• Language Models (LLMs) — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 9 episodes
• Output Bias Score — 1× in this paper
• Representation Engineering — 1× in this paper · explained in 2 episodes
• Representation-Output Alignment — 1× in this paper

Sources

• WebGPT: Browser-assisted question-answering with human feedback
• Large Language Models for Generative Recommendation: A Survey and Visionary Discussions
• A Survey on Fairness in Large Language Models
• Fairness Definitions in Language Models Explained
• Auditing Language Model Unlearning via Information Decomposition
• Measuring Individual User Fairness with User Similarity and Effectiveness Disparity
• Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation
• Ethical and social risks of harm from Language Models
• Representation Engineering: A Top-Down Approach to AI Transparency
• ContextCite: Attributing Model Generation to Context
• AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruption
• System Prompt Optimization with Meta-Learning
• Steering Language Models With Activation Engineering

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• Agents' Last Exam
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems