Agent Personas Shift Allocation Decisions About People: What Matters Is Whether the Two Strings Match
cs.AI
Submitted: 2026-05-27
Updated: 2026-11-09
Comments: 32 pages, 3 figures
Code: https://github.com/ibm-granite/granite-3.0-language-models
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Language models are increasingly used to make decisions about people, such as whom to interview, which insurance claims to approve and which patients to see first.
Terminology
Abstract
Language models are increasingly used to make decisions about people, such as whom to interview, which insurance claims to approve and which patients to see first. Research on bias in these decisions has focused on whether a person's race or gender changes the outcome. However, developers also write text that the model reads before every decision, including a persona that gives the model a role and often names a project, team or office. Whether such text affects whom a model selects remains largely unexamined. We asked what happens when a string in the persona also appears in one candidate's record. Across four kinds of decision, fourteen publicly released models and 328,480 choices between two candidates, instruction-tuned models, the kind used in chat assistants, selected the candidate whose record repeated the persona's string in 71.3% of decisions between equally qualified candidates, compared with 50.1% when no string was shared. When that candidate was one step less qualified, the rates were 31.5% and 11.7%. A made-up project code was enough. The effect persisted when the record labelled the string a conflict of interest, even in models that otherwise selected such a candidate less often. When the shared string was the candidate's race, a similar shift appeared. Bias audits built around protected attributes can at most show the resulting gaps between groups, not their source.
Sources
- Llama-Nemotron: Efficient Reasoning Models
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- AgentReputation: A Decentralized Agentic AI Reputation Framework
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- PaLM-E: An Embodied Multimodal Language Model
- Multi-agent Embodied AI: Advances and Future Directions
- TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems
- Large Language Models can Strategically Deceive their Users when Put Under Pressure
- Towards Understanding Sycophancy in Language Models
- Qwen3 Technical Report
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents