Beyond Cultural Persona Bias: Response Geometry as a Measure for Language Models and People
stat.ML, cs.AI, cs.LG, math.DG
Submitted: 2026-07-02
Updated: 2026-10-08
Project page: https://ipip.ori.org/new_ipip-50-item-scale.htm
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Evaluations of LLM personas via psychometric questionnaires typically rely on aggregate scores, discarding within-instance correlation structure.
Terminology
Abstract
Evaluations of LLM personas via psychometric questionnaires typically rely on aggregate scores, discarding within-instance correlation structure. We test whether this geometric structure is intrinsic or frame-dependent. Constructing within-instance correlation matrices from IPIP-50 responses, we analyze geometry on SPD manifolds under manipulated question orderings in GPT-4o simulating American and Chinese-American personas. We find that persona expression comprises two dissociable components: aggregated features (Big Five scores) degrade under randomization (21% drop) but are frame-robust; geometric features (SPD manifold) collapse under frame misalignment (42% drop) but recover substantially (to 84%) under shared frames, surpassing aggregated features (76%). This collapse-recovery pattern reveals that persona geometry is not intrinsic but a frame-dependent coordination pattern encoding information invisible to aggregation. Our findings establish a dual-nature framework for LLM personas, frame-dependent geometry versus frame-robust aggregates, necessitating frame-aware evaluation and challenging static trait conceptions.
Sources
- Personality Traits in Large Language Models
- Whose Opinions Do Language Models Reflect?
- Tracking the distance to criticality in systems with unknown noise
- A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- Scalable Posterior Uncertainty for Flexible Density-Based Clustering
- A Transport Geometry of Benign Overfitting
- Efficient quadratic entropy with distance sketches