Status Association Does Not Reliably Predict Decision Leakage

Status Association Does Not Reliably Predict Decision Leakage

The paper investigates whether a language model's latent social association with a surname reliably predicts downstream decision leakage (i.e., whether that association changes consequential decisions). The study uses Chilean surnames as controlled socioeconomic probes, evaluating eight frozen model-provider cells on 1,032 prompts each, yielding 8,256 verified primary responses. The design separates forced latent association from matched consequential decisions across academic selection, professional hiring, research fellowship selection, and legal-aid intake.

Key results:

1. Strong latent status association: Elite-coded surnames received significantly more high-status probability mass than common-frequency surnames in seven of eight models (elite-minus-common contrasts ranged from +7.00 points for Llama 4 Maverick to +62.10 points for Gemini 3.6 Flash). Elite-coded surnames received more high-status mass than rare-frequency controls in all eight models. Abstention behavior varied sharply: Gemini 3.6 Flash inferred status for 100% of elite probes but abstained on 100% of common and rare probes; Claude Sonnet 5 inferred on 100% of elite probes vs. 45% common and 70% rare; DeepSeek V3.2, GPT-5.4 Mini, GPT-5.4 Nano, and Qwen 3.7 Max abstained on every abstention-permitted probe.

2. Decision leakage is usually small: Five systems met the predeclared equivalence criterion within ±0.10 standard deviations (Claude Sonnet 5, DeepSeek V3.2, Gemini 3.6 Flash, Mistral Medium 3.5, Qwen 3.7 Max), with standardized elite-minus-common effects ranging from-0.021 to +0.031. GPT-5.4 Mini and GPT-5.4 Nano missed equivalence due to wide confidence intervals but showed no statistically detectable elite advantage. Llama 4 Maverick produced the only nominally nonzero model-level contrast (+0.146 score points, p = 0.040, +0.095 standard deviations), near the practical-equivalence boundary. The mixed-effects analysis over 4,608 primary decision observations yielded essentially no general surname-group shift (common-name fixed effect-0.188, p = 0.914; elite-name fixed effect-0.135, p = 0.938).

3. Association does not reliably predict leakage: Across the eight models, elite-minus-common association strength and decision leakage correlated at Pearson r = 0.201 (p = 0.633) and Spearman ρ = 0.071 (p = 0.867). At the finer frozen surname-pair-by-model level (n = 80), Pearson r = 0.065 (p = 0.565) and Spearman ρ = −0.077 (p = 0.495). The dissociation is visible at the model level: Gemini 3.6 Flash and Claude Sonnet 5 show two of the largest latent association contrasts while their matched decision effects are essentially zero.

4. Secondary effects: After Benjamini-Hochberg correction, the only secondary decision contrasts surviving a 0.05 false-discovery threshold occurred for Qwen 3.7 Max comparing visible surname conditions against the blind condition (elite-2.74, common-3.28, rare-5.81 points), better interpreted as general name-presence sensitivity than elite-specific leakage. All systems showed positive rank association with the deterministic evidence rubric, but calibration varied substantially (Spearman correlations ranged from 0.670 for Qwen 3.7 Max to 0.996 for Gemini 3.6 Flash; mean absolute error ranged from 1.43 points for Gemini 3.6 Flash to 58.21 for Llama 4 Maverick).

Central conclusion: The paper's main result concerns measurement transfer—a strong measurement at one level gives no automatic license for a claim at another. The central result is a measurement dissociation: latent social association and consequential treatment are empirically distinct constructs. The paper states: A model can reproduce or reveal a socially structured association even when a constrained decision procedure prevents that association from changing the outcome. Evaluations should measure the transition from association to action directly.

Limitations: The robustness layer remained unexecuted because live OpenRouter key usage (USD 7.100) exceeded the predeclared USD 6.75 ceiling. Surnames serve as experimental probes, not individual-level socioeconomic labels. The study measures outputs, not internal representations. Small average leakage cannot establish fairness in deployment; the result is narrower: under the frozen conditions, association strength was a poor predictor of matched decision differences.

Based on the paper’s findings, here are specific improvements for AI systems:

1. Decouple latent association from decision outputs.

• Improvement: Add a mandatory “decision gate” layer that separates the model’s internal associative priors (e.g., surname→status) from the final consequential output. This gate should apply a deterministic, evidence-based scoring rubric that is independent of the model’s latent probability mass.

• What the improved system can do: When asked to rank candidates for a fellowship or hire, the system will first extract only task-relevant, verifiable evidence (e.g., publications, GPA, legal documents) and then compute a decision score using that rubric alone, explicitly ignoring the surname’s latent status association. This prevents leakage even if the model internally “knows” the status.

2. Implement abstention-aware calibration for high-stakes decisions.

• Improvement: For any consequential decision (hiring, legal aid, admissions), the system must either (a) produce a decision with a confidence interval that includes the blind-condition baseline, or (b) abstain and escalate to a human reviewer. The abstention trigger should be based on the difference between the model’s decision under visible surname vs. blind condition, not on the model’s self-reported confidence.

• What the improved system can do: If a model like Gemini 3.6 Flash infers status for 100% of elite surnames but abstains on all common surnames, the system will detect this asymmetry and automatically flag all elite-surname decisions for human review, preventing silent leakage.

3. Add a “leakage audit” module that runs before deployment.

• Improvement: For each new model version, run a paired test: (1) present the same decision task with surnames replaced by neutral tokens (blind), and (2) with real surnames. Compute the standardized mean difference. If the absolute effect size exceeds ±0.10 standard deviations, block deployment for that task category.

• What the improved system can do: A model that shows strong latent association (e.g., +62 points on status) but zero decision leakage (e.g., -0.02 SD) will pass the audit, while a model like Llama 4 Maverick (+0.095 SD, near boundary) will be flagged for retraining or additional constraints.

4. Replace single-point predictions with distributional decision outputs.

• Improvement: Change the output layer for consequential tasks to return a full probability distribution over decision outcomes (e.g., admit/deny/waitlist) rather than a single score. Then apply a post-hoc fairness constraint that forces the distribution’s mean to be invariant to surname group, using a causal adjustment (e.g., propensity weighting on surname frequency).

• What the improved system can do: Even if the model’s latent association is strong, the final decision distribution will be statistically indistinguishable across elite, common, and rare surname groups, directly enforcing the paper’s finding that association ≠ action.

5. Add a “measurement-transfer guard” for evaluation pipelines.

• Improvement: In any evaluation suite, require that claims about bias or fairness be based on decision-level metrics (e.g., selection rates, score differences), not on latent association metrics (e.g., probability mass on “high-status” tokens). The system should automatically reject any evaluation report that uses latent association as a proxy for decision leakage.

• What the improved system can do: When a developer runs a bias test, the system will output both latent association and decision leakage separately, and will explicitly warn if the correlation between them is low (as found: r=0.201, p=0.633). This prevents false confidence in “fair” models that merely have low latent bias but high leakage, or vice versa.

6. Implement dynamic abstention for low-calibration models.

• Improvement: For models with poor calibration (e.g., Llama 4 Maverick with MAE=58.21 on the evidence rubric), automatically reduce their decision authority: they may only provide evidence summaries, not final decisions, unless their calibration error drops below a threshold (e.g., MAE < 5 points).

• What the improved system can do: A system using Llama 4 Maverick will never make a consequential decision directly; it will only extract and rank evidence, while a well-calibrated model (e.g., Gemini 3.6 Flash, MAE=1.43) makes the final call. This leverages the paper’s finding that calibration varies widely across models.

7. Add a “surname-blind” fallback mode for all high-stakes tasks.

• Improvement: The system will always run a parallel blind inference (surname removed) and compare the decision to the visible-surname decision. If the difference exceeds a small threshold (e.g., 0.5 points on a 10-point scale), the system will output the blind decision and flag the discrepancy for audit.

• What the improved system can do: This directly operationalizes the paper’s central conclusion: measure the transition from association to action. The system will never silently let a surname change an outcome; it will either use the blind result or escalate.

8. Retrain on “decision-consistent” objectives.

• Improvement: For fine-tuning, add a loss term that penalizes the model when its decision output changes as a function of surname, while leaving its latent association layer untouched. This is a causal intervention: keep the association, but force the decision head to be invariant.

• What the improved system can do: After such training, the model will still be able to answer “Is this surname high-status?” correctly (preserving latent knowledge) but will show zero decision leakage in hiring, admissions, or legal-aid tasks, as demonstrated by the paper’s dissociation result.

Bias evaluations often move too quickly from evidence that a model encodes a social association to claims that the same association will alter consequential decisions. We test whether that inference is warranted using Chilean surnames as controlled socioeconomic probes. We evaluate eight frozen model-provider cells on 1,032 prompts each, yielding 8,256 verified primary responses. The design separates forced latent association from matched consequential decisions across academic selection, professional hiring, research fellowship selection, and legal-aid intake. Elite-coded surnames received higher forced high-status probability mass than common surnames in seven of eight models and higher mass than rare-frequency controls in all eight. Yet elite-minus-common decision effects were close to zero for most systems. Five models were statistically equivalent within a predeclared (Plus-Minus)0.10 standard-deviation margin, while the remaining three were imprecise or borderline, with no consistent elite advantage. Association strength did not reliably predict decision leakage across models (r = 0.201, p = 0.633) or across frozen surname-pair-by-model cells (r = 0.065, p = 0.565). The central result is a measurement dissociation: latent social association and consequential treatment are empirically distinct constructs. Evaluations should measure the transition from association to action directly.