Investigating Social Bias Changes in Quantized Language Models
cs.CL
Submitted: 2026-02-05
Updated: 2026-08-31
Comments: Accepted to COLM 2026 (10 pages, 5 figures)
Code: https://github.com/stan-hua/PostTrainingBiasBenchmark
License: http://creativecommons.org/licenses/by/4.0/
The gist: Post-training quantization reduces the memory needed to run large language models but alters their social biases in ways that aggregate metrics fail to capture.
Terminology
Abstract
Post-training quantization reduces the memory needed to run large language models but alters their social biases in ways that aggregate metrics fail to capture. We present the first large-scale study of 50 quantized models evaluated on PostTrainingBiasBench, a unified benchmark of 13 closed- and open-ended bias datasets. We identify a phenomenon we term quantization-induced bias flipping, in which quantization causes models to change responses from biased to unbiased and vice versa, up to 21% of the time, despite no change in aggregate bias scores. These flips are strongly associated with model uncertainty, where the responses with high uncertainty are 3-11x more likely to change than the confident ones. Quantization strength amplifies this effect, with 4-bit quantized models exhibiting 4-6x more behavioral changes than 8-bit quantized models. Critically, these changes create asymmetric impacts across demographic groups, where bias can worsen by up to 18.6% for some groups while improving by 14.1% for others, yielding misleadingly neutral aggregate outcomes. Larger models show no consistent robustness advantage, and group-specific shifts vary unpredictably across model families. Our findings demonstrate that compression fundamentally alters bias patterns, requiring crucial post-quantization evaluation and interventions to ensure reliability in practice.
Sources
- Measuring Implicit Bias in Explicitly Unbiased Large Language Models
- Lessons from the Trenches on Reproducible Evaluation of Language Models
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
- FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Bias and Fairness in Large Language Models: A Survey
- Intrinsic Bias Metrics Do Not Correlate with Application Bias
- Understanding the Effect of Model Compression on Social Bias in Large Language Models
- Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications
- Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
- A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners
- Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Tasks
- The Impact of Inference Acceleration on Bias of LLMs
- Uncertainty-based Fairness Measures
- The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models
- Fairness Testing of Large Language Models in Role-Playing
- Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models
- SimPO: Simple Preference Optimization with a Reference-Free Reward
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering