Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña López
cs.CL, cs.CY, cs.LG
Submitted: 2026-07-30
Comments: 15 pages, 3 figures, 9 tables. Code and datasets publicly available
Code: https://github.com/peremartra/optipfair
License: http://creativecommons.org/licenses/by/4.0/
The gist: This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs).
Terminology
Abstract
This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias in large language models (LLMs). As a foundational empirical validation of this method, this work focuses on causal bias localization. Using minimally contrastive prompt pairs and inference-time activation capture, the method identifies neurons that react differentially when processing demographic attributes in GLU architectures, evaluating the signal at the down proj input. Empirical evaluation was conducted on models of up to 3 billion parameters (Llama-3.2 family and Salamandra-2B), combining standardized benchmark evaluation with qualitative text generation experiments. Results demonstrate that zeroing the identified neurons alters how the model responds to associated demographic variables. However, rather than producing flat mitigation, the intervention causes bidirectional bias destabilization: because BiasScore is unsigned, candidate sets mix neurons that push toward and against the stereotype, and the net effect on aggregate bias depends on which sign dominates. The intervention is extremely surgical: zeroing at most 40 neurons in Llama-3.2-1B (less than 0.031% of total MLP width) achieves a mean retention of 99.49% in reasoning and general knowledge capabilities. These findings empirically confirm that demographic bias processing and model capabilities operate on dissociable circuits, establishing the methodological foundations for transitioning from blind zeroing toward directional behavior modulation.
Sources
- Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
- Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them
- Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
- Constitutional AI: Harmlessness from AI Feedback
- Training language models to follow instructions with human feedback
- Fragile Knowledge, Robust Instruction-Following: The Width Pruning Dichotomy in Llama-3.2
- Transformer Feed-Forward Layers Are Key-Value Memories
- Language (Technology) is Power: A Critical Survey of "Bias" in NLP
- BBQ: A Hand-Built Bias Benchmark for Question Answering
- Evaluating and Mitigating Social Bias for Large Language Models in Open-ended Settings
- "The Dentist is an involved parent, the bartender is not": Revealing Implicit Biases in QA with Implicit BBQ
- EsBBQ and CaBBQ: The Spanish and Catalan Bias Benchmarks for Question Answering
- Textbooks Are All You Need
- An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
- Probing Classifiers are Unreliable for Concept Removal and Detection
- Steering Language Models With Activation Engineering
- Representation Engineering: A Top-Down Approach to AI Transparency
- SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
- A Simple and Effective Pruning Approach for Large Language Models
- LLM-Pruner: On the Structural Pruning of Large Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering