GRASP: Reinforcing Language Model Anonymizers with Group Relative Policy Optimization

arXiv:2608.06526 · cs.CL · Submitted 2026-08-06 · Read on arXiv

Sajjad Ghiasvand, Nader Sehatbakhsh

cs.CL

Submitted: 2026-08-06

Updated: 2026-08-10

License: http://creativecommons.org/licenses/by/4.0/

The gist: Large language models can infer sensitive personal attributes, such as age, location, and occupation, from ordinary text, turning everyday writing into a privacy risk.

Terminology

Abstract

Large language models can infer sensitive personal attributes, such as age, location, and occupation, from ordinary text, turning everyday writing into a privacy risk. Adversarial anonymization defends against this by rewriting a text with a capable language model that also plays the attacker, but it needs a powerful model at inference time and thus sends private text to a third party, the very exposure anonymization should prevent. Recent work distills this behavior into a small on-device model using supervised fine-tuning and direct preference optimization (DPO), but DPO only imitates the teacher's offline choices and never directly optimizes the privacy--utility objective we care about. We introduce GRASP (Group-Relative Anonymization via Self-refinement Policy-optimization), which reinforces the local anonymizer online with Group Relative Policy Optimization. A single small model acts as anonymizer, adversary, and utility judge, trained against a self-generated reward that hides attributes while preserving meaning, with a design that guards against reward hacking. Trained on Llama-3.1-8B, improves the privacy--utility trade-off over the DPO-distilled baseline, consistently across three independent LLM judges. Against adversarial anonymization driven by frontier models such as Gemini 2.5 Flash and Claude, it achieves a comparable or better overall trade-off while removing substantially more private information, and it runs entirely on-device at roughly 1% of the GPT-4o teacher's cost.

Sources

Related papers