From Profiles to Steering Vectors: Global Sparse Priors and Local Semantic Calibration for Personalized Text Generation
cs.AI
Submitted: 2026-06-01
Updated: 2026-10-29
Comments: EMNLP2026 Oral, SAC Highlight Award
Code: https://github.com/zzFestinaLente/GLASS
License: http://creativecommons.org/licenses/by/4.0/
The gist: Personalized text generation requires models to capture user-specific writing styles from historical data.
Terminology
Abstract
Personalized text generation requires models to capture user-specific writing styles from historical data. Existing approaches based on retrieval, parameter-efficient fine-tuning, or activation steering either introduce inference and storage overhead or struggle to separate stylistic signals from semantic content. We propose GLASS, a training-free framework for personalized generation via Global-Local Activation Steering with Sparse priors. GLASS uses sparse autoencoders to extract a global user-style prior from historical responses and constructs local contrastive style vectors over clustered interaction scenarios. During inference, it jointly injects global and local vectors into different model layers, enabling context-aware personalization without retrieval or parameter updates. Experiments on LaMP and LongLaMP show that GLASS outperforms retrieval-, fine-tuning-, and steering-based baselines across ROUGE metrics and LLM-as-judge evaluations. Further analyses show that SAE-based representations are more robust to topic and length shifts, suggesting better disentanglement of stylistic information from semantic residue.
Sources
- Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
- A Survey on In-context Learning
- Fints: Efficient Inference-Time Personalization for LLMs with Fine-Grained Instance-Tailored Steering
- Retrieval-Augmented Generation for Large Language Models: A Survey
- The Llama 3 Herd of Models
- LongLaMP: A Benchmark for Personalized Long-form Text Generation
- Persona-SQ: A Personalized Suggested Question Generation Framework For Real-world Documents
- Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Models
- LaMP: When Large Language Models Meet Personalization
- Can Large Language Models Understand Preferences in Personalized Recommendation?
- Steering Language Models With Activation Engineering
- Personalized Text Generation with Contrastive Activation Steering
- PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process
- A Survey of Large Language Models
- HYDRA: Model Factorization Framework for Black-Box LLM Personalization
- Representation Engineering: A Top-Down Approach to AI Transparency
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents