Learning to Simulate Individuals from Macro Social Signals
cs.LG, cs.AI, cs.CL, cs.SI
Submitted: 2026-10-05
Updated: 2026-10-05
Code: https://github.com/volcengine/verl
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral reasoning behind these responses is either inherited from pretraining or learned
Abstract
Large language models are increasingly used to simulate how individuals respond to new situations, yet the behavioral reasoning behind these responses is either inherited from pretraining or learned from individual-level annotations, which offer limited behavioral diversity and little supervision of the reasoning itself. We propose to learn behavioral reasoning from prediction markets, whose price trajectories record how populations respond to real-world events at scale. We introduce macro2mind, which trains a language model with GRPO using market signals. A social behavioral decomposition makes behavioral reasoning an explicit step of forecasting: the model infers representative groups of market participants, predicts how each interprets the news and updates its beliefs, reasons about their interactions, and aggregates these responses into a price. A hindsight-regret curriculum with difficulty-aware sampling focuses training on transitions where hindsight-identified groups substantially improve the forecast while prioritizing examples that remain learnable for the current policy. The learned reasoning applies to user simulation without further training. On SWM-Bench, macro2mind achieves state-of-the-art directional accuracy and correlation on Polymarket. Trained on market data, it transfers zero-shot to four user-simulation benchmarks (Humanual, OvertonBench, PRISM, and CAD) and has competitive performance among zero-shot methods. Used as a data generator, macro2mind also raises a downstream simulator's accuracy on unseen users by 15.5 points, outperforming data generated by its backbone by 13.2 points.
Sources
- One Adapts to Any: Meta Reward Modeling for Personalized LLM Alignment
- DecisionFlow: Advancing Large Language Model as Principled Decision Maker
- Towards Measuring the Representation of Subjective Global Opinions in Language Models
- The Ethics of Advanced AI Assistants
- S$^3$: Social-network Simulation System with Large Language Model-Empowered Agents
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Momentum-Coupled Rubric Adaptation for Detailed Image Captioning
- PyMETA: A Benchmark Dataset for Hierarchical Student Code Error Classification with Python-Interpreter-Based Labels
- AVA-Encoder: Towards Agent-Native Video Representation Learning
- SPADE: Self-Play in Adaptive Synthetic Executable Environments
- A Time Series is Worth 64 Words: Long-term Forecasting with Transformers
- LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
- Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuning
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- HumanLM: Simulating Users with State Alignment Beats Response Imitation
- Qwen3 Technical Report
- TwinMarket: A Scalable Behavioral and Social Simulation for Financial Markets
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks