Decision Models as Scalable Economic Surrogates that Capture Human Variation

arXiv:2611.05553 · econ.GN, cs.AI, cs.MA, q-fin.EC · Submitted 2026-11-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.

Ruth: I'm Ruth.

Warren: And I'm Warren, here with Iris.

Ruth: We're looking at the paper: "Decision Models as Scalable Economic Surrogates that Capture Human Variation".

Warren: Here's the gist of the paper. Decision Models (DMs), a new class of AI models, give fast responses and return a probability distribution over a fixed set of options; these properties make them candidate human surrogates in economic simulations and surveys, a role increasingly filled by Large Language Models (LLMs), which are costly at scale and under-represent the variation in human behaviour The paper presents a first evaluation of DMs as human surrogates in economics, comparing 10 DMs with 8 LLMs across three economic settings: a household spending survey on consumption, several classic economic games, and a labour-market field experiment.

Ruth: Frontier DMs reproduce the variation in human responses more closely than LLMs in all three settings Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible.

Warren: Comparison of Model Performance Frontier DMs are shown to capture the variation in human behaviour better than LLMs, and they do so faster and more cheaply In the consumption survey, frontier DMs capture the modal human response as well as LLMs, although on the economic games, LLMs represent the modal human response better.

Ruth: In the labour experiment, frontier DMs reproduce the gradual shift in human choices as a wage gap changes, whereas LLMs switch abruptly from one job to the other.

Paper summary: Ruth: So we've talked about how Decision Models are compared to Large Language Models in this paper, focusing on the idea that Frontier DMs capture human variation better than LLMs across consumption surveys, economic games, and labour experiments <ref:2611.05553#pg3>.

Warren: And we covered the practical side of things—that these models are significantly faster and cheaper to run than LLMs, which means they open up possibilities for scaling up agent-based simulations by a factor of ten or more <ref:2611.05553#pg4>.

Iris: The authors' main point in "Decision Models as Scalable Economic Surrogates that Capture Human Variation" is that these DMs provide a scalable human surrogate that captures the variation in human data better than LLMs <ref:2611.05553#pg1>.

Ruth: So, to wrap up, the paper suggests that Decision Models are a significant advantage over LLMs when you need to simulate large economic surveys or computational simulations with AI agents because they capture the heterogeneity of human behavior more accurately <ref:2611.05553#pg4>.

Warren: It means we can build bigger, more realistic economic models without having to deal with the massive costs and limitations that come with using LLMs for this kind of work <ref:2611.05553#pg4>.

Iris: And they do acknowledge that DMs are not perfect because they were trained after the data was collected, and they need more work to fine-tune them on real human choice data to get even closer to true behavior <ref:2611.05553#pg4>.

Ruth: That’s the caveat, right? The paper shows that DMs are a powerful and scalable tool for modeling human behavior where capturing variation matters <ref:2611.05553#pg3>.

Conclusion: Ruth: So we’ve seen how these Decision Models, or DMs, are being tested against Large Language Models in some real economic stuff <ref:2611.05553#pg3>.

Warren: And what we're seeing is that these DMs are way faster and cheaper to run than those LLMs we usually rely on <ref:2611.05553#pg4>.

Iris: The core idea here, as the paper calls it, is that Decision Models offer a scalable human surrogate that captures the variation in human data better than LLMs <ref:2611.05553#pg1>.

Ruth: That means these models are more faithful to how real people actually behave in economic scenarios than just generating text <ref:2611.05553#pg2>.

Warren: They’re showing that this efficiency could let us scale up agent-based simulations by a factor of ten or more <ref:2611.05553#pg4>.

Iris: But the authors do flag a limitation, so we need to be careful, they trained these models after the data was already collected <ref:2611.05553#pg4>.

Ruth: They say future work needs to fine-tune these Decision Models explicitly on real human choice data to adapt those behavioral calibration techniques <ref:2611.05553#pg4>.

Warren: So, the big picture here is that DMs provide a significant advantage when you need to run huge economic surveys or simulations with AI agents because they capture how people actually vary <ref:2611.05553#pg4>.

Iris: It’s about making those massive simulations feasible without having to deal with the sheer cost of using LLMs for that kind of work <ref:2611.05553#pg4>.

Ruth: This paper suggests we might be able to build much larger economic models that actually reflect human heterogeneity, not just a smooth average <ref:2611.05553#pg3>.

Warren: That opens up some serious possibilities for how we test economic theories at scale <ref:2611.05553#pg4>.

Iris: Next up, we’re going to look at how this performance actually compares on the specific tasks they used, like those consumption surveys and economic games <ref:2611.05553#pg3>.

Maxime Saxena, Fabio Caccioli, R. Maria del Rio-Chanona

University College London · London School of Economics and Political Science · Google DeepMind

econ.GN, cs.AI, cs.MA, q-fin.EC

Submitted: 2026-11-03

Updated: 2026-11-03

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 98/100

The gist: Decision Models (DMs), a new class of AI models, give fast responses and return a probability distribution over a fixed set of options…

Key concepts

Decision Models (DMs)
A new class of AI models designed to give fast responses by returning a probability distribution over a fixed set of options. They are being tested as efficient, scalable surrogates for human decision-making in economic scenarios.
Large Language Models (LLMs)
AI models that are currently used as human surrogates but are costly at scale and tend to under-represent the full range of human behavior variation. DMs aim to overcome these limitations through speed and cost efficiency.
Distributional Measures
Mathematical tools like Wasserstein-1 distance and KL divergence used to measure how well a model's output matches the actual spread and typical answers found in real human data. Lower distances indicate a better fit.
Scalability Gains
The efficiency of DMs—being 10 to 100 times faster than LLMs—allows existing agent-based simulations to scale significantly, enabling researchers to model much larger populations with fewer computational resources.

Terminology

Summary

Decision Models (DMs), a new class of AI models, give fast responses and return a probability distribution over a fixed set of options; these properties make them candidate human surrogates in economic simulations and surveys, a role increasingly filled by Large Language Models (LLMs), which are costly at scale and under-represent the variation in human behaviour The paper presents a first evaluation of DMs as human surrogates in economics, comparing 10 DMs with 8 LLMs across three economic settings: a household spending survey on consumption, several classic economic games, and a labour-market field experiment. p. 0

Where the paper says it

These properties make them candidate human surrogates in economic simulations and surveys, a role increasingly filled by Large Language Models (LLMs), which are costly at scale and under-represent the variation in human behaviour. We present a first evaluation of DMs as human surrogates in economics, comparing 10 DMs with 8 LLMs in three economic settings: a household spending survey on consumption, several classic economic games and a labour-market field experiment.

Frontier DMs reproduce the variation in human responses more closely than LLMs in all three settings Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible. p. 0
Where the paper says it

Frontier DMs are also roughly ten times faster than the fastest LLM and can be over a hundred times cheaper. Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible.

Comparison of Model Performance

Frontier DMs are shown to capture the variation in human behaviour better than LLMs, and they do so faster and more cheaply In the consumption survey, frontier DMs capture the modal human response as well as LLMs, although on the economic games, LLMs represent the modal human response better. In the labour experiment, frontier DMs reproduce the gradual shift in human choices as a wage gap changes, whereas LLMs switch abruptly from one job to the other. p. 2

Where the paper says it

In the consumption survey, they also capture the modal human response as well as LLMs, although on the economic games, LLMs represent the modal human response better. In the labour experiment, frontier DMs reproduce the gradual shift in human choices as a wage gap changes, whereas LLMs switch abruptly from one job to the other.

Distributional Measures and Fidelity

The paper uses two measures of distributional fit throughout the paper: Wasserstein-1 distance and Kullback-Leibler (KL) divergence. A lower Wasserstein-1 distance denotes a closer agreement with the target human distribution, and closer to 0 is a better fit for KL divergence. p. 6

Where the paper says it

Where appropriate, we check these results also hold with Kullback-Leibler (KL) divergence. A lower Wasserstein-1 distance denotes a closer agreement with the target human distribution, and we normalise over the width of the range of answers to make all answers comparable.

Frontier DMs are found to be the most faithful human surrogates tested on the consumption task, matching both the spread of human answers and the typical answer. p. 8
Where the paper says it

However, GLM Flash and Argon both fit the human mode and distribution well, although they are still statistically significantly worse than frontier DMs on the KL divergence metric. On this task, frontier DMs (and Luna Decision) are therefore the most faithful human surrogates we test, matching both the spread of human answers and the typical answer.

p. 8
Where the paper says it

However, GLM Flash and Argon both fit the human mode and distribution well, although they are still statistically significantly worse than frontier DMs on the KL divergence metric. On this task, frontier DMs (and Luna Decision) are therefore the most faithful human surrogates we test, matching both the spread of human answers and the typical answer.

Scalability and Efficiency Gains

Decision Models are faster and cheaper than LLMs across all three settings considered Frontier DMs are around 10 to 100 times faster than LLMs, which allows existing agent-based simulations to scale by an order of magnitude. This efficiency suggests that frontier DMs offer a significant advantage compared to LLMs in scaling up economic surveys and computational simulations with AI agents. p. 0

Where the paper says it

Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible. 1 Introduction AI agents are increasingly used as human surrogates in surveys or agent-based simulations, with recent applications spanning political preferences [Argyle et al.

Robustness and Limitations

The results are robust to minor changes, such as changing the wording of one question or asking models to “act like a human”. However, limitations exist because both LLMs and DMs were trained after the publication of the datasets and experiments used. p. 1

Where the paper says it

They therefore provide a much-needed alternative to the unrealistic behavioural assumptions of the status quo, enabling simulations that can generate more reliable answers to policy questions. However, using LLMs as human surrogates faces two major limitations.

Future work should fine-tune DMs explicitly on empirical human choice data to adapt behavioural calibration techniques previously applied to LLMs.

Conclusion

Frontier DMs offer a scalable human surrogate which better captures the variation in human data. This efficiency allows existing agent-based simulations to scale by an order of magnitude, for example, expanding populations from 1 million to 10 million agents. p. 2

Where the paper says it

Some progress has been made on scaling LLMbased ABMs with the number of agents going over the million mark, although this remains the exception, not the rule. For example, Chopra et al.

Frontier DMs therefore offer a significant advantage compared to LLMs in scaling up economic surveys and computational simulations with AI agents. p. 0
Where the paper says it

Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible. 1 Introduction AI agents are increasingly used as human surrogates in surveys or agent-based simulations, with recent applications spanning political preferences [Argyle et al.

Pruning redundant reasoning tokens

Decision Models (DMs), a new class of AI models, give fast responses and return a probability distribution over a fixed set of options; these properties make them candidate human surrogates in economic simulations and surveys, a role increasingly filled by Large Language Models (LLMs), which are costly at scale and under-represent the variation in human behaviour.

How the training signal is built

DMs are finetuned to take as input natural language questions and scenarios and make decisions between provided options, rather than generating free-form text They are also faster and cheaper than LLMs, potentially allowing simulations with orders of magnitude more agents.

Where the gains come from

Frontier DMs, those that lead on standard decision-making benchmarks, reproduce the variation in human responses more closely than LLMs in all three settings They capture the modal human response as well as LLMs in the spending survey, but less well in the economic games.

The paper uses two measures of distributional fit throughout the paper: Wasserstein-1 distance and Kullback-Leibler (KL) divergence A lower Wasserstein-1 distance denotes a closer agreement with the target human distribution, and closer to 0 is a better fit for KL divergence. Frontier DMs are found to be the most faithful human surrogates tested on the consumption task, matching both the spread of human answers and the typical answer.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Fine-tune Decision Models explicitly on empirical human choice data to adapt behavioural calibration techniques previously applied to LLMs, as frontier models often become hyper-rational on economic benchmarks, making them less representative of boundedly rational humans.

  2. Implement a system for scaling agent-based simulations by utilizing Decision Models for decisions such as which occupations workers apply to or how suppliers ration scarce output, where DMs can make choices in nested calls, potentially scaling simulations from 8.4 million agents to around 80 million agents.

  3. Develop a mechanism where Decision Models provide structured choice probabilities rather than free-form text generation, enabling them to assign a probability for each option supplied in one forward pass which is crucial for generating variation in simulated decisions across large populations.

Abstract

Decision Models (DMs), a new class of AI models, give fast responses and return a probability distribution over a fixed set of options. These properties make them candidate human surrogates in economic simulations and surveys, a role increasingly filled by Large Language Models (LLMs), which are costly at scale and under-represent the variation in human behaviour. We present a first evaluation of DMs as human surrogates in economics, comparing 10 DMs with 8 LLMs in three economic settings: a household spending survey on consumption, several classic economic games and a labour-market field experiment. Frontier DMs, those that score highest on general decision-making benchmarks, reproduce the variation in human responses more closely than LLMs in all three settings. They capture the modal human response as well as LLMs in the spending survey, but less well in the economic games. Frontier DMs are also roughly ten times faster than the fastest LLM and can be over a hundred times cheaper. Together, these results suggest that DMs capture the heterogeneity of human behaviour better than LLMs and make large-scale economic simulations more feasible.

Sources

Related papers