Random Exploration is Random

arXiv:2611.17011 · q-bio.NC, cs.LG · Submitted 2026-11-12 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Random Exploration is Random".

Marcus: Here's the gist of the paper. Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives <ref:2611.17011#pg3> The paper examines whether variability in random exploration reflects genuine stochasticity or a deterministic structure that traditional cognitive models fail to capture <ref:2611.17011#pg3> The core finding Limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random <ref:2611.17011#pg3> This conclusion is drawn from examining the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials <ref:2611.17011#pg4> Task and Model Comparison The Horizon Task involves participants choosing between two slot machines to maximize total reward by balancing exploitation and exploration <ref:2611.17011#pg5> The study utilized deep neural networks to test if they could recover deterministic structure from the additional variability observed in long-horizon choices <ref:2611.17011#pg4> Neural network models captured more behavioral structure than the traditional logistic model, increasing McFadden’s pseudo-R2 by 0.095–0.168 relative to the logistic regression baseline <ref:2611.17011#pg9> Testing Deterministic Structure The researchers systematically tested various sources of deterministic structure, including information from the current game and subject identity <ref:2611.17011#pg4> They also investigated whether the H6–H1 predictability gap arose from learning or strategy adaptation across repeated games, finding that it was present even when restricting analysis to the first game played by each participant <ref:2611.17011#pg9> Cross-game history effects were also tested using recurrent neural networks but did not eliminate the predictability gap between H1 and H6 <ref:2611.17011#pg9> Limitations and Future Directions A key limitation acknowledged is that even highly expressive models can only capture deterministic structure present in the measured data, as unmeasured factors like internal states or spontaneous movements could account for unexplained variability <ref:2611.17011#pg10> Future work is suggested to incorporate richer behavioral and physiological measurements, such as eye tracking or pupil dilation, to provide a stronger test of how much of the remaining variability can be explained <ref:2611.17011#pg10> Taken together, findings suggest that the additional variability observed during long-horizon exploration cannot be explained by limited model expressiveness alone, and this residual variability is consistent with a genuinely stochastic component of random exploration <ref:2611.17011#pg15> The lower predictability of H6 choices persists after accounting for several sources of deterministic structure available in the Horizon Task <ref:2611.17011#pg4> REFERENCE

5: Bellman, R.: A problem in the sequential design of experiments.

Ines: Sankhy¯a: The Indian Journal of Statistics (1933-1960) 16(3/4), 221–229 (1956) <ref:2611.17011#pg5>

8: Averbeck, B.B.: Theory of choice in bandit, information sampling and foraging tasks.

Marcus: PLoS computational biology 11(3), 1004164 (2015) <ref:2611.17011#pg9>

9: Wilson, R.C., Bonawitz, E., Costa, V.D., Ebitz, R.B.: Balancing exploration and exploitation with information and randomization.

Ines: Current Opinion in Behavioral Sciences 38, 49–56 (2021) <ref:2611.17011#pg3>

33: Wang, S., Wilson, R.C.: Separating random and deterministic sources of computational noise in explore-exploit decisions.

Paper summary: Ines: So we're looking at this paper called "Random Exploration is Random" and it's asking a really fundamental question about decision making. The core idea is whether the variability in choices when people explore randomly actually comes from genuine randomness or if there's some underlying deterministic structure that our current cognitive models just can't see.

Marcus: Right, and it sets up this comparison using a task called the Horizon Task where participants have to balance exploiting what they know with exploring uncertain options. The paper is investigating if that variability we see in exploration choices reflects true stochasticity or something else entirely.

Yuki: From a population genetics view, I wonder if this speaks to how species adapt to novel environments versus just following random noise in their gene pool. If the choice variability isn't truly random, it suggests there's a hidden mechanism driving those choices that we need to account for.

Ines: Exactly. The paper examines this using an aggregated dataset of one thousand three hundred ninety-two participants who completed over one million two hundred seventy thousand six hundred ninety-five trials in this task <ref:2611.17011#pg1>. They found that exploration becomes more valuable when participants have more future choices and choice variability increases under longer horizons <ref:2611.17011#pg3>.

Marcus: And they used deep neural network models to see if those expressive models could capture any of this underlying deterministic structure from the extra variability seen in long-horizon choices <ref:2611.17011#pg4>. The results showed that these neural networks did capture more behavioral structure than the traditional logistic model, increasing McFadden’s pseudo R squared by zero point zero nine five to one six eight relative to the logistic regression baseline <ref:2611.17011#pg9>.

Yuki: So the models were better at fitting the behavior, but they still couldn't fully explain that increased variability in those longer-horizon choices <ref:2611.17011#pg4>. That suggests that what's left over isn't just noise from a simple decision-noise parameter in traditional models <ref:2611.17011#pg3>.

Ines: That’s what they conclude: limited model expressiveness simply cannot account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random <ref:2611.17011#pg15>. It points toward a genuinely stochastic component of that exploration behavior.

Marcus: So essentially, even when you try to use more complex AI models to find a deterministic pattern in this kind of randomness, you run into a wall because the structure isn't there enough to capture <ref:2611.17011#pg3>. This means random exploration is likely substantially random <ref:2611.17011#pg15>.

Yuki: It changes how we think about uncertainty. If it's not fully deterministic, then the uncertainty isn't just an artifact of our incomplete knowledge, but something inherent in the process itself. It implies a level of randomness that resists simple capture by current theoretical frameworks <ref:2611.17011#pg3>.

Ines: And they also looked at different sources of structure, like information from the current game or subject identity <ref:2611.17011#pg4>. They also checked if this predictability gap between short horizon choices and long horizon choices came from learning or just adapting strategy across repeated games <ref:2611.17011#pg9>.

Marcus: Interestingly, even when they looked at cross-game history effects using recurrent neural networks, they didn't eliminate that predictability gap between the short and long horizons <ref:2611.17011#pg9>. That suggests the difference remains even when you account for past interactions within the same game structure.

Yuki: The limitation they mention is important here: even highly expressive models can only capture deterministic structure present in the measured data because unmeasured factors like internal states or spontaneous movements could explain some of that unexplained variability <ref:2611.17011#pg10>. So the model is only as good as what we measure.

Ines: Right, and they suggest future work should involve incorporating richer behavioral and physiological measurements, like eye tracking or pupil dilation, to provide a stronger test of how much of that remaining variability can actually be explained <ref:2611.17011#pg10>. They need more data on the biological side to pin down what's missing.

Marcus: So for someone listening who isn't in the lab, this means that when you see choices becoming more varied with more time ahead, it might just be true randomness rather than a predictable pattern we can easily model <ref:2611.17011#pg15>. It's not that our models are just too simple; they are limited by what we feed them.

Yuki: From a population perspective, if the sample size is limited to one thousand three hundred ninety-two participants and over a million trials <ref:2611.17011#pg4>, Yuki wonders how much of the true diversity in decision making is being represented by this specific set of choices <ref:2611.17011#pg1>. The findings support that randomness is substantial <ref:2611.17011#pg15>.

Ines: So to wrap up on "Random Exploration is Random," the authors are essentially saying that for these long-horizon decisions, the unexplained variability isn't a hidden deterministic signal waiting to be found in neural networks <ref:2611.17011#pg3>. It’s just random noise we haven't accounted for yet <ref:2611.17011#pg15>.

Marcus: And I think what this means for the field is that we need to stop looking for a hidden rule governing long-term exploration variability and start accepting it as genuinely stochastic, which puts constraints on how much we can model behavior accurately <ref:2611.17011#pg3>.

Yuki: It suggests that the complexity of decision making might be rooted in genuine randomness rather than just deterministic processes operating within a known set of rules <ref:2611.17011#pg3>. That's a big shift for how we view complex biological systems and human cognition alike.

Conclusion: Ines: So we've been looking at how choice variability in random exploration might be random noise instead of some hidden pattern <ref:2611.17011#pg15>.

Marcus: Right, and that paper calls itself "Random Exploration is Random," and it's focused on whether that extra variability under long horizons is genuine randomness or something else entirely.

Yuki: From a population genetics standpoint, the authors are testing if this variability reflects real stochasticity or some kind of deterministic structure that traditional cognitive models miss <ref:2611.17011#pg3>.

Ines: The core finding they land on is that limited model expressiveness simply can't account for that extra variability in long-horizon choices <ref:2611.17011#pg4>.

Marcus: They used deep neural networks to see if those more expressive models could recover any deterministic structure from the data, and while they fit the behavior better than old models, those long-horizon choices stayed consistently less predictable than short horizon choices <ref:2611.17011#pg4>.

Yuki: It means that what you observe when people are exploring randomly isn't being driven by some hidden rule we can find with current tools <ref:2611.17011#pg15>.

Ines: Exactly, the paper suggests that this residual variability is consistent with a genuinely stochastic component of random exploration rather than something else <ref:2611.17011#pg15>.

Marcus: So what does this mean for decision science? It implies that when you see increased choice variability over time, you shouldn't automatically look for a complex deterministic mechanism to explain it <ref:2611.17011#pg3>.

Yuki: It suggests that the complexity of decision making might actually be rooted in genuine randomness rather than just deterministic processes running on a set of known rules <ref:2611.17011#pg3>.

Ines: But they did flag a limitation, and that’s important: even those highly expressive models can only capture deterministic structure that is present in the measured data because unmeasured factors could explain the rest <ref:2611.17011#pg10>.

Marcus: So to wrap up on this paper, the authors conclude that the variability observed during long-horizon exploration cannot be explained by limited model expressiveness alone <ref:2611.17011#pg3>.

Yuki: It's a call to stop searching for a hidden rule in that randomness and instead accept it as truly stochastic <ref:2611.17011#pg15>.

Ines: And they point toward needing richer biological measurements, like pupil dilation, to see if we can actually explain more of that remaining variability <ref:2611.17011#pg10>.

Yingkang Luo, Hua-Dong Xiong, Hanbo Xie, Caroline Phelps, Madeline Harms, Ryan Smith, James Waltz, Robert Wilson

College of Computing, Georgia Institute of Technology. · School of Psychology, Georgia Institute of Technology. · Department of Psychology, University of Minnesota Duluth. · Laureate Institute for Brain Research. · School of Medicine, University of Maryland.

q-bio.NC, cs.LG

Submitted: 2026-11-12

Updated: 2026-11-12

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 84/100

The gist: Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives <ref:2611.17011#pg3> The paper examines whether variability in random exploration…

Key concepts

Random Exploration
This refers to decision-making where choices become more variable and less dependent on the currently 'best' option. The researchers investigated whether this variability is purely due to chance or if it follows a hidden pattern.
Horizon Task
This task involves participants choosing between two slot machines to maximize rewards by balancing exploiting known options and exploring uncertain alternatives. Variability in choices increases as the future number of possible choices (the horizon) grows longer.
Model Expressiveness
This refers to the complexity or capacity of a mathematical model, like a neural network, to capture underlying patterns in behavior. The study tested if more expressive models could uncover deterministic structures hidden within the observed variability.

Summary

Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives <ref:2611. p. 0

Where the paper says it

org; JWaltz@som. edu; Abstract Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives.

17011#pg3> The paper examines whether variability in random exploration reflects genuine stochasticity or a deterministic structure that traditional cognitive models fail to capture <ref:2611. p. 0
Where the paper says it

One strategy for exploration is random exploration, in which choices become more variable and less tied to the option that currently appears best. However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture.

17011#pg3>

The core finding

Limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random <ref:2611.17011#pg3> This conclusion is drawn from examining the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials <ref:2611. p. 0

Where the paper says it

However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture. We examined this question using the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials.

17011#pg4>

Task and Model Comparison

The Horizon Task involves participants choosing between two slot machines to maximize total reward by balancing exploitation and exploration <ref:2611.17011#pg5> The study utilized deep neural networks to test if they could recover deterministic structure from the additional variability observed in long-horizon choices <ref:2611. p. 0

Where the paper says it

We tested whether more expressive neural network models could recover deterministic structure from this additional variability. Although neural networks provided better overall fits to behavior than the traditional model, long-horizon choices remained consistently less predictable than short-horizon choices.

17011#pg4> Neural network models captured more behavioral structure than the traditional logistic model, increasing McFadden’s pseudo-R2 by 0. p. 6
Where the paper says it

The neural network models captured more behavioral structure than the baseline logistic model. We compared model fit using McFadden’s pseudo-R2 relative to the null model (Table 2).

095–0.168 relative to the logistic regression baseline <ref:2611. p. 6
Where the paper says it

Overall, the neural network models increased McFadden’s pseudo-R2 by 0. 168 relative to the logistic regression baseline.

17011#pg9>

Testing Deterministic Structure

The researchers systematically tested various sources of deterministic structure, including information from the current game and subject identity <ref:2611.17011#pg4> They also investigated whether the H6–H1 predictability gap arose from learning or strategy adaptation across repeated games, finding that it was present even when restricting analysis to the first game played by each participant <ref:2611. p. 9

Where the paper says it

6 Unexplained variance of the neural network models increases from the very first game Behavior may change as participants gain experience with the task, raising the possibility that the H1–H6 predictability gap develops through learning or strategy adaptation across repeated games [34]. To test this possibility, we repeated the analysis using only the first game played by each participant.

17011#pg9> Cross-game history effects were also tested using recurrent neural networks but did not eliminate the predictability gap between H1 and H6 <ref:2611. p. 4
Where the paper says it

We next tested for cross-game history effects using RNNs with a learned persistence parameter that carried information forward from previous games within each participant. Finally, we tracked predictive performance across all six free-choice trials in H6 to examine how predictability changed as the remaining horizon shortened.

17011#pg9>

Limitations and Future Directions

A key limitation acknowledged is that even highly expressive models can only capture deterministic structure present in the measured data, as unmeasured factors like internal states or spontaneous movements could account for unexplained variability <ref:2611.17011#pg10> Future work is suggested to incorporate richer behavioral and physiological measurements, such as eye tracking or pupil dilation, to provide a stronger test of how much of the remaining variability can be explained <ref:2611. p. 4

Where the paper says it

Finally, we tracked predictive performance across all six free-choice trials in H6 to examine how predictability changed as the remaining horizon shortened. 3 Unexplained variance of the traditional cognitive model increases with horizon We first fitted the standard Horizon Task logistic model to establish a baseline for behavioral variability explained by the model.

17011#pg10>

Conclusion on Randomness

Taken together, findings suggest that the additional variability observed during long-horizon exploration cannot be explained by limited model expressiveness alone, and this residual variability is consistent with a genuinely stochastic component of random exploration <ref:2611.17011#pg15> The lower predictability of H6 choices persists after accounting for several sources of deterministic structure available in the Horizon Task <ref:2611. p. 4

Where the paper says it

We then used neural networks (NNs) to capture additional predictable structure available in the task. To reduce model-specific limitations, we evaluated several architectures with different representational capacities.

17011#pg4>

REFERENCE

[5] Bellman, R.: A problem in the sequential design of experiments. p. 2

Where the paper says it

Some apparently random variability may therefore be systematic but unmodeled, making the model’s expressive power a key limitation. The current study addresses this problem using deep neural networks in previously collected Horizon Task data with over one million trials [1, 36–42].

Sankhy¯a: The Indian Journal of Statistics (1933-1960) 16(3/4), 221–229 (1956) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg5>

[8] Averbeck, B.B.: Theory of choice in bandit, information sampling and foraging tasks. p. 3

Where the paper says it

On the right, the instructed trials reveal an equal amount of information per machine [2 2]. Analyses focus on the first free-choice trial, highlighted in green.

PLoS computational biology 11(3), 1004164 (2015) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[9] Wilson, R.C., Bonawitz, E., Costa, V.D., Ebitz, R.B.: Balancing exploration and exploitation with information and randomization. p. 0

Where the paper says it

org; JWaltz@som. edu; Abstract Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives.

Current Opinion in Behavioral Sciences 38, 49–56 (2021) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg3>

[33] Wang, S., Wilson, R. p. 8

Where the paper says it

Each point shows the test accuracy of one architecture, computed separately for H1 and H6 after restoring the checkpoint with the lowest validation loss. Horizontal error bars indicate 95% Wilson confidence intervals.

p. 8
Where the paper says it

Each point shows the test accuracy of one architecture, computed separately for H1 and H6 after restoring the checkpoint with the lowest validation loss. Horizontal error bars indicate 95% Wilson confidence intervals.

p. 8
Where the paper says it

Each point shows the test accuracy of one architecture, computed separately for H1 and H6 after restoring the checkpoint with the lowest validation loss. Horizontal error bars indicate 95% Wilson confidence intervals.

p. 8
Where the paper says it

Each point shows the test accuracy of one architecture, computed separately for H1 and H6 after restoring the checkpoint with the lowest validation loss. Horizontal error bars indicate 95% Wilson confidence intervals.

p. 8
Where the paper says it

Each point shows the test accuracy of one architecture, computed separately for H1 and H6 after restoring the checkpoint with the lowest validation loss. Horizontal error bars indicate 95% Wilson confidence intervals.

p. 8
Where the paper says it

Each point shows the test accuracy of one architecture, computed separately for H1 and H6 after restoring the checkpoint with the lowest validation loss. Horizontal error bars indicate 95% Wilson confidence intervals.

C.: Separating random and deterministic sources of computational noise in explore-exploit decisions. p. 1
Where the paper says it

There are also spontaneous moments when we act without quite knowing what is driving us: the server arrives, and suddenly we hear ourselves ordering the fish-head soup! Every time we make a decision, we are facing the explore-exploit dilemma: choosing between exploiting what we already know and exploring uncertain alternatives that may improve future decisions [1–4].

p. 1
Where the paper says it

There are also spontaneous moments when we act without quite knowing what is driving us: the server arrives, and suddenly we hear ourselves ordering the fish-head soup! Every time we make a decision, we are facing the explore-exploit dilemma: choosing between exploiting what we already know and exploring uncertain alternatives that may improve future decisions [1–4].

PLOS Computational Biology 22, 1014026 (2026) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg4>

[35] Gershman, S.J.: Uncertainty and exploration. p. 1

Where the paper says it

Directed exploration involves deliberately favoring more informative options. Random exploration involves noisier choices that are less tied to expected value or the degree of uncertainty [5, 6, 9–19].

p. 1
Where the paper says it

Directed exploration involves deliberately favoring more informative options. Random exploration involves noisier choices that are less tied to expected value or the degree of uncertainty [5, 6, 9–19].

Decision 6(3), 277–286 (2019) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg4>

[44] Hornik, K.: Approximation capabilities of multilayer feedforward networks. p. 4

Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

p. 4
Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

p. 4
Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

p. 4
Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

22 Neural networks 4(2), 251–257 (1991) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[46] Peterson, J.C., Bourgin, D.D., Agrawal, M., Reichman, D., Griffiths, T.L.: Using large-scale experiments and machine learning to discover theories of human decision-making. p. 1

Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

Science 372(6547), 1209–1214 (2021) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[53] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. p. 7

Where the paper says it

0952 Transformer 0. 1595 Self-Attention 0.

p. 7
Where the paper says it

0952 Transformer 0. 1595 Self-Attention 0.

p. 7
Where the paper says it

0952 Transformer 0. 1595 Self-Attention 0.

p. 7
Where the paper says it

0952 Transformer 0. 1595 Self-Attention 0.

Advances in neural information processing systems 30 (2017) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[54] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. p. 4

Where the paper says it

Then, we examined whether this additional predictive capacity reduced the H1–H6 predictability gap across several complementary analyses. In the simplest case, we modeled behavior at the pooled-population level using only the instructed-trial choice–reward sequence, horizon condition, and uncertainty condition.

p. 4
Where the paper says it

Then, we examined whether this additional predictive capacity reduced the H1–H6 predictability gap across several complementary analyses. In the simplest case, we modeled behavior at the pooled-population level using only the instructed-trial choice–reward sequence, horizon condition, and uncertainty condition.

p. 4
Where the paper says it

Then, we examined whether this additional predictive capacity reduced the H1–H6 predictability gap across several complementary analyses. In the simplest case, we modeled behavior at the pooled-population level using only the instructed-trial choice–reward sequence, horizon condition, and uncertainty condition.

p. 4
Where the paper says it

Then, we examined whether this additional predictive capacity reduced the H1–H6 predictability gap across several complementary analyses. In the simplest case, we modeled behavior at the pooled-population level using only the instructed-trial choice–reward sequence, horizon condition, and uncertainty condition.

Advances in Neural Information Processing Systems 4, 3104–3112 (2014) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[55] Gelfand, A.E., Smith, A.F.M.: Sampling-based approaches to calculating marginal densities. p. 4

Where the paper says it

The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants. We also evaluated Transformer-based models to capture flexible dependencies across trials.

Journal of the American Statistical Association 85(410), 398–409 (1990) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg3>

[56] Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. p. 1

Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-6(6), 721–741 (1984) <ref:2611. p. 1
Where the paper says it

Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25]. Consistently choosing the option with the highest expected value risks becoming trapped in a familiar but suboptimal pattern.

17011#pg3>

[57] Faisal, A.A., Selen, L.P., Wolpert, D.M.: Noise in the nervous system. p. 1

Where the paper says it

Random exploration involves noisier choices that are less tied to expected value or the degree of uncertainty [5, 6, 9–19]. This noise can draw an individual away from the option with the highest expected reward.

Nature reviews neuroscience 9(4), 292–303 (2008) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg3>

[58] Beck, J.M., Ma, W.J., Pitkow, X., Latham, P.E., Pouget, A.: Not noisy, just wrong: The role of suboptimal inference in behavioral variability. p. 1

Where the paper says it

1 Introduction When dining out, do you order a familiar favorite or try something different? You can’t go wrong with a steak, but the sea urchin stir-fry might taste even better.

Neuron 74, 30–39 (2012) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg7>

[59] Wang, S., Wilson, R.C.: Separating random and deterministic sources of computational noise in explore-exploit decisions. PLOS Computational Biology 22, 1014026 (2026) <ref:2611.17011#pg4>

[60] Gershman, S.J.: Uncertainty and exploration. Decision 6(3), 277–286 (2019) <ref:2611.17011#pg4>

[45] Cybenko, G.: Approximation by superpositions of a sigmoidal function. p. 2

Where the paper says it

The current study addresses this problem using deep neural networks in previously collected Horizon Task data with over one million trials [1, 36–42]. Neural networks are powerful function approximators with scalable numbers of trainable parameters and flexible architectures [43–45].

p. 2
Where the paper says it

The current study addresses this problem using deep neural networks in previously collected Horizon Task data with over one million trials [1, 36–42]. Neural networks are powerful function approximators with scalable numbers of trainable parameters and flexible architectures [43–45].

p. 2
Where the paper says it

The current study addresses this problem using deep neural networks in previously collected Horizon Task data with over one million trials [1, 36–42]. Neural networks are powerful function approximators with scalable numbers of trainable parameters and flexible architectures [43–45].

p. 2
Where the paper says it

The current study addresses this problem using deep neural networks in previously collected Horizon Task data with over one million trials [1, 36–42]. Neural networks are powerful function approximators with scalable numbers of trainable parameters and flexible architectures [43–45].

Mathematics of Control, Signals and Systems 2, 303–314 (1989) <ref:2611.17011#pg9>

[47] Musall, S., Kaufman, M.T., Juavinett, A.L., Gluf, S., Churchland, A.K.: Singletrial neural dynamics are dominated by richly varied movements. Nature Neuroscience 22(10), 1677–1686 (2019) <ref:2611. p. 0

Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg10>

[48] Li, J.-A., Benna, M.K., Mattar, M.G.: Discovering cognitive strategies with tiny recurrent neural networks. p. 4

Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

p. 4
Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

p. 4
Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

p. 4
Where the paper says it

To reduce model-specific limitations, we evaluated several architectures with different representational capacities. The multilayer perceptron (MLP) received all trial information simultaneously, whereas recurrent neural networks (RNNs) processed choices and rewards sequentially, similar to how information was presented to participants.

Nature (2025) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[49] Wang, S., Wilson, R.C., Ebner, N.C., Lighthall, N.R.: Modeling value integration during decision making under uncertainty with the florida and georgia gambling task. Scientific Reports 15, 36826 (2025) <ref:2611. p. 0

Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[50] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. p. 1

Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

p. 1
Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

p. 1
Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

p. 1
Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

p. 1
Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

p. 1
Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

p. 1
Where the paper says it

Although introducing randomness, or stochasticity into decision-making might seem counterproductive, it provides an effective and computationally inexpensive way to explore. Stochastic methods are therefore widely used in machine-learning algorithms for search and optimization [20–25].

3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[52] Hochreiter, S., Schmidhuber, J.: Long short-term memory. p. 8

Where the paper says it

Dashed vertical lines indicate the corresponding horizon-specific test accuracy of the logistic regression baseline. Abbreviations: MLP, multilayer perceptron; RNN, recurrent neural network; LSTM, long short-term memory; GRU, gated recurrent unit; H1, Horizon 1; H6, Horizon 6.

p. 8
Where the paper says it

Dashed vertical lines indicate the corresponding horizon-specific test accuracy of the logistic regression baseline. Abbreviations: MLP, multilayer perceptron; RNN, recurrent neural network; LSTM, long short-term memory; GRU, gated recurrent unit; H1, Horizon 1; H6, Horizon 6.

p. 8
Where the paper says it

Dashed vertical lines indicate the corresponding horizon-specific test accuracy of the logistic regression baseline. Abbreviations: MLP, multilayer perceptron; RNN, recurrent neural network; LSTM, long short-term memory; GRU, gated recurrent unit; H1, Horizon 1; H6, Horizon 6.

Neural computation 9(8), 1735–1780 (1997) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg9>

[53] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) <ref:2611.17011#pg9>

[54] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 4, 3104–3112 (2014) <ref:2611.17011#pg9>

[57] Ebitz, R.B., Albarran, E., Moore, T.: Exploration disrupts choice-predictive signals and alters dynamics in prefrontal cortex. p. 1

Where the paper says it

Neural activity is inherently variable, and this variability can produce stochastic behavior [26]. During random exploration, monkeys show increased neural variability and weaker choice-related signals in prefrontal decision-making circuits [27].

Neuron 97(2), 450–4619 (2018) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg3>

[58] Tervo, D.G.R., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., Karpova, A.Y.: Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. p. 1

Where the paper says it

Neural activity is inherently variable, and this variability can produce stochastic behavior [26]. During random exploration, monkeys show increased neural variability and weaker choice-related signals in prefrontal decision-making circuits [27].

p. 1
Where the paper says it

Neural activity is inherently variable, and this variability can produce stochastic behavior [26]. During random exploration, monkeys show increased neural variability and weaker choice-related signals in prefrontal decision-making circuits [27].

p. 1
Where the paper says it

Neural activity is inherently variable, and this variability can produce stochastic behavior [26]. During random exploration, monkeys show increased neural variability and weaker choice-related signals in prefrontal decision-making circuits [27].

Cell 159, 21–32 (2014) <ref:2611. p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

p. 0
Where the paper says it

These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random. Keywords: Random Exploration, Neural Networks, Explore-exploit, Decision-making 1 arXiv:2611.

17011#pg10>

[63] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611.17011#pg9>

[64] Hornik, K.: Approximation capabilities of multilayer feedforward networks. 22 Neural networks 4(2), 251–257 (1991) <ref:2611.17011#pg9>

[65] Cybenko, G.: Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems 2, 303–314 (1989) <ref:2611.17011#pg9>

[47] Musall, S., Kaufman, M.T., Juavinett, A.L., Gluf, S., Churchland, A.K.: Singletrial neural dynamics are dominated by richly varied movements. Nature Neuroscience 22(10), 1677–1686 (2019) <ref:2611.17011#pg10>

[48] Li, J.-A., Benna, M.K., Mattar, M.G.: Discovering cognitive strategies with tiny recurrent neural networks. Nature (2025) <ref:2611.17011#pg9>

[49] Wang, S., Wilson, R.C., Ebner, N.C., Lighthall, N.R.: Modeling value integration during decision making under uncertainty with the florida and georgia gambling task. Scientific Reports 15, 36826 (2025) <ref:2611.17011#pg9>

[50] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611.17011#pg9>

[52] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997) <ref:2611.17011#pg9>

[53] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) <ref:2611.17011#pg9>

[54] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 4, 3104–3112 (2014) <ref:2611.17011#pg9>

[58] Tervo, D.G.R., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., Karpova, A.Y.: Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21–32 (2014) <ref:2611.17011#pg10>

[63] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611.17011#pg9>

[64] Hornik, K.: Approximation capabilities of multilayer feedforward networks. 22 Neural networks 4(2), 251–257 (1991) <ref:2611.17011#pg9>

[65] Cybenko, G.: Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems 2, 303–314 (1989) <ref:2611.17011#pg9>

[47] Musall, S., Kaufman, M.T., Juavinett, A.L., Gluf, S., Churchland, A.K.: Singletrial neural dynamics are dominated by richly varied movements. Nature Neuroscience 22(10), 1677–1686 (2019) <ref:2611.17011#pg10>

[48] Li, J.-A., Benna, M.K., Mattar, M.G.: Discovering cognitive strategies with tiny recurrent neural networks. Nature (2025) <ref:2611.17011#pg9>

[49] Wang, S., Wilson, R.C., Ebner, N.C., Lighthall, N.R.: Modeling value integration during decision making under uncertainty with the florida and georgia gambling task. Scientific Reports 15, 36826 (2025) <ref:2611.17011#pg9>

[50] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611.17011#pg9>

[52] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8), 1735–1780 (1997) <ref:2611.17011#pg9>

[53] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) <ref:2611.17011#pg9>

[54] Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 4, 3104–3112 (2014) <ref:2611.17011#pg9>

[58] Tervo, D.G.R., Proskurin, M., Manakov, M., Kabra, M., Vollmer, A., Branson, K., Karpova, A.Y.: Behavioral variability through stochastic choice and its gating by anterior cingulate cortex. Cell 159, 21–32 (2014) <ref:2611.17011#pg10>

[63] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611.17011#pg9>

[64] Hornik, K.: Approximation capabilities of multilayer feedforward networks. 22 Neural networks 4(2), 251–257 (1991) <ref:2611.17011#pg9>

[65] Cybenko, G.: Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems 2, 303–314 (1989) <ref:2611.17011#pg9>

[47] Musall, S., Kaufman, M.T., Juavinett, A.L., Gluf, S., Churchland, A.K.: Singletrial neural dynamics are dominated by richly varied movements. Nature Neuroscience 22(10), 1677–1686 (2019) <ref:2611.17011#pg10>

[48] Li, J.-A., Benna, M.K., Mattar, M.G.: Discovering cognitive strategies with tiny recurrent neural networks. Nature (2025) <ref:2611.17011#pg9>

[49] Wang, S., Wilson, R.C., Ebner, N.C., Lighthall, N.R.: Modeling value integration during decision making under uncertainty with the florida and georgia gambling task. Scientific Reports 15, 36826 (2025) <ref:2611.17011#pg9>

[50] Kingma, D.P., Ba, J.L.: Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings (2014) <ref:2611.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Recurrent neural networks can capture sequential dependencies within a game by propagating hidden states from one game to the next, allowing models to test whether influences from earlier games contributed to the unexplained variability.

  2. Transformer-based models can be used to capture flexible dependencies across trials, as they allow for attention mechanisms that process sequential inputs in a way that might be better suited for complex exploration patterns.

  3. Incorporating subject identity into neural network models allows systems to capture individual behavioral patterns, enabling more personalized decision-making strategies based on prior experiences.

Abstract

Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives. One strategy for exploration is random exploration, in which choices become more variable and less tied to the option that currently appears best. However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture. We examined this question using the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials. In this task, exploration becomes more valuable when participants have more future choices, and choice variability increases under longer horizons. We tested whether more expressive neural network models could recover deterministic structure from this additional variability. Although neural networks provided better overall fits to behavior than the traditional model, long-horizon choices remained consistently less predictable than short-horizon choices. These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random.

Sources

Related papers