Random Exploration is Random

summary

The gist

Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives <ref:2611.17011#pg3> The paper examines whether variability in random exploration…

In short

The study used a Horizon Task to see if increased variability in long-term choices reflects true randomness or a hidden deterministic structure. Deep neural networks showed better fit than traditional models, but long-horizon choices remained less predictable. This suggests that limited model expressiveness cannot fully explain the variability, implying random exploration is substantially random.

Key concepts

Random Exploration
This refers to decision-making where choices become more variable and less dependent on the currently 'best' option. The researchers investigated whether this variability is purely due to chance or if it follows a hidden pattern.
Horizon Task
This task involves participants choosing between two slot machines to maximize rewards by balancing exploiting known options and exploring uncertain alternatives. Variability in choices increases as the future number of possible choices (the horizon) grows longer.
Model Expressiveness
This refers to the complexity or capacity of a mathematical model, like a neural network, to capture underlying patterns in behavior. The study tested if more expressive models could uncover deterministic structures hidden within the observed variability.

Terminology used across episodes

This episode discusses

The paper

Random Exploration is Random · Read on arXiv

Yingkang Luo, Hua-Dong Xiong, Hanbo Xie, Caroline Phelps, Madeline Harms, Ryan Smith, James Waltz, Robert Wilson

College of Computing, Georgia Institute of Technology. · School of Psychology, Georgia Institute of Technology. · Department of Psychology, University of Minnesota Duluth. · Laureate Institute for Brain Research. · School of Medicine, University of Maryland.

Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives. One strategy for exploration is random exploration, in which choices become more variable and less tied to the option that currently appears best. However, it remains unclear whether this variability reflects genuine stochasticity or instead a deterministic structure that traditional cognitive models fail to capture. We examined this question using the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials. In this task, exploration becomes more valuable when participants have more future choices, and choice variability increases under longer horizons. We tested whether more expressive neural network models could recover deterministic structure from this additional variability. Although neural networks provided better overall fits to behavior than the traditional model, long-horizon choices remained consistently less predictable than short-horizon choices. These findings show that limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random.

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: We're looking at the paper: "Random Exploration is Random".

Marcus: Here's the gist of the paper. Decision making often requires balancing exploitation of familiar options with exploration of uncertain alternatives <ref:2611.17011#pg3> The paper examines whether variability in random exploration reflects genuine stochasticity or a deterministic structure that traditional cognitive models fail to capture <ref:2611.17011#pg3> The core finding Limited model expressiveness cannot fully account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random <ref:2611.17011#pg3> This conclusion is drawn from examining the Horizon Task in an aggregated dataset of 1,392 participants completing over 1,270,695 trials <ref:2611.17011#pg4> Task and Model Comparison The Horizon Task involves participants choosing between two slot machines to maximize total reward by balancing exploitation and exploration <ref:2611.17011#pg5> The study utilized deep neural networks to test if they could recover deterministic structure from the additional variability observed in long-horizon choices <ref:2611.17011#pg4> Neural network models captured more behavioral structure than the traditional logistic model, increasing McFadden’s pseudo-R2 by 0.095–0.168 relative to the logistic regression baseline <ref:2611.17011#pg9> Testing Deterministic Structure The researchers systematically tested various sources of deterministic structure, including information from the current game and subject identity <ref:2611.17011#pg4> They also investigated whether the H6–H1 predictability gap arose from learning or strategy adaptation across repeated games, finding that it was present even when restricting analysis to the first game played by each participant <ref:2611.17011#pg9> Cross-game history effects were also tested using recurrent neural networks but did not eliminate the predictability gap between H1 and H6 <ref:2611.17011#pg9> Limitations and Future Directions A key limitation acknowledged is that even highly expressive models can only capture deterministic structure present in the measured data, as unmeasured factors like internal states or spontaneous movements could account for unexplained variability <ref:2611.17011#pg10> Future work is suggested to incorporate richer behavioral and physiological measurements, such as eye tracking or pupil dilation, to provide a stronger test of how much of the remaining variability can be explained <ref:2611.17011#pg10> Taken together, findings suggest that the additional variability observed during long-horizon exploration cannot be explained by limited model expressiveness alone, and this residual variability is consistent with a genuinely stochastic component of random exploration <ref:2611.17011#pg15> The lower predictability of H6 choices persists after accounting for several sources of deterministic structure available in the Horizon Task <ref:2611.17011#pg4> REFERENCE

5: Bellman, R.: A problem in the sequential design of experiments.

Ines: Sankhy¯a: The Indian Journal of Statistics (1933-1960) 16(3/4), 221–229 (1956) <ref:2611.17011#pg5>

8: Averbeck, B.B.: Theory of choice in bandit, information sampling and foraging tasks.

Marcus: PLoS computational biology 11(3), 1004164 (2015) <ref:2611.17011#pg9>

9: Wilson, R.C., Bonawitz, E., Costa, V.D., Ebitz, R.B.: Balancing exploration and exploitation with information and randomization.

Ines: Current Opinion in Behavioral Sciences 38, 49–56 (2021) <ref:2611.17011#pg3>

33: Wang, S., Wilson, R.C.: Separating random and deterministic sources of computational noise in explore-exploit decisions.

Paper summary: Ines: So we're looking at this paper called "Random Exploration is Random" and it's asking a really fundamental question about decision making. The core idea is whether the variability in choices when people explore randomly actually comes from genuine randomness or if there's some underlying deterministic structure that our current cognitive models just can't see.

Marcus: Right, and it sets up this comparison using a task called the Horizon Task where participants have to balance exploiting what they know with exploring uncertain options. The paper is investigating if that variability we see in exploration choices reflects true stochasticity or something else entirely.

Yuki: From a population genetics view, I wonder if this speaks to how species adapt to novel environments versus just following random noise in their gene pool. If the choice variability isn't truly random, it suggests there's a hidden mechanism driving those choices that we need to account for.

Ines: Exactly. The paper examines this using an aggregated dataset of one thousand three hundred ninety-two participants who completed over one million two hundred seventy thousand six hundred ninety-five trials in this task <ref:2611.17011#pg1>. They found that exploration becomes more valuable when participants have more future choices and choice variability increases under longer horizons <ref:2611.17011#pg3>.

Marcus: And they used deep neural network models to see if those expressive models could capture any of this underlying deterministic structure from the extra variability seen in long-horizon choices <ref:2611.17011#pg4>. The results showed that these neural networks did capture more behavioral structure than the traditional logistic model, increasing McFadden’s pseudo R squared by zero point zero nine five to one six eight relative to the logistic regression baseline <ref:2611.17011#pg9>.

Yuki: So the models were better at fitting the behavior, but they still couldn't fully explain that increased variability in those longer-horizon choices <ref:2611.17011#pg4>. That suggests that what's left over isn't just noise from a simple decision-noise parameter in traditional models <ref:2611.17011#pg3>.

Ines: That’s what they conclude: limited model expressiveness simply cannot account for the increased variability associated with random exploration, suggesting random exploration truly is substantially random <ref:2611.17011#pg15>. It points toward a genuinely stochastic component of that exploration behavior.

Marcus: So essentially, even when you try to use more complex AI models to find a deterministic pattern in this kind of randomness, you run into a wall because the structure isn't there enough to capture <ref:2611.17011#pg3>. This means random exploration is likely substantially random <ref:2611.17011#pg15>.

Yuki: It changes how we think about uncertainty. If it's not fully deterministic, then the uncertainty isn't just an artifact of our incomplete knowledge, but something inherent in the process itself. It implies a level of randomness that resists simple capture by current theoretical frameworks <ref:2611.17011#pg3>.

Ines: And they also looked at different sources of structure, like information from the current game or subject identity <ref:2611.17011#pg4>. They also checked if this predictability gap between short horizon choices and long horizon choices came from learning or just adapting strategy across repeated games <ref:2611.17011#pg9>.

Marcus: Interestingly, even when they looked at cross-game history effects using recurrent neural networks, they didn't eliminate that predictability gap between the short and long horizons <ref:2611.17011#pg9>. That suggests the difference remains even when you account for past interactions within the same game structure.

Yuki: The limitation they mention is important here: even highly expressive models can only capture deterministic structure present in the measured data because unmeasured factors like internal states or spontaneous movements could explain some of that unexplained variability <ref:2611.17011#pg10>. So the model is only as good as what we measure.

Ines: Right, and they suggest future work should involve incorporating richer behavioral and physiological measurements, like eye tracking or pupil dilation, to provide a stronger test of how much of that remaining variability can actually be explained <ref:2611.17011#pg10>. They need more data on the biological side to pin down what's missing.

Marcus: So for someone listening who isn't in the lab, this means that when you see choices becoming more varied with more time ahead, it might just be true randomness rather than a predictable pattern we can easily model <ref:2611.17011#pg15>. It's not that our models are just too simple; they are limited by what we feed them.

Yuki: From a population perspective, if the sample size is limited to one thousand three hundred ninety-two participants and over a million trials <ref:2611.17011#pg4>, Yuki wonders how much of the true diversity in decision making is being represented by this specific set of choices <ref:2611.17011#pg1>. The findings support that randomness is substantial <ref:2611.17011#pg15>.

Ines: So to wrap up on "Random Exploration is Random," the authors are essentially saying that for these long-horizon decisions, the unexplained variability isn't a hidden deterministic signal waiting to be found in neural networks <ref:2611.17011#pg3>. It’s just random noise we haven't accounted for yet <ref:2611.17011#pg15>.

Marcus: And I think what this means for the field is that we need to stop looking for a hidden rule governing long-term exploration variability and start accepting it as genuinely stochastic, which puts constraints on how much we can model behavior accurately <ref:2611.17011#pg3>.

Yuki: It suggests that the complexity of decision making might be rooted in genuine randomness rather than just deterministic processes operating within a known set of rules <ref:2611.17011#pg3>. That's a big shift for how we view complex biological systems and human cognition alike.

Conclusion: Ines: So we've been looking at how choice variability in random exploration might be random noise instead of some hidden pattern <ref:2611.17011#pg15>.

Marcus: Right, and that paper calls itself "Random Exploration is Random," and it's focused on whether that extra variability under long horizons is genuine randomness or something else entirely.

Yuki: From a population genetics standpoint, the authors are testing if this variability reflects real stochasticity or some kind of deterministic structure that traditional cognitive models miss <ref:2611.17011#pg3>.

Ines: The core finding they land on is that limited model expressiveness simply can't account for that extra variability in long-horizon choices <ref:2611.17011#pg4>.

Marcus: They used deep neural networks to see if those more expressive models could recover any deterministic structure from the data, and while they fit the behavior better than old models, those long-horizon choices stayed consistently less predictable than short horizon choices <ref:2611.17011#pg4>.

Yuki: It means that what you observe when people are exploring randomly isn't being driven by some hidden rule we can find with current tools <ref:2611.17011#pg15>.

Ines: Exactly, the paper suggests that this residual variability is consistent with a genuinely stochastic component of random exploration rather than something else <ref:2611.17011#pg15>.

Marcus: So what does this mean for decision science? It implies that when you see increased choice variability over time, you shouldn't automatically look for a complex deterministic mechanism to explain it <ref:2611.17011#pg3>.

Yuki: It suggests that the complexity of decision making might actually be rooted in genuine randomness rather than just deterministic processes running on a set of known rules <ref:2611.17011#pg3>.

Ines: But they did flag a limitation, and that’s important: even those highly expressive models can only capture deterministic structure that is present in the measured data because unmeasured factors could explain the rest <ref:2611.17011#pg10>.

Marcus: So to wrap up on this paper, the authors conclude that the variability observed during long-horizon exploration cannot be explained by limited model expressiveness alone <ref:2611.17011#pg3>.

Yuki: It's a call to stop searching for a hidden rule in that randomness and instead accept it as truly stochastic <ref:2611.17011#pg15>.

Ines: And they point toward needing richer biological measurements, like pupil dilation, to see if we can actually explain more of that remaining variability <ref:2611.17011#pg10>.

More episodes

← Home