Agentic AI in Automated Markets
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Economics. The day in economics: theory, mechanisms and the evidence beneath them.
Ruth: We're looking at the paper: "Agentic AI in Automated Markets".
Warren: Here's the gist of the paper. Automated markets increasingly host autonomous, learning trading agents and are powered by formulaic automated market makers (AMMs) These agentic AI systems are studied to understand how heterogeneous agent behavior shapes stress, risk-adjusted performance, liquidity, and volatility in these types of markets The pilot on hourly cryptocurrency data motivates hypotheses about the distinct footprints left by rational arbitrageurs, manipulators, and retail herd-followers Agent Archetypes and Modeling Three behavioral archetypes are modeled as agentic reinforcement learning (RL) systems: a rational arbitrageur, a manipulator, and a retail herd-follower Each archetype is defined by its reward function within a common, asset-agnostic exposure-control environment These archetypes are trained with a value-based Deep Q-Network (DQN) and with REINFORCE using a learned baseline The rational arbitrageur maximizes risk-adjusted return with additional penalties on realized volatility and turnover The manipulator receives a bonus proportional to turnover, κturn et−et−1, and a short discount horizon, which favors rapid position cycling and short-term gains The retail herd-follower receives a bonus for aligning exposure with recent momentum, κmom sign(momt) (et − 0.5) Trading is formulated as a discrete exposure-control Markov decision process with riskadjusted, cost-aware rewards and market-stress penalties defined over features available in any market The base reward is a volatility-normalized net return: rbase t = etRt+1 − c et − et−1 σˆt + ϵ - λstress 1
stresst: et DQN is a value-based method that estimates Q(s, a) for the three actions REINFORCE with baseline uses a two-layer policy network and a separate two-layer value network to estimate the expected discounted return and serve as a baseline Pilot Findings on Market Stress The pilot on hourly cryptocurrency data motivates hypotheses about the distinct footprints left by rational arbitrageurs, manipulators, and retail herd-followers Retail agents show the most frequent stress episodes, but these are short, consistent with shallow drawdowns from momentum overreaction Rational agents show a moderate frequency, with controlled exposure limiting both occurrence and duration Manipulators show the fewest episodes, but stress persists the longest when it occurs, most visibly around the major crash regimes near hours 1,450–1,500 Regulatory Implications The pilot suggests that the regulation of automated and agentic trading should target behaviors rather than apply uniform limits on volatility or turnover Recommendations include: 1.
Ruth: For herding (H1), tools such as risk disclosure at the point of trade, leverage and margin limits for retail accounts, and single-stock circuit breakers are suggested 2.
Warren: For high-turnover and manipulative strategies (H2), surveillance should prioritize abnormal turnover and order-cycling patterns, and venues could consider fees that scale with order-to-trade or cancellation ratios 3.
Ruth: For rational risk management (H3, H4), rules that penalize all volatility may suppress the adaptive repositioning that keeps prices informative Limitations and Future Work The study has several limitations, including the lack of price impact or interaction between agents because they are price-takers in separate environments Archetype intensity is a nonlinear function of public signals, and controls for size, value, momentum, volatility, and turnover do not eliminate the risk that intensity proxies for a known characteristic A shared limit-order-book simulator with all three archetypes trading together is suggested as the natural extension to address the lack of price impact or interaction The pilot scale used one run per configuration on about 1,750 hours per asset, and multi-seed training and reward-weight tuning are needed before drawing formal conclusions regarding policy collapse The findings support regulation that targets herding and manipulation while preserving adaptive risk-taking The framework proposed here offers a way to measure that distinction using public data The cryptocurrency pilot shows distinct footprints: frequent, shallow stress for herd-followers; rare, persistent stress for manipulators; and volatile but stable risk-adjusted performance for rational agents References Brad M.
Paper summary: Ruth: So we're looking at this paper, "Agentic AI in Automated Markets," and what it claims is that automated markets are increasingly being run by autonomous learning trading agents powered by those formulaic automated market makers, or AMMs.
Warren: Right. The core idea here is studying how these different agent behaviors shape things like stress, risk adjusted performance, liquidity, and volatility in these types of markets.
Iris: It seems they’re not just looking at the aggregate data because we don't have labels for those behavior types in public datasets.
Ruth: Exactly. Instead of hand-coding trading rules, they model three behavioral archetypes using agentic reinforcement learning systems: a rational arbitrageur, a manipulator, and a retail herd-follower.
Warren: Each archetype gets its own reward function defined within this common environment where the agents trade without knowing anything about the specific asset they're trading.
Iris: And to figure out what those agents are doing, they train each one using a value based deep q network called DQN and with REINFORCE using a learned baseline.
Ruth: The rational arbitrageur, for example, maximizes risk adjusted return but adds penalties for realized volatility and turnover.
Warren: Then you have the manipulator who gets a bonus proportional to turnover, which means rapid position cycling and short term gains because of that short discount horizon they use.
Iris: And the retail herd-follower gets a bonus for aligning exposure with recent momentum, which is basically just betting on what's happening right now.
Ruth: The paper makes five main contributions, starting with formulating trading as a discrete exposure control Markov decision process with risk adjusted cost aware rewards and market stress penalties.
Warren: They define these penalties using features that are available in any market, which is pretty broad because it’s asset agnostic.
Iris: They compare a value based method called DQN with a policy gradient method called REINFORCE with baseline on each archetype and use shap to attribute each learned policy to its inputs.
Ruth: What's really interesting is the pilot study they ran on hourly cryptocurrency data, which motivates their hypotheses about the footprints left by rational arbitrageurs, manipulators, and retail herd-followers.
Warren: So what did that pilot actually show regarding market stress?
Iris: Retail agents showed the most frequent stress episodes, but those are short and consistent with shallow drawdowns from momentum overreaction.
Ruth: Rational agents showed a moderate frequency because their controlled exposure limits both how often they get stressed and how long it lasts.
Warren: And the manipulators show the fewest episodes, but when stress does happen, it persists the longest, most visibly around major crash regimes near hours one thousand four hundred fifty to one thousand five hundred on that cryptocurrency data.
Iris: That's a key finding because it suggests different types of agents leave distinct signatures in market stress.
Ruth: And this leads them to some regulatory recommendations, suggesting we should target behaviors instead of just applying uniform limits on volatility or turnover.
Warren: They suggest tools for herding, like risk disclosure at the point of trade and leverage limits for retail accounts because that relates to the herd follower hypothesis.
Iris: Then for high turnover and manipulative strategies, they recommend surveillance focused on abnormal turnover patterns, and venues could consider fees that scale with order to trade or cancellation ratios.
Ruth: For rational risk management, they suggest rules that penalize all volatility might actually suppress the adaptive repositioning that keeps prices informative.
Warren: It seems like they’re trying to find a way to distinguish between agents who are just taking risks and those who are actively trying to move the market in a way that causes stress.
Iris: They also pointed out some limitations, specifically that because these agents trade independently in separate environments, they don't account for price impact or interaction between the agents themselves.
Ruth: That's fair; they admit that an agent-based simulator with all three archetypes trading together would be the natural next step to address that lack of interaction.
Warren: They also mentioned that the intensity of each archetype is a nonlinear function of public signals, and there are no controls for size, value, momentum, volatility, or turnover in their current setup.
Iris: And they note that the study only used one run per configuration on about one thousand seven hundred fifty hours per asset without multi seed training or reward weight tuning before drawing formal conclusions.
Ruth: So to summarize this paper "Agentic AI in Automated Markets," it models three distinct trading agents using reinforcement learning, showing how their different objectives—arbitrage, manipulation, and herding—create different patterns of stress and risk across markets.
Warren: And the conclusion they draw is that effective regulation needs to target the behaviors themselves rather than just setting uniform limits on volatility or turnover.
Iris: It’s interesting because they use public data to measure these distinctions, showing that retail agents cause frequent shallow stress, while manipulators cause rare but persistent stress episodes.
Ruth: So what this means for us is that we need to be careful not to just look at how much volatility a strategy generates, but whether it’s coming from the rational risk management agent or the herd follower pushing things too hard.
Conclusion: Ruth: So we're wrapping up this look at "Agentic AI in Automated Markets." The authors are looking at how different types of trading agents behave when they run automated markets powered by those AMMs, and they model three distinct agent personalities: the rational arbitrageur, the manipulator, and the retail herd-follower.
Warren: Right. The core finding is that these behaviors leave very different footprints on market stress and risk adjusted performance. It’s not just about how much volatility a strategy generates; it’s about what kind of stress it causes and why those agents are behaving that way in the first place.
Iris: I think the paper's main contribution is using this agent-based approach to link specific reward functions—the things they want each agent to achieve—directly to observable patterns in public market data, like that cryptocurrency pilot.
Ruth: Exactly. The implication for me is that we can start thinking about regulation not just as setting limits on total turnover or volatility, but as trying to target the *behavior* of the agents themselves.
Warren: That makes sense because if you can distinguish between a rational agent managing risk and a herd follower chasing momentum, then the policy needs to change depending on which one is causing trouble.
Iris: And they suggest that for herding behavior, tools like risk disclosure or leverage limits for retail accounts might be more useful than just blanket volatility rules.
Ruth: That’s where it gets interesting; they’re arguing that we need to focus on the specific intent behind the trading action, not just the outcome of high volatility.
Warren: So what this means for anyone watching the markets is that future oversight might have to be more nuanced, trying to spot those distinct signatures of manipulation versus genuine risk management.
Iris: And they’ve pointed out some limitations too; they haven't modeled how these agents actually interact with each other in a real market, which means the next step has to be modeling that interaction itself.
Irene Aldridge, Lourenco Belleau-Bhowmik, James Chan, Josh Chang, Kelly Deng, Pranav Dhingra, Xinda Ding, Haodong Fang, Satya Hanumanchi, Carter He, § Shuang He, § Yadong Hou, § Vallab Karanam, Rithvik Kasarla, Tishya Khanna, Jihyun Kim, Sarah Konety, Abhinav Kotta, Anderson Lo, Jake Luther, Omar Mian, § Ishan Patwardhan, § Matteo Perona, § Chenyue Qi, § Shashank Ramachandran, ☿ Shubham Rateria, Jake Swain, David Thai, Michael Twersky, Susan Wu, Dean Xu, Helenna Yin, 内 Jiachun Zheng
Cornell Tech
econ.EM, cs.ET, q-fin.RM, q-fin.TR
Submitted: 2026-10-15
Updated: 2026-11-01
Comments: 16 pages
License: http://creativecommons.org/licenses/by/4.0/
Importance score: 84/100
The gist: Automated markets increasingly host autonomous, learning trading agents and are powered by formulaic automated market makers (AMMs) These agentic AI systems are studied to understand how…
Key concepts
- Agent Archetypes
- Three distinct trading styles were modeled: a rational arbitrageur seeking risk-adjusted returns, a manipulator focused on rapid turnover for short gains, and a retail herd-follower reacting to recent market momentum. These are defined by how they change their trading rewards based on their goals.
- Exposure-Control MDP
- Trading is treated as a decision process where an agent chooses how much of an asset to hold (exposure) at each step. The system uses risk-adjusted rewards and penalties for market stress to guide the agent's actions, aiming to manage its position between zero and one.
- Market Stress Penalties
- The reward function includes a penalty term ($ ext{λstress}$) triggered during specific market crash regimes. This allows the model to learn how different agents react when the market is under extreme duress, helping distinguish between behaviors that cause frequent short stress versus those that cause persistent stress.
Terminology
Summary
Automated markets increasingly host autonomous, learning trading agents and are powered by formulaic automated market makers (AMMs) These agentic AI systems are studied to understand how heterogeneous agent behavior shapes stress, risk-adjusted performance, liquidity, and volatility in these types of markets The pilot on hourly cryptocurrency data motivates hypotheses about the distinct footprints left by rational arbitrageurs, manipulators, and retail herd-followers
Agent Archetypes and Modeling
Three behavioral archetypes are modeled as agentic reinforcement learning (RL) systems: a rational arbitrageur, a manipulator, and a retail herd-follower Each archetype is defined by its reward function within a common, asset-agnostic exposure-control environment These archetypes are trained with a value-based Deep Q-Network (DQN) and with REINFORCE using a learned baseline The rational arbitrageur maximizes risk-adjusted return with additional penalties on realized volatility and turnover The manipulator receives a bonus proportional to turnover, κturn et−et−1, and a short discount horizon, which favors rapid position cycling and short-term gains The retail herd-follower receives a bonus for aligning exposure with recent momentum, κmom sign(momt) (et − 0.5)
Methodology and Algorithms
Trading is formulated as a discrete exposure-control Markov decision process with riskadjusted, cost-aware rewards and market-stress penalties defined over features available in any market The base reward is a volatility-normalized net return: rbase t = etRt+1 − c et − et−1 σˆt + ϵ - λstress 1[stresst] et DQN is a value-based method that estimates Q(s, a) for the three actions REINFORCE with baseline uses a two-layer policy network and a separate two-layer value network to estimate the expected discounted return and serve as a baseline
Pilot Findings on Market Stress
The pilot on hourly cryptocurrency data motivates hypotheses about the distinct footprints left by rational arbitrageurs, manipulators, and retail herd-followers Retail agents show the most frequent stress episodes, but these are short, consistent with shallow drawdowns from momentum overreaction Rational agents show a moderate frequency, with controlled exposure limiting both occurrence and duration Manipulators show the fewest episodes, but stress persists the longest when it occurs, most visibly around the major crash regimes near hours 1,450–1,500
Regulatory Implications
The pilot suggests that the regulation of automated and agentic trading should target behaviors rather than apply uniform limits on volatility or turnover Recommendations include:
-
For herding (H1), tools such as risk disclosure at the point of trade, leverage and margin limits for retail accounts, and single-stock circuit breakers are suggested
-
For high-turnover and manipulative strategies (H2), surveillance should prioritize abnormal turnover and order-cycling patterns, and venues could consider fees that scale with order-to-trade or cancellation ratios
-
For rational risk management (H3, H4), rules that penalize all volatility may suppress the adaptive repositioning that keeps prices informative
Limitations and Future Work
The study has several limitations, including the lack of price impact or interaction between agents because they are price-takers in separate environments Archetype intensity is a nonlinear function of public signals, and controls for size, value, momentum, volatility, and turnover do not eliminate the risk that intensity proxies for a known characteristic A shared limit-order-book simulator with all three archetypes trading together is suggested as the natural extension to address the lack of price impact or interaction The pilot scale used one run per configuration on about 1,750 hours per asset, and multi-seed training and reward-weight tuning are needed before drawing formal conclusions regarding policy collapse
Conclusion
The findings support regulation that targets herding and manipulation while preserving adaptive risk-taking The framework proposed here offers a way to measure that distinction using public data The cryptocurrency pilot shows distinct footprints: frequent, shallow stress for herd-followers; rare, persistent stress for manipulators; and volatile but stable risk-adjusted performance for rational agents
References
Brad M. Barber and Terrance Odean. p. 9 References
Brad M. Barber and Terrance Odean. References
Brad M. Barber and Terrance Odean. Barber and Terrance Odean. All that glitters: The effect of attention and news on the
buying behavior of individual and institutional investors. Barber and Terrance Odean. All that glitters: The effect of attention and news on the
buying behavior of individual and institutional investors. Review of Financial Studies, 21(2):
785–818, 2008. Lin William Cong, Xi Li, Ke Tang, and Yang Yang. Retail investors chase attention and recent performance
(Barber and Odean, 2008). Evidence of manipulation and wash trading exists in both equities
and crypto (Gandal et al. Retail investors chase attention and recent performance
(Barber and Odean, 2008). Evidence of manipulation and wash trading exists in both equities
and crypto (Gandal et al. Management Science,
69(11):6427–6454, 2023. Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. Deep direct reinforcement
learning for financial signal representation and trading. IEEE Transactions on Neural Networks
and Learning Systems, 28(3):653–664, 2016. Neil Gandal, J. Neil Gandal, J. Hamrick, Tyler Moore, and Tali Oberman. Hamrick, Tyler Moore, and Tali Oberman. Price manipulation in the Bitcoin
ecosystem. Journal of Monetary Economics, 95:86–96, 2018. Lawrence R. Lawrence R. Glosten and Paul R. Glosten and Paul R. Bid, ask and transaction prices in a specialist market
with heterogeneously informed traders. Bid, ask and transaction prices in a specialist market
with heterogeneously informed traders. Journal of Financial Economics, 14(1):71–100, 1985. Journal of Financial Economics, 14(1):71–100, 1985. Continuous auctions and insider trading. Econometrica, 53(6):1315–1335, 1985. Blake LeBaron. 2 Related Work
Heterogeneous agents and market quality. Agent-based computational finance shows that
interactions among fundamentalists, chartists, and noise traders can generate fat tails, volatility
clustering, and crashes (LeBaron, 2006). Agent-based computational finance. In Leigh Tesfatsion and Kenneth L. In Leigh Tesfatsion and Kenneth L. Judd,
editors, Handbook of Computational Economics, volume 2, pages 1187–1233. Interpretability. SHAP (Lundberg and Lee, 2017) gives additive feature attributions for neural
policies. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In
Advances in Neural Information Processing Systems, volume 30, 2017. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, et al. Fidjeland, Georg Ostrovski, et al. Human-level
control through deep reinforcement learning. Nature, 518:529–533, 2015. John Moody and Matthew Saffell. John Moody and Matthew Saffell. Learning to trade via direct reinforcement. Learning to trade via direct reinforcement. IEEE Transactions
on Neural Networks, 12(4):875–889, 2001. IEEE Transactions
on Neural Networks, 12(4):875–889, 2001. Simple statistical gradient-following algorithms for connectionist reinforcement
learning.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
--- Page 1 ---
Agentic AI in Automated Markets Irene Aldridge∗, Lourenco Belleau-Bhowmik, James Chan, Josh Chang, Kelly Deng, Pranav Dhingra, Xinda Ding, Haodong Fang, Satya Hanumanchi, Carter He et al. Cornell Tech Abstract Automated markets increasingly host autonomous learning trading agents and are powered by formulaic automated market makers (AMMs) We study how heterogeneous agent behavior shapes stress risk-adjusted performance liquidity and volatility in these types of markets Three behavioral archetypes are modeled as agentic reinforcement learning (RL) systems: a rational arbitrageur, a manipulator, and a retail herd-follower Each archetype is defined by its reward function within a common asset-agnostic exposure-control environment
--- Page 2 ---
These questions are hard to answer from aggregate data because behavior types are not labeled in public datasets We address this with an agent-based approach Rather than hand-coding trading rules, we specify each archetype through its objective and let a reinforcement learning agent discover a policy that maximizes it The learned policies then serve as measurement devices applied to observable state variables they yield a stock-day estimate of how intensely each archetype would be trading The framework is asset-agnostic any market with a price series, a volume series, and a liquidity proxy supports it A pilot on hourly cryptocurrency data provides evidence and motivates our hypotheses
--- Page 3 ---
This paper makes five contributions 1. We formulate trading as a discrete exposure-control Markov decision process with riskadjusted cost-aware rewards and market-stress penalties defined over features available in any market 2. p. 1 This paper makes five contributions:
1. We formulate trading as a discrete exposure-control Markov decision process with riskadjusted, cost-aware rewards and market-stress penalties, defined over features available in
any market. We formulate trading as a discrete exposure-control Markov decision process with riskadjusted, cost-aware rewards and market-stress penalties, defined over features available in
any market. We define three trader archetypes through reward shaping: rational arbitrageur, manipulator,
and retail herd-follower. We explicitly assume that the markets are moderated by Automated
Market Makers, which are deterministic and not agentic at present. We compare a value-based method (DQN) with a policy-gradient method (REINFORCE with
a baseline) on each archetype and use SHAP to attribute each learned policy to its inputs.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 4 ---
4.1 Environment Trading is a discrete exposure-control problem at each step t the agent observes state st and chooses an action at ∈ th, which moves exposure et by −0. p. 2 1 Environment
Trading is a discrete exposure-control problem. At each step t the agent observes state st and
chooses an action at ∈ {sell, hold, buy}, which moves exposure et by −0. 5 within [0, 1]. Exposure 0 is flat, 0.Where the paper says it
Where the paper says it
--- Page 5 ---
4.2 Algorithms DQN (value-based) A two-layer multilayer perceptron estimates Q(s, a) for the three actions A replay buffer breaks the temporal correlation between updates and a target network synchronized every N steps stabilizes bootstrapped targets Exploration is ε-greedy with linear decay REINFORCE with baseline (policy gradient) A two-layer policy network outputs a softmax over the three actions A separate two-layer value network estimates the expected discounted return and serves as a baseline The advantage is the Monte Carlo return minus the baseline estimate which reduces gradient variance without introducing bias Returns are standardized within each batch to stabilize training
--- Page 6 ---
4.3 Trader archetypes Each archetype modifies the base reward (1) Archetypes are trained in separate environments so behavioral effects are isolated strategic interaction is not modeled Section 8 Rational arbitrageur Maximizes risk-adjusted return with additional penalties on realized volatility and turnover Manipulator receives a bonus proportional to turnover, κturn et−et−1, and a short discount horizon, which favors rapid position cycling and short-term gains Retail herd-follower receives a bonus for aligning exposure with recent momentum, κmom sign(momt) (et − 0. p. 2 It receives a bonus proportional to turnover, κturn |et−et−1|, and a short discount horizon, which favors rapid position cycling
and short-term gains. Retail herd-follower.Where the paper says it
--- Page 7 ---
4.4 Hyperparameters Reward weights (κturn, κmom, λstress) are fixed across assets and set for the validation window Each configuration is trained with five random seeds the reported results average across those seeds
--- Page 8 ---
5 Experimental Design We trained three archetypes of agents and evaluated the performance of each strategy 5.1 Strategy-level metrics These metrics describe the strategies not the market Because agents are price-takers in Stage 1 a strategy’s Sharpe ratio measures how well it exploits the market, not how efficient the market is Stage 2 addresses market-level questions
--- Page 9 ---
5.2 Hypotheses Table 2 Testable hypotheses ID Hypothesis Expected sign Source H1 Retail intensity raises crash frequency but not persistence βRET > 0 on frequency Pilot H2 Manipulator intensity raises crash persistence βMAN > 0 on persistence Pilot H3 Rational intensity lowers inefficiency (variance-ratio deviation) βRA < 0 on VR − 1 Pilot H4 Rational intensity raises volatility without raising crash frequency βRA > 0 on volatility, ≈ 0 on frequency Pilot
--- Page 10 ---
6 Results This section reports the cryptocurrency pilot for the rational manipulator and retail archetypes (Sections 6.1–6.4) Pilot findings are based on one training run per configuration and visual inspection of rolling metrics so they are motivating evidence rather than formal tests
--- Page 11 ---
6.2 Feature attributions We applied SHAP DeepExplainer to the REINFORCE action probabilities and the DQN Q-values using randomly sampled historical states as the background distribution and the test period as the evaluation set Mean absolute SHAP values identify the three most influential features per archetype Time of day ranks in the top three for every archetype which likely reflects intraday liquidity and volatility seasonality in crypto
--- Page 12 ---
6.3 Crisis frequency and persistence Retail agents show the most frequent stress episodes, but these are short, consistent with shallow drawdowns from momentum overreaction (Figure 1) Rational agents show a moderate frequency, with controlled exposure limiting both occurrence and duration Manipulators show the fewest episodes, but stress persists the longest when it occurs, most visibly around the major crash regimes near hours 1,450–1,500 All strategies have far fewer stress hours than the market’s own crash regime because none are fully invested at all times
--- Page 13 ---
6.4 Risk-adjusted performance and volatility Rolling 7-day Sharpe ratios swing widely for every agent, roughly between −20 and +20 annualized (Figure 2) Rational agents are most stable where they trade Manipulators show spikes under favorable conditions, followed by sharp deteriorations Retail agents are weakest during reversals; for example, on ETC between hours 300 and 800 Return volatility tells a different story from stress (Figure 3) The active rational agents have the highest baseline volatility reflecting continual exposure adjustments Manipulators have lower baseline volatility, punctuated by sharp spikes during stress periods, up to about 0. p. 4 The active rational agents (DQN
on AAVE, REINFORCE on ETC) have the highest baseline volatility, reflecting continual exposure
adjustments. Manipulators have lower baseline volatility, punctuated by sharp spikes during stress
periods, up to about 0.Where the paper says it
--- Page 14 ---
(a) AAVE (b) ETC (c) SUI Figure 1: Hours with net strategy return below −2% in a rolling 24-hour window by agent. Dashed line: hours the market is in a crash regime
(b) ETC (c) SUI Figure 2: Rolling 7-day annualized Sharpe ratio of net strategy returns by agent
(c) SUI Figure 3: Rolling 24-hour standard deviation of net strategy log returns by agent
--- Page 15 ---
Discussion and Regulatory Implications The pilot suggests that the regulation of automated and agentic trading should target behaviors rather than apply uniform limits on volatility or turnover Each recommendation below is tied to a hypothesis that the CRSP tests can confirm or reject Herding (H1). If retail intensity predicts frequent short stress episodes, the case strengthens for investor-protection tools that slow pro-cyclical flows: risk disclosure at the point of trade, leverage and margin limits for retail accounts, and single-stock circuit breakers such as the U. p. 8 Herding (H1). If retail intensity predicts frequent short stress episodes, the case strengthens for
investor-protection tools that slow pro-cyclical flows: risk disclosure at the point of trade, leverage
and margin limits for retail accounts, and single-stock circuit breakers such as the U. Crypto venues largely lack these tools, so the pilot setting is where the
gap is widest. High-turnover and manipulative strategies (H2). High-turnover and manipulative strategies (H2). If manipulator intensity predicts persistent stress, surveillance should prioritize abnormal turnover and order-cycling patterns, and venues
could consider fees that scale with order-to-trade or cancellation ratios.Where the paper says it
Where the paper says it
Where the paper says it
--- Page 16 ---
Overall, effective policy should distinguish between stabilizing and destabilizing behavior The framework proposed here offers a way to measure that distinction using public data
--- Page 17 ---
9 Conclusion We model three trader archetypes as agentic reinforcement learning systems whose behavior follows from their objectives, and we propose using the trained policies to measure archetype intensity in public market data The cryptocurrency pilot shows distinct footprints: frequent, shallow stress for herd-followers; rare, persistent stress for manipulators; and volatile but stable risk-adjusted performance for rational agents The findings support regulation that targets herding and manipulation while preserving adaptive risk-taking
--- Page 18 ---
References Brad M. Barber and Terrance Odean. All that glitters: The effect of attention and news on the buying behavior of individual and institutional investors. Review of Financial Studies, 21(2):785–818, 2008 Lin William Cong, Xi Li, Ke Tang, and Yang Yang. p. 9 Review of Financial Studies, 21(2):
785–818, 2008. Lin William Cong, Xi Li, Ke Tang, and Yang Yang. Management Science,
69(11):6427–6454, 2023. Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. Deep direct reinforcement
learning for financial signal representation and trading.Where the paper says it
Where the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Improve agent training by using multi-seed training to mitigate policy collapse, as seen in REINFORCE on AAVE and SUI where agents converged to zero exposure. This will ensure more robust and diverse learned policies across different initial conditions.
-
Enhance model interpretability by systematically analyzing SHAP attributions for each archetype's learned policies, identifying the
three most influential features per archetype
as described in Table 3. This allows researchers to verify that each behavior relies on the signals its objective implies, moving beyond aggregate performance metrics alone. -
Develop a risk-aware regulatory framework by distinguishing between stabilizing and destabilizing behavior based on archetype intensity rather than uniform limits on volatility or turnover. This system can provide specific recommendations: targeting herding for retail flows and manipulating strategies for abnormal turnover patterns, while preserving the adaptive risk-taking of rational agents.
-
Design a shared limit-order-book simulator where all three archetypes trade together to study their strategic interaction, addressing the limitation that
Agents are price-takers in separate environments.
This will allow for analysis of how different agent behaviors affect market microstructure and liquidity simultaneously.
Abstract
Automated markets increasingly host autonomous, learning trading agents and are powered by formulaic automated market makers (AMMs). We study how heterogeneous agent behavior shapes stress, risk-adjusted performance, liquidity, and volatility in these types of markets. Three behavioral archetypes are modeled as agentic reinforcement learning (RL) systems: a rational arbitrageur, a manipulator, and a retail herd-follower. Each archetype is defined by its reward function within a common, asset-agnostic exposure-control environment, and each is trained with a value-based Deep Q-Network (DQN) and with REINFORCE using a learned baseline. A pilot on hourly cryptocurrency data (AAVE, ETC, SUI) motivates the hypotheses. Herd-followers produce the most frequent but shortest stress episodes, manipulators the fewest but most persistent, and rational agents the most stable rolling Sharpe ratios despite higher return volatility. SHAP attributions tie these patterns to drawdown and cross-asset signals, short-term volatility and position state, and momentum, respectively. The results motivate behavior-specific regulation rather than uniform limits on volatility or turnover.
Related papers
- Stationarity with Occasionally Binding Constraints
- Backtesting High-Dimensional Expected Shortfall
- Higher-Order Neyman Orthogonality in Moment-Condition Models
- Online Bayesian Inference for a Dirichlet Process Mixture of Binary Logit Models
- Debiased Conditional Independence Tests for Causal Effects
- Sharp Testable Implications of Encouragement Designs