ARTIS: Agentic Risk-Aware Test-Time Scaling via Iterative Simulation

The gist The proposed framework, ARTIS, decouples exploration from commitment by enabling test-time exploration through simulated interactions prior to realworld execution.

ARTIS Overview

ARTIS is a novel inference paradigm tailored to improving performance on action-centric and environment-interactive settings by repeatedly simulating, evaluating, and refining candidate action plans before committing to a real execution We propose ARTIS, a decision-time inference framework that improves the reliability of tool-using agents by repeatedly simulating, evaluating, and refining candidate action plans before committing to a real execution Unlike conventional TTS methods that primarily focus on expanding reasoning paths to obtain a correct final answer, our approach explicitly targets action-level correctness in agentic settings where tool executions can be irreversible and costly

Iterative Simulation Loop

The core of ARTIS involves an iterative simulation loop where the agent performs multiple simulated attempts before producing a single committed execution This framework operates in three stages: (i) iterative simulation, (ii) trial summarization, and (iii) final execution The agent generates up to N candidate attempts to explore potential action space, considering two modes: Sequential Iteration and Parallel Iteration Sequential iteration allows the agent to iteratively refine its action proposals by conditioning each new attempt on the outcomes of all previous simulated attempts, which encourages diverse and nonredundant attempts Parallel iteration generates multiple simulated attempts independently, allowing for efficient parallel computation and avoiding context-length growth, although it may lead to redundant proposals if the underlying policy has low sampling diversity

Risk-Aware Tool Simulator

A key component of ARTIS is a risk-aware tool simulator designed to emphasize high-impact error prediction over average-case accuracy This simulator models the probability of an output given an action, state, and history, denoted as p(o a, s) To achieve this fidelity on failure-inducing actions, the simulator employs a failure-driven data generation strategy This involves collecting diverse real-world tool implementations and generating targeted tool invocations for rare errors using heterogeneous agents The training data is rebalanced by assigning each instance a sampling weight inversely proportional to the frequency of its outcome type, specifically downweighting successful executions and upsampling samples corresponding to rare failure cases

Summarization and Final Execution

Before the final real execution, a summarization step aggregates the simulated outcomes and distills them into a single execution strategy or high-level recommendation denoted as S This summary is designed to maximize robustness while minimizing execution risk by compressing diverse simulated experiences into a concise guidance signal The summarized recommendation is then injected into the agent prompt as additional guidance, and the agent performs executions in the real environment without further simulation or rollback

Performance and Efficiency Gains

Experiments on multi-turn and multistep agentic benchmarks demonstrate that iterative simulation substantially improves agent reliability when a high-fidelity simulator is available ARTIS consistently outperforms standard inference and conventional TTS baselines, validating the necessity of both iterative simulation and risk-aware simulation The sequential iteration variant achieves the strongest performance on both benchmarks, demonstrating its effectiveness in complex agentic scenarios Furthermore, ARTIS requires substantially fewer self-evaluation calls than Sequential Revision, which needs to perform one self-evaluation per action when N becomes large, leading to significantly higher API usage and token consumption for the baseline This design choice causes Sequential Revision to incur significantly higher API usage and token consumption overall ARTIS not only achieves consistently better performance across all N values but also demonstrates superior token efficiency compared to the Sequential Revision baseline

Generalizability and Model Dependence

The benefits of agentic test-time scaling are not tied to a specific backbone architecture, as ARTIS with a perfect simulator consistently yields the best results across both benchmarks However, smaller or less capable models may fail to realize similar gains, as shown by the Qwen3-4B results which exhibit unstable and inconsistent performance across cases Notably, performance improvements are closely correlated with the evaluation pass rate, suggesting that a stronger ability to identify valid action trajectories benefits final execution The work addresses the reliability of agentic LLMs that interact with external environments through tool usage by enabling agents to explore candidate actions through simulation before committing to real world execution

Impact Statement

By enabling agents to explore candidate actions through simulation before committing to real world execution, the proposed approach may reduce unintended errors and irreversible side effects in real-world deployments From an ethical perspective, our method emphasizes riskaware decision making by prioritizing the anticipation of high-impact failure modes This design aligns with widely accepted principles of responsible AI However, improved simulation and planning capabilities may also increase the effectiveness of autonomous agents in ways that could be misused if deployed without appropriate safeguards Our approach does not eliminate the need for external governance mechanisms such as access control, monitoring, and human oversight

Implementation Details

The simulator is instantiated as an LLM conditioned on a structured prompt representing a hypothetical tool execution, modeling p(o a, s) The training data yields 524 distinct tools from various sources like MCP-Universe and Python-based function tools Training is performed using parameter-efficient fine-tuning with LoRA to mitigate overfitting The framework includes detailed prompts for the action agent, self-evaluation stage, tool simulator, and summarization stage The implementation of baselines follows Snell et al. (2024b) to design the prompts directly for agentic tool use

Conclusion

In this work, we introduce ARTIS, a test-time scaling framework designed for complex agentic scenarios in which agents are able to iteratively explore multiple action attempts within a simulated environment before committing to real world execution Central to ARTIS is a risk-aware simulator trained using failure-driven data generation and rebalanced objectives, enabling more accurate modeling of critical error cases Extensive experiments on two agentic benchmarks demonstrate that ARTIS substantially outperforms existing baselines, generalizes across diverse backbone models, and remains effective under different scaling configurations The impact of this work is to enhance the reliability of agentic LLMs that interact with external environments through tool usage by enabling agents to explore candidate actions through simulation before committing to real world execution

References

Acikgoz, E. C., Qian, C., Ji, H., Hakkani-Tür, D., and Tur, G. Self-improving llm agents at test-time AI@Meta. Llama 3 model card Barres, V., Dong, H., Ray, S., Si, X., and Narasimhan, K. τ2-Bench: Evaluating Conversational Agents in a DualControl Environment Bi, Z., Han, K., Liu, C., Tang, Y., and Wang, Y. Forestof-thought: Scaling test-time compute for enhancing llm reasoning Chae, H., Kim, N., Ong, K. T.-i., Gwak, M., Song, G., Kim, J., Kim, S., Lee, D., and Yeo, J. Web agents with world models: Learning and leveraging environment dynamics in web navigation Chakraborty, S., Pourreza, M., Sun, R., Song, Y., Scherrer, N., Huang, F., Bedi, A. S., Beirami, A., Gu, J., Palangi, H., and Pfister, T. On the Role of Feedback in Test-Time Scaling of Agentic AI Workflows Chen, A., Liu, Z., Zhang, J., Prabhakar, A., Liu, Z., Heinecke, S., Savarese, S., Zhong, V., and Xiong, C. Grounded test-time adaptation for llm agents Chen, C., Hao, X., Liu, W., Huang, X., Zeng, X., Yu, S., Li, D., Wang, S., Gan, W., Huang, Y., et al. Acebench: Who wins the match point in tool learning? Chen, M., Sun, H., Li, T., Yang, F., Liang, H.: Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning Chen, Y., Chen, J., Meng, R., Yin, J., Li, N.: Tumix: Multi-agent test-time scaling with tool-use mixture Chen, Y., Pan, X., Li, Y., Ding, B.: Simple and provable scaling laws for the test-time compute of large language models Ding, J., Zhang, Y., Shang, Y.

1. Bold header: Agentic TTS via Iterative Simulation

This framework decouples exploration from commitment by enabling test-time exploration through simulated interactions prior to realworld execution, allowing computation to be scaled over simulated action trajectories, directly targeting action-level correctness.

2. Bold header: Risk-Aware Tool Simulator Development

The simulator must prioritize failure prediction by being decision-useful rather than uniformly accurate, with emphasis on anticipating failure modes through a strategy that emphasizes fidelity on failure-inducing actions via targeted data generation and rebalanced training.

3. Bold header: Iterative Simulation Modes

The framework supports two exploration strategies: Sequential Iteration, which allows for adaptive exploration of the action space, where the agent can explicitly avoid previously unsuccessful strategies, and Parallel Iteration, which enables efficient parallel computation and avoiding context-length growth.

4. Bold header: Risk-Aware Data Generation Pipeline

The simulator training must employ a strategy to address data imbalance by using a failure-driven data generation strategy that downweights successes and upsamples failures, ensuring the simulator captures rare but high-impact failure modes.

5. Bold header: Final Execution Strategy

To maximize robustness, the framework uses a summarization step to aggregate the simulated outcomes and distills them into a single execution strategy or high-level recommendation, which is then injected as guidance before real execution, ensuring agents perform commit[ted] action[s] for final task completion.

6. Bold header: Model-Specific Evaluation Tuning

Performance gains are optimized by using larger models for the evaluator and summarizer when substituting them, as performance improvements are closely correlated with the evaluation pass rate, specifically noting that the Qwen3-32B evaluator achieves the highest pass rate.

Current test-time scaling (TTS) techniques enhance large language model (LLM) performance by allocating additional computation at inference time, yet they remain insufficient for agentic settings, where actions directly interact with external environments and their effects can be irreversible and costly. We propose ARTIS, Agentic Risk-Aware Test-Time Scaling via Iterative Simulation, a framework that decouples exploration from commitment by enabling test-time exploration through simulated interactions prior to real-world execution. This design allows extending inference-time computation to improve action-level reliability and robustness without incurring environmental risk. We further show that naive LLM-based simulators struggle to capture rare but high-impact failure modes, substantially limiting their effectiveness for agentic decision making. To address this limitation, we introduce a risk-aware tool simulator that emphasizes fidelity on failure-inducing actions via targeted data generation and rebalanced training. Experiments on multi-turn and multi-step agentic benchmarks demonstrate that iterative simulation substantially improves agent reliability, and that risk-aware simulation is essential for consistently realizing these gains across models and tasks.