An Extensible Agentic Framework for Expert-Level Automation of Atomistic Simulations
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Condensed Matter. The day in condensed matter: materials, phases and the measurements under them.
Kai: We're looking at the paper: "An Extensible Agentic Framework for Expert-Level Automation of Atomistic Simulations".
Mira: Here's the gist of the paper. Atomistic simulations are being automated using Paimon, a platform designed for agentic integration in materials optimization and nanoscale simulations This framework substantially improves the reliability of agentic workflows by suppressing silent errors, which are plausible yet physically incorrect results Paimon affords researchers a continuous, science-centric workflow throughout the entire simulation lifecycle by translating natural language queries into simulation workflows that execute tasks, analyze data, and deliver results following expert procedures Platform Architecture Paimon is structured around three independent components: expert knowledge, an orchestrator, and executors The orchestrator serves as the central controller by combining user queries with expert knowledge, performing dynamic planning and task decomposition to manage complex simulation workflows Each subtask record consists of persistent files, including the memory of the executor agent, which constitutes an episodic memory of Paimon comprising a summary of the entire run and key simulation scripts produced at each stage Execution Reliability Analysis The system categorizes errors into "benign", "loud", and "silent" errors based on their impact and detectability Silent errors are the most concerning because they leave no signal in outputs and can only be caught by inspecting the simulation script itself The stochastic nature of LLMs introduces run-to-run variance, necessitating robust statistical analysis through repeated sampling to enhance system reliability Executor Specialization and Tooling Executors are agents that handle specialized software packages, such as LAMMPS or ASE, ensuring that the core design philosophy is to fully take advantage of established scientific packages rather than unreliable re-implementation Each executor is equipped with dedicated tools for accessing software documentation and managing execution and I/O, including generating simulation scripts via retrieval-augmented generation from official software documentation End-to-End Workflow Validation The end-to-end evaluation of Paimon on the Li-ion diffusivity task demonstrates that adding agentic components like the critic and episodic memory progressively improves success rates across workflow stages The effect of the file contract is concentrated in the production MD stage, where it eliminates handoff errors by resolving semantic mismatches between different packages Extensibility Mechanisms Paimon achieves extensibility through two properties: first, all agents communicate through a shared interface so introducing a new executor requires no modification to the rest of the system second, the simulation procedure is consolidated into a knowledge module rather than embedded in the agents, meaning teaching Paimon a new workflow amounts to obtaining its natural-language description The capability of Paimon can be extended without modifying the existing agents and without predefined mappings between them The gated evaluation method involves setting validation gates at each stage, where a gate must be satisfied to advance, bypassing expensive MD simulations when criteria are met The diffusivity gate requires the reported diffusivity to match the reference within 0.001 × 10−6 cm2/s This approach allows for an ablation study at scale by bypassing expensive MD simulations when Paimon passes a gate The success rates at medium effort all surpass their low-effort counterparts, suggesting that both architectural components and reasoning effort improve success rates Universal Machine Learning Potentials The selection of universal machine learning interatomic potentials is modular, allowing an agent to retrieve potentials as needed, defaulting to the SevenNet family if not specified elsewhere The liquid electrolyte task uses SevenNet-0 with Grimme’s D3 dispersion correction with Becke–Johnson damping applied following reference
69: The solid electrolyte task uses SevenNet-0 without the D3 dispersion correction, following ref.
73: The orchestrator defaults to the SevenNet family and autonomously selects a potential if it is not specified elsewhere Implementation Details Paimon runs on a host computer separate from the computer cluster and communicates with it over SSH, integrated with the Slurm job scheduler Intermediate and final output files are saved in the working directory of each executor, and software versions are fixed by the Apptainer image Users interact with Paimon through a web interface that displays the orchestrator chat and the file system of the run The Paimon source code is available at https://github.
Paper summary: Kai: So, we're looking at this paper today, "An Extensible Agentic Framework for Expert-Level Automation of Atomistic Simulations." It seems to be about taking those atomistic simulations that used to be super slow and hard to set up and automating them using agentic AI.
Mira: Right. The main idea they're pushing is that this framework aims to make the whole simulation lifecycle, from the initial query all the way through analysis, a continuous, science-centric workflow by turning natural language questions into actual tasks that get done following expert procedures.
Lev: From where I sit on the hardware side, it sounds like they're trying to solve that problem of making sure these complex workflows don't just fail silently or produce physically wrong results because the AI is guessing too much.
Kai: Exactly. The paper claims this approach substantially improves reliability by suppressing those silent errors, which are those plausible but actually incorrect results that you can't easily spot in the final output files alone.
Mira: They’re using a system where the orchestrator combines user queries with expert knowledge, and it does dynamic planning and task decomposition to manage these complex simulation workflows.
Lev: I wonder how robust that dynamic planning is when we're dealing with multi-step tasks that are long horizon, because if the LLM builds the procedure probabilistically, you get a lot of failure risk on real hardware.
Kai: That’s a huge concern for me. The paper addresses this by incorporating agentic AI components like self-reflection and dynamic task decomposition to significantly enhance the execution accuracy of those large language models.
Mira: And they use an episodic memory within the orchestrator, which acts like a summary of the entire run and keeps track of key simulation scripts produced at each stage, which is pretty smart for maintaining context.
Lev: So it's not just one big LLM doing everything; it’s a whole system of agents—an orchestrator, executors, and expert knowledge—that passes the work along, right?
Kai: That’s right. You have specialized executor agents that handle specific software packages like LAMMPS or ASE, ensuring they take advantage of established scientific packages instead of trying to re-implement everything themselves.
Mira: And the paper shows this end-to-end validation on a Li-ion diffusivity task, where adding components like a critic and episodic memory progressively improves the success rates across all those workflow stages.
Paper summary: Lev: I noticed they also talk about how their file contract works, because that seems to be where they eliminate handoff errors between different packages when dealing with things like production molecular dynamics simulations.
Kai: It really does. The effect of that file contract seems concentrated in the production molecular dynamics stage, where it resolves those semantic mismatches between the different simulation packages involved.
Mira: But the authors also point out that because these LLMs have limited parametric knowledge, they often fall short of the specialized expertise needed to execute those sophisticated expert-level simulations on their own.
Lev: So even with all this agentic scaffolding, we still need a solid foundation of expert knowledge embedded somewhere because the raw language models just don't inherently know how to run a specific simulation perfectly.
Kai: That’s where the integration of that external expert knowledge is key; it provides the specialized context that guides the agents through those complex scientific procedures.
Mira: The framework achieves extensibility in two ways: first, all agents communicate through a shared interface, so you can introduce a new executor without modifying the rest of the system.
Lev: That’s good for hardware because it means we don't have to rewrite the execution layer every time we want to support a new simulation tool or software package.
Kai: And second, they consolidate the entire simulation procedure into this knowledge module instead of embedding it directly in every agent, meaning teaching Paimon a new workflow is just about giving it a natural language description.
Mira: That’s an interesting way to handle the knowledge base; you're not retraining the execution logic, you're just updating the description of what to do.
Lev: It seems like they’ve managed to create something that handles both the high-level reasoning and the low-level execution details without having to write every single instruction in code.
Kai: And when it comes to evaluating this system, they use a gated evaluation method where you set validation gates at each stage so you only run those expensive molecular dynamics simulations when the criteria are actually met.
Mira: For instance, for the diffusivity task, the gate requires the reported diffusivity to match a reference within zero point zero zero one times ten to the negative six centimeters squared per second.
Lev: That level of precision is demanding; if Paimon passes that gate, it means we're saving a lot of compute time because we don't have to run those long simulations for every single test case.
Paper summary: Kai: It does. And they found that the success rates at medium effort all surpass their low-effort counterparts, which suggests that both the architectural components and the reasoning effort improve the final outcome.
Mira: So, looking at this paper "An Extensible Agentic Framework for Expert-Level Automation of Atomistic Simulations," what’s it really saying about how we automate scientific discovery?
Kai: It’s saying we can move away from manual, error-prone setup procedures and use agentic AI to manage the entire process, making it much more reliable than just feeding a prompt into a large language model.
Mira: The authors are arguing that this framework allows for continuous science-centric workflows throughout the simulation lifecycle by translating natural language into executable tasks that follow expert procedures.
Lev: For us building real hardware, the implication is that if we can automate the input preparation and data analysis steps reliably, it really shifts our focus to optimizing the core physics simulations themselves rather than debugging setup errors.
Kai: The paper’s title points to this idea of an extensible framework for expert-level automation, showing how you can plug in different agents and knowledge bases without redesigning the entire system.
Mira: The authors are basically showing that agentic AI, when structured correctly with planning and memory, can bridge the gap between abstract scientific questions and concrete simulation execution.
Lev: It’s about making sure that when we use these tools to run things like Li-ion diffusivity tasks, we aren't just getting a random output; we are getting a reproducible result based on expert guidance.
Kai: So, if you were driving and needed to know what this means for your day, it means complex scientific modeling can become less about spending hours writing setup scripts and more about defining the high-level goal.
Mira: That’s the core shift they’re proposing; the heavy lifting of translating a research idea into a runnable simulation is being offloaded to this agentic structure.
Lev: It suggests that for quantum error correction research, we might eventually see systems where these automated workflows handle the complex parameter sweeps and initial data processing much more efficiently than current deterministic tools allow.
Kai: I think the most important thing is that this moves atomistic simulation automation from being a series of brittle scripts to something more adaptive and capable of handling the inherent uncertainty in scientific modeling.
Conclusion: Kai: So we're wrapping up on this paper, "An Extensible Agentic Framework for Expert-Level Automation of Atomistic Simulations." Essentially, they've built a system that lets you take natural language instructions and turn them into full simulation workflows, like running complex atomistic stuff.
Mira: It’s about making the whole process—from asking the question to getting the final data—a continuous loop where the AI follows expert procedures rather than just guessing.
Lev: From my side, it’s interesting because they focused on making sure those agentic steps don't introduce silent errors, which are those tricky results that look fine but are actually wrong in the simulation script itself.
Kai: Yeah, they categorize those errors into benign loud and silent ones, and the silent ones are the most worrying because you can’t see them in the final numbers.
Mira: They use an orchestrator to manage everything, combining user requests with expert knowledge to plan out all the subtasks step by step.
Lev: And that orchestrator keeps memory of what happened at every stage, like keeping a summary and the key scripts produced so it doesn't forget things mid-run.
Kai: They show how adding components like a critic agent and that memory makes the success rate better across different stages, especially in those medium effort tasks.
Mira: The paper is really showing how you can plug in new simulation tools or methods without rewriting the main agent structure, which is pretty flexible.
Lev: That extensibility through shared interfaces means we can add new software executors easily without messing up the core execution layer we've built for real hardware.
Kai: And they use a gating method for testing, so you don't waste time running expensive molecular dynamics simulations if the initial steps already look promising based on some criteria.
Mira: So it boils down to using AI to manage the entire lifecycle of a simulation with expert-level guidance while keeping an eye on those tricky hidden errors.
Lev: It suggests that for things like complex material science, we can offload a lot of the tedious setup and error checking to this framework, letting us focus more on the physics itself.
Kai: That’s what it points toward—moving from writing brittle scripts to defining the high-level goal for the simulation.
Mira: It opens up a way for researchers to explore materials much faster because they don't have to manually manage every single handoff between different software packages.
Yutack Park, Yeonwoo Chung, Jinmu You, Jisu Kim, Suyeon Ju, Seungwu Han
Seoul National University · Research Institute of Advanced Materials Seoul National University Research Institute of Advanced Materials Seoul National University Center for AI and Natural Sciences Korea Institute for Advanced Study
cond-mat.mtrl-sci
Submitted: 2026-06-08
Updated: 2026-11-09
Comments: Minor revisions
Code: https://github.com/MDIL-SNU/Paimon
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 86/100
The gist: Atomistic simulations are being automated using Paimon, a platform designed for agentic integration in materials optimization and nanoscale simulations This framework substantially improves the…
Key concepts
- Silent Errors
- These are plausible but physically incorrect results that do not produce obvious output signals during a simulation run. They are the most dangerous errors because they can only be found by manually inspecting the underlying simulation script, making them hard to catch automatically.
- Orchestrator
- The central controller of Paimon that combines user requests with expert knowledge. It performs dynamic planning and breaks down complex simulation tasks into manageable steps, ensuring the entire workflow is managed logically from start to finish.
- Executors
- Specialized agents designed to handle specific scientific software packages like LAMMPS or ASE. They ensure the core design uses established tools instead of unreliable custom code and are equipped with tools to generate necessary scripts.
Terminology
Summary
Atomistic simulations are being automated using Paimon, a platform designed for agentic integration in materials optimization and nanoscale simulations This framework substantially improves the reliability of agentic workflows by suppressing silent errors, which are plausible yet physically incorrect results Paimon affords researchers a continuous, science-centric workflow throughout the entire simulation lifecycle by translating natural language queries into simulation workflows that execute tasks, analyze data, and deliver results following expert procedures
Platform Architecture
Paimon is structured around three independent components: expert knowledge, an orchestrator, and executors The orchestrator serves as the central controller by combining user queries with expert knowledge, performing dynamic planning and task decomposition to manage complex simulation workflows Each subtask record consists of persistent files, including the memory of the executor agent, which constitutes an episodic memory of Paimon comprising a summary of the entire run and key simulation scripts produced at each stage
Execution Reliability Analysis
The system categorizes errors into benign
, loud
, and silent
errors based on their impact and detectability Silent errors are the most concerning because they leave no signal in outputs and can only be caught by inspecting the simulation script itself The stochastic nature of LLMs introduces run-to-run variance, necessitating robust statistical analysis through repeated sampling to enhance system reliability
Executor Specialization and Tooling
Executors are agents that handle specialized software packages, such as LAMMPS or ASE, ensuring that the core design philosophy is to fully take advantage of established scientific packages rather than unreliable re-implementation Each executor is equipped with dedicated tools for accessing software documentation and managing execution and I/O, including generating simulation scripts via retrieval-augmented generation from official software documentation
End-to-End Workflow Validation
The end-to-end evaluation of Paimon on the Li-ion diffusivity task demonstrates that adding agentic components like the critic and episodic memory progressively improves success rates across workflow stages The effect of the file contract is concentrated in the production MD stage, where it eliminates handoff errors by resolving semantic mismatches between different packages
Extensibility Mechanisms
Paimon achieves extensibility through two properties: first, all agents communicate through a shared interface so introducing a new executor requires no modification to the rest of the system second, the simulation procedure is consolidated into a knowledge module rather than embedded in the agents, meaning teaching Paimon a new workflow amounts to obtaining its natural-language description The capability of Paimon can be extended without modifying the existing agents and without predefined mappings between them
Methodology for Evaluation
The gated evaluation method involves setting validation gates at each stage, where a gate must be satisfied to advance, bypassing expensive MD simulations when criteria are met The diffusivity gate requires the reported diffusivity to match the reference within 0.001 × 10−6 cm2/s This approach allows for an ablation study at scale by bypassing expensive MD simulations when Paimon passes a gate The success rates at medium effort all surpass their low-effort counterparts, suggesting that both architectural components and reasoning effort improve success rates
Universal Machine Learning Potentials
The selection of universal machine learning interatomic potentials is modular, allowing an agent to retrieve potentials as needed, defaulting to the SevenNet family if not specified elsewhere The liquid electrolyte task uses SevenNet-0 with Grimme’s D3 dispersion correction with Becke–Johnson damping applied following reference [69] The solid electrolyte task uses SevenNet-0 without the D3 dispersion correction, following ref. [73] The orchestrator defaults to the SevenNet family and autonomously selects a potential if it is not specified elsewhere
Implementation Details
Paimon runs on a host computer separate from the computer cluster and communicates with it over SSH, integrated with the Slurm job scheduler Intermediate and final output files are saved in the working directory of each executor, and software versions are fixed by the Apptainer image Users interact with Paimon through a web interface that displays the orchestrator chat and the file system of the run The Paimon source code is available at https://github.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Enhance reliability by introducing a critic committee module that reviews each subtask at both pre-submission and completion stages, aggregating votes from multiple LLMs to obtain a reliable verdict of Pass, Fail, or Revise.
-
Improve execution accuracy by having executors generate simulation scripts via retrieval over official documentation instead of embedding simulation logic into predefined tool functions, shifting the extensibility bottleneck from tool implementation to documentation coverage.
-
Increase robustness against silent errors by employing repeated sampling where an LLM checks each script against a predefined set of binary pass or fail criteria and human experts assign error severity, which
measures how often the intended simulation procedure is correctly executed across N independent trials with the same prompt.
-
Enhance adaptability to new simulation domains by ensuring all agents communicate through a shared interface and simulation procedures are consolidated into natural language rather than embedded in agents, allowing Paimon to accommodate a new capability without modifying existing agents.
-
Improve workflow management by combining Paimon with external workflow management systems to store workflows in a structured form instead of relying on text-based episodic memory judged by LLMs, which would allow a calculation to be retrieved by a query rather than judged by an LLM.
Sources
- Evaluating Large Language Models Trained on Code
- Large Language Model Agent: A Survey on Methodology, Applications and Challenges
- CASCADE: Cumulative Agentic Skill Creation through Autonomous Development and Evolution
- DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulation
- PARC: An Autonomous Self-Reflective Coding Agent for Robust Execution of Long-Horizon Tasks
- S1-MatAgent: A planner driven multi-agent system for material discovery
- SimMOF: AI agent for Automated MOF Simulations
- Retrieval-Augmented Generation for Large Language Models: A Survey
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- OpenAI GPT-5 System Card
- Evaluating LLM-generated code for domain-specific languages: molecular dynamics with LAMMPS
- Harnessing AtomisticSkills for Agentic Atomistic Research
Related papers
- Structural contribution to light-induced gap suppression in Ta 2 NiSe 5
- Anatomy of fast current-induced skyrmion motion in synthetic antiferromagnets
- Magnetic control of an exciton-polariton condensate in a van der Waals magnet
- High-pressure elastic properties of GeO2 polymorphs up to 120 GPa
- Decoupling Strain-Rate Sensitivity and Deformation Length Scale Effects in Neutron-Irradiated Tungsten: A Coupled Nano-Indentation, HR-EBSD and Crystal Plasticity Study
- Magnetism modulation in Co3Sn2S2 by current-assisted domain wall motion