Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

arXiv:2610.12369 · cs.RO, cs.CV · Submitted 2026-10-08 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: Today's paper: "Embodied Turing Machines".

Dev: The gist The embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy.

Rosa: First, who's behind it and why it matters.

Title and authors: Rosa: To get into more detail on this paper, we’re looking at how they introduce this idea of COAP as a new way to handle these embodied problems.

Dev: They are proposing that instead of relying on a large model running at test time, the code itself acts as the policy, storing all the state in code variables and rules written directly in code logic >

Taro: That means no need for an external model call during execution, which is a big deal for real-time robotics because it’s much faster than having a vision language model constantly make decisions >

The paper's summary: Rosa: The main idea they summarize is that this approach gives the system explicit state, meaning the state isn't hidden inside a black box model; it’s stored in code that you can inspect and save >

Dev: They emphasize three main advantages for COAP: first, explicit state, which lets you store and check it easily; second, execution, because the code makes decision making controllable so you can recover from failures flexibly by backtracking and re-attempting; and third, extensibility where new tasks can reuse or extend a shared library >

Taro: So it’s not just about making decisions fast; it’s about having a structure where if something goes wrong, the code itself knows how to try again in a controlled way, which is important when the world doesn't behave as expected >

The paper's improvements: Rosa: When we look at what they suggest as improvements to this idea, the focus shifts toward recursive self-improvement.

Dev: They argue that these explicit state and controllable execution features make COAP suitable for recursive self-improvement, where coding agents develop the library in a closed loop, and each change is explicit and controllable >

Taro: That closed loop idea is interesting; it means the AI system can actually learn to improve its own tools by developing a new library all on its own through this code interaction >

Conclusion: Rosa: So, to wrap up this discussion on "Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement," the paper suggests that COAP can be a powerful new direction for embodied tasks by making the state explicit and using code as the universal interface >

Dev: It shows that this approach, with a success rate of seventy point two four percent on RoboDojo's forty-two bimanual tasks, has real potential to benefit other systems like VLAs or Agent Harnesses because it produces successful trajectories quickly >

Taro: I just think the idea of coding agents developing the library in a closed loop through COAP is what really sets this apart for long-term progress in robotics >

Rosa: Well, it moves us toward a system where code isn't just an interface but the actual decision-maker, which could open up a lot of possibilities for how robots learn and adapt to novel situations.

Dev: I agree, because that explicit state management and fast execution means we can finally build systems that are truly controllable online without needing massive models constantly running in the background.

Taro: It seems like the real promise is in those coding agents being able to iterate on their own libraries, making progress happen through explicit, controllable changes rather than just hoping a big model finds the right path.

Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

Nanyang Technological University

cs.RO, cs.CV

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: 31 pages, 19 figures

Code: https://github.com/Awomo-WestlakeDI/Awomo-0.5

Project page: https://mmlab.hk/research/PhysicalRSI

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 80/100

The gist: The gist The embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy.

Key concepts

Embodied Turing Machine Model
This model treats a robot and its surroundings like a computer tape. The tape stores two types of information: the environment's state (like where objects are) and the robot's state (like its arm position). Rules, or policy, dictate how the robot reads this tape and changes it.
Code-Only-as-Policy (COAP)
COAP is a method where code directly implements the Embodied Turing Machine. Instead of using a large model at runtime, the policy is written as explicit code storing state in variables and rules in logic. This makes every decision transparent and executable without needing a separate model during operation.
Explicit State
In COAP, the robot's situation is stored directly within the code itself. This state is inspectable, persistent, and accurate because it resides in standard code variables rather than being inferred by a black-box model. This explicit state allows for precise action planning and reliable recovery.
Recursive Self-Improvement (RSI)
COAP enables RSI where coding agents develop the robot's policy in a loop. The code acts as a universal interface, allowing agents to modify the library, run tests on development layouts, find improvements, and update the code iteratively. This creates a closed loop for continuous self-improvement.

Terminology

Summary

The gist The embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy.

Embodied Turing Machine Model

The paper models a robot and its environment as an Embodied Turing Machine, where the tape holds two kinds of state: the environment state (e.g., object poses and their spatial relations) and the robot state (e.g., arm pose and reachable poses) Both kinds of state are accurately observable, with the robot state available directly through proprioception and object poses recovered from RGB images using measured object geometry. A policy that acts on this state gains two advantages: more accurate state enables more precise action, and when objects are rearranged, only the tape changes while the same rules (policy) still apply.

Code-Only-as-Policy (COAP)

Code-Only-as-Policy (COAP) proposes a paradigm where code is a white box that implements the Embodied Turing Machine directly, storing the state in code variables and rules as code logic. This approach differs from Vision-Language Models (VLAs) and Agent Harnesses because it makes every decision from explicit state stored in code, meaning no model is required at runtime. The policy is therefore code only and reaches a 70.24% success rate on RoboDojo.

Advantages of COAP

COAP offers three main advantages over VLAs and Agent Harnesses: (i) Explicit State, (ii) Execution, and (iii) Extensibility.

(i) Explicit State means the state can be stored in code, which is inspectable, persistent, and accurate.

(ii) Execution allows for controllable decision-making, flexible recovery from failures by backtracking and re-attempting, and fast/cheap online execution.

(iii) Extensibility enables new tasks to reuse, inherit, or extend the shared library, allowing capabilities to accumulate over tasks.

Code as a Universal Interface and RSI

The explicit state and controllable execution make COAP suitable for recursive self-improvement (RSI), where coding agents develop the library in a closed loop, and each change is explicit and controllable. The code becomes a universal interface to the physical world, taking over the roles of VLAs, prompts and skills of Agent-as-Policy, and memory of Harness VLA. Code iterates efficiently, and each change is controllable through a development loop where coding agents run the current code on development layouts to find improvements.

Performance and Limitations

COAP achieves a success rate of 70.24% on RoboDojo's 42 bimanual tasks, which is 38.9 percentage points above the current state-of-the-art method. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. Limitations include simulation only, as perception relies on public configuration and assets of the simulated benchmark, offline development costs time, and success on randomized layouts is lower than on standard layouts.

Data Engine Potential

COAP can serve as an efficient data engine for VLAs and Agent Harnesses because once a task program works, it runs on a CPU without a model and can be executed in parallel, yielding successful trajectories quickly. These trajectories provide the explicit state and decision behind every action, which can train a VLA or supervise an Agent Harness inside.

Conclusion

The paper proposes that the embodied world is an Embodied Turing Machine, introducing Code-Only-as-Policy (COAP) as a new direction where the state is explicit, rules are code, and coding agents improve the code. COAP's success in 70.24% of episodes and its advantages in Explicit State, Execution, and Extensibility suggest it can benefit VLAs, Agent-as-Policy, Harness VLA, and RSI.

References

[1] Michael Ahn et al. Do as I can, not as I say: Grounding language in robotic affordances. In Conference on Robot Learning (CoRL), 2022

[7] Tianxing Chen et al. RoboDojo: A unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. arXiv preprint arXiv:2607.04434, 2026

[59] Ruiyang Si et al. Fewer tokens, better action: GPT-6 Astra robot agents with 14% higher success rate but 65% fewer tokens. arXiv preprint arXiv:2610.01939, 2026

[47] Yuran Wang et al. OpenWAM: An open, modular exploration towards systematic world-action model pretraining. arXiv preprint arXiv:2609.07398, 2026

[55] Yixian Zhang et al. Harness VLA: Steering frozen VLAs into reliable manipulation primitives via memory-guided agents. arXiv preprint arXiv:2607.08448, 2026

[39] Zeyu Shen et al. EmbodiedSWE: Coding agents for long horizon dexterous robotics. arXiv preprint arXiv:2609.27308, 2026

[45] Guanzhi Wang et al. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research (TMLR), 2024

[58] Xueyang Zhou et al. LIBERO-PRO: Towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827, 2025

[48] Wei Wu et al. A pragmatic VLA foundation model. arXiv preprint arXiv:2601.18692, 2026

[57] He Zhang et al. Hy-Embodied-0.5-VLA: From vision-language-action models to a real-world robot learning stack. arXiv preprint arXiv:2606.14409, 2026

[53] Shunyu Yao et al. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

[54] Cheng Yin et al. SimpleMemVLA: A simple but effective native-video memory for vision-language-action models. arXiv preprint arXiv:2609.05533, 2026

[41] Ishika Singh et al. ProgPrompt: Generating situated robot task plans using large language models. In IEEE International Conference on Robotics and Automation (ICRA), 2023

[43] Jiayi Su et al. GPT 6 Astra as an embodied policy. Technical report, https://anonymous-report-421.github.io/public-website/, 2026

[50] John Yang et al. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), 2024

[55] Yixian Zhang et al. Harness VLA: Steering frozen VLAs into reliable manipulation primitives via memory-guided agents. arXiv preprint arXiv:2607.08448, 2026 <ref:2610.

Improvements for AI systems

  1. The system can achieve recursive self-improvement (RSI) by having coding agents develop the library in a closed loop, and each change is explicit and controllable, as this leverages Code-Only-as-Policy's advantages in Extensibility and Execution.

  2. The AI system can operate without a model during online execution because it utilizes Code-Only-as-Policy (COAP), which allows the robot to run on a CPU with a median decision time of 0.3 ms, achieving efficiency that is one CPU core, no model call.

  3. The system can recover from failures flexibly by utilizing explicit state tracking, as the code backtracks and re-attempts, enabling recovery when something went wrong in episodes where the program noticed the problem itself in 76%.

  4. The AI system can be rapidly adapted to new tasks by reusing a shared library, as described by COAP's extensibility where a new task reuses, inherits, or extends the shared library, leading to development times that are fast to evolve and allowing capabilities to accumulate over tasks.

  5. The system can function as an efficient data engine for other AI systems by producing successful trajectories that can be used to train other models, as COAP produces successful trajectories quickly and reproducibly which can then train a VLA, and the recorded states and decisions can supervise the model inside an Agent Harness.

Sources

Related papers