Embedded Arena: Iterative Optimization via Hardware Feedback
summary
The gist
Embedded devices from wildlife monitoring stations to clinical wearables require local AI inference due to latency, communication…
In short
This work introduces a hardware-in-the-loop agent arena where an LLM autonomously refines embedded software and models on real hardware. Unlike previous attempts that failed without physical feedback, this system successfully deploys optimized vision and audio models, achieving significant compression while meeting strict constraints on memory, power, and temperature. This enables battery-free operation for devices like wildlife monitors.
Key concepts
- Hardware-in-the-Loop Arena
- A testing environment where an LLM agent generates code or model changes, compiles them, flashes them onto real microcontrollers (MCUs), and receives physical feedback signals. This allows the agent to close the loop and iteratively optimize software against actual hardware limitations.
- Task Instantiations on MCUs
- Specific optimization goals tailored for resource-constrained devices. These include maximizing inference accuracy, minimizing peak power consumption, and managing thermal rise under a given load. The agent's goal is to find a deployable artifact that minimizes the defined objective function while satisfying all physical constraints.
- Agentic Co-optimization
- Using an LLM agent to autonomously navigate the complex pipeline of model compression, firmware compilation, and hardware testing. The agent treats the deployment process as a search problem, iteratively refining artifacts based on real-world performance metrics rather than relying solely on theoretical or synthetic scores.
Terminology used across episodes
This episode discusses
- Embedded Arena: Iterative Optimization via Hardware Feedback · Paper Radio
- MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
- ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
- Evaluating Large Language Models Trained on Code
- Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimization
- TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
- Distilling the Knowledge in a Neural Network
- Towards a Science of Scaling Agent Systems
- Skilled AI Agents for Embedded and IoT Systems Development
- MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement
- EmbedAgent: Benchmarking Large Language Models in Embedded System Development
- Neural Architecture Search with Reinforcement Learning
The paper
Embedded Arena: Iterative Optimization via Hardware Feedback · Read on arXiv
Zhihan Zhang, Alexander Le Metzger, Jiuyang Lyu, Chun-Cheng Chang, Jiayi Shao, Yujia Liu, Emmanuel Azuh Mensah, Edward Wang Kurtis Heimerl Gregory D. Abowd Shwetak Patel Natasha Jaques Vikram Iyer
University of Washington University of California San Diego Northeastern University
As AI moves into the physical world, from robots to environmental sensors to medical wearables, it increasingly relies on local inference to meet latency, communication, or privacy constraints. Optimizing models for heterogeneous edge hardware such as microcontrollers (MCUs) requires simultaneously satisfying hard physical constraints on memory, power, and temperature while preserving accuracy, a multidimensional optimization that is today performed manually by experts. We ask whether an LLM agent can autonomously navigate this complex, multi-turn pipeline guided by real hardware feedback, and introduce a hardware-in-the-loop agent arena in which the agent iteratively refines both model and firmware by compiling, flashing, and measuring on real hardware, enabling closed-loop optimization. Frontier models, including Claude Opus 4.7 and Gemini 3.1 Pro, fail entirely without hardware feedback (0% deployment success), whereas our hardware-in-the-loop formulation achieves the first successful deployment within three iterations and can surpass human expert results within seven. This agentic co-optimization achieves 250x compression for vision models with <3.3% accuracy loss and 400x for audio with <6% Feature Error Rate loss, enabling battery-free operation on a commercial MCU via solar harvesting. We demonstrate practical impact in two real-world systems: an elk-detection camera trap (96.7% accuracy) and a phonetic-transcription wearable (8.37% FER) for child development research.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Embedded Arena: Iterative Optimization via Hardware Feedback".
Jane: Here's the gist of the paper. Embedded devices from wildlife monitoring stations to clinical wearables require local AI inference due to latency, communication, or privacy constraints Optimizing models for heterogeneous microcontrollers (MCUs) requires simultaneously satisfying hard physical constraints on memory, power, and temperature while preserving accuracy We ask whether an LLM agent can autonomously navigate this complex, multi-turn pipeline guided by real hardware feedback, and introduce a hardware-in-the-loop agent arena in which the agent iteratively refines both model and firmware—compiling, flashing, and measuring on real hardware—to enable closed-loop optimization Frontier models, including Claude Opus 4.7 and Gemini 3.1 Pro, fail entirely without hardware feedback (0% deployment success), whereas our hardware-in-the-loop formulation achieves the first successful deployment within three iterations and can surpass human expert results within seven This agentic co-optimization achieves 250× compression for vision models with <3.3% accuracy loss and 400× for audio with <6% Feature Error Rate loss, enabling battery-free operation on a commercial MCU via solar harvesting Hardware-in-the-Loop Arena We present a hardware-in-the-loop arena in which an LLM agent must generate, compile, flash, and iteratively refine software subject to physical feedback signals returned by real hardware The formalism we develop in §2.1 is general to any constrained hardware target The agent’s goal is to find a deployable artifact s⋆ that minimizes the task objective: s⋆ = arg min s∈S f(M(s, h)) s.t.
Tom: Φ(s, h) = 1 The benchmark score B for a task is defined as B = f(M(s⋆, h)) Task Instantiations on MCUs The framework is instantiated across three physically distinct axes: model compression, power minimization, and thermal management The agent’s task is to find a deployable artifact s⋆ that minimizes the task objective: s⋆ = arg min s∈S f(M(s, h)) s.t.
Jane: Φ(s, h) = 1 Task 1: Model Compression involves maximizing ondevice inference accuracy macc = Macc(s, D) ∈
0, 1: subject to Φ(s, h) = 1 Task 2: Power Minimization targets the model inference workload with objective fpeak(m) = Ipeak subject to macc ≥ m0 acc Task 3: Thermal Management minimizes peak temperature rise ∆T = Tpeak − Tamb under a controlled ambient load Model Compression Results HIL feedback is necessary for successful deployment, as frontier LLMs fail entirely without it For vision models on the MAX78000, Gemini 3.1 Pro reaches a top mAP50-95 of B = 18.03% The agentic pipeline achieves 250× compression for vision models with <3.3% accuracy loss and 400× for audio with <6% Feature Error Rate loss Power Minimization Analysis Agents may change clock frequencies, sleep scheduling, peripheral gating, and firmware scheduling to minimize peak current or total energy HIL raises successful deployments from 15.9% (Sco) and 38.3% (Doc) to 46.7% (Table 3) for Peak-current minimization on the MAX78000 Thermal Management Evaluation The objective is jointly penalizes peak temperature rise and total task latency: fheat(m) = w∆T ∆T + wτ τ HIL achieves the highest success rates in both conditions, reaching 26.7% at room temperature and 33.3% under contact heating Applications and Real-World Deployments We demonstrate practical impact in two real-world deployments developed in direct collaboration with domain practitioners The HIL pipeline compressed a YOLOv8 elk-detection model to the MAX78000, achieving 96.7% accuracy at under 36 mW during active inference and 7 mW average power Discussion and Conclusion Hardware-in-the-loop feedback raises deployment success from 0% to surpassing human expert results within seven iterations The underlying recipe of reducing a physical objective to a programmatic score that LLM agents can iteratively optimize could generalize well beyond this setting Limitations Our evaluations have three limitations, including the cap of Nmax = 10 iterations due to compounding real resource costs We also evaluated six closed-source frontier models, and our instantiations cover three MCU families and two input modalities Broader Applicability The underlying recipe of reducing a physical objective to a programmatic score that LLM agents can iteratively optimize could generalize well beyond this setting First, additional physical constraints can be folded into the loop, from electromagnetic emissions and vibration tolerance to long-horizon properties like component aging Second, adding a vision frontend to wrap instruments such as mechanical balances, manual titrations, and legacy measurements common in wet labs can enable agents to use a vast array of legacy non-digital measurements Third, this formulation can be extended to scientific domains with unbounded optimization landscapes, such as material design or catalyst discovery References
1: KoelLabs/xlsr-english-01 · Hugging Face, June 2025
3: Andrej.
Tom: karpathy/autoresearch, June 2026
48: John Yang, Carlos E.
Title and authors: Tom: We’re talking about Embedded Arena: Iterative Optimization via Hardware Feedback. The title itself tells you exactly what this paper is about—it’s all about using hardware feedback to iteratively optimize software for embedded devices >
Jane: It suggests a closed loop system where the AI agent isn't just guessing; it's constantly testing and getting real physical feedback on how well its code actually performs on the target hardware >
Lu: The authors are a solid team from places like Washington University and San Diego, all focusing heavily on embedded systems and machine learning representation learning >
Meng: I’m interested in the authors because they seem to have a good grasp of the practical side, since this whole thing is about making things actually deployable in the real world >
Lalam: The paper sets up this arena as a general formalism that works for any constrained hardware target, which is really ambitious because it’s not tied to one specific chip like just one model >
Tom: So what does that mean for us? It means they aren't just solving a problem for the MAXseventy-eight thousand; they are giving us a blueprint that could apply to any MCU family >
Jane: Right, it suggests that the core idea here is translating these complex physical constraints—memory, power, and temperature—into a mathematical score that an AI agent can then optimize toward >
The paper's summary: Tom: The summary points out the core challenge: optimizing models for heterogeneous microcontrollers requires meeting hard physical constraints on memory, power, and temperature simultaneously while keeping accuracy high >
Jane: And the main question they pose is whether an LLM agent can navigate this complex, multi-turn pipeline guided by real hardware feedback to solve that optimization problem autonomously >
Lu: They introduce the hardware in the loop arena where the agent iteratively refines both model and firmware through compiling, flashing, and measuring on real hardware >
Meng: So instead of just optimizing for a fixed number or a benchmark score, they are creating a system where the reward is measured by instruments attached to the device under test >
Lalam: They formalize this as finding an artifact s star that minimizes the task objective subject to satisfying constraints, which they call phi of s and h equaling one >
Tom: That’s a very cool way to frame it; transforming deployment into a constrained optimization problem over real hardware >
The paper's improvements: Jane: One big improvement is that they show that frontier models, like Claude Opus four point seven and Gemini three point one Pro, fail entirely without this hardware feedback loop > <ref:2606.16190#pg1,Claude Opus 4.7 and Gemini 3.1 Pro, fail>
Lu: That’s a strong result because it proves that the current methods used by practitioners to navigate these trade-offs are not robust enough on their own >
Meng: It means that the manual expertise people spend weeks optimizing single models is currently just brittle and hard to scale when you want to deploy things quickly >
Lalam: They show that this agentic co-optimization can achieve two hundred fifty times compression for vision models with less than three point three percent accuracy loss and four hundred times for audio with less than six point zero percent feature error rate loss >
Tom: So the improvements boil down to showing that this method works where previous methods failed, allowing them to find artifacts that actually work on the device, not just those that pass software checks >
Conclusion: Jane: So to wrap up, the main implication is moving away from manual expert optimization toward an AI agent finding a deployable artifact through this closed loop feedback mechanism >
Tom: They’re showing that this system can actually achieve deployment success rates of eighty percent on the hardest tasks compared to zero without hardware feedback >
Lu: It moves the process into a much more accessible workflow where the physical world dictates what we are looking for by making a programmatic score for an artifact that works on hardware >
Meng: From my side, it means we can automate things that used to require weeks of manual tuning, like power minimization and thermal management without needing an expert present >
Lalam: The paper "Embedded Arena: Iterative Optimization via Hardware Feedback" shows how this iterative optimization of models and firmware can lead to real-world impacts like a solar-powered camera trap for elk detection achieving ninety six point seven percent accuracy under seven milliwatts >
Tom: It’s a lot of stuff, but it really shows how we need that kind of system to get complex AI onto the edge where the constraints are this tight >
More episodes
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game
- 1905.10490-Finding new routes for integrating Multi-Agent Systems using Apache Camel
- 2401.10816-Co-Pilot for Health: Personalized Algorithmic AI Nudging to Improve Health Outcomes
- 2503.01935-MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents
- 2509.19405-Improving Outdoor Multi-cell Fingerprinting-based Positioning via Mobile Data Augmentation
- 2602.15198-Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems