Measure the Deployment, Not Just the Model: Rethinking Hardware-Aware Model Selection for Microcontrollers
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: I'm Rosa.
Dev: And I'm Dev, here with Taro.
Rosa: We're looking at the paper: "Measure the Deployment, Not Just the Model".
Dev: Here's the gist of the paper. Deploying neural networks on low-power microcontrollers (MCUs) requires selecting model architectures under tight memory, latency, and energy constraints <ref:2606.15004#pg2> Existing workflows often simplify this process along one or more axes: static proxy costs such as FLOPs or parameters, treating one MCU as representative, and continuous-inference tests instead of deployed sensing schedules <ref:2606.15004#pg4> These assumptions can mis-rank Pareto-front candidates, miss infeasible deployments, and obscure schedule-dependent energy <ref:2606.15004#pg2> We present CREST (Cross-platform Runtime Evaluation and Search Tool), a deployment-realistic hardware-in-the-loop (HIL) neural architecture search (NAS) framework for MCU sensing systems <ref:2606.15004#pg3> CREST keeps the optimizer, HIL measurement boundary, logging, and replay workflow fixed while exposing workload, model family, target backend, schedule, quantization, and scoring policy as configurable axes <ref:2606.15004#pg4> This makes deployment effects experimentally separable within one reusable workflow <ref:2606.15004#pg4> CREST shows that deployment-realistic MCU NAS must jointly optimize model architecture, target platform, runtime schedule, and deployment policy rather than relying only on static proxy costs or continuous-inference measurements <ref:2606.15004#pg4>.
Rosa: Pruning redundant reasoning tokens The paper introduces CREST to address the fragmentation in MCU deployment by making the selection process configurable across multiple axes <ref:2606.15004#pg4>.
Dev: The authors simplify current hardware-aware model selection workflows in three ways: using proxy costs instead of measured deployment cost, treating one platform as representative of a broader class, and measuring continuous inference instead of sensing schedules <ref:2606.15004#pg4>.
Rosa: CREST's approach keeps the search loop, HIL measurement path, logging, and replay workflow fixed while exposing workload, model family, target backend, schedule, quantization, and scoring policy as configurable axes <ref:2606.15004#pg4>.
Title and authors: Rosa: Now we're going to dig into the specifics of what CREST is actually doing, since we know the old methods are flawed. So, basically, this paper is about how we choose AI models for tiny microcontrollers—like those in your smart sensors or small robots—and they’re saying the old way of choosing is pretty flawed.
Dev: Right. They’re talking about how we used to just rely on static measurements, like FLOPs or parameter counts, to pick a model architecture. But that doesn't actually tell you what it takes to run it in the real world on that specific hardware <ref:2606.15004#pg1>.
Rosa: Exactly. CREST, which is their framework, keeps the search loop and how they measure things fixed—that’s the HIL setup and the logging path—but they let you change all these other things: workload, model family, target backend, schedule options—and how you score the results <ref:2606.15004#pg4>.
Dev: So it’s not just about finding a small model; it’s about finding a model that works for your specific operating conditions on that specific chip. They show that deployment cost isn't one single number anymore <ref:2606.15004#pg2>.
Rosa: That’s the big thing, Dev. They found that deployment cost depends on the architecture, the target platform, how you schedule it—like running it continuously versus in scheduled bursts—and even your scoring policy <ref:2606.15004#pg4>.
Dev: And they prove this by showing that when you use static proxies or only look at continuous inference measurements, those measurements can actually misorder which models are best for deployment <ref:2606.15004#pg2>.
Rosa: Case Study one showed that the proxy ranking was totally wrong. It ranked the energy fronts in a way that didn't match what happened when you actually put the model on the board and ran it under real conditions <ref:2606.15004#pg4>.
Dev: That means if you’re using those initial static estimates to guide your search, you might end up chasing dead ends or missing the actual best deployment options <ref:2606.15004#pg4>.
Rosa: Case Study two looked at scheduling on a specific chip like that STM32 N657. They found that changing the runtime schedule completely changed the measured energy trade-offs, showing how much better a cadenced execution can be compared to continuous inference <ref:2606.15004#pg2>.
Dev: That highlights that you need to test different operational schedules because they fundamentally alter what you’re measuring in terms of power and latency on the chip itself <ref:2606.15004#pg2>.
Rosa: And Case Study three showed that your scoring policy—how you decide which model wins—can actually change which model gets selected based on things like validation metrics versus the measured inference energy budget <ref:2606.15004#pg4>.
Dev: So, it’s not just about the architecture itself; it’s about how that architecture interacts with the whole deployment environment to give you a final choice <ref:2606.15004#pg2>.
Rosa: The implication here is that for building anything on microcontrollers, you can't just optimize one thing in isolation anymore. You have to consider all these deployment factors together when you decide what model to build <ref:2606.15004#pg4>.
Dev: It shifts the entire process toward a hardware-in-the-loop search that respects the operational reality of the deployment environment rather than just theoretical performance numbers <ref:2606.15004#pg2>.
The paper's summary: Rosa: So we covered how CREST shows that deployment cost is complicated—it depends on workload, schedule, and policy—and now we’re looking at what they suggest to make those testing scenarios even more powerful. They are suggesting ways to make those testing scenarios more powerful by letting you move beyond just continuous inference measurements.
Dev: Right. One improvement is using the same setup to compare how energy changes when you run a model continuously versus running it in scheduled bursts on the same chip, like that STM32 N657 <ref:2606.15004#pg4>.
Rosa: That’s cool because they found that the native cadenced execution saves about three point six milliJoules per window, or maybe twenty-one percent compared to that continuous inference replay front <ref:2606.15004#pg4>.
Dev: That makes a big difference for battery-powered devices where you aren't running constantly, it shows you how much energy you actually save by being smart about the schedule <ref:2606.15004#pg2>.
Rosa: Then there’s this idea for application-level selection, which is Case Study three from before. They suggest replacing that basic Pareto exploration with a scalar score based on things like validation metrics and measured inference energy budget <ref:2606.15004#pg4>.
Dev: So instead of just looking at a curve, you can define a specific trade-off you care about—like scoring it as the validation metric minus some small fraction of the energy budget <ref:2606.15004#pg4>.
Rosa: Yeah, that lets users select models based on their own quality-energy exchange rate, making the choice more tailored to what they actually need <ref:2606.15004#pg4>.
Dev: That’s a nice addition because it moves beyond just finding a minimum energy point and lets you inject specific performance requirements directly into the search process <ref:2606.15004#pg4>.
Rosa: And they also showed that the conclusion you get from scoring isn't fixed by your policy, which is interesting because it means target-specific architectures can be selected based on what you choose to measure <ref:2606.15004#pg4>.
Dev: That’s a key point—the exact policy you pick can dictate whether the system chooses a compact classifier for one board or a wider one for another <ref:2606.15004#pg4>.
Rosa: So, they're pushing for this joint optimization of architecture and policy because that’s where the real deployment reality is found <ref:2606.15004#pg4>.
Dev: It moves the search from just finding a good model to finding the right model for your specific operational reality on that hardware <ref:2606.15004#pg4>.
Rosa: And it keeps things experimental by making those deployment effects separable, which is super helpful for testing different design choices in a controlled way <ref:2606.15004#pg4>.
The paper's improvements: Dev: So to wrap up, this paper, "Measure the Deployment, Not Just the Model: Rethinking Hardware-Aware Model Selection for Microcontrollers," basically argues that you can't just pick a model architecture in a vacuum anymore.
Rosa: That’s right. They show that deployment cost isn't just one static number like FLOPs; it depends on the workload, the target chip, how you schedule the operation—continuous versus burst—and even your scoring policy <ref:2606.15004#pg4>.
Dev: They use this CREST framework to keep everything fixed except those deployment variables so they can test them separately and see how they interact <ref:2606.15004#pg4>.
Taro: I just want to push on what this means for real-world failure modes, Rosa. If you're designing an autonomous system, knowing that your model choice depends so much on how you schedule it in the field is crucial for graceful recovery when things go wrong <ref:2606.15004#pg4>.
Rosa: It is crucial because it gives engineers a structured way to test if a certain architecture is actually going to work under your specific operational schedule, not just on paper <ref:2606.15004#pg4>.
Dev: The safety case changes because now you have to account for the scheduling decisions as part of the risk assessment for the entire system <ref:2606.15004#pg4>.
Taro: That makes sense because it ties directly into how we design robust control barrier functions; if the deployment environment dictates a specific schedule, your controller needs to be built around that reality <ref:2606.15004#pg4>.
Rosa: So what this paper does is move us past just finding the smallest model and toward finding the right model for a specific operational reality on low-power hardware <ref:2606.15004#pg4>.
Dev: It's about making selection decisions that actually work when you deploy them, which is something we need when we’re dealing with real-time loops and latency constraints <ref:2606.15004#pg4>.
Taro: And that moves the search from just finding a good model to finding the right model for your specific operational reality on that hardware <ref:2606.15004#pg4>.
Rosa: Exactly. It keeps things experimental by making those deployment effects separable, which is super helpful for testing different design choices in a controlled way, even when you're outside the lab <ref:2606.15004#pg4>.
Dev: This work sets up a much more realistic search space for hardware-aware model selection on microcontrollers by considering all these operational factors together <ref:2606.15004#pg4>.
Taro: The paper's title, "Measure the Deployment, Not Just the Model: Rethinking Hardware-Aware Model Selection for Microcontrollers," really sums up this shift in focus <ref:2606.15004#pg1>.
Rosa: Right. It’s a big step toward making AI deployment more practical and grounded in real-world constraints rather than just abstract math <ref:2606.15004#pg4>.
Dev: We'll take a quick break, and when we come back, we’re going to look at how other papers are trying to handle these complex scheduling issues in power distribution networks.
Conclusion: Rosa: So we've looked at how the paper "Measure the Deployment, Not Just the Model: Rethinking Hardware-Aware Model Selection for Microcontrollers" shows that you can't just pick a model architecture in a vacuum anymore.
Dev: That's right. They show that deployment cost isn't just one static number like FLOPs; it depends on the workload, the target chip, how you schedule the operation—continuous versus burst—and even your scoring policy.
Taro: I just want to push on what this means for real-world failure modes, Rosa. If you're designing an autonomous system, knowing that your model choice depends so much on how you schedule it in the field is crucial for graceful recovery when things go wrong.
Rosa: It is crucial because it gives engineers a structured way to test if a certain architecture is actually going to work under your specific operational schedule, not just on paper.
Dev: The safety case changes because now you have to account for the scheduling decisions as part of the risk assessment for the entire system.
Taro: That makes sense because it ties directly into how we design robust control barrier functions; if the deployment environment dictates a specific schedule, your controller needs to be built around that reality.
Rosa: So what this paper does is move us past just finding the smallest model and toward finding the right model for a specific operational reality on low-power hardware.
Dev: It's about making selection decisions that actually work when you deploy them, which is something we need when we’re dealing with real-time loops and latency constraints.
Taro: And that moves the search from just finding a good model to finding the right model for your specific operational reality on that hardware.
Rosa: Exactly. It keeps things experimental by making those deployment effects separable, which is super helpful for testing different design choices in a controlled way, even when you're outside the lab.
Dev: This work sets up a much more realistic search space for hardware-aware model selection on microcontrollers by considering all these operational factors together.
Taro: The paper's title, "Measure the Deployment, Not Just the Model: Rethinking Hardware-Aware Model Selection for Microcontrollers," really sums up this shift in focus.
Rosa: Right. It’s a big step toward making AI deployment more practical and grounded in real-world constraints rather than just abstract math.
Dev: We'll take a quick break, and when we come back, we’re going to look at how other papers are trying to handle these complex scheduling issues in power distribution networks.
Joseph Q. Zales, Pragya Sharma, Mani Srivastava
University of California, Los Angeles
eess.SY, cs.LG, cs.SY
Submitted: 2026-06-12
Updated: 2026-11-09
Comments: 23 pages, 11 figures, 14 tables, including appendices
Code: https://github.com/nesl/crest
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 91/100
The gist: Deploying neural networks on low-power microcontrollers (MCUs) requires selecting model architectures under tight memory, latency…
Key concepts
- CREST
- A framework for Hardware-in-the-Loop (HIL) neural architecture search (NAS). It keeps the core search loop fixed but makes deployment variables like workload, target backend, runtime schedule, and scoring policy adjustable. This allows researchers to experimentally separate how different deployment factors influence the final model selection.
- Deployment Cost
- The total energy and latency incurred when running a neural network on a specific microcontroller under real-world conditions. The paper argues that this cost is not just about the model's size (parameters) but is deeply intertwined with the target hardware, the execution schedule, and how it interacts with the workload.
- Runtime Schedule
- The specific timing of when inference tasks run on a device. CREST treats this as a search variable rather than assuming continuous operation. By defining deployment as a scheduled window (including sleep and non-inference time), it allows researchers to quantify how scheduling impacts energy consumption and performance metrics.
- Pareto Frontier
- The set of optimal trade-off points between competing objectives, such as model accuracy and energy consumption. CREST shows that the true Pareto frontier for deployment is not found by looking at static proxies or continuous inference alone, but by considering the interplay of workload, schedule, and hardware.
Terminology
Summary
Deploying neural networks on low-power microcontrollers (MCUs) requires selecting model architectures under tight memory, latency, and energy constraints <ref:2606. p. 015004#pg2> Existing workflows often simplify this process along one or more axes: static proxy costs such as FLOPs or parameters, treating one MCU as representative, and continuous-inference tests instead of deployed sensing schedules <ref:2606. p. 015004#pg4> These assumptions can mis-rank Pareto-front candidates, miss infeasible deployments, and obscure schedule-dependent energy <ref:2606. p. 015004#pg2> We present CREST (Cross-platform Runtime Evaluation and Search Tool), a deployment-realistic hardware-in-the-loop (HIL) neural architecture search (NAS) framework for MCU sensing systems <ref:2606. p. 015004#pg3> CREST keeps the optimizer, HIL measurement boundary, logging, and replay workflow fixed while exposing workload, model family, target backend, schedule, quantization, and scoring policy as configurable axes <ref:2606. p. 015004#pg4> This makes deployment effects experimentally separable within one reusable workflow <ref:2606. p. 015004#pg4> CREST shows that deployment-realistic MCU NAS must jointly optimize model architecture, target platform, runtime schedule, and deployment policy rather than relying only on static proxy costs or continuous-inference measurements <ref:2606. p. 015004#pg4>.
Pruning redundant reasoning tokens
The paper introduces CREST to address the fragmentation in MCU deployment by making the selection process configurable across multiple axes <ref:2606.15004#pg4>. The authors simplify current hardware-aware model selection workflows in three ways: using proxy costs instead of measured deployment cost, treating one platform as representative of a broader class, and measuring continuous inference instead of sensing schedules <ref:2606. p. 015004#pg4>. CREST's approach keeps the search loop, HIL measurement path, logging, and replay workflow fixed while exposing workload, model family, target backend, schedule, quantization, and scoring policy as configurable axes <ref:2606.15004#pg4>.
How the training signal is built
CREST is built for controlled comparison where a study can vary the workload, board, runtime schedule, or scoring policy while keeping the optimizer, HIL measurement boundary, logging path, and replay workflow fixed <ref:2606.15004#pg4>. The system follows a sample–evaluate–score loop in which each trial starts from a candidate proposed by the optimizer and ends with metrics that are scored and returned as optimizer feedback <ref:2606. p. 315004#pg4>. CREST uses Optuna [2] as the optimizer, supporting both scalar and multi-objective studies through the same metric interface <ref:2606. p. 315004#pg4>.
Where the gains come from
The evaluation demonstrates that deployment cost is a property of the model architecture, target, schedule, workload, and policy together <ref:2606.15004#pg4>. Case Study 1 shows that static-proxy ranking misorders measured-energy fronts <ref:2606. p. 815004#pg4>. Case Study 2 reveals that continuous-inference measurement is insufficient for a sensing deployment, as runtime schedule changes the measured Pareto geometry for STM32/OxIOD <ref:2606. p. 915004#pg4>. Case Study 3 shows that application-level scoring selects different audio models on different boards based on the policy <ref:2606. p. 015004#pg4>.
Deployment-Realistic Hardware-in-the-Loop NAS for Embedded Sensing Systems
CREST ships with built-in modules for two workload/model-family pairs, three MCU backends, continuous and cadenced runtime schedules, float32 and int8 PTQ representations, proxy and HIL measurement paths, and scalar or multi-objective policies <ref:2606.15004#pg4>. The same boundaries also define how CREST is extended by implementing corresponding module interfaces to address the monolithic-pipeline challenge C3 <ref:2606. p. 415004#pg4>.
Runtime Schedule as a Search Variable
CREST treats runtime schedule as part of the study definition because sensing deployments usually run as scheduled windows rather than saturated inference loops <ref:2606.15004#pg4>. The authors define a general sensing window as the sum of its phase costs, including inference, non-inference work, and sleep energy <ref:2606. p. 415004#pg4>. This controlled scope isolates the schedule effect tested in CS2 while keeping workload, model family, backend, and measurement path fixed <ref:2606. p. 515004#pg4>.
Evaluation of Deployment Effects
The case studies test specific deployment variables: CS1 varies policy and target backend, CS2 varies runtime schedule on the same target, and CS3 changes the workload, model family, task metric, target backend, and scoring policy <ref:2606.15004#pg4>. The results show that CREST exposes architecture-selection differences that are invisible to proxy-only, single-target, and single-schedule NAS workflows <ref:2606. p. 115004#pg4>. The right architecture is a property of the workload, target backend, runtime schedule, and policy together <ref:2606. p. 5 p. 515004#pg4>.
CREST Impact
The work presents CREST as an open-source HIL NAS framework for MCU sensing models <ref:2606.15004#pg4>. Across inertial odometry and audio classification, CREST shows that deployment cost is not captured by a static proxy, a single board, or a continuous-inference measurement <ref:2606. p. 115004#pg4>. By keeping the optimizer, HIL measurement boundary, logging, and replay workflow fixed while varying workload, target backend, runtime schedule, and scoring policy, CREST makes these deployment effects experimentally separable <ref:2606. p. 015004#pg4>. The right architecture is a property of the workload, target backend, runtime schedule, and policy together <ref:2606.15004#pg4>.
Acknowledgments
This research was funded by DEVCOM ARL under award noW911NF1720196, NIH under award no1P41EB028242, and NSF under award CNS no2325956 <ref:2606.15004#pg4>. The authors used generative AI tools, including Codex/ChatGPT, to assist with software development tasks such as documentation, unit test generation, boilerplate implementation, debugging, and parts of the codebase implementation <ref:2606.15004#pg4>. The authors also used generative AI tools during manuscript preparation for brainstorming, outlining, and editing assistance <ref:2606. p. 715004#pg4>. The authors conceived the architecture, designed the study, conducted the experiments, and verified all AI-assisted code and text <ref:2606.15004#pg4>. The authors take full responsibility for the final artifact and manuscript <ref:2606. p. 015004#pg4>.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Deployment-realistic NAS framework CREST keeps
the optimizer, HIL measurement boundary, logging, and replay workflow fixed while exposing workload, model family, target backend, schedule, quantization, and scoring policy as configurable axes.
This allows forexperimentally separable
deployment effects across different deployment conditions. -
The system can evaluate how runtime schedule changes the measured Pareto geometry by comparing continuous-inference energy per inference against cadenced execution energy per window on the same target. This reveals that
the native cadenced front saves a median 3.6 mJ per window, or 2.1%, relative to the continuous-inference replay front.
-
CREST supports application-level deployment selection by allowing users to replace
Pareto exploration with a scalar score over validation macro-F1 and measured inference energy
in Case Study 3. This enables selection of models based on a user defined quality energy exchange rate, such asscore = MacroF1val − 0.10 · Einf/Einf budget.
-
The system can select target-specific model architectures by showing that the score-based conclusion is not unique to a fixed policy setting. For instance, the Portenta H7 CM7 selects a compact classifier while the STM32 N657 selects a wider classifier, illustrating how
the exact policy changes which model is selected.
Sources
- MLPerf Tiny Benchmark
- Measuring what Really Matters: Optimizing Neural Networks for TinyML
- Edge Impulse: An MLOps Platform for Tiny Machine Learning
- CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
- Not All Ops Are Created Equal!
- HW-NAS-Bench:Hardware-Aware Neural Architecture Search Benchmark
- $\mu$NAS: Constrained Neural Architecture Search for Microcontrollers
- TinyML Platforms Benchmarking
- TinyTNAS: GPU-Free, Time-Bound, Hardware-Aware Neural Architecture Search for TinyML Time Series Classification
- Hello Edge: Keyword Spotting on Microcontrollers
Related papers
- Industrial Load Modeling and Optimization for Market-Based Interaction with Power Systems
- Exact and Approximate Convex Reformulation of Linear Stochastic Optimal Control with Chance Constraints
- Certified Set Convergence for Piecewise Affine Systems
- Symbolic Control for Stochastic Systems via Finite Parity Games
- Multiobjective optimization-based design and dispatch of islanded, hybrid microgrids for remote, off-grid communities in sub-Saharan Africa
- Physiology-informed layered sensing for intelligent human-exoskeleton interaction