A Hierarchical Optimisation Framework for Integrated Electric-Hydrogen-Transport Systems

arXiv:2607.25776 · eess.SY, cs.SY · Submitted 2026-07-28 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "A Hierarchical Optimisation Framework for Integrated Electric-Hydrogen-Transport Systems".

Dev: Integrated electric–hydrogen–transport systems require a hierarchical optimisation framework that couples vehicle scheduling and downstream energy dispatch through dynamically generated demand to achieve low service delays and minimum operational costs.

Rosa: First, who's behind it and why it matters.

Paper summary: Rosa: We’ve covered how this paper models an integrated electric-hydrogen-transport system, or EHTS, and proposes a hierarchical optimisation framework to link vehicle scheduling and energy dispatch through a sequential, demand-driven two-layer structure. Dev It basically argues that the strong coupling between these two areas causes operational headaches when you try to manage them together in real time.

Taro: The paper introduces a vehicle scheduling layer where they use a solver-free greedy heuristic, or SFGH algorithm, specifically designed to avoid solving optimization problems repeatedly. Rosa This allows for nonpreemptive service and sequential assignment of EV charging and HV refueling requests within an interval without needing heavy solvers running constantly.

Dev: The output from that scheduling layer—the charging and refueling demands—is then passed down to the energy dispatch layer, which is where the multi-energy management happens. Taro This second layer employs a deep reinforcement learning approach, specifically a TD3 agent, to figure out how to manage battery operation, hydrogen tank operation, and even PV generation allocation.

Rosa: The goal for this second layer is to minimize the overall operational cost of the EHTS while still making sure all the scheduled transport demands are satisfied by meeting those energy needs. Dev So they’re optimizing cost against service satisfaction simultaneously.

Taro: They define a specific state space for that DRL agent, which includes things like battery and hydrogen tank states, solar generation, and the current vehicle requests. Rosa It’s a complex set of inputs the AI agent needs to process to make its decisions about charging or refueling actions.

Dev: The action space for the TD3 agent is also continuous and multi-dimensional, involving actions for both the battery—like charging or discharging—and the hydrogen tank—determining flow requests. Taro This means it’s learning a sophisticated way to coordinate energy storage and fuel supply based on all those inputs.

Rosa: The paper models the physical aspects of this system too, which includes PV output calculations based on temperature-dependent panel efficiency, and how the electrolyser converts allocated PV power into hydrogen. Dev They model the state-of-charge dynamics for both the lithium-ion battery and the hydrogen tank based on supply from electrolysis or pipelines and demand from refuellers.

Taro: These models help create a complete picture of what's happening physically within the EHTS, connecting the abstract optimization to concrete physical constraints like energy conversion and storage dynamics. Rosa It’s about making sure the AI isn’t just making theoretical decisions but those decisions are physically possible within the system's limitations.

Dev: The paper also sets up two main hierarchical objectives: an energy dispatch objective focusing on reducing costs, C ed t, and a vehicle scheduling objective targeting minimizing total service delay, C vs t. Taro This shows they are optimizing two distinct but related things at different levels of the hierarchy.

Rosa: The results show that the SFGH algorithm consistently finds the lowest service delay among all compared scheduling strategies, and the TD3 approach achieves the lowest annual operational cost among evaluated DRL and rule-based methods. Dev These findings confirm their claims about each layer's specific performance metrics in isolation.

Taro: And when looking at generalisation, they show that these performance metrics hold up across different scenarios without needing to be retrained, which is a really strong point for practical use. Rosa It moves the discussion toward how this framework can actually be used in diverse real-world conditions like those tested.

Dev: So the paper’s core contribution is providing a specific structure that explicitly models and links these two layers sequentially to solve the operational challenge of integrated EHTSs.

Conclusion: Rosa: The paper, "A Hierarchical Optimisation Framework for Integrated Electric-Hydrogen-Transport Systems," essentially presents a structured way to handle the complexity of electric and hydrogen vehicles coexisting in transport systems. Dev It claims that by coupling vehicle scheduling with energy dispatch through dynamically generated demand, you can achieve low service delays and minimum operational costs simultaneously.

Taro: The key elements are the SFGH heuristic for scheduling, which is efficient because it avoids repeated optimization solves, and the TD3 agent for dispatching energy resources. Rosa This combination of a solver-free approach with a deep reinforcement learning method is what drives their claimed success in this framework.

Dev: What this means simply is that instead of looking at the cars and the power system separately, you build a system where one layer manages the scheduling, and that schedule feeds directly into another layer managing the energy. Taro The authors stress that this design is modular so you can easily swap out parts if you want to use a different scheduling algorithm or dispatch method later on without rebuilding everything.

Rosa: They point out that the framework is designed to preserve operational dependency between these two layers while keeping things computationally tractable for real-time operation. Dev So, it’s an architecture that works because of how the parts talk to each other, not just because of one magic algorithm.

Taro: It’s about having a clear separation between the scheduling decisions and the energy management decisions happening at different levels in time. Rosa This framework gives us a practical blueprint for how to design these integrated infrastructure systems going forward.

Fulong Yao, Yiming Xu, Liana Cipciganb, Maurizio Albanob, Naeima Hameda, Nima Valizadeha, Omer Rana

School of Computer Science and Informatics Cardiff University School of Engineering Cardiff University

eess.SY, cs.SY

Submitted: 2026-07-28

Updated: 2026-10-17

Project page: https://flyao123.github.io/EHTS-Dashboard

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: Integrated electric–hydrogen–transport systems require a hierarchical optimisation framework that couples vehicle scheduling and downstream energy dispatch through dynamically generated demand to

Key concepts

Vehicle Scheduling Layer: This layer uses a solver-free greedy heuristic (SFGH) to assign electric vehicles (EVs) and hydrogen vehicles (HVs). Its goal is to create feasible schedules that minimize vehicle waiting times and charging/refuelling delays by considering vehicle availability and infrastructure constraints.
Energy Dispatch Layer: This layer uses a Twin Delayed Deep Deterministic Policy Gradient (TD3) agent to manage energy resources. It decides how much power from solar PV, grid, electrolyser, battery storage, and hydrogen tank should be used or allocated in real-time to meet the demands generated by the scheduling layer while minimizing total operational costs.
Sequential Coupling: The two layers are linked sequentially: vehicle scheduling first generates the required charging and refuelling demands. These dynamically generated demands then serve as input for the energy dispatch layer, which optimizes energy usage based on those specific needs. This dependency ensures that energy management directly supports transport operations.
Solver-Free Greedy Heuristic (SFGH): This is a method used in the vehicle scheduling layer to make quick, feasible decisions without needing to solve complex mathematical optimization problems repeatedly. It prioritizes assignments based on waiting times and resource availability, making it fast enough for real-time operation.

Terminology

Ablation — 1× in this paper · explained in 1 episode
Deep Reinforcement Learning — 1× in this paper · explained in 3 episodes
Deep Reinforcement Learning (DRL) — 1× in this paper · explained in 2 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Robustness — 1× in this paper · explained in 5 episodes

Summary

Integrated electric–hydrogen–transport systems require a hierarchical optimisation framework that couples vehicle scheduling and downstream energy dispatch through dynamically generated demand to achieve low service delays and minimum operational costs.

How it works

The proposed framework consists of two interdependent layers: a vehicle scheduling layer and an energy-dispatch layer, which are sequentially coupled through dynamically generated charging and refuelling demand. The vehicle scheduling layer uses a solver-free greedy heuristic (SFGH) algorithm to determine feasible assignments based on vehicle waiting times, charging/refuelling requirements, and pile availability. This layer aims to minimise service delay by scheduling heterogeneous EV and HV charging/refuelling requests.

Energy Dispatch Layer

The energy dispatch layer coordinates the battery storage, hydrogen tank, and surplus PV generation during the same interval to reduce overall operational costs while satisfying real-time demands. A Twin Delayed Deep Deterministic Policy Gradient (TD3) agent is employed to learn an effective dispatch policy. The state space for the DRL agent is defined as s ed t = h p elc t−L:t, ph2 t−L:t, Pum t−L:t, Hrc,hrs t−L:t, Pev,ua t, Hhv,ua t, SOCbt t+, SOCtk, hsin, hcos, dsin, dcos(i). The action space is a three-dimensional continuous vector a ed t = a bt t+, atk, a pv, t+ where the battery action determines the charging/discharging request and PV allocation, and the hydrogen-tank action determines the flow request.

Key Components of Modelling

The EHTS model integrates energy supply (PV generation, grid connection, and hydrogen pipeline), energy storage (battery and hydrogen tank), energy conversion (electrolyser), and vehicle service infrastructures.

PV Modelling involves calculating the PV output P pv t using equations that account for temperature-dependent panel conversion efficiency.
Electrolyser Modelling converts allocated PV power into hydrogen, with H el,tk t defined by P pv,el t and η el.
Battery Modelling describes the state-of-charge (SOC) dynamics of the lithium-ion battery considering charging/discharging from PV and grid electricity.
Hydrogen Tank Modelling models the SOC dynamics of the hydrogen tank based on supply from electrolyser or pipeline and demand from refuellers.

Hierarchical Objectives and Performance

The framework establishes two hierarchical objectives: an energy dispatch objective focused on reducing operational costs C ed t = C gd t + C pl t + C pv t, and a vehicle scheduling objective aiming to minimise the total service delay C vs(t) = X i∈S wt,ev t w ev i,t+1 − w ev i,t+1 + X j∈S wt,hv t w hv j,t+1 − w hv j,t+1. The SFGH algorithm consistently achieves the lowest service delay among compared scheduling strategies and the TD3 approach delivers the lowest annual operational cost among evaluated DRL and rule-based approaches.

Generalisation and Robustness

The framework demonstrates robustness under diverse conditions, as shown in comparative studies across different scenarios. The SFGH consistently achieves the lowest or tied-lowest cumulative service delay across all evaluated scenarios, demonstrating good generalisation to transport-demand distributions different from those used during training and the TD3-based approach continues to achieve the lowest annual operational cost among all compared approaches in all three scenarios.

Deployment Flexibility

The hierarchical architecture is modular, preserving the operational dependency between transport services and energy management while enabling each layer to be designed, updated, and executed independently for real-time deployment. This separation allows for the incorporation of alternative scheduling algorithms or energy-dispatch approaches without redesigning the entire framework. The solver-free scheduling algorithm avoids repeatedly solving mathematical optimisation problems at every decision interval, making the framework particularly suitable for real-time operation in scalable EHTSs.

The proposed hierarchical optimisation framework makes explicit the dependency between station-level vehicle scheduling and downstream multi-energy dispatch. The paper proposes a hierarchical optimisation framework for integrated electric–hydrogen–transport systems that makes explicit the dependency between station-level vehicle scheduling and downstream multi-energy dispatch. The proposed SFGH algorithm consistently achieved the lowest service delay among the compared scheduling algorithms, while the TD3 approach delivered the lowest annual operational cost among evaluated DRL and rule-based approaches. The proposed framework also maintained strong performance in ablation studies and under reduced transport-demand scenarios without retraining, demonstrating good robustness and generalisation capability. The proposed framework should be regarded as a general hierarchical operational architecture rather than a task-specific optimisation method. This design preserves the operational dependency between the two optimisation layers while maintaining computational tractability, modularity, and the flexibility to incorporate alternative scheduling or dispatch algorithms without redesigning the overall framework. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. This design preserves the operational dependency between the two optimisation layers while maintaining computational tractability, modularity, and the flexibility to incorporate alternative scheduling or dispatch algorithms without redesigning the overall framework. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. This design preserves the operational dependency between the two optimisation layers while maintaining computational tractability, modularity, and the flexibility to incorporate alternative scheduling or dispatch algorithms without redesigning the overall framework. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer. The proposed framework intentionally adopts a sequential hierarchical architecture in which transport-service scheduling determines the charging and refuelling demand subsequently handled by the energy-dispatch layer.

Improvements for AI systems

The proposed SFGH algorithm can be improved by further refining its candidate selection process, as it currently selects a limited set of highpriority vehicles, which could be made more sophisticated to better anticipate future bottlenecks. This refinement would allow the scheduler to make even better initial assignments, potentially leading to a higher g = ∆W + ∆O score and thus achieving even greater service delay reductions.
The TD3-based dispatch agent can be enhanced by incorporating a more comprehensive state representation that integrates both historical data and predicted future states, moving beyond the current history record-based state representation. This would allow the agent to better handle compounded uncertainty from stochastic vehicle arrivals and heterogeneous service requirements without relying solely on historical sequences.
The reward function can be improved by transitioning from the current benefit-based formulation to a more direct cost minimization, as shown in TD3r,cost replaces the proposed benefit-based reward in (51) with the raw cost formulation in (34). This change could lead to the largest performance degradation being avoided and potentially yield better long-term economic outcomes by directly optimizing the objective function.
The hierarchical architecture can be made more flexible by allowing for independent updates of the layers, as described in Section 4.6: the separation of the two optimisation layers allows future scheduling algorithms or energy-dispatch approaches to be incorporated independently without redesigning the entire framework. This enables rapid iteration and adaptation of specific components without compromising the overall system structure.
The simulation setup can be improved by explicitly modeling network-level constraints, as suggested in Section 4.6: Future work will extend the framework to multi-station systems with network-level constraints and investigate multi-agent coordination for large-scale deployments. This would allow the framework to be applied to more complex, real-world EHTS environments where inter-station dependencies exist.