Benchmarking Learning-Based Formula 1 Race Strategies

arXiv:2512.21570 · eess.SY, cs.SY · Submitted 2025-12-25 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Benchmarking Learning-Based Formula 1 Race Strategies".

Dev: Here's the gist of the paper. This paper presents two complementary frameworks to optimize Formula 1 race strategies by jointly accounting for energy allocation, tire wear and pit stop timing.

Rosa: The learned policy’s suboptimality is assessed with respect to the optimal solution, both in a nominal scenario and with an unforeseen disturbance.

Dev: Modeling the Race Scenario The race scenario is modeled using lap time maps and a dynamic tire wear model capturing the main trade-offs arising during a race.

Rosa: The system states include the battery energy Eb, the available fuel energy Ef, the car’s mass mcar and the race time Trace.

Paper summary: Rosa: To recap, the paper "Benchmarking Learning-Based Formula one Race Strategies" presents this dual approach where they use a mixed-integer nonlinear program to find an optimal solution for race strategies, and then they train a reinforcement learning agent to solve the same problem in an environment <ref:2512.21570#pg1,Learning-Based Formula 1 Race Strategies>.

Dev: They show that this method is useful because the computational burden of finding the optimal strategy can be shifted entirely into the training phase, allowing for very fast inference when you actually need a decision during a race.

Taro: It’s about getting a solution that is robust and reliable when things get messy, which they prove by testing it against an unforeseen event where tire wear suddenly increases at lap twenty-two.

Rosa: The abstract says the learned policy shows suboptimality of approximately five seconds on one point five hours of race time in both the normal case and with that disturbance, mainly because of how they allocate their battery energy <ref:2512.21570#pg1>.

Dev: That's a concrete number showing that even though it’s not perfect, this RL approach is performing very well compared to the optimal solution found by the MINLP solver, which takes sixty-eight seconds to run.

Taro: The way they model tire wear is pretty specific, using a function fj(TW, mcar) with terms involving current tire wear and the car's mass ratio to update that wear state lap by lap.

Rosa: And they handle the lap time calculation by combining nominal maps with a correction term based on that tire wear model, where they use functions N1, N2, and N3 for each compound.

Dev: The reward function for the reinforcement learning agent is set up to minimize total race time by defining a per-step reward as Tlap constant minus the lap time at that step.

Taro: It seems like they’re trying to bridge the gap between complex, slow optimization methods and systems that can actually react in real-time when you are on track or in the pit lane.

Rosa: And what this means for us is that for anyone who only listens to the show, it shows how learning-based approaches can be practical when you need a fast decision maker instead of waiting for a heavy solver.

Conclusion: Dev: So looking at "Benchmarking Learning-Based Formula one Race Strategies," the authors are basically saying that combining a direct optimization method like the MINLP with an RL agent trained on the same problem is a solid way to check how good the learning approach is <ref:2512.21570#pg1,Learning-Based Formula 1 Race Strategies>.

Rosa: They show that while they find an optimal solution, it can still be slow, taking about sixty-eight seconds to solve for the entire race strategy.

Taro: But by training an agent on this exact scenario, they get a policy that's robust enough to handle unexpected events and pit stop timing changes reasonably well.

Dev: The implication here is that we can use this RL approach because its inference time is very small, which is what makes it suitable for real races where you need rapid decision-making.

Rosa: They suggest a path forward by integrating the two approaches, using the RL agent's discrete decisions to parametrize a continuous nonlinear program within a model predictive control framework.

Taro: That would get rid of those integer variables in the online optimization, which is what makes it feasible for real-time deployment during a race event.

Dev: So in simple terms, this paper provides a benchmark showing that learning approaches are reliable and fast enough to support human decision-making by providing near-optimal strategies without needing massive computational power on the fly.

Institute for Dynamic Systems and Control, ETH Zurich

eess.SY, cs.SY

Submitted: 2025-12-25

Updated: 2026-10-28

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: This paper presents two complementary frameworks to optimize Formula 1 race strategies by jointly accounting for energy allocation, tire wear and pit stop timing.

Key concepts

Mixed-Integer Nonlinear Program (MINLP)
This is a complex mathematical optimization technique used to find the absolute best race strategy. It considers multiple variables like when to pit, which tire compound to use, and how much energy or fuel to allocate. Solving this problem takes significant computing time.
Reinforcement Learning (RL) Agent
An AI agent trained using a soft actor-critic algorithm learns the best race strategy through trial and error in a simulated environment. It learns to make decisions (like when to pit) based on the current race state to minimize total race time.
Tire Wear Model
This mathematical function predicts how quickly tires degrade based on factors like lap time and car mass. The model uses specific functions for different tire compounds and helps determine the optimal timing for pit stops to maintain performance.
Reward Function
The objective of the AI is to minimize the total race time. A reward is given at each step based on how much faster or slower the current lap time was compared to a constant target, guiding the agent toward a strategy that results in the fastest overall finish.

Terminology

Summary

This paper presents two complementary frameworks to optimize Formula 1 race strategies by jointly accounting for energy allocation, tire wear and pit stop timing. The learned policy’s suboptimality is assessed with respect to the optimal solution, both in a nominal scenario and with an unforeseen disturbance.

Modeling the Race Scenario

The race scenario is modeled using lap time maps and a dynamic tire wear model capturing the main trade-offs arising during a race. The system states include the battery energy Eb, the available fuel energy Ef, the car’s mass mcar and the race time Trace. Key inputs to define the strategy are the allocated battery energy per lap ∆Eb,all, allocated fuel energy per lap ∆Ef,all and the pit stop decision variable PS. The physical states evolve according to equations such as Eb[k + 1] = Eb[k] + ∆Eb,all[k].

Formulating the Optimization Problem

The race strategy is formulated as a mixed-integer nonlinear program (MINLP) where pit stops, compound choice and energy allocation are considered. Auxiliary variables like bPS[k] and binlap[k] are defined to reformulate logical equations needed to define the minimum race time problem. The resulting optimal control problem (OCP) is solved using CasADi [27] and BONMIN [28], with computational time ranging from 50 s to 5 min on a commercial laptop.

Training the Reinforcement Learning Agent

The same race scenario is embedded into a reinforcement learning environment, on which an agent is trained. The state space S includes Eb[k], Ef[k], mcar[k], Trace[k], bcomp[k] and TC[k]. The observation space O consists of sk Tlap[k] Nlaps − k. The action space A includes the normalized fuel energy allocation F, normalized battery energy allocation B and the pit stop action PS. The agent is trained using a soft actor-critic (SAC) algorithm [30].

Benchmarking and Results

The RL agent is benchmarked with the MINLP solution, showing a suboptimality of 4.96 s in the nominal case. In a scenario with an unexpected disturbance, such as an increase in tire wear at lap 22, the RL agent adapts its strategy to pit for soft tires earlier than the previous case study. The agent shows its superiority by providing close-to-optimal pit stop strategy with a negligible computational time.

Future Research Directions

For future research, ideas include adding probabilistic models such as pit stop timing, traffic or weather predictions. Additional inputs or models used in real racing may also be integrated. The paper suggests integrating the two approaches by solving a continuous NLP parametrized by the RL agent’s discrete pit stop decisions within an MPC framework. This eliminates integer variables from the online optimization, achieving real-time feasibility. The paper also suggests exploring scenarios with multiple agents interacting with each other.

Tire Wear Modeling Details

Tire wear is modeled by the function fj(TW, mcar) = aj · TW[k] + bj · mcar[k]/mcar[0] + cj. The tire wear update equation is given by TW[k + 1] = (1 − bPS)· 1/2 · z2 · z3 · f3(TW, mcar) - z1 · z3 · f2(TW, mcar) + 1/2 · z1 · z2 · f1(TW, mcar). The correction term for lap time is modeled as ∆T j[k] = Nj(TW[k]). The fitting function N denotes a twice differentiable fitting function, one for each compound j.

Lap Time Modeling

The nominal lap time maps determine if the current lap is normal, an inlap or an outlap via the selection of auxiliary variables. The correction term for lap time is calculated by combining the nominal map and the correction term ∆T = 1/2 · z2 · z3 · N1(TW) - z1 · z3 · N2(TW) + 1/2 · z1 · z2 · N3(TW). The velocity of the pit lane is restricted by regulations to be 80 km/h.

State Transition Dynamics

The next state sk+1 is obtained by the deterministic transition function T: S × A → S, such that sk+1 = T(sk, ak). The episode terminates when no laps remain, i.e., Done = 1 if k = Nlaps, 0 otherwise. The Markov property is satisfied because the state sk explicitly contains all quantities that influence future evolution as for eq. (48).

Reward Function Definition

The goal is to minimize the total race time, so a per-step reward is defined as R(sk, ak,sk+1) = rk = Tlap,const − Tlap(k). The cumulative return J = E [X Nlaps k=0 γ k · rk] with discount factor γ = 1 corresponds to minimizing the overall race time.

Computational Feasibility Comparison

Table 3 summarizes the computational burden needed to evaluate the strategy for the entire race, showing that while the MINLP solver requires 68 s, the RL agent computes in less than a second. This difference highlights that real-time feasibility is achieved by shifting computational burden to the training phase. The agent's performance in adapting to disturbances confirms its robustness, as it changes its pit stop strategy almost identically to the MINLP version after the disturbance. The RL agent shows its superiority by providing close-to-optimal pit stop strategy with a negligible computational time. The agent keeps a battery level of Eb = 0 during the race, while the MINLP chooses to stay around Eb = 1.2 in normalized units.

Conclusion and Outlook

In this paper, we filled the literature gap of race strategies where energy allocation, pit stop and tire wear are jointly considered. The MINLP serves as a ground-truth benchmark for the RL agent, and delivers the optimal solution. The RL approach is robust and reliable, with a negligible inference time. For future research, we can now explore scenarios with multiple agents interacting with each other. This motivates the integration of the two approaches by solving a continuous NLP parametrized by the RL agent’s discrete pit stop decisions within an MPC framework. By eliminating integer variables from the online optimization, real-time feasibility can be achieved. The paper is structured as follows: In Section 6, we conclude the paper in Section 6 with an outlook on potential extensions of the presented work.

Improvements for AI systems

  1. Evaluation of learned policies against optimal solutions can be improved by developing specialized models that are suited for optimization to assess suboptimality, as mentioned in Section 1.2. This would allow for more direct and quantitative assessments of policy suboptimality beyond the current comparison with the MINLP benchmark.

  2. Integration of continuous optimization within a Model Predictive Control (MPC) framework can be implemented by solving a continuous NLP parametrized by the RL agent’s discrete pit stop decisions, as suggested in Section 6. This would allow real-time feasibility while handling integer variables through the agent.

  3. The RL agent's action space mapping can be refined by incorporating target pace or brake temperature dynamics as inputs to improve lap time modeling, as suggested in Section 6. This would allow the driver to push the car for faster lap times at the expense of increased tire wear or better brake performance.

Sources

Related papers