Learning-based Autonomous Oversteer Control and Collision Avoidance
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.
Rosa: We're looking at the paper: "Learning-based Autonomous Oversteer Control and Collision Avoidance".
Paper summary: Rosa: So we're talking about this paper called Learning-based Autonomous Oversteer Control and Collision Avoidance. Basically, the thesis here is that you can tackle oversteer control and collision avoidance at the same time using an end-to-end approach.
Dev: That sounds ambitious, Rosa. So what's the core problem they're trying to solve with this paper?
Rosa: The problem is that oversteer—when your rear tires lose traction and cause unwanted yaw—is a huge safety challenge because if you don't control it, it can lead to really bad traffic accidents.
Dev: I've seen that before. And existing methods, like Electronic Stability Control, they can only try to lower the slip probability, but they can't completely eliminate the oversteer or get the car back to a stable grip state after it happens.
Rosa: Right, and most current techniques focus on just tracking a path even when you're oversteering. But in real life with obstacles around, that might not be enough to avoid collisions.
Dev: Exactly. So this paper claims they've introduced a novel end-to-end approach that handles both stability control and obstacle avoidance simultaneously using a single neural network for path planning and vehicle control.
Taro: It sounds like it tries to solve the problem of controlling the vehicle when it's already unstable while also making sure you don't hit things around you.
Rosa: That’s right. The paper introduces a new algorithm called Q-Compared Soft Actor-Critic, or QC-SAC, which is a new hybrid learning technique that learns from imperfect demonstration data and adapts quickly to new situations.
Dev: Okay, so before we get into how it works, what are the authors saying about the limitations of the existing methods they're trying to improve upon?
Rosa: They point out that Imitation Learning struggles when expert demonstrations are scarce or poor quality, and Reinforcement Learning suffers from slow training speed and low sample efficiency.
Dev: And Hybrid Learning techniques have another limitation: they require the expert demonstrations to be nearly optimal, or even optimal, which is almost impossible under oversteer conditions since human drivers struggle to react perfectly when things go wrong.
Taro: That makes sense. The dynamics are too complex and unpredictable for simple learning approaches right now.
Rosa: So the QC-SAC architecture is built around three core elements: Q-Compared Objective, Q-Network from Demonstration, and Selective Demonstration Data Update.
Dev: How does the Q-Compared Objective actually work to handle that issue with imperfect data?
Rosa: The objective function they use is J pi phi equals JSac phi plus Jbc phi, where the Jbc term uses a Q compared weighting factor called C sd ad to prioritize demonstrations based on their quality difference.
Taro: So it’s not just using the data; it’s selectively picking the most useful parts of that data for behavior cloning.
Paper summary: Rosa: Precisely. And they enhance the Q-network training with a Q-Network from Demonstration, or QNfD, which uses demonstration data to get better estimates for that quality difference factor C sd ad.
Dev: That sounds like they're trying to make the learning from demonstrations more meaningful by using that quality metric to guide the behavior cloning part of the objective.
Rosa: And then they have this Selective Demonstration Data Update, or SDDU, which allows for rapid learning from new situations by iteratively selecting successful episodes and using that aggregated data as demonstration data.
Taro: So it’s like the system learns from what worked well in previous runs and keeps adding those successful experiences to its knowledge base.
Dev: That iterative process is designed so that the updated dataset, D prime, has a better expected return than the original dataset, D, which helps make sure they're actually improving.
Rosa: Yes, that’s right. And they also use something called Focused Experience Replay to handle data imbalance by using a half-normal distribution for sampling and prioritizing recent data to boost training speed and stability.
Taro: So when you look at the results, what are the concrete numbers they give us about how well QC-SAC actually performs compared to other methods?
Dev: The benchmark they use involves inducing oversteer on a slippery road while avoiding randomly placed obstacles ahead. In those test runs, QC-SAC records a control and avoidance success rate of about eighty-one point eight percent.
Rosa: That's significantly better than the Behavior Cloning approach, which got only thirty-four point six percent, and much better than Soft Actor-Critic at nineteen percent or the basic Behavior Cloning at zero percent.
Taro: So it seems like this method shows a very strong ability to handle that complex task of controlling oversteer while avoiding obstacles in a real-world setting.
Dev: The ablation study they did confirms that all three core elements—the Q-Compared Objective, the QNfD, and the SDDU—are necessary for achieving that best performance.
Rosa: So, to summarize this paper on Learning-based Autonomous Oversteer Control and Collision Avoidance: it proposes QC-SAC as a new hybrid learning algorithm that uses selective demonstration data updates to effectively learn from immature data and adapt quickly to novel situations.
Dev: And the main implication is that this technique shows promise for developing optimal driving policies even when you start with suboptimal expert demonstrations, which is a big hurdle in real-world autonomous systems.
Taro: It means we might be able to deploy vehicles that can handle sudden oversteer events and still safely navigate around unexpected obstacles.
Rosa: That’s the hope. In conclusion, this paper introduces a pioneering autonomous driving technology that aims to reduce accidents caused by oversteer control issues by achieving near-optimal performance in their test scenarios.
Conclusion: Rosa: So, to wrap up this look at the paper "Learning-based Autonomous Oversteer Control and Collision Avoidance," they're proposing this whole end-to-end system that handles losing grip and avoiding crashes at the same time.
Dev: Yeah, it’s about taking a problem that used to require separate systems—one for stability, one for avoiding obstacles—and putting them into one neural network.
Taro: The authors are trying to move away from just mimicking perfect human driving because that data is too hard to get when you're actually losing control.
Rosa: Exactly, they’re using this hybrid learning approach called Q-Compared Soft Actor-Critic, or QC-SAC, which lets the AI learn effectively even when the demonstration data isn't perfect.
Dev: The results they show are pretty strong for a hybrid method. They compare it against imitation learning and reinforcement learning baselines on this specific test scenario of oversteer on a slippery road with obstacles.
Taro: The success rate they report, about eighty-one point eight percent, is what caught our attention because it’s way higher than the other methods tested.
Rosa: It really shows that even when you start with imperfect driving examples, this technique can actually learn a policy that works well in the real world under those unstable conditions.
Dev: It suggests a path toward making autonomous systems more robust when things go wrong unexpectedly, not just when they follow a perfect script.
Taro: But we still have to ask how long this stability holds up in different kinds of weird situations outside of the lab environment they tested.
Seokjun Lee, Seung-Hyun Kong
Korea Advanced Institute of Science and Technology
cs.RO, cs.AI, cs.LG
Submitted: 2025-05-21
Updated: 2026-11-16
Comments: 14 pages, 8 figures, 4 tables. Accepted for publication in IEEE Transactions on Intelligent Vehicles
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 80/100
The gist: Learning-based Autonomous Oversteer Control and Collision Avoidance introduces a novel end-to-end (E2E) autonomous driving approach that tackles oversteer control and collision avoidance…
Key concepts
- Oversteer
- Oversteer occurs when a vehicle's rear tires lose traction, causing the vehicle to rotate unintentionally in an unintended direction. This is dangerous because it requires both proper steering and counter-steering to control, leading to potential accidents.
- Q-Compared Objective (QCO)
- This objective function focuses on using only the most useful demonstration data for learning. It estimates the difference in action values between what demonstrations show and what the current policy does, prioritizing demonstrations that are more valuable for improvement.
- Selective Demonstration Data Update (SDDU)
- This method allows the system to learn quickly from new situations. It iteratively selects successful driving episodes and uses this aggregated data to update the demonstration dataset, ensuring that the new data improves performance over old data.
Terminology
Summary
Learning-based Autonomous Oversteer Control and Collision Avoidance introduces a novel end-to-end (E2E) autonomous driving approach that tackles oversteer control and collision avoidance simultaneously The experimental results show QC-SAC attains near-optimal driving policies, significantly surpassing state-of-the-art IL, RL, and HL baselines Our method demonstrates the world’s first safe autonomous oversteer control with obstacle avoidance. p. 0
Problem Context
Oversteer occurs when a vehicle’s rear tires lose traction and induce unintentional excessive yaw, posing critical safety challenges because failing to control it often leads to severe traffic accidents Controlling oversteer requires both appropriate pedal manipulation and adequate counter-steering at the same time Existing methods like Electronic Stability Control (ESC) can only lower the slip probability but cannot eliminate it entirely or return a vehicle to a stable, grip state after entering an oversteer condition Furthermore, existing techniques often focus on tracking predefined paths even under oversteer conditions, which may not be sufficient in real-world scenarios where surrounding obstacles are present
Learning Paradigms Limitations
Imitation Learning (IL) struggles when expert demonstration data is scarce or poor quality, leading to significant performance degradation Reinforcement Learning (RL) suffers from low sample efficiency and slow training speed, especially as the task complexity increases due to broader state and action spaces and higher transition variance Hybrid Learning (HL) techniques face the limitation that expert demonstrations must be nearly optimal or contain optimal demonstration data for effective policy development These limitations are critical because it is nearly impossible under oversteer to collect optimal demonstration data, as human drivers find it difficult to react appropriately
QC-SAC Architecture
The proposed QC-SAC algorithm consists of three core elements: Q-Compared Objective (QCO), Q-Network from Demonstration (QNfD), and Selective Demonstration Data Update (SDDU) The QCO selectively utilizes only the beneficial demonstration data for behavior cloning by estimating the action value difference between demonstration actions and policy actions The objective function is formulated as Jπ(ϕ) = JSAC (ϕ) + JBC (ϕ), where JBC (ϕ) uses a Q compared weighting factor C(sd, ad) to prioritize demonstrations with higher Q values
Data Utilization and Adaptation
The QNfD method enhances the Q-network training by utilizing demonstration data to improve accurate estimates for C(sd, ad), which is crucial for the effectiveness of QCO The SDDU method enables rapid learning from new situations by iteratively selecting successful episodes and using aggregated data as demonstration data, ensuring that the updated dataset D' satisfies E(s,a)∼D′ [r(s, a)] > E(s,a)∼D[r(s, a)] This iterative process prevents overfitting by continuously updating the demonstration data with successful episodes
Benchmark and Evaluation
The benchmark is inspired by real driver training using a kick plate to induce oversteer on a slippery road while avoiding randomly placed obstacles ahead The state space includes vehicle state variables like side slip angle, longitudinal velocity, and steering angle, alongside a 90-dimensional surrounding state vector representing the drivable distance for each azimuth angle The action space consists of normalized pedal values and steering angular velocity to constrain the maximum steering angular velocity at 700°/s Performance evaluation shows that QC-SAC records a control & avoidance success rate of about 81.8% in test runs, significantly outperforming BC-SAC (34. p. 76%), SAC (19. p. 00%), and BC (0.0%) The ablation study confirms that all three core elements—QCO, QNfD, and SDDU—are essential for achieving the best performance
Conclusion
QC-SAC addresses the challenges of immature demonstration data and high transition variance by effectively utilizing immature demonstration data and rapidly adapting to novel situations The study proposes a pioneering autonomous driving technology that can significantly contribute to reducing accidents caused by oversteer This technique achieves near-optimal performance in the proposed benchmark, demonstrating its robustness for developing optimal driving policies
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C3008370) The authors thank IPG Automotive for providing CarMaker test licenses for the dissertation
REFERENCES
arXiv:2505.15275v1 [cs. p. 0RO] 21 May 2025
Seokjun Lee and Seung-Hyun Kong, Senior Member, IEEE Learning-based Autonomous Oversteer Control and Collision Avoidance Abstract—Oversteer, wherein a vehicle’s rear tires lose traction and induce unintentional excessive yaw, poses critical safety challenges Failing to control oversteer often leads to severe traffic accidents Although recent autonomous driving efforts have attempted to handle oversteer through stabilizing maneuvers, the majority rely on expert-defined trajectories or assume obstacle-free environments, limiting real-world applicability This paper introduces a novel end-to-end (E2E) autonomous driving approach that tackles oversteer control and collision avoidance simultaneously Existing E2E techniques, including Imitation Learning (IL), Reinforcement Learning (RL), and Hybrid Learning (HL), generally require near-optimal demonstrations or extensive experience Yet even skilled human drivers struggle to provide perfect demonstrations under oversteer, and high transition variance hinders accumulating sufficient data Hence, we present Q-Compared Soft Actor-Critic (QC-SAC), a new HL algorithm that effectively learns from suboptimal demonstration data and adapts rapidly to new conditions To evaluate QC-SAC, we introduce a benchmark inspired by real-world driver training: a vehicle encounters sudden oversteer on a slippery surface and must avoid randomly placed obstacles ahead Experimental results show QC-SAC attains near-optimal driving policies, significantly surpassing state-of-the-art IL, RL, and HL baselines Our method demonstrates the world’s first safe autonomous oversteer control with obstacle avoidance Index Terms—Autonomous driving, oversteer, collision avoidance, imitation learning, reinforcement learning I. INTRODUCTION WHEN slip occurs between a vehicle’s tires and the road surface, causing the tires to skid, the vehicle becomes highly unstable Particularly, when the rear tires lose traction, the vehicle rotates more than the driver intends and heads in an unintended direction; this phenomenon is known as oversteer (Refer to Fig. p. 01a) Oversteer primarily occurs due to causes such as reduced road friction caused by road icing or hydroplaning, tire slippage from sharp steering or pedal manipulation beyond the vehicle’s limits, or rear-end collisions Controlling oversteer requires both appropriate pedal manipulation and adequate counter-steering (a sophisticated driving technique where the driver steers in the opposite direction of the skid) at the same time [1] However, it is very difficult for untrained ordinary drivers to use such a specialized driving technique As a result, oversteer often leads to severe traffic accidents; according to the National Highway Traffic Safety Administration (NHTSA) Fatality Analysis Reporting System (FARS), oversteer accounted for more than 18,852 fatal accidents in the United States from 2011 to 2020, making it the 8th leading cause of fatal accidents To mitigate oversteer, various methods have been proposed over the years One of the earliest and most prevalent solutions is the implementation of Electronic Stability Control (ESC), which is now standard equipment in most modern vehicles ESC helps prevent vehicle skidding by braking individual wheels or cutting engine power However, it can only lower the slip probability but not eliminate it entirely Moreover, since ESC adjusts wheel forces rather than steering inputs, it lacks the ability to return a vehicle that has already entered an oversteer condition back to a stable, grip state To address the limitations of ESC, researchers have explored autonomous driving technologies that enable automatic counter-steering Most of these technologies focus on control techniques designed to track predefined paths even under oversteer conditions [3]–[9] While such methods can help an oversteering vehicle return to its original lane as shown in Fig. p. 02a, they may not be sufficient in real-world scenarios where surrounding obstacles (e. p. 1g., other vehicles) are present In these situations, merely controlling the oversteer is not enough to avoid collisions When there is an obstacle in the driving lane, as illustrated in Fig. p. 12b, the vehicle must not only regain stability but also recognize the surrounding environment and execute an evasive maneuver that ensures collision avoidance and is feasible for an oversteering vehicle to perform To tackle this challenge, our study adopts an endto-end (E2E) approach that directly maps the vehicle’s state and surrounding environment to control actions by integrating path planning and vehicle control into a single neural network This means that the path planning process simultaneously considers the vehicle’s controllability, enabling the generation of paths that are both safe and executable under oversteer conditions In the realm of E2E autonomous driving, imitation learning (IL) and reinforcement learning (RL) have emerged as prominent techniques for training neural networks to perform complex driving tasks [10]–[15] IL involves learning control policies by mimicking expert demonstrations, and RL enables an agent to learn optimal policies through trial-and-error interactions with the environment, guided by a reward function However, both IL and RL have fundamental limitations The action policies trained via IL often face significant performance degradation when they encounter novel situations where expert demonstration data is scarce or when the quality of the demonstration data is poor On the other hand, even though RL requires sufficient experience of successful episodes, as the task becomes more complex, the state and action spaces become broader, and the transition dynamics have higher variance, the probability of experiencing successful episodes through random actions diminishes Consequently, developing an optimal action policy using RL becomes exceedingly difficult To overcome these shortcomings, recent hybrid learning (HL) techniques combine IL and RL, utilizing expert demonstration data with RL for more efficient and improved policy development [18]–[23] However, HL techniques have a limitation in that the expert demonstrations must be nearly optimal or, even if noisy, must contain optimal demonstration data Such limitations in IL, RL, and HL become more critical when it comes to the vehicle oversteer situation Firstly, it is nearly impossible under the oversteer situation to collect optimal demonstration data Moreover, there is an extensive amount of information to be perceived regarding the surroundings and the vehicle state, and there is a high transition variance due to the nonlinear dynamics Not only does this make it challenging to develop an optimal driving policy using RL alone, but it also means that agents are unlikely to encounter the same situation multiple times, making it difficult to learn through repeated experiences Therefore, there is a critical need for methods that enable fast learning from new situations and can rapidly incorporate new successful experiences into the learning process To overcome the aforementioned limitations, we propose an innovative HL technique, Q-Compared Soft Actor-Critic (QC-SAC), capable of learning optimal policies even from immature demonstration data and adapting swiftly to novel situations Using the proposed QC-SAC, we trained an autonomous driving agent capable of oversteer control and collision avoidance In scenarios where even human drivers often fail to control the vehicle, a test vehicle equipped with the driving policy developed by QC-SAC successfully avoids obstacles when it suddenly starts to spin (oversteer) on a slippery road The major contributions of the paper are as follows • We introduce the world’s first safe driving technology capable of successful autonomous control of vehicles that must avoid obstacles ahead while in an oversteer condition • We propose the first benchmark for training and evaluating autonomous driving agents capable of oversteer control and collision avoidance, inspired by the actual driver training process • We present a novel HL technique, QC-SAC, that effectively utilize immature demonstration data and adapt swiftly to novel situations
Q-Value Comparison Objective
The QCO selectively utilizes only the beneficial demonstration data for behavior cloning (BC) by estimating the action value (Q) difference between the actions stored in the demonstration data and the actions generated by the driving policy being developed It prioritizes demonstration data that have higher Q values than the driving policy being developed, weighting them in proportion to their quality difference The objective is formulated as Jπ(ϕ) = JSAC (ϕ) + JBC (ϕ), where JBC (ϕ) uses a Q compared weighting factor C(sd, ad) to prioritize demonstrations with higher Q values The weight C(sd, ad) is calculated as C(sd, ad) = max Q− (sd, ad − Q (sd, a ∼ πϕ (sd)), 0 This mechanism ensures that the more action a d has a higher Q value than the RL policy's action a ∼ πϕ(sd), the more important it is considered and given higher priority
Q-Network Enhancement
The QNfD method utilizes demonstration data to enhance the Q-network training by combining two batches for the Q-network update, sampling from both the replay buffer DRL and a batch BBC sampled from the demonstration dataset D This combined batch B = BRL ∪ BBC is used in updating both Q-networks using conventional objective function used in RL This enhancement significantly improves the performance of QC-SAC by producing more accurate Q value estimates for C(sd, ad) and JSAC (ϕ)
Rapid Experience Incorporation
The SDDU method enables rapid learning from new situations by iteratively selecting successful episodes and using aggregated data as demonstration data for training This iterative process expands the knowledge base with higher-quality data, enhancing learning stability and performance After the iterative process of SDDU, we obtain the updated dataset D′, which satisfies E(s,a)∼D′ [r(s, a)] > E(s,a)∼D[r(s, a)] This leads to JBC (ϕ) being considered more importantly and promoting active learning from expert behavior Focused Experience Replay (FER) is also employed to address the data imbalance problem by using a half-normal distribution for sampling, prioritizing recent data to enhance training speed and stability
Performance Comparison
The performance evaluation compares QC-SAC against Behavior Cloning (BC), Soft Actor-Critic (SAC), and Behavior Cloned Soft Actor-Critic (BC-SAC) While SAC shows higher rewards than BC, it fails to develop the optimal driving policy due to high transition variance in oversteer control BC-SAC performs slightly better than SAC but still cannot achieve high rewards as it continuously considers immature demonstration data that disturbs the training QC-SAC records a control & avoidance success rate of about 81.8%, demonstrating superior performance compared to existing techniques The ablation study proves that all three core elements—QCO, QNfD, and SDDU—are essential for achieving the best performance
ACKNOWLEDGMENTS
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Q-Compared Soft Actor-Critic (QC-SAC) can learn optimal policies even from immature demonstration data by selectively utilizing only beneficial data for behavior cloning through its Q-Compared Objective (QCO). This allows the system to function effectively when human demonstrations are suboptimal or noisy, addressing the limitation where
the action policy becomes suboptimal when immature demonstration data is provided.
-
The QNfD method enhances the performance of QC-SAC by utilizing a combined batch for the Q-network update, sampling from both an environment replay buffer and a demonstration dataset. This improves the
Q value estimates for C(sd, ad) and JSAC (phi)
in the QCO, which is crucial whenthe state and action spaces are broad and the transition variance is high.
-
The Selective Demonstration Data Update (SDDU) allows for rapid incorporation of new successful experiences by iteratively selecting episodes with higher rewards than the average dataset reward. This mechanism ensures that
the agent effectively expands its knowledge base with higher-quality data, enhancing the learning stability and performance of the driving policy,
preventing overfitting by continuously updating the demonstration data. -
The system can achieve near-optimal control and collision avoidance by successfully navigating scenarios where human drivers often fail, as demonstrated by a success rate of about 81.8% in testing, significantly surpassing baseline techniques like BC-SAC (34.6% success rate).
Sources
- Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations
- Residual Reinforcement Learning from Demonstrations
- Reinforcement Learning from Imperfect Demonstrations
- Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
Related papers
- Rethinking Actions: Continuity-Aware Priors for World-Model Planning
- MOSAIC: Modular Obstacle and Spatial-Aware Framework for Manipulation In Cluttered Scenes
- SAGAS: Semantic-Aware Graph-Assisted Stitching for Offline Temporal Logic Planning
- DexSeed: Expanding Human Hand-Object Interactions into Robot Demonstrations for Dexterous Policy Learning
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- CIALLO: Learning Shared Intent Representations for Cross-Embodiment Manipulation