AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust

arXiv:2605.24301 · cs.RO · Submitted 2026-05-23 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust".

Dev: Here's the gist of the paper. Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight, perching, and sensing.

Rosa: The proposed method achieves the lowest position deviation and settling time across all evaluated baselines by modulating a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes.

Dev: Separate policies are trained via reinforcement learning for nominal-to-inverted and inverted-to-nominal transitions.

Rosa: In JAX-based simulation, the proposed method reduces position root mean square error (RMSE) by 32% and settling time by 57% relative to the strongest optimization-based baseline.

Paper summary: Rosa: So we were just talking about how this paper, "AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust", aims to show how bidirectional thrust gives quadrotors a second equilibrium condition and increased control authority, which expands the envelope for aggressive maneuvers like inverted flight or sensing.

Dev: Exactly. The core claim is that prior geometric control approaches struggle with actuator saturation and motor reversal delay when inversions happen, which means they need these heuristic thrust posture schedules and waypoint tuning.

Taro: So this paper proposes a learning-based framework that modulates a constant reference trajectory to perform compact, position constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across different flight regimes.

Rosa: And the way they do it is by having separate policies trained via reinforcement learning for nominal to inverted and inverted to nominal transitions.

Dev: That setup means they have one policy for getting into an inverted state, and another for coming back out, which is smart because those two transitions are very different behaviors.

Taro: I wonder how robust these learned policies are when the world doesn't behave exactly as expected during those critical transition phases.

Rosa: They tested this extensively in JAX-based simulation, and the proposed method consistently outperformed classical baseline inversion strategies for both nominal to inverted and inverted to nominal transitions across every performance metric.

Dev: And they reported that their approach reduces position root mean square error by thirty-two percent and the settling time by fifty-seven percent relative to the strongest optimization-based baseline in that simulation.

Taro: So, it’s not just a small improvement; it's a significant jump when you look at how fast they settle compared to what was previously achievable.

Rosa: And on hardware, they showed successful inversion across multiple yaw configurations with position root mean square error below zero point three five meters and compatibility with downstream trajectory generation and control through circular flight in both regimes.

Dev: That’s the validation part, showing that the simulation success translates pretty well to physical hardware for this specific task.

Taro: I think what's interesting is that they are using a learning framework to modulate a constant reference trajectory instead of just relying on pre-defined control allocations during these tricky maneuvers.

Rosa: And it’s important to remember the caveat that their hardware validation showed that the average maximum position deviations increase by at most forty percent relative to the simulation for nominal to inverted transitions.

Dev: That gap between simulation and reality is something we need to account for when we talk about deploying this kind of system in the field.

Conclusion: Rosa: So, wrapping up on "AcroRL: Learning Aggressive Quadrotor Inversion using Bidirectional Thrust", the main thing is that this paper demonstrates a learning-based method that uses bidirectional thrust to achieve compact inversions while maintaining compatibility with existing trajectory generation techniques.

Dev: The authors are Gabriel Rodriguez, Henri Sayag, Abhishek Rathod, John Stecklein, Siddharth Saha, Christopher Barngrover and Wennie Tabib from Carnegie Mellon University and Shield AI.

Taro: It shows how learning can be used to handle the complex control allocation challenges that arise when you introduce bidirectional thrust into quadrotor dynamics.

Rosa: They showed that even though there are some real-world limitations, like those position offsets without an integrator, the method still reliably inverts at arbitrary yaw angles with position root mean square error below zero point three five meters.

Dev: It means that for applications involving complex surfaces or cluttered environments, we have a system that can potentially handle these aggressive maneuvers with decent precision when the world is behaving reasonably well.

Taro: So it’s about using learning to manage those tricky control allocation issues so the system doesn't just fail when things get messy.

Gabriel Rodriguez, Henri Sayag, Abhishek Rathod, John Stecklein, Siddharth Saha, Christopher Barngrover, Wennie Tabib

Carnegie Mellon University · Shield AI

cs.RO

Submitted: 2026-05-23

Updated: 2026-11-03

Comments: 17 pages, 8 figures, Accepted to the 10th Annual Conference on Robot Learning (CoRL 2026)

Code: https://github.com/rislab/acrorl

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 95/100

The gist: Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight…

Key concepts

Bidirectional Thrust
This refers to using the quadrotor's propellers to generate both positive and negative thrust, allowing for more complex control over movement, such as hovering or moving in ways that are impossible with standard single-direction thrust. This capability is key to achieving inverted flight and aggressive maneuvers.
Hopf Fibration-Based Control Algorithm (HFCA)
This is a control algorithm used to track a specific hybrid trajectory that combines position and orientation. It uses the Hopf fibration concept to manage the quadrotor's attitude control across all possible orientations, including negative thrust cases, ensuring stable tracking of the desired path.
Markov Decision Process (MDP)
The inversion task is framed as an MDP where the quadrotor's current state and its recent actions form the observation. The goal is to learn an optimal policy that selects actions (like modulating thrust) to minimize a cost function, which penalizes deviation from the desired compact inversion while ensuring smooth control inputs.
Proximal Policy Optimization (PPO)
PPO is a reinforcement learning algorithm used to train the separate policies for transitioning between flight modes (nominal-to-inverted and inverted-to-nominal). It learns how to adjust thrust posture and trajectory modulation based on the observed state history to achieve the desired inversion maneuvers effectively.

Terminology

Summary

Bidirectional thrust grants quadrotors a second equilibrium condition and increased control authority, expanding the envelope of possible aggressive maneuvers and enabling inverted flight, perching, and sensing. p. 0 The proposed method achieves the lowest position deviation and settling time across all evaluated baselines by modulating a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes. p. 0 Separate policies are trained via reinforcement learning for nominal-to-inverted and inverted-to-nominal transitions. p. 0 In JAX-based simulation, the proposed method reduces position root mean square error (RMSE) by 32% and settling time by 57% relative to the strongest optimization-based baseline. p. 0 Hardware experiments demonstrate successful inversion across multiple yaw configurations with position RMSE below 0. p. 035 m and compatibility with downstream trajectory generation and control through circular flight in both regimes. p. 0

System Dynamics and Control Architecture

The quadrotor dynamics are modeled as a 3D rigid body with control inputs u = [fc, τ] and state x = [r, r˙, R, ω], governed by equations (1). The thrust mapping is defined by u = M(omega)T where M(omega) is the mixer matrix. p. 2 Thrust modeling utilizes a second-order asymmetric thrust model that accounts for both (+) and (−) operating regimes of the propeller. p. 2 Transient dynamics are adapted by defining distinct operating regime specific slew rates α(±) and a switching logic S based on a switching threshold omega0 and dead-zone width ∆omega. p. 3 The Hopf Fibration-Based Control Algorithm (HFCA) tracks the specified hybrid-mode differentially flat reference trajectory σκ(t) = [rκ(t), ψκ(t)]⊤. p. 4 Attitude control is achieved by defining four coordinate charts to cover all of SO(3) and handle the negative thrust case. p. 4

Learning Framework and Cost Function

The quadrotor inversion task is formulated as a Markov decision process (MDP) with observation space O comprising the quadrotor state x and a finite action history ahist. The action space A provides a modulation rδκ to a constant flat reference σκ and thrust posture η. p. 3 The cost function C is designed to produce the desired positionally compact inversion while maintaining smoothness in control action for favorable sim2real transfer. p. 3 The cost includes translational penalties like wr∥r∥1+wr˙LH(r˙) and orientation penalties such as wη1−η2. p. 3 Separate policies πθ(atx, ahist), parameterized by θ and implemented as multi-layer perceptrons (MLPs), are trained using Proximal Policy Optimization (PPO) for nominal-to-inverted (NTI) and inverted-to-nominal (ITN) transitions. p. 3

Performance Evaluation and Results

The simulation experiment compares the proposed method against three baselines: step thrust posture command with direct inversion control allocation, step thrust posture with optimal control allocation, and a minimum-snap trajectory baseline. Table 1 reports parallelized method comparison in JAX-based simulation where our approach consistently outperforms classical baseline inversion strategies for both NTI and ITN transitions. p. 6 Results show that the learned policy consistently outperforms both baselines in every performance metric across all test cases. p. 6 Hardware experiments demonstrate that the proposed method transfers favorably from simulation to hardware, achieving successful zeroshot quadrotor inversions. p. 7 The average maximum position deviations increase by at most 40% relative to the simulation for NTI transitions. p. 7

Hardware Validation and Limitations

The hardware validation confirms that the proposed method can reliably invert at arbitrary yaw angles with position RMSE below 0.35 m. The system performance trends observed in simulation are preserved on hardware. p. 7 However, the average maximum position deviations increase by at most 40% relative to the simulation for NTI transitions. p. 7 It is noted that the absence of an integrator in the reference controller produces steady-state position offsets on hardware. p. 8 The current reward structure is tailored to inversion maneuvers and would require reward reshaping for more general acrobatic behaviors. p. 8

Acknowledgements

This work was supported in part by the Army Research Laboratory and the Army Research Office under contract/grant number W911NF-25-2-0153. The work also received gift funding from Shield AI. p. 8

References

[1] P. Yu, G. Chamitoff, and K. p. 8 Wong. Perching upside down with bi-directional thrust quadrotor. p. 8

[2] J. Bass and A. p. 1 p. 1 L. Desbiens. Adaptative friction shock absorbers and reverse thrust for fast multirotor landing on inclined surfaces. p. 8

[3] M. Watterson, A. p. 1 Zahra, and V. p. 8 Kumar. Geometric control and trajectory optimization for bidirectional thrust quadrotors. p. 8

[4] W. Jothiraj, I. p. 4 Sharf, and M. p. 8 Nahon. Control allocation of bidirectional thrust quadrotor subject to actuator constraints. p. 8

[5] M. Maier. Bidirectional thrust for multirotor mavs with fixed-pitch propellers. p. 8

[6] J. Bass and A. L. Desbiens. Improving multirotor landing performance on inclined surfaces using reverse thrust. p. 8

[7] P. Yu and K. Wong. An implementation framework for vision-based bat-like inverted perching with bi-directional thrust quadrotor. p. 9

[8] M Gong, Z Shao, B Li, J Wang, and Y Wang. p. 9 Aggressive perching trajectory planning and control for quadrotor. p. 9

[9] B Habas et al. p. 9 From ceilings to walls: Universal dynamic perching of quadrotors on surfaces with variable orientations. p. 9

[10] B Habas and B Cheng. p. 9 From flies to robots: Inverted landing in small quadcopters with dynamic perching. p. 9

[11] B Habas et al. p. 9 Inverted landing in a small aerial robot via deep reinforcement learning for triggering and control of rotational maneuvers. p. 9

[12] J Mao et al. Aggressive visual perching with quadrotors on inclined surfaces. p. 9

[13] Y Zou et al. Perch a quadrotor on planes by the ceiling effect. p. 9

[14] Y Li et al. Design and control of a perching drone inspired by the prey-capturing mechanism of venus flytrap. p. 9

[15] Y Li et al. Tail-mav: Design and control of a perching micro aerial vehicle inspired by the tail-suspended behavior of primates. p. 9

[16] J L Paneque et al. p. 9 Perception-aware perching on powerlines with multirotors. p. 9

[17] A Battiston et al. p. 9 Attitude estimation for collision recovery of a quadcopter unmanned aerial vehicle. p. 9

[18] M W Mueller. Multicopter attitude control for recovery from large disturbances. p. 9

[19] M Faessler et al. p. 9 Automatic re-initialization and failure recovery for aggressive flight with a monocular vision-based quadrotor. p. 9

[20] F Liao et al. Reversible thrust-based fault tolerant control for quadrotor uavs against motor failure. p. 9

[21] Z Chen et al. Robust control for bidirectional thrust quadrotors under instantaneously drastic disturbances. p. 0

[22] Y Zhao et al. A novel anti-disturbance control framework for bidirectional quadrotors. p. 0

[23] Y Zhao et al. Bidirectional thrust control for quadrotor safety. p. 1

[24] J Wehbeh and I Sharf. p. 8 An mpc formulation on so(3) for a quadrotor with bidirectional thrust and nonlinear thrust constraints. p. 5

[25] J Wehbeh and I Sharf. p. 8 Geometric mpc techniques for reduced attitude control on quadrotors with bidirectional thrust. p. 0

[26] J Wehbeh and I Sharf. p. 8 Nonlinear scenario-based model predictive control for quadrotors with bidirectional thrust. p. 1

[27] L Xu et al. The control method of a quadrotor driven by bidirectional electronic speed controllers. p. 0

[28] L Xu et al. The motion planning, learning and control of a bidirectional thrust quadrotor with special tasks. p. 0

[29] W Jothiraj et al. p. 4 Enabling bidirectional thrust for aggressive and inverted quadrotor flight. p. 0

[30] K Mao et al. Trajectory planning for the bidirectional quadrotor as a differentially flat hybrid system. p. 4

[31] K Mao et al. Toppquad: Dynamically-feasible timeoptimal path parametrization for quadrotors. p. 0

[32] S Ji et al. Flipping an upright-inverted bimodal bicopter uav: Attitude control and optimization. p. 4

[33] R Mahony et al. Multirotor aerial vehicles: Modeling, estimation, and control of quadrotor. p. 9

[34] P J Huber. p. 3 Robust Estimation of a Location Parameter. p. 3

[35] J Schulman et al. Proximal policy optimization algorithms. p. 3

[36] J Heeg et al. Learning quadrotor control from visual features using differentiable simulation. p. 8

[37] M Watterson and V Kumar. p. 1 Control of quadrotors using the hopf fibration on so(3). p. 1

[38] A Spitzer and N Michael. Rotational error metrics for quadrotor control. p. 7

[39] Z Nurlanov. so(3) transformations and jacobian of extended log map. p. 4

[40] D Bertsekas. Nonlinear Programming. p. 1 p. 1

[41] J Bradbury et al. JAX: composable transformations of Python+NumPy programs. p. 0 p. 0

[42] G Cano Lopes et al. Intelligent control of ˜a quadrotor with proximal policy optimization reinforcement learning. p. 3

[43] J Milnor. Analytic proofs of the ”hairy ball theorem” and the brouwer fixed point theorem. p. 7

[44] D W Lyons. An elementary introduction to the hopf fibration. p. 0

[45] V Ganesh et al. EdgeOS: A highperformance middleware framework for autonomous robotics. p. 0

[46] O R developers. Onnx runtime.

[47] D Mellinger. Trajectory generation and control for quadrotors PhD thesis. p. 1 p. 1

[48] se-2 Copyright - Database copyright ProQuest LLC; ProQuest does not claim copyright in the individual underlying works; Last updated - 2023-03-03.

[49] D Mellinger. Trajectory generation and control for quadrotors PhD thesis.

[50] D Bertsekas. Nonlinear Programming.

[51] J Bradbury et al. JAX: composable transformations of Python+NumPy programs.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Modulate a constant reference trajectory to perform compact, position-constrained quadrotor inversions while remaining compatible with traditional trajectory generation and tracking across flight regimes using a learned reference modulation policy. This allows for aggressive maneuvers anchored by an almost globally stable reference controller, exploring maneuvers while anchored by an almost globally stable reference controller.

  2. Train separate policies via reinforcement learning for nominal-to-inverted (NTI) and inverted-to-nominal (ITN) transitions with separate training weights to handle regime switching. This allows the system to manage the transition dynamically, as seen in the text where separate control gains are used for different flight regimes.

  3. Use a piecewise first-order model adapted from [4] to account for trans-regime transient dynamics variability rather than explicitly modeling stochastic timing of reversal delay. This provides a computationally efficient model that remains structurally simple and easy to fit to little experimental data.

  4. Employ a learned reference modulation policy that outputs both position reference modulation and thrust posture, allowing the policy to determine thrust reversal timing, removing the need for heuristics present in baseline classical methods.

  5. Integrate an optimal control allocation (OCA) that solves a weighted least squares formulation with Tikhonov regularization to compute optimal thrust commands under saturation conditions. This allows for effective allocation even when actuator limits are active, as shown by the cost function being augmented with a Tikhonov regularization term and parameter lambda.

  6. Achieve successful zero-shot quadrotor inversions on hardware at varying initial yaw angles with position RMSE below 0.35 m and compatibility with downstream trajectory generation through circular flight in both regimes. This demonstrates a robust sim2real transfer, as the paper states, The proposed method transfers favorably from simulation to hardware.

Sources

Related papers