Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate

arXiv:2404.13003 · physics.flu-dyn · Submitted 2024-04-19 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Physics. The day in general physics: instruments, methods and results across the field.

Kai: We're looking at the paper: "Deep reinforcement learning-based active flow control of an elliptical cylinder".

Mira: Here's the gist of the paper. Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate is studied to understand the adaptability of DRL technology for bluff body flows with complex geometries.

Kai: The research findings indicate that, for elliptical cylinders with Ar between 1.75 and 0.75, the reduction in drag coefficient ranges from 0.9% to 15.7%, and the reduction in lift coefficient ranges from 95.2% to 99.7%.

Mira: Problem Setup and Governing Equations The study investigates the flow characteristics around a two-dimensional elliptical cylinder using the Navier-Stokes equations to describe the dynamics of this incompressible viscous fluid flow.

Kai: The Reynolds number is set to Re = 100 in this study.

Title and authors: Kai: So let's talk about who wrote this. The paper is titled "Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate," written by Wang Jia and Hang Xu.

Mira: It’s interesting that the authors are coming from different backgrounds, with one focusing on ocean and civil engineering, which is where they study these physical fluid dynamics problems.

Lev: And the fact that they're using PPO, Proximal Policy Optimization, which is a very common reinforcement learning algorithm right now. I wonder if that choice was intentional for balancing exploration and exploitation in this fluid control task.

Kai: It seems intentional. The paper says they utilize the Proximal Policy Optimization algorithm to precisely control the mass flow rates of synthetic jets located on the upper and lower surfaces of a cylinder to achieve reduction in drag, minimization of lift, and suppression of vortex shedding >

Mira: That setup is key. They are using those synthetic jets—blowing or suction—to actively manipulate the flow field around the body. It’s a direct intervention method that DRL is learning to optimize >

Lev: From an error correction perspective, controlling mass flow rates in real-time requires very fast inference and reliable actuation, which is a significant overhead when you move from simulation to actual hardware implementation.

Kai: Right, the paper mentions their numerical setup uses OpenFOAM as the incompressible flow solver with a hybrid meshing approach combining structured and unstructured meshes >

Mira: That’s standard practice for these kinds of CFD problems, but it shows they are grounding their learning agent in a fairly rigorous computational fluid dynamics environment before applying the DRL control layer >

The paper's summary: Kai: So what’s the core finding here? The paper summarizes that DRL excels in many active flow control problems because its focus on long-term rewards lets the control models balance short-term gains with future outcomes >

Mira: It’s about how DRL handles the inherent nonlinearities in flow around bluff bodies, like flow separation and recirculation, which traditional methods often struggle to manage effectively >

Lev: So they are essentially using this AI framework to learn complex control policies that map a fluid state directly to an optimal jet actuation sequence over time >

Kai: Precisely. They’ve shown that for elliptical cylinders with an aspect ratio Ar between one point seven five and zero point seven five, the DRL-based AFC technology is very effective at suppressing vortex shedding and enhancing flow stability in the wake field across different aspect ratios > <ref:2404.13003#pg1,Ar between 1.75 and 0.75, the>

Mira: And they quantify this effectiveness by stating that for those specific shapes, the control achieves substantial reductions in drag coefficient ranging from zero point nine percent to fifteen point seven percent and lift coefficient reductions between ninety-five point two percent and ninety-nine point seven percent > <ref:2404.13003#pg1>

Lev: Those numbers are what we need to look at when we try to translate this. A fifteen point seven percent drag reduction is a lot of energy savings, provided the control action itself doesn't introduce massive parasitic losses in the actuation system.

Kai: And they also noted that when the aspect ratio decreases further, like Ar values of zero point five, zero point two five, zero point one and zero, those reduction rates for drag and lift coefficient are significantly higher >

The paper's improvements: Mira: What the paper suggests as an improvement is really about showing that the DRL framework itself is robust enough to handle geometric transitions. They systematically test it on elliptical cylinders with Ar ranging from zero to two, which verifies the robustness of this technology across different shapes >

Lev: That verification across a wide range of shapes, from a slender cylinder to a flat plate, is what moves this beyond just being an empirical success on one specific geometry >

Kai: And they also explore the physical mechanisms behind the control in Section D. They analyze how the controlled flow actually works when they are applying these jet actions >

Mira: The paper indicates that by using this DRL-based AFC, they can maintain high stability by transforming those originally periodic oscillations of the drag coefficient into a minimum value and stabilizing them >

Lev: That stabilization aspect is really important for practical applications because it means you aren't just reducing a single measured value; you’re smoothing out the entire flow response over time, which is what real systems need to handle >

Kai: So, they are showing that the agent isn't just reacting; it’s learning a policy that results in a stable and symmetrical recirculation bubble instead of whatever chaotic thing was happening before >

Conclusion: Mira: So wrapping up this paper on "Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate," the main implication is that this DRL approach shows strong adaptability for bluff body flows with complex geometries >

Kai: It demonstrates that we can train these agents to learn optimal control strategies directly through interaction with the environment, which means they can adapt their control policies in real time to variations in the flow state >

Lev: From a hardware perspective, it’s promising because if you could manage the computational overhead of PPO inference on a real-time system, this framework offers a way to achieve those significant drag and lift reductions we saw on paper >

Mira: For Ar between one point seven five and zero point seven five, the mass flow rate ratios of synthetic jets relative to the inlet flow rate are less than one percent, which points to a very energy-saving attribute for this control method > <ref:2404.13003#pg1,Ar between 1.75 and 0.75, the>

Kai: It shows that DRL-based AFC can moderately alleviate vortex shedding and reduce the amplitude and frequency of fluctuations in drag and lift coefficients for aspect ratios ranging from zero point five up to zero >

Lev: The paper’s limitation, which they mention, is that while they show effectiveness across shapes, the study doesn't fully detail how this DRL framework would scale up when you move from two-dimensional simulations to full three-dimensional Navier-Stokes models with realistic boundary conditions >

Mira: That’s a fair point. The real challenge moving forward will be translating these 2D control policies into robust, high-fidelity three-dimensional systems where those physical assumptions start breaking down > <ref:2404.13003#pg1>

Wang Jia, Hang Xu

School of Ocean and Civil Engineering, Shanghai Jiao Tong University

physics.flu-dyn

Submitted: 2024-04-19

Updated: 2024-09-26

Journal ref: Physics of Fluids 36, 074117 (2024)

DOI: 10.1063/5.0214664

Code: https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 80/100

The gist: Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate is studied to understand the…

Key concepts

Deep Reinforcement Learning (DRL)
DRL combines deep learning with reinforcement learning to allow an AI agent to learn how to make decisions through trial and error. In this study, the DRL agent learns the best way to adjust jet flow rates in real-time based on the fluid dynamics it observes, enabling adaptive flow control.
Active Flow Control (AFC)
AFC involves using actuators, like synthetic jets on a cylinder's surface, to actively manipulate fluid flow. The DRL system uses these jets as controls to suppress unwanted phenomena like vortex shedding and improve the overall aerodynamic performance of the cylinder.
Drag and Lift Coefficients (CD and CL)
These are key measures of aerodynamic forces experienced by the cylinder. Drag coefficient (CD) measures resistance to flow, while lift coefficient (CL) measures the force perpendicular to the flow. The DRL system aims to minimize CD and control CL for better efficiency.
Aspect Ratio (Ar)
The aspect ratio is a geometric parameter describing the shape of an elliptical cylinder, essentially its length relative to its width. The study investigates how well the DRL control adapts when this ratio changes, from elongated shapes to flatter ones.

Terminology

Summary

Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate is studied to understand the adaptability of DRL technology for bluff body flows with complex geometries. p. 0 The research findings indicate that, for elliptical cylinders with Ar between 1. p. 075 and 0.75, the reduction in drag coefficient ranges from 0. p. 09% to 15.7%, and the reduction in lift coefficient ranges from 95. p. 02% to 99.7%.

Problem Setup and Governing Equations

The study investigates the flow characteristics around a two-dimensional elliptical cylinder using the Navier-Stokes equations to describe the dynamics of this incompressible viscous fluid flow. The Reynolds number is set to Re = 100 in this study. p. 1 Boundary conditions are defined with a parabolic velocity profile at the upstream boundary and a slip-wall boundary condition at the upper and lower boundaries. p. 3 The lift coefficient (CL) and drag coefficient (CD) are defined as physical important quantities in active flow control. p. 3

Numerical Simulation Methodology

The numerical simulation utilizes the open-source CFD package OpenFOAM as the incompressible flow solver. A hybrid meshing approach is adopted for discretization, integrating both structured (quadrilateral) and unstructured (triangular) meshes. p. 4 Grid independence tests were performed using three different mesh resolutions to validate the numerical method. p. 4

Deep Reinforcement Learning Framework

The DRL model marries the perceptual power of deep learning with the decision-making and optimization capabilities of reinforcement learning. The Proximal Policy Optimization (PPO) algorithm is used to precisely control the mass flow rates of synthetic jets located on the upper and lower surfaces of a cylinder. p. 0 The reward function is formulated to evaluate the agent based on the average drag coefficient, time-averaged drag coefficient, and time-averaged lift coefficient during the action. p. 6

Flow Control Performance Analysis

The DRL-based AFC technology demonstrates consistent and effective suppression of vortex shedding and enhancement of flow stability in the wake field across different aspect ratios. For Ar values between 1. p. 475 and 0.75, the control achieves substantial reductions in CD ranging from 0. p. 29% to 15.7% and CL exceeding 95%. When Ar decreases to 0. p. 85, 0.25, 0.1, and 0, the reduction rates of CD and CL are significantly higher but vortex shedding phenomena have not yet stabilized compared to cases with larger Ar values. p. 0

Conclusion on Control Efficacy

The DRL-based AFC implemented in this study is demonstrated across various aspect ratios, showing robust adaptability and the capability to learn optimal control strategies directly through interaction with the environment. The control policies based on the PPO algorithm significantly reduce CL around the elliptical cylinder, greatly diminish CD, and completely suppress vortex shedding in the wake. p. 1 For Ar ranging from 1. p. 275 to 0.75, the mass flow rate ratios of synthetic jets relative to the inlet flow rate are less than 1%, showcasing its significant energy-saving attribute. p. 6 The study concludes that DRL-based AFC strategies can moderately alleviate vortex shedding and reduce the amplitude and frequency of fluctuations in CD and CL for Ar values ranging from 0. p. 15 to 0. The control results underscore the robust adaptability of DRL-based AFC strategies to various geometric shapes. p. 2 The paper concludes by summarizing the key findings and discussing their implications. p. 2 The authors express gratitude to Dr. Jean Rabault (University of Oslo, Oslo, Norway), Dr. p. 0 Jichao Li (National University of Singapore, Singapore), and Mr. p. 0 Qiulei Wang (The University of Hong Kong, Hong Kong SAR, China) for making their open-source codes for DRL and numerical simulation available online. p. 0 The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github. p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0 p. 0com/venturi123/DRLinFluids39,65 >. The authors report no conflict of interest. p. 1 Wang Jia conceptualized the work (equal); Hang Xu conceptualized the work (equal). p. 0 Wang Jia led data curation (lead); Hang Xu led validation (lead). p. 0 The data supporting the findings are available from the corresponding author upon reasonable request. Wang Jia's ORCID is https://orcid.org/0009-0008-2786-397X and Hang Xu's is https://orcid.org/2413.15 >. The references provided support the research presented in this paper. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.com/npuljc/RL control Nek500038,64, and https://github.com/venturi123/DRLinFluids39,65 >. The relevant code repositories can be accessed at https://github.com/jerabaul29/Cylinder2DFlowControlDRLParallel34,63, https://github.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Deep reinforcement learning agents can learn optimal control strategies directly through interaction with environments, as demonstrated by DRL-based AFC technology shows adaptability and potential in addressing flow control problems for this type of bluff body flow. The improved system can adapt its control policies in real time to variations in the flow state.

  2. The AI system can achieve precise drag reduction and lift suppression across a range of elliptical cylinder aspect ratios, specifically achieving reductions from 0.9% to 15.7% in drag coefficient for Ar between 1.75 and 0.75, and up to a 68% reduction in lift coefficient for Ar = 0.5 to Ar = 0 based on Table III results.

  3. The system can maintain high stability by transforming oscillating recirculation bubbles into stable ones; The originally periodic oscillations of the drag coefficient are reduced to their minimum value and stabilized and the oscillating recirculation bubble in the baseline flow is gradually elongated and transformed into a stable and symmetrical recirculation bubble.

  4. The DRL-based AFC framework can dynamically respond to flow state variations, as it dynamically responds in real time to variations in the flow state, allowing for adaptive adjustments based on observed states st.

Sources

Related papers