Aerodynamic Optimization of Airfoil in Wide Range of Operating Conditions Based on Reinforcement Learning

Abstract

Airfoil aerodynamic design represents an essential domain in aircraft development, where the pursuit of advanced and intelligent optimization strategies is important for achieving significant advancements. In this paper, we demonstrate the effectiveness and versatility of reinforcement learning (RL)-based optimization methods in enhancing aerodynamic performance for both transonic and supersonic airfoils. We introduced a novel methodology using RL to optimize airfoil designs, leveraging ADflow as the aerodynamic solver and constructing an RL environment where Class-Shape Transformation (CST) parameters describe the airfoil geometry, transforming it into a finite state variable. Key flow field features, especially shock waves, were incorporated to guide the optimization process, enabling the RL model to iteratively improve designs based on real-time feedback from simulations. Applied to transonic airfoils, this method yielded remarkable results, including a 70.20% increase in the lift-to-drag ratio for one airfoil, with consistent improvements across various initial geometries and flight conditions. Extending to the NASA SC(2)-0404 supersonic airfoil, the optimized design achieved significant geometric changes that resulted in a 6.25% increase in the lift-to-drag ratio, with improvements ranging from 4.90% to 25.46% across different lift coefficients. These findings highlight the robustness and adaptability of RL techniques in addressing the unique challenges of both transonic and supersonic aerodynamics while maintaining structural integrity.

1. Introduction

The aviation industry is undergoing a transformative shift towards more efficient and environmentally friendly solutions. High-speed aircrafts capable of significantly reducing travel time have gathered considerable attention from both academic communities and industries. Green transonic and supersonic aircrafts represent a critical direction for the future development of civil aviation, offering not only faster travel but also addressing the pressing need for sustainable flight technologies. Enhancing aerodynamic performance in transonic and supersonic conditions is pivotal to achieving green and sustainable aviation, as it can lead to reduced fuel consumption, lower emissions, and improved operational efficiency.

Aerodynamic shape optimization plays a crucial role in this endeavor. Traditional methods for designing airfoils have relied on iterative wind tunnel testing and empirical adjustments, which are not only costly and time-consuming but also limited by their reliance on designer experience. The advent of advanced computational tools, particularly computational fluid dynamics (CFD), has revolutionized the field. CFD-based numerical optimization methods have become indispensable for achieving better aerodynamic performance in aircraft design.

However, the complexity and nonlinearity of aerodynamic optimization problems pose significant challenges that traditional methods struggle to address effectively. This is where reinforcement learning (RL) emerges as a powerful alternative [1,2,3,4]. RL offers a dynamic approach that can learn from interactions with complex environments, making it highly suitable for tackling the intricate problems associated with airfoil aerodynamic design. Unlike traditional optimization methods that require predefined rules or extensive simulations, RL enables the development of adaptive strategies that improve over time through trial and error, guided by rewards that reflect desired outcomes.

Several pioneering studies have demonstrated the potential of RL in aerodynamic optimization. Yonekura et al. [5] introduced a deep reinforcement learning (DRL) framework for optimizing wing angles in 2D, steady, incompressible flows, highlighting DRL’s generalization ability. Viquerat et al. [6] extended DRL to shape optimization, showing that well-crafted reward functions enable agents to autonomously produce optimal shapes. Li et al. [7] incorporated physical attributes like shock wave position and peak suction Mach number into state variables for supercritical airfoil design, achieving drag minimization via a Hicks–Henne function controlled by three parameters. Dussauge et al. [8] modeled airfoil optimization as a Markov decision process (MDP) by using neural networks to approximate values and policy functions, which improved learning efficiency. Liu et al. [9] proposed a multi-point DRL-based optimization framework for supercritical airfoils, integrating surrogate models to lower computational costs while managing complex constraints. Su et al. [10] explored a deformable airfoil reverse design with Deep Q-Networks (DQNs), demonstrating its practicality compared to conditional generative adversarial networks (CGANs). Liu et al. [11] applied an actor-critic algorithm to rotor wing optimization, improving lift coefficients and reducing drag and torque by addressing dynamic stall characteristics. These studies highlight RL’s transformative role in advancing aerodynamic optimization techniques for enhanced airfoil performance.

This paper explores the application of RL to optimize the design of transonic and supersonic airfoils. The proposed method introduces a RL-based framework for aerodynamic optimization, which differs fundamentally from traditional optimization techniques such as gradient-based or evolutionary algorithms. One of the key advantages of our approach lies in its ability to autonomously explore a broad design space without relying on explicit gradient information or population-based heuristics. This makes it particularly suitable for optimization problems commonly encountered in airfoil aerodynamic design. By leveraging RL, we aim to develop intelligent algorithms capable of efficiently navigating the complex space of aerodynamic design parameters. The focus is on creating a novel framework that integrates ADflow for aerodynamic simulations and CST parameters to describe airfoil geometry. This approach allows the RL model to iteratively refine designs based on real-time feedback from simulations. We investigate how this method can improve airfoil shapes under various flow conditions, with the goal of enhancing lift-to-drag ratios and reducing shock wave drag. This study will examine the effectiveness of RL in optimizing both transonic and supersonic airfoils.

2. Method

2.1. Aerodynamic Analysis Approach

The open-source code ADflow is used for aerodynamic analysis, which is a finite volume CFD solver for structured multiblock and overlapping grids [12]. ADflow solves the compressible Euler equations, laminar Navier–Stokes equations, and Reynolds-averaged Navier–Stokes (RANS) equations with second-order accurate spatial discretization. It employs a variety of numerical methods to achieve robust and efficient convergence to steady-state solutions, including multigrid, approximate Newton–Krylov, and full Newton–Krylov algorithms. The integration of these iterative methods enhances ADflow’s performance, making it both robust and fast [13,14].

For validation, we performed a comparison study against experiments under specific flow conditions: a Mach number of Ma = 0.729 and a Reynolds number of Re = 6.5 × 10⁶. The RANS equations and the Spalart–Allmaras turbulence model equation are solved on a converged grid. Figure 1a shows the geometry of the RAE2822 airfoil, and the converged pressure coefficient distribution is given in Figure 1b, where the curve located on the upper side represents the pressure coefficient distribution on the upper surface of the airfoil, and the curve located on the lower side represents the pressure coefficient distribution on the lower surface of the airfoil, from which it is known that the calculated results from ADflow in our study exhibit (the blue solid line) good agreement with the experimental data (the black X-shaped markers in Figure 1b) [15].

2.2. Geometric Parameterization

The Class-Shape Transformation (CST) method [16] is utilized to parameterize the airfoil, which can represent the airfoil shape using Bernstein polynomials with relatively few parameters:

$${\left({\displaystyle \frac{\mathrm{z}}{\mathrm{c}}}\right)}_{\mathrm{u}\mathrm{p}\mathrm{p}\mathrm{e}\mathrm{r}}={\left({\displaystyle \frac{x}{c}}\right)}^{N1}{\left(1-{\displaystyle \frac{x}{c}}\right)}^{N2}\sum _{i=1}^{n}A{u}_i{\displaystyle \frac{n!}{i!\left(n-i\right)!}}{\left({\displaystyle \frac{x}{c}}\right)}^i{\left(1-{\displaystyle \frac{x}{c}}\right)}^{n-i}+{\displaystyle \frac{x}{c}}{\displaystyle \frac{∆z}{c}}$$

(1)

$${\left({\displaystyle \frac{\mathrm{z}}{\mathrm{c}}}\right)}_{\mathrm{l}\mathrm{o}\mathrm{w}\mathrm{e}\mathrm{r}}={\left({\displaystyle \frac{x}{c}}\right)}^{N1}{\left(1-{\displaystyle \frac{x}{c}}\right)}^{N2}\sum _{i=1}^{n}A{l}_i{\displaystyle \frac{n!}{i!\left(n-i\right)!}}{\left({\displaystyle \frac{x}{c}}\right)}^i{\left(1-{\displaystyle \frac{x}{c}}\right)}^{n-i}+{\displaystyle \frac{x}{c}}{\displaystyle \frac{∆z}{c}}$$

(2)

where $\mathrm{\Delta }\mathrm{z}$ represents half the thickness of the trailing edge of the airfoil and $c$ denotes the chord length of the airfoil. For a general airfoil, the parameters $N1$ and $N2$ are typically set to 0.5 and 1.0, respectively. The polynomial order is denoted by $n$, while $Au$ and $Al$ represent the CST parameters for the upper and lower surfaces of the airfoil, respectively.

In order to balance the accuracy of airfoil geometry description and the exploration scope of reinforcement learning, we select 10 CST coefficients as design variables for subsequent optimization methods based on reinforcement learning.

2.3. Reinforcement Learning

Reinforcement learning (RL) is an interactive learning paradigm in which an agent learns optimal behaviors by interacting with its environment [17]. The agent selects actions based on the current state of the environment, observes the resulting outcomes, and receives rewards or penalties. Positive rewards reinforce beneficial actions, while negative rewards deter unfavorable ones. Over time, this iterative feedback loop enables the agent to develop strategies that maximize cumulative rewards, leading to improved decision-making and performance.

The RL process utilized in this paper can be summarized in Figure 2. Firstly, we use the CST method to parametrize the airfoil, serving as the state of reinforcement learning, which constitutes the current state of the environment. Then, the intelligent agent generates new airfoils by modifying CST parameters to perform actions on the environment. After this operation, the aerodynamic performance of the modified airfoil is estimated via the RANS solver (ADflow), such as the lift-to-drag ratio (L/D) and flow field characteristics. These aerodynamic performance indicators serve as the basis for rewards. At the end, the score can be calculated based on the cumulative performance indicators of the airfoil generated at each iteration to estimate the overall performance of the agent.

3. Formulation of Optimization Problem

The performance of an airfoil is closely linked to its flow field. An effective airfoil design strategy relies on understanding the aerodynamic and flow characteristics of the airfoil, allowing for the development of solutions that can adapt to various operating conditions. Specifically, the physical properties of the flow field, particularly the pressure distribution on the airfoil surface, serve as critical factors for airfoil optimization, enabling more efficient design outcomes. These characteristics also enhance the interpretability of the design process for engineers. In our study, we aim to demonstrate how the flow field characteristics influence the optimization process in RL by incorporating the airfoil surface pressure coefficient into the reward function.

To achieve this, we selected several key features from the airfoil’s flow field as part of the RL reward function based on Ref. [7], which includes three features, i.e., the pressure coefficient at the leading edge of the airfoil lower surface, the strength of the shock wave, and the minimum pressure coefficient on the airfoil’s upper surface.

To assess the impact of incorporating flow field features into the reward function, we compared the optimization performance of RL using these features against a baseline method without such features. For this analysis, we used the RAE2822 airfoil with the following conditions: a Mach number of Ma = 0.75 and a Reynolds number of Re = 6.4 × 10⁶. The goal was to maximize the lift-to-drag ratio at a fixed lift coefficient of C_L = 0.50.

The overall optimization problem is formulated as follows:

Design objective: The design objective is to maximize the lift-to-drag ratio at a Mach number of Ma = 0.75, a Reynolds number of Re = 6.4 × 10⁶, and a lift coefficient of C_L = 0.50.

Design variables: The airfoil CST coefficients are selected as design variables.

Design constraints: The design constraints include the maximum airfoil thickness and a target lift coefficient range to satisfy performance requirements. These constraints are embedded directly into the reward function or through penalties within the reinforcement learning framework.

Figure 3 shows the optimized airfoil geometry and the pressure coefficient distribution after optimization.

Table 1. Optimization results with and without features.

Feature	CD	L/D
Initial	0.0116	43.35
Three Features	0.0097	51.35
Without Features	0.0104	48.02

summarizes the optimization results. As can be seen from the results, including flow field features in the reward function leads to superior optimization performance by reducing shock wave intensity, decreasing wave drag, and lowering the drag coefficient, thereby enhancing the lift-to-drag ratio.

4. Airfoil Aerodynamic Optimization Case Studies

4.1. Case Study for Different Initial Airfoils

To further explore the effectiveness of the RL-based optimization design, we selected two initial airfoils—RAE2822 and HQ2012—with a relative thickness of 12%. These airfoils were optimized under flight conditions of C_L = 0.5, Ma = 0.75, and Re = 6.4 × 10⁶. The RL optimization method incorporated the three key flow field features discussed in Section 3 while ensuring that the airfoil thickness is no less than the original value.

The comparison of airfoil geometry and pressure distribution before and after optimization is illustrated in Figure 4.

Table 2. Optimization results and comparison.

Airfoil	Initial		Optimization		%
	CD	L/D	CD	L/D
RAE2822	0.0116	43.45	0.0097	51.35	18.17%
HQ2012	0.0216	23.27	0.0131	39.61	70.20%

summarizes the optimization results, highlighting improvements in aerodynamic performance across all selected airfoils. The RL-based optimization method proved highly effective in improving the aerodynamic performance of the two selected airfoils, resulting in notable increases in the lift-to-drag ratio. From the pressure coefficient distributions, it is clear that RL optimization weakened the shock waves, leading to decreased drag and an increase in the lift-to-drag ratio. Notably, the HQ2012 airfoil showed a much more significant improvement, with a 70.20% increase in the lift-to-drag ratio. In comparison, the RAE2822 airfoil only shows an 18.17% improvement in the lift-to-drag ratio. The main reason is that the initial RAE2822 airfoil already possesses good transonic performance, so the optimization effect is not as significant as that of the HQ2012 airfoil.

4.2. Transonic Airfoil Optimization Results

We conducted aerodynamic analysis on the optimization results of the RAE2822 airfoil, which was previously optimized using RL, at different Mach numbers: 0.30, 0.75, 0.78, and 0.80. The aerodynamic polar graphs obtained from this analysis are presented in Figure 5. The polar provides insights into how the optimized RAE2822 airfoil performs across a range of lift coefficients and Mach numbers. This analysis is crucial for understanding the robustness and versatility of the optimized design under varying flight conditions.

The RL-based optimization design for the RAE2822 airfoil shows significant aerodynamic performance improvements across various lift coefficients and Mach numbers. Starting with low-speed conditions such as takeoff and descent, at Ma = 0.30, the optimized airfoil achieved its maximum lift-to-drag ratio improvement at C_L = 0.60, resulting in a modest yet meaningful 2.21% increase. This demonstrates that RL-based optimization enhances aerodynamic performance even during critical flight phases where efficiency is paramount. Moving into transonic conditions, at Ma = 0.75, the intended cruising Mach number, the optimized airfoil achieved its most substantial lift-to-drag ratio improvement at C_L = 0.58, with a remarkable 26.22% increase, underscoring its exceptional performance under these conditions. At slightly higher transonic speeds, specifically Ma = 0.78, the maximum lift-to-drag ratio improvement was recorded at C_L = 0.47, achieving a significant enhancement of 35.95%, indicating robust performance even as Mach numbers increase. Finally, at Ma = 0.80, despite challenges posed by more obvious shock waves, the optimized airfoil maintained substantial performance benefits, with a 26.98% gain in the lift-to-drag ratio at C_L = 0.39. These findings highlight the versatility and effectiveness of the RL-based optimization method. The optimized RAE2822 airfoil not only achieves substantial improvements in the lift-to-drag ratio near its cruising Mach number but also demonstrates enhanced performance across a broader range of operating conditions. This comprehensive improvement highlights the potential of RL techniques to significantly advance airfoil design, offering better aerodynamic efficiency and adaptability for diverse flight scenarios.

4.3. Supersonic Airfoil Optimization Results

We applied the RL-based optimization design to the supersonic airfoil NASA SC(2)-0404 under conditions of C_L = 0.5, Ma = 1.40, and Re = 11.9 × 10⁶. The optimization incorporated three key flow field features introduced earlier—pressure, coefficient, and shock wave—while ensuring the airfoil’s thickness remained above 4% to preserve structural integrity. Figure 6 compares the airfoil’s geometry before and after optimization. RL effectively optimized the NASA SC(2)-0404 airfoil under supersonic conditions, resulting in key geometric modifications: decreased upper surface curvature, increased lower surface curvature, and a sharper leading edge. These changes led to a notable 6.25% increase in the lift-to-drag ratio, demonstrating the effectiveness of RL-based optimization in enhancing aerodynamic performance while addressing the unique challenges of supersonic aerodynamics.

To further analyze the optimization results, we examined the NASA SC(2)-0404 airfoil across a wide range of lift coefficients. The polar presented in Figure 7 illustrates the relationship between lift and drag coefficients, providing comprehensive insights into the aerodynamic performance. Additionally, Figure 8 shows the pressure coefficient distribution on the airfoil surface for selected lift coefficients, offering detailed visualizations of the improvements achieved through optimization. From Figure 7 and Figure 8, we can observe that the optimization results show varying degrees of improvements in the lift-to-drag ratio under different lift coefficients, while the changes in the pressure coefficient distribution exhibit similar trends. These analyses underscore the versatility and robustness of the RL-based method, highlighting its potential to significantly advance supersonic airfoil design by balancing aerodynamic efficiency with structural requirements.

5. Conclusions

In this paper we have studied the effectiveness and versatility of reinforcement learning (RL)-based optimization methods in enhancing aerodynamic performance for both transonic and supersonic airfoils. The main conclusions are summarized as follows.

• We introduced a novel methodology using RL to optimize airfoil designs. This approach leverages ADflow as the aerodynamic solver and constructs an RL environment where CST parameters describe the airfoil’s geometry, transforming it into a finite state variable. Key flow field features, especially shock waves, were incorporated into the training process to guide the optimization. This framework enables the RL model to iteratively learn and improve airfoil designs based on real-time feedback from aerodynamic simulations, leading to significant performance improvements.
• The application of RL-based optimization to transonic airfoils yielded remarkable results. We applied optimization to two initial airfoils—REA2822 and HQ2012—under flight conditions of C_L = 0.5, Ma = 0.75, and Re = 6.4 × 10⁶. The optimized airfoils showed reduced upper surface curvature and more rounded leading edges, aligning with traditional transonic optimization practices. Across various initial airfoils, the method consistently improved lift-to-drag ratios, with one notable case achieving a 70.20% increase. Performance improvements were observed across a wide range of lift coefficients and Mach numbers, demonstrating the robustness and adaptability of the optimized designs under diverse flight conditions.
• Extending the RL-based optimization to supersonic airfoils, specifically the NASA SC(2)-0404 airfoil under flight conditions of C_L = 0.5, Ma = 1.40, and Re = 11.9 × 10⁶, resulted in significant geometric changes that enhanced aerodynamic performance. The optimized design leads to a significant increase in the lift-to-drag ratio. In the range of the lift coefficient from 0.30 to 0.60, lift-to-drag ratio improvements varied from 4.90% to 25.46%, with the most significant enhancement observed at C_L = 0.10. These results highlight the effectiveness of RL techniques in addressing the unique challenges of supersonic aerodynamics while maintaining structural integrity.

While our reinforcement learning-based framework has demonstrated robust improvements in lift-to-drag performance for both transonic and supersonic airfoils, several limitations remain. First, the reliance on high-fidelity CFD solvers incurs substantial computational costs; in future studies, we will investigate the use of trained surrogate models or reduced-order solvers to accelerate the optimization loop without compromising accuracy. Second, the current work considers only clean, two-dimensional airfoil sections; extending the methodology to three-dimensional wing geometries and including high-lift devices (e.g., flaps and slats) will be essential to address the takeoff, landing, and maneuvering phases. Finally, the present formulation assumes steady-state flow; the incorporation of unsteady effects, as well as aeroelastic coupling, would further enhance the realism and applicability of the approach. These future research directions will guide our ongoing efforts to deliver a comprehensive and efficient design tool for future aircraft configurations.