Discovering novel control strategies for turbulent wings through deep reinforcement learning

Project Information

DEEPCONTROL

Grant agreement ID: 101043998

DOI

10.3030/101043998

EC signature date 31 March 2022

Start date 1 April 2022

End date 31 March 2027

Funded under

• European Research Council (ERC)

Total cost

€ 1 999 748,00

EU contribution

€ 1 999 748,00

Coordinated by

KUNGLIGA TEKNISKA HOEGSKOLAN
Sweden

Periodic Reporting for period 1 - DEEPCONTROL (Discovering novel control strategies for turbulent wings through deep reinforcement learning)

Reporting period: 2022-04-01 to 2024-09-30

Summary of the context and overall objectives of the project

DEEPCONTROL aims at using high-fidelity simulations and deep reinforcement learning to develop a framework for real-time prediction and control of the flow around wing sections and three-dimensional wings based only on sparse measurements. We first perform high-order spectral-element simulations of wing sections and three-dimensional wings at high Reynolds numbers. Using sparse measurements at the wall, we reconstruct the velocity fluctuations above the wall within a region of interest. To this end, we employ a generative adversarial network (GAN), together with a fully-convolutional network (FCN) and modal decomposition. Then, we perform flow control based on deep reinforcement learning (DRL), which enables discovering novel solutions in terms of flow actuation and design of winglet geometry. In order to assess the robustness of the framework for real-time applications, we also carry out detailed wind-tunnel experiments.

This framework will constitute a breakthrough in aviation sustainability, and will enable developing more efficient aeronautical solutions worldwide.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

First of all, we have carried out high-fidelity simulations of turbulent wings in various configurations. We have conducted high-resolution large-eddy simulations (LESs) of a NACA4412 wing section at a Reynolds number based on freestream velocity and chord length of over 1 million, with several angles of attack up to 14 degrees. This enables analyzing, in a very high level of detail, both attached and detached cases. Furthermore, we have carried out high-resolution LES of a NACA0012 wing with a rounded wing tip, i.e. including the effect of wing-tip vortices, at a Reynolds number of 200,000 and angles of attack up to 10 degrees, again being able to assess both attached and detached cases. This is a unique database, with important implications in the context of wall-bounded turbulence.

When it comes to sensing the flow from the wall, we have further developed several deep-learning-based techniques, allowing very accurate predictions with a wide range of cases, including flat-plate turbulence and also three-dimensional wakes. In addition to being able to sense two-dimensional (2D) planes from the wall, we can also predict the three-dimensional (3D) volume above the wall, thus significantly enhancing the applicability of the sensing strategy. Also, when it comes to sensing an predicting the flow, we have developed a methodology based on beta variational autoencoders (betaVAE) and transformers to create reduced-order models (ROMs) which enable very robust predictions of the flow, outperforming classical methods e.g. based on proper-orthogonal decomposition (POD). Furthermore, we have also developed a novel methodology based on explainable deep learning which enables identifying the most important regions of turbulent flows, leading to results questioning the classical knowledge regarding coherent structures in turbulence.

Regarding the usage of deep reinforcement learning (DRL) for flow control, we have established a multi-agent reinforcement learning (MARL) methodology leading to extremely good flow-control results in a number of relevant turbulent cases, including: turbulent channel flow (where we reduce the drag more than the classical opposition control); turbulent separation bubble (where we reduce the separation length more than the classical periodic control); three-dimensional cylinder (where we reduce the drag more than the classical periodic control); and even Rayleigh-Béndard convection (where we reduce the Nusselt number more than the classical proportional control). This method, leading to breakthroughs in the context of flow control, is currently being applied to turbulent wings, leading to very promising results.

We have also started performing LES of more complex wing geometries, including high-lift devices and realistic wings. Finally, we have started the design of the experimental work, involving turbulent wings with jet actuators to reduce the drag, and also cases with separation aimed at reducing the stall conditions.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

So far, we can highlight three main breakthroughs in the field of turbulent-flow studies, facilitated by our deep-learning capabilities.

First, the framework to develop reduced-order models (ROMs) relying on beta variational autoencoders (beta VAEs) and transformers enables very compact reduced representations of the high-dimensional fluid flows under study. The autoencoder enables compressing the flow in a non-linear manner, thus fewer latent representations are needed to reconstruct a significant fraction of the energy in the flow compared with a linear method such as proper-orthogonal decomposition (POD). Furthermore, the beta VAE enables disentangling the latent space, a fact that produces essentially independent latent vectors, facilitating the interpretability of the ROM and also improving the temporal predictions. Such predictions are conducted with a transformer, which can more effectively identify and exploit the temporal patterns in the data, since it is not limited to sequential relations like the well-known long short-term memory (LSTM) neural network. This work has been recently published in Nature Communications.

Second, the method to identify regions of importance in the flow based on explainable deep learning has revealed that the structures studied in turbulence from a classical perspective may actually not be the most important. These structures include vortical motions and also Q events, i.e. regions of intense Reynolds shear stress. We train a deep neural network (in particular, a U-net) to predict the 3D flow in the future based on the present state, and use SHAP to assess the most important regions of the input. Our results show that the most important regions, assessed via SHAP, are actually not the same as those identified based on the classical knowledge. Furthermore, these SHAP regions can be used to motivate flow-control strategies. This work has also been recently published in Nature Communications.

Finally, we have established a new standard regarding what can be achieved with deep reinforcement learning (DRL) for developing flow-control strategies in turbulent flows. In DRL an agent (the neural network) interacts with an environment (the flow, either numerical or experimental) through actions (the control), in order to maximize a reward (in this case, drag reduction or separation-length reduction). By iteratively interaction with the environment, the goal is to obtain a policy which, given the state of the system (some observation) chooses the optimal actions to maximize the reward in the long term. We have implemented DRL to perform flow control in turbulent channels, turbulent separation bubbles, 3D cylinders, Rayleigh-Bénard convection and now in wings, outperforming classical control. Our results are published in fluid-mechanics journals, and we have some of this work under revision in Nature journals.