Can Artificial Intelligence Accelerate Fluid Mechanics Research?

Abstract

The significant growth of artificial intelligence (AI) methods in machine learning (ML) and deep learning (DL) has opened opportunities for fluid dynamics and its applications in science, engineering and medicine. Developing AI methods for fluid dynamics encompass different challenges than applications with massive data, such as the Internet of Things. For many scientific, engineering and biomedical problems, the data are not massive, which poses limitations and algorithmic challenges. This paper reviews ML and DL research for fluid dynamics, presents algorithmic challenges and discusses potential future directions.

1. Introduction

In the past few years, machine learning (ML) and deep learning (DL) have been pursued in most social, scientific, engineering, and industry branches. Initially established at the point where computer and statistical science meet, ML evolution has been driven by advancements in Artificial Intelligence (AI) and its related algorithms, fortified by the “big data” and access to cost-effective computational architectures. ML-based approaches span fields such as construction engineering, aerospace, biomedicine, materials science, education, financial modelling, and marketing [1,2].

Machine learning techniques fall into three groups: unsupervised, semi-supervised, and supervised [2,3]. This paper considers DL as a subset of the broader ML field. The supervised approaches rely on labelled data to train the ML algorithm effectively. This is the most common category, with wide applicability in science and technology [4,5,6,7]. Unsupervised ML involves extracting features from high-dimensional data sets without the need for pre-labelled training data. Well-established clustering methods, such as K-means, hierarchical agglomerative clustering, and DBSCAN [8,9], apply here. Furthermore, dimensionality reduction techniques such as proper orthogonal decomposition (POD) [10], principal component analysis (PCA) [11,12] and dynamic mode decomposition (DMD) [13] emerged from fundamental statistical analysis, and Reduced Order Model techniques have also been bound to unsupervised ML. Meanwhile, semi-supervised learning combines elements from supervised and unsupervised ML and has been successfully incorporated in applications concerning time-dependence and image data [14,15].

At the heart of ML theories, the algorithms proposed range from linear feed-forward models to complex DL implementations. To distinguish the complexity of various models, the algorithms may fall into shallow or deep ML, with possible combinations. Shallow learning (SL) algorithms, both in classification and regression tasks, involve linear and multiple linear algorithms, their linear variants such as LASSO and Ridge, support vector machines (SVM), stochastic methods based on Gaussian processes and Bayes’ theorem, and decision trees [16]. Ensemble and super-learner methods, combining multiple shallow algorithms in more complex instances, such as the random forest, gradient boosting, and extremely randomized trees, have also increased in popularity in the past years [17].

Notwithstanding its wide acceptance, the applicability of SL is limited to small-to-medium-sized data sets, which might be a problem in demanding fluid mechanics tasks. Thus, the need for more robust methods has emerged, and DL has come to the fore [18]. The idea of DL is based on the multi-layer perceptron, the artificial neural network (ANN) architecture that can be seen as the corresponding artificial mechanism that mimics the biological functions of the neural networks of the human brain [6].

DL models have embedded complex mathematical concepts and advanced computer programming techniques in a great number of interconnected nodes and layers [19], making them an appropriate choice for dealing with applications that involve image and video processing [20,21,22,23], speech recognition [24], biological data manipulation [25], and flow reconstruction [26]. Nevertheless, a physical interpretation of the outcome is still lacking, which limits their application in physical sciences, where interpretability has a central role. In such cases, it would be beneficial to adopt models that produce meaningful results, i.e., mathematical expressions, which can spot correlations with existing empirical relations and propose an analytical approach bound to physical laws.

Conventional neural networks operate solely on data and are often considered “black box” models. On the other hand, symbolic regression (SR) is a class of ML algorithms in which mathematical equations bound exclusively to data are derived. In contrast to usual regression procedures, SR searches within a wide mathematical operator set and constants to connect input features to produce meaningful outputs [27] based on evolutionary algorithms. This process of extracting results that can be interpreted and exploited simultaneously has led to the development of numerous genetic programming-based tools that implement SR for disciplines from materials science to construction engineering to medical science [28].

ML methods have been employed in fluid dynamics research but have not been in engineering practice. Computational fluid dynamics (CFD) discretizes space using meshes: as the grid density increases, so does the accuracy, but at the expense of computational cost. Such techniques have benefited the aerospace, automotive and biomedical industries. The challenges are numerical and physical, and several reviews have been written on specific industrial applications or areas of physics [29,30,31,32]. However, despite significant advances in the field, there is still a need to improve efficiency and accuracy.

The most critical example is turbulence, a flow regime characterised by velocity and pressure fluctuations, instabilities, and rotating fluid structures (eddies) at vast, macroscopic and molecular scales. Accurately describing transient flow physics using CFD would require many grid points. Such an approach where the Navier–Stokes equations are directly applied is called direct numerical simulation (DNS) and is accompanied by tremendous computational overhead. As such, it is limited to specific problems and requires large high-performance computing (HPC) facilities and long run times. Consequently, reduced turbulent models are often used at the expense of accuracy.

Furthermore, while many such techniques exist, no universal model works consistently across all turbulence flows, so while many computational methods exist for different problems, the computational barrier is often unavoidable. Recent efforts in numerical modelling frequently focus on how to speed up computations, a task that would benefit many industries. A growing school of thought is to couple these numerical methods with data-driven algorithms to improve accuracy and performance. This is where ML becomes relevant.

The topic of ML and its applications in fluid mechanics is broad, and a single review article does not suffice to cover everything. Our primary aim is to touch on some applications as indicative examples, summarise the ML methods used, and identify challenges and future perspectives.

2. A Brief Overview and Methods Classification

Many articles and reviews are produced each year on subjects related to ML. This review covers the past five years when a vast increase in reviews on these subjects was observed. Figure 1 shows the number of publications referring to ML, DL and the number of relevant reviews, with data taken from Scopus. Further analysis of the results is presented in the pie charts in Figure 2. The key terms used in the Scopus search were: Machine Learning and Review, from 2017 to 2023.

The findings from this survey revealed some interesting points. A great number of reviews meant that an even greater number of original research articles had been published. If we limited the results of the Scopus search to reviews related to fluids, three major fields of application emerge that attracted research interest in applying ML methods to (a) theoretical and industrial oil and gas [33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52], (b) CFD simulations [53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68], and (c) health and medical applications [69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86] (Figure 2a).

Another popular subject is fluid property estimation with ML algorithms, a promising alternative to prohibitively expensive experimental measurements.Most reviews have exploited historical data from experiments or simulations to predict fluid behaviour and calculate the properties of interest [87,88,89,90,91,92,93,94,95,96]. Furthermore, heat transfer applications, concerning the addition of nanoparticles in simple fluids [97,98,99,100,101,102] or the investigation of flows in microfluidic channels [103,104,105,106,107,108] play a central role. Of equal significance, though with fewer instances, several recent reviews refer to aerodynamics [109,110,111,112,113], geosciences, geoengineering and environmental subjects [114,115,116,117,118,119,120,121,122,123,124,125,126], energy applications [127,128,129,130], biological processes [131,132,133,134], control processes [135,136,137], multiscale simulations [138,139,140,141], and general ML [2,142].

Common to all these applications is the difficulty of obtaining accurate data; industrial processes usually obtain them from field sensors. In addition to data acquisition and resolution, compression might be needed to handle quantity effectively, especially in real-time applications [143]. Data availability is a prerequisite because, for example, a lack of available data in industrial multiphase oil and gas flows makes it hard to experiment with ML techniques, and the need for interdisciplinary collaboration between industry and academia is highlighted [42]. Using only high-quality data is important to avoid incorporating erroneous, missing, or redundant data points. Before data are applied on a specific ML algorithmic platform, an extensive statistical analysis is conducted on the implied dataset, and feature engineering is employed to curate the data. Feature engineering involves raw data manipulation and extracting characteristic features before transforming them into an acceptable ML format. This process encompasses a set of statistical tools (e.g., feature importance selection, correlation tests, and variable transformation and normalization) that can be used to predict the desired outcome and enhance the performance of the model.

All the studies above have incorporated several supervised ML algorithms, while only a few unsupervised algorithms have been exploited for dimensionality reduction (DR) tasks. In Figure 2b, most of them went through various algorithm reviews that argued for their applicability to a specific subject. Property prediction with historical data can be realized with SL algorithms such as decision trees, linear regression [144] and simple NNs like the multi-layer perceptron (MLP) [11]. In general, the most widely incorporated SL algorithms in fluid research can be categorized into five areas [96]: (i) linear models based mainly on linear regression techniques and their variants such as LASSO and Ridge, (ii) algorithms that employ kernel functions to transform input data into a higher-dimensional space (e.g., support vector machines (SVMs)), (iii) decision-tree methods, which exploit the non-linear tree-structure to achieve accurate predictions, (iv) neural networks built on the perceptron architecture, and (v) instance-based, which exploits the proximity of instances inside the training set such as K-nearest neighbours.

A new category, the ensemble technique, has emerged to boost prediction accuracy further. This approach incorporates more than one base ML algorithm to construct a combined estimator, ensuring accuracy, robustness and lower variance [145]. Individual predictions are combined in classification or averaging in regression tasks. In this way, each base learner in an ensemble model can achieve better results as it is trained in a specific region of the dataset, thereby improving prediction accuracy and generalizability. Some widely applied ensemble techniques involve random forest (RF), gradient boosting machine (GBM), and extreme gradient boosting (XGBoost) [146].

On the other hand, a DL architecture with convolutional NNs (CNNs) or Recurrent NNs (RNNs) is incorporated to address flow reconstruction problems [26]. Moreover, as imaging applications in most branches of science, engineering and medical studies become more demanding, DL techniques are further combined with ensemble techniques [147]. Physics-informed neural networks (PINNs), another class of algorithm, are gaining ground since they add physical intuition in common NNs, and can bind experimental data with Navier–Stokes equations for fluid mechanics [140,141,148,149,150]. Algorithms based on genetic programming (GP) are best suited for flow control in various applications [151] and can be an excellent choice when the output needs to be interpreted as a mathematical equation [8].

Another research direction that has attracted attention is Physics-informed machine learning (PIML), which integrates ML algorithms with high-fidelity numerical simulations [58]. PIML could potentially reduce computing time in complex turbulence simulations.

3. Applications of ML in Fluid Mechanics

As mentioned in the Introduction, a long-standing problem in fluid dynamics is turbulence, a flow regime characterised by unpredictable, time-dependent velocity and pressure fluctuations [152,153,154,155]. The use of data-driven methods has increased significantly over the last few years, providing new modelling tools for complex turbulent flows [156]. ML has been used to model turbulence, such as the use of ANNs to predict the instantaneous velocity vector field in turbulent wake flows [157] and turbulent channel flows [158,159]. The popularity of many ML methods, especially deep learning, as a promising method for solving complex fluid flow problems is also supported by high-quality open-source data sets published by researchers [160,161,162].

Much effort is currently put into using ML algorithms to improve the accuracy of Reynolds-averaged Navier–Stokes (RANS) models for which GPs have been used to quantify and reduce uncertainties [163]. Tracey et al. [164] used an ANN that inputs flow properties and reconstructs the SA closure equations. The authors studied how different components of ML algorithms, such as the choice of the cost function and feature scaling, affected their performance. Other examples include using DL to model the Reynolds stress anisotropy tensor [165,166]. Bayesian inference has also been used to optimize RANS model coefficients [167,168], as well as to derive functions that quantify and reduce the gap between model predictions and high-fidelity data, such as DNS data [169,170,171]. Furthermore, a Bayesian neural network (BNN) has been used to improve the predictions of RANS models and to specify the uncertainty associated with the forecast [172].

In the field of large eddy simulations (LES), an ANN was used to replace these subgrid models [173,174]. Convolutional neural networks (CNNs) have also been used to calculate subgrid closure terms [175]. Moreover, DL modelling using input vorticity and stream function stencils was implemented to predict a dynamic, time- and space-dependent closure term for the subgrid models [176]. More specifically, starting from the detailed DNS simulation output, the main focus was to employ low-resolution LES data to train a NN model and reconstruct fluid dynamics to replace DNS where possible. To this end, many relevant applications have emerged. Bao et al. [177] have suggested combining physics-guided, two-component, PDE-based techniques with recurrent temporal data models and super-resolution spatial data methods. An extension of this approach includes a multi-scale temporal path UNet (MST-UNet) model that can reconstruct both temporal and spatial flow information from low-resolution data [177].

Fukami et al. [178] have applied super-resolution techniques, common in image reconstruction applications, on 2D course-flow fields and managed to reconstruct laminar and turbulent flows. A more recent version of this method employed autoencoders to achieve order reduction for fluid flows [179]. A complete super-resolution case study is presented in another work, covering all the details of a CNN-based, super-resolution architecture for turbulent fluid flows with DNS training data for various Reynolds numbers [180]. Nevertheless, generative models are often incorporated because DNS data are hard to obtain. In the review of Buzzicotti [53], three prevalent models are discussed: variational autoencoders (VAE), GANS, and diffusion models. These models are based on CNNs, and their implementation is focused on reproducing the multiscale and multifrequency nature of fluid dynamics, which is more complex than classical image recognition applications.

Another exciting application of ML in fluid mechanics is related to food processing [181]. The ML models aim to optimize process parameters and kinetics for reduced energy consumption, speed up the processing time, and continuously improve product quality. Several food processing operations, including drying, frying, baking, and extrusion, each require the development of ML algorithms and their training and validation against accurate data.

Furthermore, ML has found application in many chemical engineering flows. These applications include transport in porous media, catalytic reactors, batteries, and carbon dioxide storage [182]. Many of these problems are linked to energy transition efforts such as batteries, where long-term safety, energy density, and cycle life are particularly interesting. Diverse ML applications in fluid mechanics include nanofluids and ternary hybrid nanofluids [183] as well as metallurgy processes [184]. The domain of applications is broad and goes beyond the scope of a single review article.

4. Challenges

The data from fluid simulations or flow sensors are vast and have prohibitive degrees of freedom to cope with. Bearing in mind also that 2D flow fields in the pictorial form are currently gaining ground as research items for fluid flow characterization, with added complexity when they evolve in time (e.g., multiple images), it becomes clear that great effort is needed to harness and derive meaning from processing and analysis. On the contrary, there are cases where information from data is sparse, and the implied ML algorithm must be accurate. Methods and techniques from reinforced learning could be exploited here, such as GANs [185,186], which can train an ML model with fictitious data without affecting accuracy.

We summarized the challenges of ML in fluid mechanics in Figure 3 and detail them below.

• ML algorithms are well-defined mathematically and widely supported by software. An open question on their applicability in high-precision fluid mechanics, such as in biological or aerodynamic flows, is the availability of high-fidelity data to train ML models effectively [187].
• Open databases should become available to the scientific community and play a crucial role in machine learning, enabling knowledge sharing and advanced development.
• Modern computers and experimental devices have higher speed, power, and flexibility, making access to data easy. However, concerns about storage, retrieval, post-processing, and other challenges may arise, especially when dealing with massive turbulence data.
• ML in fluids extends beyond image processing as fluid imaging is not static and includes dynamic information [3]. Furthermore, when flow images are provided in time sequence, the computational load increases dramatically, and algorithmic selection is essential.
• Algorithms perform differently depending on the problem they are called on to solve. For example, there is no need to dive into a complex DL architecture when the task is the property prediction of a specific fluid based on several variable-type input parameters [188].
• Differential equation solutions with PINNs may fail in complex physical phenomena, such as turbulence, compared to traditional numerical methods [189]. There seems to be much to do to consider PINNs as the dominant PDE solver for fluid dynamics.
• At the nanoscale, problems originate from time and length scales. Complex aqueous environments and fluid/solid interfaces can only come to light through atomistic simulation techniques with first-principles accuracy. Machine learning potentials have been successfully introduced over the past years and are now close to standardization as ab-initio simulation alternatives [190].
• There are many instances in the physical sciences where decision-making is based on empirical relations. The challenge for future ML methods is to provide accurate data-derived physics-based equations rather than empirical ones. Symbolic regression can help towards this goal. [191].

5. Perspectives

Following the discussion on state-of-the-art ML in fluid dynamics and considering the emerging challenges, it is imperative to summarize where scientists and engineers should focus next for ML fluid dynamics applications.

As was made clear in the literature review, AI and ML methods have been widely incorporated in the industry - from oil and gas to energy production and aerospace engineering -particularly in research. There is ample historical and newly generated data to be analyzed in these fields, and ML has much to offer to promote domain knowledge. The statistical nature of ML, either in interpolation or short extrapolation instances, will continue to evolve.

Nevertheless, these applications exploit traditional ML methods. Although widely incorporated, they were developed in a different era and cannot keep pace with current research needs. Moreover, issues originating from big data, such as size, heterogeneity, real-time processing speed or various complex forms (e.g., 2D and 3D), will continue to pose serious obstacles [192].

Novel algorithmic implementations are expected to continue to adapt to these profound demands. This is unavoidable from a computer scientist’s and engineer’s point of view since new versions of the traditional algorithms should be proposed to handle big data. But here comes the impact of domain knowledge. Data and hidden patterns inside it may be enlightening, but it is doubtful if it can replace physical understanding. Therefore, fluid dynamics researchers should work in parallel to embed fluid concepts and physical laws onto the backbone of the proposed algorithms.

The increasing interest in SR algorithmic implementations can significantly boost computational science by suggesting practical and physically meaningful analytical equations that outperform established empirical or approximate equations [8]. This direction unveils the “black box” nature of ML, enabling these approaches to play a prominent role in predicting properties and guiding fluid dynamics research. This field of investigation still has much to offer, and extensive research on real fluids and mixtures, although computationally demanding and challenging to interpret, is to come.

In another direction, Vinuesa and Brunton [193] have highlighted areas of future research on DNS acceleration, turbulence closure modelling improvement, and new reduced-order model (ROM) establishment, i.e., the areas where numerical computations are highly demanding and in many cases impossible to perform. Care has to be taken here since physics-informed procedures only sometimes adhere to physical consistency. When complex NNs are constructed for an intensive application, it is not guaranteed that they will lead to generalization. Although NNs may produce valuable predictions, they remain a “black-box” model.

Another engineering area of increasing importance concerns the effort and cost of performing expensive numerical simulations and laboratory experiments. This cost could be reduced by using ML models that use coarse data to produce refined predictions. This is particularly important when data are limited or exhibit noise or both. The above factors significantly influence the accuracy of interpolation methods and reduced-order models, including ML.

Poulinakis et al. [194] presented a study on these issues and concluded that ML and DL can be significantly more promising than interpolation under certain data sparsity and noise conditions. Otherwise, ML and DL can be more inaccurate than simple interpolation. For example, DL is significantly more robust to noise than cubic Splines. Therefore, DL can lead to a ’true’ function hidden under the noise, thus making it a valuable tool in engineering applications. Furthermore, we know that increasing the data will increase the accuracy of DL to a certain level. What this level is will depend on the application problem. Therefore, future emphasis should be placed on establishing the limitations and best practices for DL methods in different applications. The explainability of DLs remains problematic, and research should be carried out in this direction.

6. Conclusions

As AI methods and algorithms evolve, simulation and experimental data will play increasingly important roles in practical designs, covering demanding applications in science and technology. Fluid dynamics research should take advantage of AI development but carefully explore the possibilities without hyping the potential benefits. ML and DL methods should adapt to specific problems, and their accuracy and limitations should be investigated. The key is that AI and ML approaches should fly above simple prediction tasks and dive deep into interpretability and causation. Their potential impact will be high only if the output abides by physical laws.

Although many of the issues discussed here apply generally to ML and DL methods, there are some specific areas where these methods can benefit fluid dynamics and areas where caution should be exercised:

1.

Exploring how to accelerate numerical simulations by introducing ML.

2.

Investigating the accuracy limitations of ML methods in experiments and numerical simulations.

3.

Exploring ML methods when using big fluid dynamics data and when data are scarce.

4.

Developing a better theoretical understanding of ML methods that will allow better explainability of the results.

5.

Avoiding AI hyping such as “new equations will be discovered through AI”, as first principles will drive fluid dynamics (and other physics) research.

Research in AI evolves fast, and fluid dynamics will benefit from it. Still, its applicability should follow rigorous verification and testing and allow researchers to publish not only their successes in using AI but also their failed attempts.