Data-Driven Modeling of Geometry-Adaptive Steady Heat Convection Based on Convolutional Neural Networks

Abstract

Heat convection is one of the main mechanisms of heat transfer, and it involves both heat conduction and heat transportation by fluid flow; as a result, it usually requires numerical simulation for solving heat convection problems. Although the derivation of governing equations is not difficult, the solution process can be complicated and usually requires numerical discretization and iteration of differential equations. In this paper, based on neural networks, we developed a data-driven model for an extremely fast prediction of steady-state heat convection of a hot object with an arbitrary complex geometry in a two-dimensional space. According to the governing equations, the steady-state heat convection is dominated by convection and thermal diffusion terms; thus the distribution of the physical fields would exhibit stronger correlations between adjacent points. Therefore, the proposed neural network model uses convolutional neural network (CNN) layers as the encoder and deconvolutional neural network (DCNN) layers as the decoder. Compared with a fully connected (FC) network model, the CNN-based model is good for capturing and reconstructing the spatial relationships of low-rank feature spaces, such as edge intersections, parallelism, and symmetry. Furthermore, we applied the signed distance function (SDF) as the network input for representing the problem geometry, which contains more information compared with a binary image. For displaying the strong learning and generalization ability of the proposed network model, the training dataset only contains hot objects with simple geometries: triangles, quadrilaterals, pentagons, hexagons, and dodecagons, while the testing cases use arbitrary and complex geometries. According to the study, the trained network model can accurately predict the velocity and temperature field of the problems with complex geometries, which has never been seen by the network model during the model training; and the prediction speed is two orders faster than the CFD. The ability of accurate and extremely fast prediction of the network model suggests the potential of applying reduced-order network models to the applications of real-time control and fast optimization in the future.

1. Introduction

There are many applications of forced convection in daily life and in the industry. Attributed to the fast development of computational technique and computational ability, numerical simulations have been one of the main methods for solving complex convective heat transfer problems. It is well known that numerically solving differential equations of heat convection is time-consuming, which may become prohibitive for optimization problems involving a large number of design parameters. While during the early stage of design/optimization it usually does not require high-fidelity simulation results, what is favored is that the numerical prediction should be fast for quick iteration. A popular strategy is to use the framework of reduced order modeling (ROM) to enable a fast fluid flow and heat transfer predictions [1,2,3].

In general, a ROM attempts to convert and represent the high-dimension dynamic system into the linear subspace domain by choosing an appropriate transforming coordinate system [4,5]. An important feature of this transformed space is that it allows for decoupling of spatial and temporal modes effectively [6]. The most common choice for the construction of these spatial transformation bases is the method of proper orthogonal decomposition (POD). The POD method has provided powerful tools for building ROM for fluid flow and heat transfer problems [7,8,9]. Indeed, the low computational expense and small memory requirement of this method make it particularly suitable for optimization and aftertreatment analysis. However, the POD method is also limited because it is a linear combination of eigenvectors and eigenvalues and does not explicitly account for the nonlinear interactions of the highly nonlinear dynamic system. That is to say that the POD method is significantly effective for quasi-steady-state, time-periodic problems, but might be challenging for highly nonstationary and nonlinear problems [10].

In recent years, machine learning (ML)- and deep learning (DL)-based techniques/methods have been showing power on building a surrogate/reduced-order model for highly nonlinear dynamic problems, including applications of aerodynamics, heat transfer, and fluid flow. The big difference between DL-enabled and POD-based ROM is that DL can be used to directly set up a nonlinear relationship between abundance of inputs and outputs of a target system. This process of fitting to available data, known as model training, generates a low-dimensional subspace, which records the mean behavior of the underlying phenomena of flows; and this training process allows for the representation of complex relationships/features, which cannot be expressed explicitly in a functional form [11]. In practice, the DL-enabled ROM has been demonstrated to be able to accurately capture the spatial and temporal nonlinear features of fluid flow. Wang et al. [12] presented a model identification of reduced-order fluid dynamic systems by using deep learning, and proved that the framework is capable of capturing the features of complex fluid dynamics with less computational cost. Fukami et al. [13] used machine learning to perform super-resolution analysis of grossly under-resolved turbulent flow data and reconstructed the high-resolution flow field; that is, their model successfully builds a nonlinear mapping between a low-resolution and high-resolution turbulent field. Alternatively, an approach known as DL-based closure modeling has been used to improve the performance of the traditional ROM methods [14,15,16].

The deep learning (DL)-based reduced-order modeling method has started to catch the attention of the thermal engineering community, although there are only a few publications available. San and Maulik [6] applied the machine learning method for developing a data-driven closure modeling for stabilizing projection-based reduced-order models for the Boussinesq equations, that is, improving the performance of the traditional ROM by DL. Gao et al. [17] proposed a physics-constrained CNN architecture to learn solutions of parametric PDEs on irregular domains, where the PDEs include a heat transfer equation and Navier–Stokes equations. Their results demonstrate the effectiveness of the proposed DL approach in predicting the temperature field and velocity field.

In the current study, we apply CNN to build a reduced-order, geometry-adaptive, and steady-state heat convection model, since CNN has demonstrated strong feasibility in geometry representation and per-pixel prediction in two-dimensional fields [18,19,20]. In Section 2, we introduce the network architecture of the reduced-order model, the preparation of the datasets, and the training algorithm. In Section 3 and Section 4, we present and discuss the outstanding performance of the network model.

2. Methods

In this paper, we propose that a CNN-based ROM directly builds a mapping between physical fields and signed distance function (SDF), which represents the problem geometry. The networks and its training are implemented using Tensorflow. The “ground truth” (training dataset) is generated through CFD simulation using OpenFOAM.

2.1. Design of the CNNs-ROM Framework

2.1.1. Framework Workflow

Figure 1 depicts the schematic of the proposed reduced-order model, which shows the learning/training strategy and the method of a new prediction using the network model. For the model training, we use the signed distance function (SDF), which carries physical information of the geometry as the network input and the results (temperature/velocity) of numerical simulations as the learning target (output/label) of the CNNs. After training with the proper amount of dataset, the trained CNNs can predict the temperature/velocity fields based on the new SDF values.

2.1.2. Signed Distance Function

The signed distance function (SDF) is proposed as the geometry representation at the network input. Using the SDF, the geometry of the object is represented as a level-set function defined over the simulated space in which the object is embedded [21,22]. Compared with the boundaries and geometric parameters representation, the SDF provides more physical and mathematical information for CNNs. In the SDF representation method, the zero-level set is created to represent the location of the boundary of the object. That is, the boundary of the object, $\sum$, is represented as the zero-level set of a continuous level-set function “$\varphi$” defined in a domain $\Omega \subset R^2$, and $R^2$ stands for a 2D bounded domain; that is,

$$\sum =\left\{\mathit{X}\in R^2:\varphi \left(\mathit{X}\right)=0\right\}$$

(1)

The level-set function $\varphi \left(\mathit{X}\right)$ is defined everywhere in the domain $\Omega$, and $\varphi \left(\mathit{X}\right)=0$ if and only if $\mathit{X}\left(x_i,y_i\right)$ is on the object boundary, such as the edge of the polygon shown in Figure 2a. The SDF associated with the level-set function $\varphi \left(\mathit{X}\right)$ is defined as

$$D\left(\mathit{X}\right)=\underset{{\mathit{X}}_2\in \sum }{\min }\left|\mathit{X}\left(x_i,y_i\right)-X_2\left(x^{\prime },y^{\prime }\right)\right|sign\left(\varphi \left(\mathit{X}\right)\right)$$

(2)

$D\left(\mathit{X}\right)$ is an oriented distance function, X₂ is a point on the boundary of the object, and the sign function “$sign$” is defined as:

$$sign\left(\varphi \left(\mathit{X}\right)\right)=\left\{\begin{array}{c}1 if \varphi \left(\mathit{X}\right)>0\\ 0 if \varphi \left(\mathit{X}\right)=0\\ -1 if \varphi \left(\mathit{X}\right)<0\end{array}\right.$$

(3)

By using the above level-set function, one can represent an arbitrary shape in the fixed design domain, $\Omega$ [23]. Figure 2b gives the distribution of the SDF representation of a training dataset.

2.1.3. Architecture of CNNs

The proposed network architecture consists of convolution layers and transposed convolution (also called deconvolution) layers. To ensure the high nonlinearity of the proposed reduced-order model, we use a multilayer deep structure for our neural network [24]. In the model, multiple convolutional layers are applied to extract a highly encoded [25] geometry representation from the SDF (input matrix of the network), and the encoded geometry representation is decoded by multiple deconvolutional layers [26] to predict the physical fields. The choice of the network structure is highly dependent on the problem, the data quality, and even the dataset size [27]. For example, it would be necessary to build deep convolutional neural networks in our problem due to the big size and complexity of the training dataset. Figure 3 shows the structure and components of the CNN model, and

Table 1. Parameters of the neural network model of each layer, where $w\times h$ denotes the size of the convolution kernel (F), $n_F$ denotes the number of F, and $N_l\times N_l\times n_F$ is the size of the lth layer after operation.

Name of Layer	Executing Operation $\mathit{w}\times \mathit{h}\times {\mathit{n}}_{\mathit{F}}/\mathit{S}\mathit{t}\mathit{r}\mathit{i}\mathit{d}\mathit{e}$	Shape ${\mathit{N}}_{\mathit{l}}\times {\mathit{N}}_{\mathit{l}}\times {\mathit{n}}_{\mathit{F}}$
Input of model	--	250 × 250 × 1
Conv1	4 × 4 × 32/3	83 × 83 × 32
Conv2	5 × 5 × 64/2	40 × 40 × 64
Conv3	6 × 6 × 128/2	18 × 18 × 128
Conv4	6 × 6 × 256/2	7 × 7 × 256
Conv5	3 × 3 × 512/2	3 × 3 × 512
Conv6	2 × 2 × 1024/1	2 × 2 × 1024
Conv7	1 × 1 × 1024/1	2 × 2 × 1024
Deonv1	2 × 2 × 1024/1	3 × 3 × 1024
Deonv2	3 × 3 × 512/2	7 × 7 × 512
Deonv3	6 × 6 × 256/2	18 × 18 × 256
Deconv4	6 × 6 × 128/2	40 × 40 × 128
Deconv5	5 × 5 × 32/2	83 × 83 × 32
Output	4 × 4 × 1/3	250 × 250

displays the parameters of each layer of the network model. We will describe each part of the network in the following subsection.

2.1.4. Encoding Part

The encoding part aims to reduce the dimension of the input data and extract the potential spatial features between neighboring points of the SDF matrix and convolution layers. According to the governing equations, the steady-state heat convection is dominated by convection and thermal diffusion terms; thus the distribution of the physical fields would exhibit stronger correlations between adjacent points. As the CNNs are very powerful for handing two dimensional data with a locality structure [28,29], in the current approach, the data-driven model is developed using the CNNs. Figure 4 shows the schematic of the convolution (down sample) operation by the network.

In Figure 4, the $2\times 2\times n_F$ light blue matrix denotes the convolutional kernel, the white matrix is the feature map matrix (also called the next layer’s input matrix), and the blue translucent matrix represents the output of the convolution operation. Besides the size of the kernel, $F$($w\times h$), the convolutional output is also influenced by the kernel stride, $S$; the size of the channels, $n_F$; and the padding size, $P$. The padding operation adds zeros around the border of the input matrix, and the kernel stride controls the sliding step size of the kernel. The size of the output matrix after the convolutional operation can be calculated as:

$$N_{l+1}=\frac{N_l-F+2P}{S}+1$$

(4)

where $N_l$ is the size of feature map at the $l^{th}$ layer (l = 0 represents the layer of the input matrix of the network model). In the current work, zero padding size (P = 0) is used. From the above equation, it can be seen that after several convolutional operations, the size of the original input can be reduced significantly, and the features of the original input are also highly encoded; thus the memory space required by the CNN training was reduced.

To increase the nonlinear capability of CNNs, each convolution or deconvolution layer involves the nonlinear activation operations. Mathematically, the nonlinear activation operation can be expressed as:

$${\mathit{a}}_l=\sigma \left({\mathit{W}}_l\ast {\mathit{a}}_{l-1}+{\mathit{b}}_l\right)$$

(5)

where ${\mathit{W}}_l$ is the weights or convolutional kernel of the current layer; $\sigma$ denotes the nonlinear activation function; ${\mathit{a}}_l$ and ${\mathit{a}}_{l-1}$ are the input and the output of the layer, respectively; $\ast$ is the convolutional operator; and ${\mathit{b}}_l$ is the bias term. The introduction of $\sigma$ is crucial for neural networks to possess nonlinearity. In this paper, we apply the rectified linear unit (RELU) activation function:

$$\sigma \left(x\right)=\left\{\begin{array}{c}0, x<0\\ x, x\ge 0\end{array}\right.$$

(6)

The advantages of RELU are that its computation cost is cheap as the function has no complicated math, and it converges fast; thus the model takes less time to train or run. It should be noticed that, because the final prediction of the network model needs to be a continuous regression, there is no activation function between the output layer and the “Deconv5” layer.

2.1.5. Decoding Part

The decoding part is designed to analyze the highly encoded features and decode them to be a recognizable velocity and temperature fields. The decoding part is composed by multiple deconvolutional layers, and these up-sample operations (deconvolution) unravel the high-level features encoded by the encoding part. Figure 5 shows the schematic of the deconvolution operation of one channel.

The deconvolution operation needs appropriate basis functions to learn invariant subspaces of the feature map, and then compensates the minutia information to predict the physical fields by multilayer iterative optimization. For instance, as shown in Figure 5, for adjusting the width and height of the output feature map (pool blue block), random values (optimized by the algorithm) will be padded at the white block around the block (dark blue block) of the input feature map. Therefore, highly encoded information is recovered by applying the deconvolutional decoder. Meanwhile, the CNNs learn/train their ability to predict physical fields based on the SDF through this process.

2.2. Model Training

The model training is an iterative process that continuously minimizes the loss between the outputs of the network predicted ($\widehat{\psi }$) and the ground truth ($\mathit{\psi }$) for obtaining the optimal model parameters, $\theta$. Further, we need to condition our ROM prediction based on the SDF, as the flow field within the geometry should be ignored and must not affect the loss function. The SDF-based conditioning operation is defined as follows:

$$J=\frac{1}{N}{\displaystyle \sum }_{n=1}^N{\left(\left({\psi }_n\left(x,y\right)-{\widehat{\psi }}_n\left(x,y\right)\right)·\delta \left(x,y\right)\right)}^2+\lambda {\left|\left|W\right|\right|}_2$$

(7)

$$\delta \left(x,y\right)=\left\{\begin{array}{c}1, \varphi \left(x,y\right)\ge 0\\ 0, \varphi \left(x,y\right)<0\end{array}\right.$$

(8)

where (x, y) is the index of the space point and $n$ is the index of the case number, $N$ is the size of the (batch) dataset, $\psi$ indicates the result by the numerical simulation, $\widehat{\psi }$ is the result predicted by the network model, $W$ denotes all the weight of the network layers, $\lambda$ is the regularization coefficient, and $\lambda {\left|\left|W\right|\right|}_2$ is the L2 regularization term for preventing model overfitting. Such a conditioning operation eases the training process and improves the prediction accuracy [30].

During the training process, the SDF (shape: 250 × 250) and velocity/temperature field (shape: 250 × 250) are the input and the output of the CNN model. Specifically, the velocity/temperature field is regarded as the ground truth (label) of the network model, while the SDF is treated as the input matrix of the CNNs. After the model training, the network extracts the features from the input matrix and builds a proper mapping between the SDF and the physical fields. Obviously, the training process of the CNNs is to reduce the deviation of Equation (7).

To train the model efficiently, a minibatch-based learning strategy is used. Small batch training has been shown to provide improved generalization performance and reduces the memory cost significantly [31]. The loss function is computed using a randomly selected subset (i.e., the batch size), and the whole optimization process forms an iteration way. Moreover, the adaptive moment estimation (Adam) method is implemented as the optimization algorithm for the model training; the Adam method has been proved to be robust and well suited for large datasets and/or optimization problems of high-dimensional parameter spaces [32]. The hyperparameters of the optimization algorithm for the network training are shown in

Table 2. Hyperparameters of the optimization algorithm. Regularization coefficient ($\lambda$) and learning rate ($\beta$).

Hyperparameter	Value
Batch size	64
$\lambda$	0.0001
$\beta$	0.00008

.

2.3. Preparation of Dataset

The simulation domain of the studied heat convection problem is shown in Figure 2a, where the geometry of the hot object is changeable. The flow direction is flow left to right. The up and bottom boundaries are a cold wall, whose temperature is same to that of the inlet flow. Such a configuration is chosen as it is a typical forced convection problem and can be seen in many engineering applications [33,34]. The training and test datasets consist of 5 types of 2D simple hot objects in a square simulated domain (see Figure 2 as an example): triangles, quadrilaterals, pentagons, hexagons, and dodecagons. With the simulated domain kept unchanged, each set of the hot objects is randomly different in size, shape, orientation, and location. In general, it is more often that the object locates near the center of the studied domain; therefore, the location of the object is generated following a normal distribution in the function of x- and y- coordinates and using the center of the domain as the mean. Each 2D simple object is preprocessed into a 250 × 250 pixel matrix based on the SDF. The training and testing datasets contain a total of 10,000 samples (2000 random samples for each type of object), where the testing dataset contains 1000 samples (random separated from the whole training and testing dataset). The simulated domain is discretized using an unstructured mesh tool, SnappyHexMesh [35,36], and the numerical simulation is performed using OpenFOAM (openfoam.org), where the SIMPLE algorithm is used. Regarding the validation dataset, which is dedicated to validating and indicating the generalization ability of our CNN model, more complex geometries of the hot objects are chosen (see Figure 6). Those samples have never been seen by the network model during the training process. All the processes of the mesh generation and numerical simulation for generating the validation dataset are similar to that for the training dataset.

The physics or governing equations of the steady-state heat convection can be given as:

$$\nabla ·\mathit{v}=0$$

(9)

$$\left(\mathit{v}·\nabla \right)\mathit{v}=-\nabla p+\nu \nabla ·\nabla \left(\mathit{v}\right)$$

(10)

$$\left(\mathit{v}·\nabla \right)T=\alpha \nabla ·\nabla \left(T\right)$$

(11)

where $\mathit{v}$ is the velocity vector, $p$ is the pressure, $T$ is the temperature, $\nabla ·$ is the divergence operator, and $\nabla$ is the gradient operator; $\nu$ and $\alpha$ are the momentum and thermal diffusivity. It should be noticed that the viscous dissipation has been ignored. Based on the following parameters:

$${\mathit{v}}^{\ast }=\frac{\mathit{v}}{v_0}; T^{\ast }=\frac{T-T_0}{T_1-T_0}; P=\frac{p}{v_0{}^2}; Re=\frac{v_0{L}_r}{\nu }; Le=\frac{\alpha }{v_0{L}_r} ;{\nabla }^{\ast }·=L_r\nabla ·;{\nabla }^{\ast }=L_r\nabla$$

The governing equations can be normalized as follows:

$$\nabla ·\mathit{v}=0$$

(12)

$$\left(\mathit{v}·\nabla \right)\mathit{v}=-\nabla P+\frac{1}{Re}\nabla \times \nabla \left(\mathit{v}\right)$$

(13)

$$\left(\mathit{v}·\nabla \right)T=Le\nabla \times \nabla \left(T\right)$$

(14)

where $L_r$ is the reference length, $T_0$ is the temperature of the cold wall and $T_1$ is the temperature of the hot object, $v_0$ is the inlet mean velocity, $Re$ is the Reynolds number, and $Le$ is the Lewis number, which is the ratio of thermal diffusivity to convective mass transport. Notice that the asterisks have been ignored for simplicity. In the following study, we keep the Reynolds number and the Lewis number at 10 and 15, respectively. The differential equation is solved based on the SIMPLE algorithm. The boundary conditions used here are listed in

Table 3. Boundary conditions of the numerical model.

Boundary Type	Temperature	Velocity	Pressure
Wall	Fixed value	Fixed value	Zero gradient
Inlet	Fixed value	Fixed value	Zero gradient
Outlet	Zero gradient	Zero gradient	Fixed value
Object	Fixed value	Fixed value	Zero gradient

. Moreover, the grid independency study is carried out to determine the appropriate meshes to use. We only show the results for the case of a complex geometry (Car), as shown in

Table 4. Average temperature at the outlet (O_t) with different meshes.

Mesh	Grid Number	Outlet Temp (Ot)
Grid-one	8849	311.4296
Grid-two	18,786	311.3324
Grid-three	25,068	311.2711
Grid-four	31,514	311.2264

. To speed up the data generation, grid-one (approximate value) is chosen for further studies.

3. Results

We use the average, mean, and maximum relative error to measure the prediction accuracy of the network model. For a studied case, the relative error for velocity and temperature is calculated as:

$$E\left(x,y\right)=\frac{\left|v\left(x,y\right)-\widehat{v}\left(x,y\right)\right|\delta \left(x,y\right)}{v_0}; E\left(x,y\right)=\frac{\left|T\left(x,y\right)-\widehat{T}\left(x,y\right)\right|\delta \left(x,y\right)}{\left|T\left(x,y\right)\right|}$$

(15)

The maximum point relative error is defined as:

$$E_{max}=\max \left(E\left(x,y\right)\right)$$

(16)

The mean relative error is defined as:

$$E_{mean}=\frac{{{\displaystyle \sum }}_x{{\displaystyle \sum }}_{y}E\left(x,y\right)}{{{\displaystyle \sum }}_x{{\displaystyle \sum }}_y\delta \left(x,y\right)}$$

(17)

For evaluating the performance of the network model over several different cases, we also define the average relative error as:

$$E_{avg}=\frac{1}{N}{\displaystyle \sum }_{n=1}^N{\left(E_{mean}\right)}_n$$

(18)

where $n$ is the index of the studied cases. Furthermore, from Equation (15), it should be noticed that when calculating the error, we only consider the domain outside the hot object.

3.1. Feasibility of the CNN-Based Model

In this section, we verify the feasibility of the proposed framework on three aspects: (1) the outstanding performance of the CNN model in predicting the velocity and temperature fields, (2) the advantage of the SDF representation, and (3) the optimization process of the CNN model structure.

3.1.1. Performance of the Temperature Field Prediction

We first evaluate the prediction performance of the temperature field of the network model. After proper model training, the prediction accuracy of the testing dataset is higher than 98.75%. Figure 7 visualizes two typical temperature fields of the test dataset predicted by the network model and numerical simulation (OpenFOAM) and the corresponding relative error distribution (see Figure 7). We can see that the network model has given a satisfied accurate prediction on the temperature field. Furthermore, the figure of the relative error distribution indicates that the large error mainly locates in the region with a large temperature gradient.

Figure 8 shows the temperature fields of the validation cases by the network model and the numerical simulation. The displayed validation cases include: Car, Car2, Airplane, and Locomotive. For the cases shown in Figure 8, the maximum relative error ($E_{max}$) is lower than 5.2%; the large error happens more frequently near the hot objects, especially at the boundary with large curvature. Figure 9 shows the temperature profile distribution along the y-direction at a different x-position by the network model (symbols) and numerical simulation (lines). The good coincident of the symbols and lines quantitatively affirms the accuracy of the network model. From the results, a comparatively poor prediction performance appears in the wake (x = 0.65, blue line), but the error is still within the acceptable range.

Overall, considering that the network model has only “seen” triangles, quadrilaterals, pentagons, hexagons, and dodecagons during the training process, such a high-prediction accuracy on the validation cases with much complex geometries indicates that the network model has an outstanding ability of generalization and extensionality.

Furthermore, we study the computational cost of the field prediction by the network model and CFD.

Table 5. Time consumption for predicting the steady-state temperature field by the network model (GPU, 2080ti) and the numerical simulation (CPU, Ryzen 7 3700X, OpenFOAM).

Geometry Object	CNNs (s)	OpenFOAM (s)	Grid Quantity	Speedup
Car	0.2354	38	10,144	161
Car2	0.2423	31	10,000	128
Airplane	0.2314	23	9032	99
Locomotive	0.2284	20	7758	88
Bionic Fish	0.2463	28	8252	114
Missile	0.2324	26	8528	112
Ship	0.2364	22	8956	93
Submarine	0.2394	26	9876	109

quantitatively compares the time consumption for predicting the steady-state temperature field by the network model and the numerical simulation. Overall, the network model can speed up the prediction for two orders; and as the geometry of the problem becomes more complex (that is, it requires a finer mesh for the CFD to converge the simulation), the prediction speedup by the network model becomes more outstanding. In addition, we compare the prediction time costs by the network model using different GPUs, 2080ti and 1660s (see

Table 6. Time consumption for predicting the steady-state temperature field by the network model on GPU 1660s and GPU 2080ti.

Geometry Object	1660s Time (s)	2080ti Time (s)
Car	0.3983	0.2354
Car2	0.3723	0.2423
Airplane	0.3523	0.2314
Locomotive	0.3793	0.2284
Bionic Fish	0.3194	0.2463
Missile	0.3164	0.2324
Ship	0.317	0.2364
Submarine	0.3184	0.2394

). From the table, we can see that the performances of these two equipment have no big difference.

3.1.2. Performance of the Velocity Field Prediction

The steady-state velocity fields of the validation cases predicted by the network model and numerical simulation are shown in Figure 10. From the results, it can be observed that the velocity field by the network model shows good agreement with the CFD simulation. The right column in Figure 10 shows the distributions of the relative error: we can see that the maximum error happens frequently on the edge of the objects, because the flow field there is usually accompanied by a large velocity gradient.

We further verify the ability of generalization and extensionality of the network model with validation datasets (see Figure 6). The velocity fields of the validation dataset predicted by the network model and numerical simulation (OpenFOAM) and the corresponding relative error distribution are shown in Figure 11. For the six cases shown in the figure, the maximum mean relative error ($E_{mean}$) is less than 6%, and the maximum point relative error is less than 10%. In Figure 12, we quantitatively compare the max ($E_{max}$) and mean relative error of the studied validation cases. Compared with the testing cases, since the complexity of the geometric objects increases, the prediction error of the network model becomes larger. Furthermore, as the predicted shape gradually deviates from the polygon (e.g., Ship, Submarine), the prediction error increases. Overall, the low mean error suggests that the network model is able to provide an accurate prediction on cases with arbitrary complex geometries.

3.1.3. Advantages of the SDF Representation

In previous subsections, we showed the outstanding performance of the proposed network model, where the SDF is used as the geometry representation and the neural network inputs, while according to the literature review, in most previous works a binary image is used as the input of the CNNs [37,38]. The binary image represents the geometry as follows: its element is 0 if and only if the position is on the object boundary or inside the object. To indicate the effectiveness of the SDF, we trained a model using a binary image as the geometric representation with the exact same network architecture. The average relative error ($E_{avg}$) by two different network models on the testing and validation datasets is summarized in

Table 7. Average relative error ($E_{avg}$) on the prediction of the testing and validation datasets by the network models using the SDF and binary image representation.

Datasets	Prediction Field	SDF	Binary
Testing	Velocity	2.79%	6.56%
	Temperature	0.83%	4.62%
Validation	Velocity	5.12%	18.27%
	Temperature	1.91%	10.82%

.

Table 7 indicates that the SDF representation is much more effective than the binary representation. Regarding the validation dataset, the error of using the SDF representation is significantly smaller than the model using the binary representation. This can be attributed to the reason that each value in the SDF matrix carries a certain level of global information, while for a binary image, only the values on the boundary of the object carry useful information. Therefore, the SDF representation achieves a much better performance than the binary representation. In the following investigation, only the SDF representation is used as the model input.

3.1.4. Influence of the CNN Model Structure

The outstanding performance of the prediction results is sufficient to illustrate the feasibility of the current network structure. To further gain insight into the performance and the characteristic of the network model, we study the optimization process of the network structure. To avoid showing the tedious trial and error, the results presentation will be simplified, and we will only study the performance of the network model with different numbers of the layers, which represents the depth of the network. Four types of network depth, L = {3, 4, 5, 6}, are studied, where each layer is accompanied by a symmetric decoding layer. All cases are trained in the same iterations.

Figure 13 shows that, as the number of the network layer increases, the validation loss decreases obviously, especially when the layer number changes from 3 to 4. Figure 14 shows the prediction of the temperature fields of each network model and proves the above observation. The studied geometry is a car placed on the lower left of the calculation domain, and the network with six layers has the smallest error. In addition, in terms of error distribution, the result of a six-layer network is more reasonable than the result of a three-layer model, which implies that a deeper neural network model has stronger learning capabilities than a shallow one. On the other hand, it should also be noticed that the parameters of the network, the training time, and the prediction time rise accordingly (see

Table 8. Parameters memory, training time, and prediction time costs for the network model with different numbers of layers.

Layer	Parameters Memory (MB)	Prediction Time (s)	Training Time (min)
3	7.23	0.1197	43.2
4	31.7	0.1497	51.1
5	58.7	0.1825	64.4
6	247	0.2643	86.3

). It can be found that when the number of the layers increases from five to six, there is no significant improvement in the validation loss, but the prediction time cost increases significantly. Hence, when optimizing the structure of the neural network, the prediction accuracy and the time cost need to be balanced according to the demand. In this paper, the main aim is to improve the performance of the network model; therefore, the current network structure has been designed to have six layers.

3.2. Influence of the Space Distribution of the Hot Object

In this part, first, we investigate the effect of the space distribution of the hot object on the performance of the network model. Although the network model has shown good prediction accuracy on the validation dataset, the predicted objects above are fixed at the center of the entire field. As mentioned in the section of data preparation, the location of the object is generated following a normal distribution in the function of x- and y-coordinates and using the center of the domain as the mean. Hence, in this section, we study the effect of the distance between the center of the hot object and the center of the studied domain. Figure 15 shows the prediction results by the network model and numerical simulation (OpenFOAM) and the corresponding relative error. The result shows that the maximum error is less than 6%, which illustrates that the network model is robust to the randomness of the space distribution of the hot object in the entire flow field.

Second, since all the training cases only contain a single hot object, here we also investigate the performance of the network model in predicting the problems with multiple hot objects. Compared with studying the spatial distribution of the hot object, the problem with multiple hot objects is providing a huger challenge to the network model. Figure 16 shows the temperature fields of the case with two circles predicted by the network model and numerical simulation and the corresponding relative error. It can be observed that the network model can still give a less accurate but still reasonable prediction. Using more training datasets to train the network model is an effective solution to further improve the prediction accuracy of the cases with multiple hot objects.

3.3. Influence of Incorporating Velocity into Input Matrices

For most thermal applications, people are more interested in the temperature than the velocity field. In this section, we study the effect of additionally incorporating the velocity field as a portion of the input matrices. The prepared cases have the same hyper-parameters and network architecture. Figure 17 shows the mean ($E_{mean}$) and maximum ($E_{max}$) relative prediction error on the temperature field by the network models with and without incorporating the velocity field as the portion of the input matrices. Figure 18 plots the relative prediction error of the two studied situations along the y-direction at different x-positions. The relative error ($E\left(x,y\right)$) is calculated by Equation (15). From Figure 17 and Figure 18, it can be observed that the overall prediction accuracy increases as the network model further incorporates velocity information, while the improvement is moderate. Therefore, for current problems, incorporating velocity information is not economic.

It can be concluded from the above results that additionally incorporating the velocity field as the portion of the input matrices can enhance the accuracy of the temperature prediction of the neural network. Here, we further investigate the impact of the type of the embedded velocity field on the performance of the neural network model. In the above portion, the input matrix of the SDF combined with the velocity magnitude is studied; then, we investigate another two types of input matrix: SDF-combined x-velocity, SDF-combined y-velocity. Figure 19 shows the training history of the model with different input matrices. Figure 20 shows the temperature profile distribution along the y-direction at the dimensionless x-position of 0.8, predicted by the network model with different input matrices and numerical simulations. It can be seen from Figure 19 and Figure 20 that the case of the SDF-combined velocity shows the fastest convergence speed and the highest accuracy, as it contains the most physical information.

4. Discussion

Reduced-order modelling is a data-driven method, and it provides a way of fast predicting physical fields without solving partial differential equations (PDEs) iteratively and numerically. An efficient reduced-order prediction model is useful for production design and optimization and real-time simulation/prediction in some control scenarios. The results discussed in the last section indicate that the proposed CNN-based reduced-order model achieves high accuracy and robustness, even though the geometry of the problem is much more complex than that of the training dataset. The strong ability of generalization and extensionality is one of the major advantages of the CNNs. Furthermore, compared with the CFD, the time consumption for the field prediction by the network model can be negligible.

Although the proposed model has shown outstanding extensionality, the prediction accuracy is still affected obviously as the physical conditions (geometric in the current paper) gradually deviate from the training dataset; and of course, this also applies to the traditional data-driven method, such as POD and DMD [39,40]. Certainly, for relieving this problem, a CNN-based ROM should be trained using as exhaustive a dataset as possible, while it is time-consuming to retrain the network model once we get new/additional data. One solution to this problem is to make full use of the trained reduced-order model for fast reconstruction/retraining of the model.

This is a preliminary attempt to develop the reduced-order model predicting the heat convection fields with arbitrary geometries based on the CNNs. As the beginning of the exploration, the studied problem is kept two-dimensional and laminar, and the performance of the network model is satiated and exciting, while the problems of engineering optimization and design in real applications are usually three-dimensional and turbulent, which are greatly challenging for the network model. Therefore, paying more attention to machine-learning-based reduced-order modeling and making it able to better serve the physics or engineering community is warranted.

5. Conclusions

In this paper, based on the deep convolutional neural networks (CNNs), we proposed a data-driven model for predicting the steady-state heat convection of a hot object with an arbitrary complex geometry in a two-dimensional space, and the signed distance function (SDF) is applied to represent the geometry of the problem. The training and validation datasets only contain the hot objects with simple geometries, including triangles, quadrilaterals, pentagons, hexagons, and dodecagons. According to the results, the velocity and temperature fields of the problems with complex geometries predicted by the network model agree well with the CFD simulation. As the location and the number of hot objects vary, the network model is still able to give a satisfied prediction. In addition, further incorporating velocity information into the input matrix will moderately improve the performance of the network model in the temperature field prediction. Although the training process takes about 42 min in the current work, the time cost of the CNNs to predict the heat convection fields is negligible compared with the CFD simulation; specifically, the prediction speed of the network model is two orders faster. Therefore, it is potentially possible that this approach can provide some enlightenment for fast optimization, real-time simulation, and control.