Novel Method to Improve the Convergence of Physics-Informed Neural Networks for Complex Thermal Simulations

Abstract

In the context of developing PINN methods for real-time digital twins in manufacturing processes, we propose a new approach that combines two complementary weighting strategies to significantly improve their convergence. The first method, called SD-PINN, balances the loss terms associated with the governing equations, boundary conditions, and initial conditions, ensuring that their contributions are dimensionally consistent and therefore comparable in magnitude. The second method, called SDFEET-PINN, rescales the terms of the governing equations during the early stages of training. This facilitates learning by temporarily modifying the equations to make terms comparable in amplitude, and then progressively restoring the original formulation, thereby preserving the influence of lower-magnitude terms that are often neglected in standard PINN approaches. We apply these methods to transient thermal problems, which are critical for predicting defects in Powder Bed Fusion (PBF). A range of 2D configurations with complex boundary conditions is used to test robustness, and a practical case study is carried out on heat transfer in a complex 3D geometry previously investigated both numerically and experimentally in PBF. Results show that the combined SD-PINN and SDFEET-PINN approach achieves higher predictive accuracy and stability compared to classical PINNs. Furthermore, we introduce an Adaptive Learning Rate strategy that reduces the step size after initial stabilization, further enhancing predictive performance and enabling efficient convergence across all test cases.

1. Introduction

The thermal history of a material is crucial in various thermomechanical processes, particularly in additive manufacturing. In this context, the material is subjected to multiple thermal loads, leading to complex microstructures that significantly affect the performance of the manufactured parts [1,2,3,4]. This study is part of the ongoing development of a digital twin of additive manufacturing, aimed at the real-time correction of manufacturing defects in a closed-loop system.

Figure 1 illustrates the principle of the digital twin of manufacturing processes. On the left, the physical entity (the manufacturing machine) is connected bidirectionally to the virtual entity (the digital twin) on the right. The physical entity is equipped with sensors that capture data, which are then transferred to the “brain” of the virtual entity, i.e., the expert model. This model can incorporate the expertise of metallurgists, tribologists, and data scientists, enabling real-time decision-making without human intervention [5,6]. These decisions may involve adjusting process parameters, such as laser power [7]. In addition to data from physical sensors, it is essential to rely on virtual sensor data provided by simulators, which serve as the digital model of the process [8]. Virtual data help reduce instrumentation costs and provide spatial and temporal information that physical sensors cannot capture. A concrete example of this is the real-time prediction of the microstructure state during the part fabrication process. For instance, no physical equipment currently exists that allows real-time measurement of the microstructure texture of alloys during the Laser Powder Bed Fusion (L-PBF) additive manufacturing process.

Given the challenges associated with digital twin technology, several scientific barriers can be identified:

• Real-time calculations: To enable real-time decision-making, predictions must be made in real time.
• Incorporating domain knowledge: The digital twin must integrate domain knowledge such as metallurgy, tribology, numerical analysis, and data science to make informed decisions [9].
• Online knowledge development: Some information is only available online and must be processed in real time, which creates the challenge of handling large volumes of data.
• Multi-scale models: The digital twin must cover all scales, from microstructure to machine level, to identify defects.

This project aims to simulate real-time additive manufacturing based on the multi-scale Finite Element approach proposed by Bresson et al. [10] for thermal modeling of L-PBF.

Numerical models often require significant computational time. For real-time calculations, reduction methods (e.g., linearizing equations or relaxing certain assumptions) or surrogate models (such as neural networks, Gaussian processes, polynomial chaos expansions, and more recently, random forests [11]) must be used. Among these tools, neural networks stand out as versatile optimizers, enabling real-time computations and the preservation and transfer of knowledge across various problems. Physics-Informed Neural Networks (PINNs) are particularly effective in extending predictions beyond the initial training ranges of the network [12]. Since the work of Raissi et al. [13], PINNs have gained widespread attention, with numerous studies and dedicated libraries emerging. Despite their advantages, PINNs face significant convergence issues, particularly related to minimizing the loss function, a scalar computed from physical equations, boundary conditions, initial conditions, and sometimes experimental data. As a result, these issues are often addressed on a case-by-case basis.

The balance between physical equations, initial conditions, and boundary conditions has been at the heart of many studies in the literature. The weak form of the physical equations has been used in PINNs by Xu et al. [14] and Rong et al. [15] to reduce the order of the physical equations in relation to the boundary conditions. Other studies have employed arbitrary or adaptive weighting, dynamically adjusting the importance of different terms in the loss function during training. The latter approach has been applied in various studies [16,17,18,19], including the work of Wang et al. [20], which proposes adaptive weighting for solving Navier–Stokes equations. In large deformation problems, Gao et al. [21] also used adaptive weighting to improve model performance. Zhou et al. [16] introduced an auto-adaptive method for large deformation analysis, while Hou et al. [17] discusses adaptive movement of collocation points and adaptive weighting of the loss. Hooshyar and Elahi [22] took a similar approach by sequencing initial conditions in PINNs, progressively adjusting the weights for more accurate results in complex physical scenarios.

Another related approach is Curriculum Learning, where the model is gradually exposed to tasks of increasing complexity [22,23]. For example, Guo et al. [24] used Curriculum Transfer Learning to simulate physical behaviors. The method proposed by Münzer and Bard [25] for progressive collocation point distribution in PINNs is also a related concept, allowing the model to adapt gradually to problem complexity.

The challenges of PINNs are further amplified by the complexities of additive manufacturing [26]. The process itself is challenging to model using numerical approaches, and presents specific challenges for PINNs due to the dynamic changes in the part’s geometry during production. The significant variability in part geometries in Laser Powder Bed Fusion (L-PBF) almost necessitates the development of an individual model for each part.

Only a few studies in the literature have focused on applying PINNs to changing geometries [26,27,28,29]. Recently, Peng and Panesar [26] used PINNs to simulate the layer-by-layer thermal process of the Directed Energy Deposition (DED) additive manufacturing process. The model is 2D with a rectangular geometry and a layer-by-layer sequential approach, where the initial conditions are predicted by a previously trained network. Although the author proposes an extension to 3D, the phenomenon remains almost entirely 2D.

To address these challenges, it is essential to develop advanced PINN models capable of converging effectively despite the complexities of L-PBF additive manufacturing. The existing literature reveals a lack of universally applicable methods. Moreover, most studies simply present results that work without explaining why they work or why others do not. Often, the models are not detailed enough to allow for a comparative study of different approaches.

In this study, we introduce two novel methods for Physics-Informed Neural Networks (PINNs), SD-PINN and SDFEET-PINN, designed to enhance convergence. A new Adaptive Learning Rate strategy is also proposed. Conventional PINNs often face major challenges, such as unbalanced loss terms of different magnitudes, difficulties in handling mixed boundary conditions (Dirichlet, Neumann, and Newton), and poor convergence on complex 3D geometries. The proposed SD-PINN and SDFEET-PINN methods directly address these issues by automatically balancing the different physical loss components and adapting the learning dynamics during training. These improvements lead to faster and more stable convergence and enable accurate predictions in realistic, industrially relevant configurations. The approaches can be used with any method from the literature dealing with the loss function balancing problem. They simply replace classical normalization, which is, in fact, used by all PINN methods. The effectiveness of these approaches is evaluated on four representative problems, including a complex 3D case involving the hydraulic joint studied by Bresson et al. [10] in the context of multi-scale thermal simulation in additive manufacturing. Although the heat conduction equation is relatively simple in its physical formulation, it remains particularly challenging for PINNs when involving mixed boundary conditions (Dirichlet, Neumann, and Newton) and realistic 3D geometries. Such configurations often lead to unbalanced loss terms and poor convergence in standard PINN frameworks. Therefore, this study focuses on improving convergence robustness in these practical but numerically demanding cases. The remainder of the document is organized as follows. Section 2 introduces the proposed methods and their numerical implementation. Section 3 presents the four benchmark problems used for evaluation, along with their numerical formulations. Section 4 discusses the numerical results obtained with the proposed PINN-based approaches, highlighting their robustness and accuracy.

2. Modeling and Methods

2.1. Classical PINN Method

The model’s geometry and the definition of the inlet, outlet, and wall boundaries are shown in Figure 2. The dimensions of the plate are $L_0=9 \mathrm{m}\mathrm{m}$ in length and $l=6 \mathrm{m}\mathrm{m}$ in width. The Partial Differential Equation (PDE) for transient heat diffusion is formulated as follows:

$$\left\{\begin{array}{cc}\rho c{\displaystyle \frac{\partial T}{\partial t}}=\lambda \left({\displaystyle \frac{{\partial }^{2}T}{\partial x^2}}+{\displaystyle \frac{{\partial }^{2}T}{\partial y^2}}\right)\hfill & (x,y)\in \mathrm{\Omega },t\in [0,1.2]\hfill \\ T(x,y,0)=f(x,y)\hfill & (x,y)\in \mathrm{\Omega }\hfill \\ \mathcal{B}T=g(x,y,t)\hfill & (x,y)\in \partial \mathrm{\Omega },t\in [0,1.2]\hfill \end{array}\right.$$

(1)

where the density $\rho$, specific heat c, and thermal conductivity $\lambda$ are $4210 \mathrm{k}\mathrm{g}/{\mathrm{m}}^3$, $770 \mathrm{J}/\mathrm{k}\mathrm{g}/\mathrm{K}$, and $26 \mathrm{W}/\mathrm{m}/\mathrm{K}$, respectively. $\mathcal{B}$ is a boundary condition operator. For Dirichlet conditions, $\mathcal{B}T=T$ on the boundary, while for Neumann conditions, $\mathcal{B}T={\displaystyle \frac{\partial T}{\partial n}}$ on the boundary. The function $f(x,y)$ represents the prescribed initial temperature, while $g(x,y)$ can represent either a prescribed temperature or flux, depending on the operator $\mathcal{B}$.

The architecture of a standard PINN is shown in Figure 3, where the loss function is composed of losses related to the diffusion equation, ${\mathcal{L}}_{\mathrm{pde}}$, the initial conditions, ${\mathcal{L}}_{\mathrm{ic}}$ and the boundary conditions, ${\mathcal{L}}_{\mathrm{in}}$, ${\mathcal{L}}_{\mathrm{out}}$ and ${\mathcal{L}}_{\mathrm{wall}}$. The network inputs should also include boundary conditions and other parameters (such as material properties) to enable training with multiple configurations, making it possible to predict any configuration in real-time, even unseen ones [30]. In this work, however, we will restrict the inputs to positions x, y, and time t, as the focus is on validating the new methods rather than real-time computation. For the classical PINN method, Equation (1) is also nondimensionalized, just like the inputs and outputs of the neural networks, in order to maintain the same order of magnitude between the loss functions ${\mathcal{L}}_{\mathrm{pde}}$, ${\mathcal{L}}_{\mathrm{ic}}$, ${\mathcal{L}}_{\mathrm{in}}$, ${\mathcal{L}}_{\mathrm{out}}$ and ${\mathcal{L}}_{\mathrm{wall}}$. It should be noted that the neural network architecture used in SD-PINN is identical to that of the standard PINN. The improvement comes solely from the scale-driven normalization of the loss terms, which aligns their physical dimensions and magnitudes, thereby enhancing the stability and convergence of the training process.

2.2. SD-PINN Method

The Same Dimensions PINN (SD-PINN) method is our first approach, based on the non-dimensionalization of the neural network’s input and output variables, as is commonly adopted in the standard PINN method, with the particularity of aligning the physical dimensions of the loss functions for the physical equations, boundary conditions, and initial conditions. Therefore, in the SD-PINN method, we use dimensional equations instead of the non-dimensional ones applied in the standard PINN approach. The preservation of the order of magnitude between the loss functions is achieved by weighting the equations with appropriate coefficients. This approach ensures that the loss functions are of the same physical dimensions, such as power or energy. The selection of these coefficients, which depends on the physical problem and boundary conditions, will be detailed in the various problems discussed in this document.

It is important to note that the proposed weighting ensures that all residuals—those of the governing equation, boundary, and initial conditions—are expressed in the same physical dimension (power). The scaling coefficients are defined so that the residual magnitudes are of the same order, approximately 1% of their characteristic physical terms. This normalization aligns the loss terms, prevents one from dominating the others, and thereby enhances training stability and convergence.

2.3. SDFEET-PINN Method

While the SD-PINN method weights the loss functions, the Same Dimensions From Equal Equation Terms PINN (SDFEET-PINN) method instead weights the terms of the heat equation, as well as those of the initial and boundary conditions, using identical coefficients that gradually evolve toward their true values (corresponding to the coefficient values used in SD-PINN) over the epochs. The distinction between the two methods lies in the use of the From Equal Equation Terms (FEET) strategy in SDFEET-PINN.

In this work, the coefficients evolve linearly toward their true value by the midpoint of the epochs. Consequently, as long as the true coefficient values have not been reached, the FEET strategy temporarily alters the physical problem. However, it helps prevent a term that is initially too small compared to the others from being neglected throughout the training process, particularly in equations with more than two terms.

3. Benchmark Problems for Evaluation

The geometric model illustrated in Figure 2 corresponds to a reference two-dimensional benchmark designed in this study to evaluate the performance of the proposed PINN formulations. It represents a transient heat conduction problem with combined Dirichlet and convection boundary conditions, inspired by standard academic configurations used for validation in thermal analysis. To assess the two proposed methods, various configurations and problem types will be tested. The first group focuses on models with Dirichlet boundary conditions. The second group includes both Dirichlet and Newton boundary conditions, accounting for convective heat exchange with the external environment. The third group extends the second by incorporating an advection phenomenon. Finally, the last group applies the methods to a 3D additively manufactured part that has been extensively studied in experimental and numerical works conducted by Bresson et al. [10,31] and Benoist et al. [32].

To test the ability of our methods to perform under the most challenging conditions, the Adam optimization algorithm is used throughout this study instead of the commonly recommended L-BFGS algorithm for PINNs [33,34]. A simple neural network architecture with six hidden layers of 40 neurons each is employed. The results obtained from the neural network models are compared with those from numerical simulations carried out using the Abaqus finite element software to validate the predictions.

3.1. Problem with Dirichlet Boundary Conditions

Here, we start with a simple problem by applying Dirichlet boundary conditions at the inlet and outlet. To ensure a unidirectional phenomenon, an adiabatic condition is imposed on the wall. This initial problem will allow us to test several activation functions and select the most suitable one for further studies. At the inlet and outlet, temperatures $T_{\mathrm{inlet}}=1200 °\mathrm{C}$ and $T_{\mathrm{outlet}}=20 °\mathrm{C}$ are prescribed, respectively. The initial temperature is set to $T_{\mathrm{initial}}=20 °\mathrm{C}$, and the total physical simulation time is $1.2 \mathrm{s}$. With these conditions, the diffusion thermal problem is expected to be constant along the y-direction. Even though the adiabatic condition is imposed on the wall, this problem is referred to as ’Dirichlet boundary conditions’ due to the conditions imposed at the inlet and outlet.

In Equation (1), for Dirichlet boundary conditions, $\mathcal{B}T=T$, and $g(x,y,t)$ represents the imposed temperature, while $f(x,y)$ is the prescribed initial temperature. For the adiabatic conditions at the wall, $\mathcal{B}T={\displaystyle \frac{\partial T}{\partial n}}$ (where n is the normal outward), with $g(x,y,t)=0$ indicating no heat flux. Equation (1) then becomes:

$$\left\{\begin{array}{cc}\rho c{\displaystyle \frac{\partial T}{\partial t}}=\lambda \left({\displaystyle \frac{{\partial }^{2}T}{\partial x^2}}+{\displaystyle \frac{{\partial }^{2}T}{\partial y^2}}\right)\hfill & (x,y)\in \mathrm{\Omega },t\in [0,1.2]\hfill \\ T(x,y,0)=T_{\mathrm{initial}}\hfill & (x,y)\in \mathrm{\Omega }\hfill \\ T(x,y,t)=T_{\mathrm{inlet}}\hfill & (x,y)\in \partial {\mathrm{\Omega }}_{\mathrm{inlet}},t\in [0,1.2]\hfill \\ T(x,y,t)=T_{\mathrm{outlet}}\hfill & (x,y)\in \partial {\mathrm{\Omega }}_{\mathrm{outlet}},t\in [0,1.2]\hfill \\ {\displaystyle \frac{\partial T(x,y,t)}{\partial n}}=0\hfill & (x,y)\in \partial {\mathrm{\Omega }}_{\mathrm{wall}},t\in [0,1.2]\hfill \end{array}\right.$$

(2)

Introducingresidual errors denoted as $\mathcal{R}$, we define:

$$\left\{\begin{array}{cc}{\mathcal{R}}_{\mathrm{pde}}(x,y,t)=\rho c{\displaystyle \frac{\partial T}{\partial t}}-\lambda \left({\displaystyle \frac{{\partial }^{2}T}{\partial x^2}}+{\displaystyle \frac{{\partial }^{2}T}{\partial y^2}}\right)\hfill & (x,y)\in \mathrm{\Omega },t\in [0,1.2]\hfill \\ {\mathcal{R}}_{\mathrm{ic}}(x,y,0)=T(x,y,0)-T_{\mathrm{initial}}\hfill & (x,y)\in \mathrm{\Omega }\hfill \\ {\mathcal{R}}_{\mathrm{in}}(x,y,t)=T(x,y,t)-T_{\mathrm{inlet}}\hfill & (x,y)\in \partial {\mathrm{\Omega }}_{\mathrm{inlet}},t\in [0,1.2]\hfill \\ {\mathcal{R}}_{\mathrm{out}}(x,y,t)=T(x,y,t)-T_{\mathrm{outlet}}\hfill & (x,y)\in \partial {\mathrm{\Omega }}_{\mathrm{outlet}},t\in [0,1.2]\hfill \\ {\mathcal{R}}_{\mathrm{wall}}(x,y,t)={\displaystyle \frac{\partial T(x,y,t)}{\partial n}}\hfill & (x,y)\in \partial {\mathrm{\Omega }}_{\mathrm{wall}},t\in [0,1.2]\hfill \end{array}\right.$$

(3)

To normalize the network by making its inputs and outputs dimensionless, we use reference quantities such as temperature $T_0=T_{\mathrm{inlet}}$, the length $L_0$ of the model geometry, and the time $t_0=1.2 \mathrm{s}$. The new dimensionless variables are expressed as: $T^{*}={\displaystyle \frac{T}{T_0}}$, $x^{*}={\displaystyle \frac{x}{L_0}}$, $y^{*}={\displaystyle \frac{y}{L_0}}$, and $t^{*}={\displaystyle \frac{t}{t_0}}$. Utilizing these new variables, the physical equation transforms into:

$$\rho c{\displaystyle \frac{T_0}{t_0}}{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}=\lambda {\displaystyle \frac{T_0}{L_0^2}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)$$

(4)

Consequently, the dimensionless equation is derived as:

$${\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}={\displaystyle \frac{\lambda }{\rho c}}{\displaystyle \frac{t_0}{L_0^2}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)$$

(5)

Here, we observe that the non-dimensionalization was carried out by dividing the entire equation by the coefficient of the transient term. We define PINN1, which refers to normalization by the coefficient of the transient term, and PINN2, which refers to normalization by the coefficient of the conduction term.

The Root Mean Square Error (RMSE) of the residuals is expressed as:

$$\left\{\begin{array}{c}{\mathcal{L}}_{\mathrm{pde}}=∥{\mathcal{R}}_{\mathrm{pde}}^{*}∥=∥{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}-{\displaystyle \frac{\lambda }{\rho c}}{\displaystyle \frac{t_0}{L_0^2}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{ic}}=∥{\mathcal{R}}_{\mathrm{ic}}^{*}∥=∥T^{*}-{\displaystyle \frac{T_{\mathrm{initial}}}{T_0}}∥\hfill \\ {\mathcal{L}}_{\mathrm{in}}=∥{\mathcal{R}}_{\mathrm{in}}^{*}∥=∥T^{*}-{\displaystyle \frac{T_{\mathrm{inlet}}}{T_0}}∥\hfill \\ {\mathcal{L}}_{\mathrm{out}}=∥{\mathcal{R}}_{\mathrm{out}}^{*}∥=∥T^{*}-{\displaystyle \frac{T_{\mathrm{outlet}}}{T_0}}∥\hfill \\ {\mathcal{L}}_{\mathrm{wall}}=∥{\mathcal{R}}_{\mathrm{wall}}^{*}∥=∥{\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}∥\hfill \end{array}\right.$$

(6)

In fact, both dimensional and non-dimensional equations correspond to PDEs of different orders. Dirichlet boundary conditions and initial conditions impose temperature, making them zero-order PDEs, whereas adiabatic boundary conditions impose temperature gradients, which correspond to first-order PDEs. For the heat equation, the conduction term involves second-order temperature gradients, making it a second-order PDE.

With Dirichlet boundary conditions, the residual errors are defined by ${\mathcal{R}}^{*}=T^{*}-{\displaystyle \frac{T_{\mathrm{prescribed}}}{T_0}}$. The acceptable order of magnitude for the value of the residual errors can be deduced from the imposed term ${\displaystyle \frac{T_{\mathrm{prescribed}}}{T_0}}$, assuming that residual errors are considered acceptable if they are one percent of ${\displaystyle \frac{T_{\mathrm{prescribed}}}{T_0}}$. For instance, the inlet residual error is acceptable if ${\mathcal{R}}_{\mathrm{in}/\mathrm{optimal}}^{*}=0.01{\displaystyle \frac{T_{\mathrm{inlet}}}{T_0}}$, which results in ${\mathcal{R}}_{\mathrm{in}/\mathrm{optimal}}^{*}=0.01$. Furthermore, for the outlet, the residual is considered acceptable when ${\mathcal{R}}_{\mathrm{out}/\mathrm{optimal}}^{*}=0.01{\displaystyle \frac{T_{\mathrm{outlet}}}{T_0}}$, leading to ${\mathcal{R}}_{\mathrm{out}/\mathrm{optimal}}^{*}=0.000167$. This reveals a significant difference between the two acceptable residual errors, whose magnitudes can be determined in advance. Moreover, it illustrates how acceptable residual errors from higher-order equations can vary considerably in magnitude. For instance, in the case of the adiabatic boundary condition at the wall, where the residual error is determined by ${\mathcal{R}}_{\mathrm{wall}}^{*}={\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}$, the acceptable residual error cannot be precisely estimated but is expected to differ significantly from ${\mathcal{R}}_{\mathrm{in}/\mathrm{optimal}}^{*}$ and ${\mathcal{R}}_{\mathrm{out}/\mathrm{optimal}}^{*}$, depending on the physical problem. This highlights the limitations of the classical method in terms of generalizability.

To address this issue, we propose the SD-PINN approach, which expresses initial and boundary conditions in the same physical dimension as the governing heat equation. In fact, the terms in the heat equation, Equation (1), have the dimension of power. The objective is to transform the boundary and initial condition equations so that they are also expressed in terms of power.

In the heat equation, the transient term corresponds to the time derivative of temperature, ${\displaystyle \frac{\partial T}{\partial t}}$, multiplied by the specific heat c. For prescribed temperature boundary conditions, this derivative can be approximated numerically by considering the residual error $T-T_{\mathrm{prescribed}}$ as a temperature variation and dividing it by a finite change in time, $\Delta t$. With $\Delta t$ chosen such that its order of magnitude reflects a physically acceptable time increment for the discretization of the problem.

Similarly, for Neumann boundary conditions at the wall, the residual error ${\displaystyle \frac{\partial T}{\partial n}}-0$, can be divided by $\Delta l$ (finite change in length) to compute a numerical second-order temperature gradient, which will then be multiplied by ${\displaystyle \frac{\lambda }{\rho }}$ to represent the conductive power in the heat equation. Here, $\Delta l$ is chosen to be of the same order of magnitude as the mesh element size. Mathematically, this yields:

$$\left\{\begin{array}{cc}c{\displaystyle \frac{T-T_{\mathrm{prescribed}}}{\Delta t}}\hfill & \mathrm{for}\mathrm{initial}\mathrm{and}\mathrm{Dirichlet}\mathrm{boundary}\mathrm{conditions}\hfill \\ {\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{\frac{\partial T}{\partial l}-0}{\Delta l}}\hfill & \mathrm{for}\mathrm{the}\mathrm{Neumann}\mathrm{condition}\mathrm{at}\mathrm{the}\mathrm{wall}\hfill \end{array}\right.$$

(7)

It is important to emphasize that the expressions in Equation (7) are designed to scale the residuals of the initial and boundary conditions, namely $\mathcal{R}\mathrm{ic}$, $\mathcal{R}\mathrm{in}$, $\mathcal{R}\mathrm{out}$, and $\mathcal{R}\mathrm{wall}$, so that they are comparable in magnitude to the terms $\rho c{\displaystyle \frac{\partial T}{\partial t}}$ and $\lambda \left({\displaystyle \frac{{\partial }^{2}T}{\partial x^2}}+{\displaystyle \frac{{\partial }^{2}T}{\partial y^2}}\right)$ in the heat equation residual $\mathcal{R}\mathrm{pde}$, rather than to the residual $\mathcal{R}\mathrm{pde}$ itself. Since the overall residual $\mathcal{R}\mathrm{pde}$ of the heat equation is naturally much smaller than its individual terms, the second step involves multiplying the boundary and initial condition residuals ($\mathcal{R}\mathrm{ic}$, $\mathcal{R}\mathrm{in}$, $\mathcal{R}\mathrm{out}$, and ${\mathcal{R}}_{\mathrm{wall}}$) by 0.01, under the assumption that the residual of the physical equation can be considered acceptable once it reaches about 1% of its constituent terms. Thus, the residuals for the SD-PINN method are expressed as:

$$\left\{\begin{array}{c}{\mathcal{R}}_{\mathrm{pde}}=c{\displaystyle \frac{\partial T}{\partial t}}-{\displaystyle \frac{\lambda }{\rho }}\left({\displaystyle \frac{{\partial }^{2}T}{\partial x^2}}+{\displaystyle \frac{{\partial }^{2}T}{\partial y^2}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{ic}}=0.01c{\displaystyle \frac{T-T_{\mathrm{initial}}}{\Delta t}}\hfill \\ {\mathcal{R}}_{\mathrm{in}}=0.01c{\displaystyle \frac{T-T_{\mathrm{inlet}}}{\Delta t}}\hfill \\ {\mathcal{R}}_{\mathrm{out}}=0.01c{\displaystyle \frac{T-T_{\mathrm{outlet}}}{\Delta t}}\hfill \\ {\mathcal{R}}_{\mathrm{wall}}=0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{\frac{\partial T}{\partial n}-0}{\Delta l}}\hfill \end{array}\right.$$

(8)

or using non-dimensional variables:

$$\left\{\begin{array}{c}{\mathcal{R}}_{\mathrm{pde}}=c{\displaystyle \frac{T_0}{t_0}}{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}-{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0^2}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{ic}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{initial}}}{T_0}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{in}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{inlet}}}{T_0}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{out}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{outlet}}}{T_0}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{wall}}=0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0\Delta l}}{\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}\hfill \end{array}\right.$$

(9)

We define the coefficients ${\mathcal{C}}_{\mathrm{tran}}$, ${\mathcal{C}}_{\mathrm{cond}}$, ${\mathcal{C}}_{\mathrm{ic}}$, ${\mathcal{C}}_{\mathrm{in}}$, ${\mathcal{C}}_{\mathrm{out}}$, and ${\mathcal{C}}_{\mathrm{wall}}$ to obtain the following equations:

$$\left\{\begin{array}{c}{\mathcal{R}}_{\mathrm{pde}}={\mathcal{C}}_{\mathrm{tran}}{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}-{\mathcal{C}}_{\mathrm{cond}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{ic}}={\mathcal{C}}_{\mathrm{ic}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{initial}}}{T_0}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{in}}={\mathcal{C}}_{\mathrm{in}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{inlet}}}{T_0}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{out}}={\mathcal{C}}_{\mathrm{out}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{outlet}}}{T_0}}\right)\hfill \\ {\mathcal{R}}_{\mathrm{wall}}={\mathcal{C}}_{\mathrm{wall}}{\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}\hfill \end{array}\right.$$

(10)

Hence, SD-PINN loss functions are articulated as:

$$\left\{\begin{array}{c}{\mathcal{L}}_{\mathrm{pde}}=∥{\mathcal{C}}_{\mathrm{tran}}{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}-{\mathcal{C}}_{\mathrm{cond}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{ic}}=∥{\mathcal{C}}_{\mathrm{ic}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{initial}}}{T_0}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{in}}=∥{\mathcal{C}}_{\mathrm{in}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{inlet}}}{T_0}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{out}}=∥{\mathcal{C}}_{\mathrm{out}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{outlet}}}{T_0}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{wall}}=∥{\mathcal{C}}_{\mathrm{wall}}{\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}∥\hfill \end{array}\right.$$

(11)

As previously discussed, $\Delta t$ and $\Delta l$ should ideally be of the same order of magnitude as the numerical discretization parameters, namely the physical time step and the mesh element size, respectively. However, as a first approximation, and in order to disregard any specific mesh and time discretization, we assume that 1% of the model’s characteristic temporal and spatial scales provides a representative magnitude for the time increment and element size. Accordingly, we set $\Delta t=0.01,t_0$ and $\Delta l=0.01,L_0$, which leads to the following coefficient values:

$$\left\{\begin{array}{cc}{\mathcal{C}}_{\mathrm{tran}}=c{\displaystyle \frac{T_0}{t_0}}\hfill & =770,000\hfill \\ {\mathcal{C}}_{\mathrm{cond}}={\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0^2}}\hfill & =91,492.92\hfill \\ {\mathcal{C}}_{\mathrm{ic}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =770,000\hfill \\ {\mathcal{C}}_{\mathrm{in}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =770,000\hfill \\ {\mathcal{C}}_{\mathrm{out}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =770,000\hfill \\ {\mathcal{C}}_{\mathrm{wall}}=0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0\Delta l}}\hfill & =91,492.92\hfill \end{array}\right.$$

(12)

Finally, in the SDFEET-PINN method we propose, the coefficients ${\mathcal{C}}_{\mathrm{tran}}$, ${\mathcal{C}}_{\mathrm{cond}}$, ${\mathcal{C}}_{\mathrm{ic}}$, ${\mathcal{C}}_{\mathrm{in}}$, ${\mathcal{C}}_{\mathrm{out}}$, and ${\mathcal{C}}_{\mathrm{wall}}$ vary over the epochs. The idea is to initially assign high importance to each term and then gradually decrease it to its appropriate value. In this work, we have chosen to reach the desired values at half of the total epochs, a point that corresponds to the initial stabilization of the learning process. To determine an appropriate starting value, we compute the maximum coefficient from the heat equation as ${\mathcal{C}}_{\max }=max\left({\mathcal{C}}_{\mathrm{tran}},{\mathcal{C}}_{\mathrm{cond}}\right)$ and enforce all coefficients to begin at this value until they reach their final values midway through training. Since the true maximum value of the initial and boundary conditions coefficients may exceed ${\mathcal{C}}_{\max }$, we apply a scaling factor of ${\displaystyle \frac{{\mathcal{C}}_{\max }}{max\left({\mathcal{C}}_{\mathrm{ic}},{\mathcal{C}}_{\mathrm{in}},{\mathcal{C}}_{\mathrm{out}},{\mathcal{C}}_{\mathrm{wall}}\right)}}$ to them. Although this adjustment is not strictly necessary, it ensures that the physical equation dictates the highest coefficient value, resulting in more realistic coefficient values overall.

The evolution of the weighting coefficients implemented for SD-PINN and SDFEET-PINN is presented in Figure 4.

The models, SD-PINN, SDFEET-PINN, PINN1, and PINN2, were trained for 1100 epochs with a network comprising six hidden layers of 40 neurons each. The ADAM optimization algorithm was employed alongside various activation functions, including Exponential Linear Unit (ELU), Sigmoid, and tanh. The learning rate was set to a fixed value of 1 × 10⁻⁴. It is worth noting that in numerical simulations, boundary conditions are strictly enforced in the initial increment. To properly replicate this with neural networks, the initial increment should ideally be divided into multiple steps to accurately describe this evolution. For simplicity, all boundary conditions are interpolated linearly from the initial value to the imposed value over the entire simulation duration. All neural network computations were carried out on an NVIDIA A100-PCIE-40GB GPU server, utilizing the entire set of domain and time points simultaneously, without batching. The computation times for all models are comparable, averaging around 3 min.

3.2. Problem with Dirichlet and Newton Boundary Conditions

The multi-scale L-PBF additive manufacturing model proposed by Bresson et al. [10] simulates the process across various scales, considering the influence of the surrounding powder through Newton boundary conditions. In addition to incorporating these boundary conditions into the model, we start with a warm initial condition of $T_{\mathrm{initial}}=1200 °\mathrm{C}$ to simulate the cooling of the workpiece during additive manufacturing. A Newton boundary condition is applied at the outlet, and a Dirichlet boundary condition with linear cooling to $T_{\mathrm{inlet}}=20 °\mathrm{C}$ is imposed at the inlet. For the Newton boundary condition, the external temperature $T_{ext}$ is set to decrease from $T_{ext}=1200 °\mathrm{C}$ to $T_{ext}=20 °\mathrm{C}$, in accordance with the previously mentioned rule for the progressive application of boundary conditions. To increase the complexity, we use a deliberately high wall heat transfer coefficient of h = 10,000 W m⁻² K⁻¹. The physical time is set to $0.3 \mathrm{s}$.

Only the outlet loss function differs from the previous equations and is expressed as:

$${\mathcal{L}}_{\mathrm{out}}=∥0.01{\displaystyle \frac{h}{\rho }}{\displaystyle \frac{T_0}{\Delta l}}\left({\displaystyle \frac{T_{ext}}{T_0}}-T^{*}\right)-0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0\Delta l}}{\displaystyle \frac{\partial T^{*}}{\partial x^{*}}}∥$$

(13)

The previous output coefficient, ${\mathcal{C}}_{\mathrm{out}}$, is then replaced by two distinct coefficients: one for the heat exchange component, ${\mathcal{C}}_{\mathrm{out}/\mathrm{h}}=0.01{\displaystyle \frac{h}{\rho }}{\displaystyle \frac{T_0}{\Delta l}}$, and another for the conduction component, ${\mathcal{C}}_{\mathrm{out}/\mathrm{k}}=0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0\Delta l}}$. The values of the coefficients are:

$$\left\{\begin{array}{cc}{\mathcal{C}}_{\mathrm{tran}}=c{\displaystyle \frac{T_0}{t_0}}\hfill & =3,080,000\hfill \\ {\mathcal{C}}_{\mathrm{cond}}={\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0^2}}\hfill & =91,492.92\hfill \\ {\mathcal{C}}_{\mathrm{ic}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =3,080,000\hfill \\ {\mathcal{C}}_{\mathrm{in}}=0.01c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =3,080,000\hfill \\ {\mathcal{C}}_{\mathrm{out}/\mathrm{h}}=0.01{\displaystyle \frac{h}{\rho }}{\displaystyle \frac{T_0}{\Delta l}}\hfill & =316,706.28\hfill \\ {\mathcal{C}}_{\mathrm{out}/\mathrm{k}}=0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0\Delta l}}\hfill & =91,492.92\hfill \\ {\mathcal{C}}_{\mathrm{wall}}=0.01{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0\Delta l}}\hfill & =91,492.92\hfill \end{array}\right.$$

(14)

Here, we also propose a new Adaptive Learning Rate (ALR) approach, which will be compared to a Fixed Learning Rate (FLR) of 1 × 10⁻⁴. The ALR strategy starts training with an initial learning rate of 1 × 10⁻³ and reduces it by half every 100 epochs if the global loss function, $\mathcal{L}$, is higher than its value from 100 epochs earlier. This adjustment strategy is only activated after half of the total epochs have been completed, allowing the model to reach a preliminary convergence before applying learning rate modifications. We maintained the architecture of the previous network but trained it for 10,000 epochs. Once again, the simulation time is comparable across all methods, averaging approximately 25 min.

3.3. Problemwith Advection, Dirichlet, and Newton Boundary Conditions

In certain physical problems, the numerical complexity of solving the momentum equation can be circumvented by modeling the material flow using well-established principles. This allows the thermal problem to be solved numerically while incorporating an advection term to account for material movement. As a result, thermal advection can be applied to various thermal problems in fluid mechanics and even in solid mechanics when using the Eulerian formulation. In this subsection, advection has been incorporated into the previous problem. The imposed parabolic advection velocity field is shown in Figure 5. The PDE loss function becomes:

$${\mathcal{L}}_{\mathrm{pde}}=∥c{\displaystyle \frac{T_0}{t_0}}{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}+c{u}_{\max }{\displaystyle \frac{T_0}{L_0}}{u}^{*}{\displaystyle \frac{\partial T^{*}}{\partial x^{*}}}-{\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0^2}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}\right)∥$$

(15)

The coefficients of the heat equation become:

$$\left\{\begin{array}{cc}{\mathcal{C}}_{\mathrm{tran}}=c{\displaystyle \frac{T_0}{t_0}}\hfill & =3,080,000\hfill \\ {\mathcal{C}}_{\mathrm{cond}}={\displaystyle \frac{\lambda }{\rho }}{\displaystyle \frac{T_0}{L_0^2}}\hfill & =91,492.92\hfill \\ {\mathcal{C}}_{\mathrm{conv}}=c{u}_{\max }{\displaystyle \frac{T_0}{L_0}}\hfill & =30,800,000\hfill \end{array}\right.$$

(16)

We retained the same network architecture and training for 20,000 epochs. The computation times of the models are similar and are equal to 42 min.

3.4. Application to a Complex 3D Geometry

In this subsection, we apply the methods to a complex 3D geometry. As mentioned earlier, the component used in this study is an additively manufactured hydraulic joint that has been the subject of several numerical and experimental studies. The goal of this project is to replace the multi-scale numerical models for simulating the L-PBF additive manufacturing process, proposed by [10], with neural network-based models for real-time predictions. Therefore, the material data used in this study are derived from the work of [10]. In this case, we focus on the hydraulic joint, considering a macroscopic scale simulation and a specific layer.

Figure 6 shows the geometry of the part discretized using voxel elements. The bottom of the part is fixed at the plate temperature, $T_{\mathrm{bottom}}=170 °\mathrm{C}$. The conduction in the surrounding powder that is not explicitly modeled is represented as a convective exchange with a coefficient of $h_{\mathrm{powder}}=4 \mathrm{W}{\mathrm{m}}^{-2}{\mathrm{K}}^{-1}$, assuming the powder temperature is equal to the chamber temperature, $T_{\mathrm{ext}}=20 °\mathrm{C}$. The cooling of the top surface of the part by heat transfer due to the gas flow is represented by a convective exchange with $h_{\mathrm{gas}}=200 \mathrm{W}{\mathrm{m}}^{-2}{\mathrm{K}}^{-1}$, with the gas temperature also assumed to be equal to the chamber temperature. A heating flux of $P_{\mathrm{heat}}=3000 \mathrm{W}{\mathrm{m}}^{-2}$ is applied. The physical simulation time is 4 s. All boundary conditions are gradually applied to reach their specified values by the end of the simulation. The loss functions are expressed as follows:

$$\left\{\begin{array}{c}{\mathcal{L}}_{\mathrm{pde}}=∥{\mathcal{C}}_{\mathrm{tran}}{\displaystyle \frac{\partial T^{*}}{\partial t^{*}}}-{\mathcal{C}}_{\mathrm{cond}}\left({\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {x^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {y^{*}}^2}}+{\displaystyle \frac{{\partial }^2{T}^{*}}{\partial {z^{*}}^2}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{ic}}=∥{\mathcal{C}}_{\mathrm{ic}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{initial}}}{T_0}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{bot}}=∥{\mathcal{C}}_{\mathrm{bot}}\left(T^{*}-{\displaystyle \frac{T_{\mathrm{bottom}}}{T_0}}\right)∥\hfill \\ {\mathcal{L}}_{\mathrm{pow}}=∥{\mathcal{C}}_{\mathrm{pow}/\mathrm{h}}\left({\displaystyle \frac{T_{\mathrm{ext}}}{T_0}}-T^{*}\right)-{\mathcal{C}}_{\mathrm{pow}/\mathrm{k}}{\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}∥\hfill \\ {\mathcal{L}}_{\mathrm{top}}=∥{\mathcal{C}}_{\mathrm{top}/\mathrm{p}}+{\mathcal{C}}_{\mathrm{top}/\mathrm{h}}\left({\displaystyle \frac{T_{\mathrm{ext}}}{T_0}}-T^{*}\right)-{\mathcal{C}}_{\mathrm{top}/\mathrm{k}}{\displaystyle \frac{\partial T^{*}}{\partial n^{*}}}∥\hfill \end{array}\right.$$

(17)

The coefficients are determined by:

$$\left\{\begin{array}{cc}{\mathcal{C}}_{\mathrm{tran}}=\rho c{\displaystyle \frac{T_0}{t_0}}\hfill & =1620.85\hfill \\ {\mathcal{C}}_{\mathrm{cond}}=\lambda {\displaystyle \frac{T_0}{L_0^2}}\hfill & =12.31\hfill \\ {\mathcal{C}}_{\mathrm{ic}}=0.01\rho c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =1620.85\hfill \\ {\mathcal{C}}_{\mathrm{bot}}=0.01\rho c{\displaystyle \frac{T_0}{\Delta t}}\hfill & =1620.85\hfill \\ {\mathcal{C}}_{\mathrm{pow}/\mathrm{h}}=0.01h_{\mathrm{powder}}{\displaystyle \frac{T_0}{\Delta l}}\hfill & =0.123\hfill \\ {\mathcal{C}}_{\mathrm{pow}/\mathrm{k}}=0.01\lambda {\displaystyle \frac{T_0}{L_0\Delta l}}\hfill & =12.31\hfill \\ {\mathcal{C}}_{\mathrm{top}/\mathrm{p}}=0.01{\displaystyle \frac{P_{\mathrm{heat}}}{\Delta l}}\hfill & =46.15\hfill \\ {\mathcal{C}}_{\mathrm{top}/\mathrm{h}}=0.01h_{\mathrm{gaz}}{\displaystyle \frac{T_0}{\Delta l}}\hfill & =6.15\hfill \\ {\mathcal{C}}_{\mathrm{top}/\mathrm{k}}=0.01\lambda {\displaystyle \frac{T_0}{L_0\Delta l}}\hfill & =12.31\hfill \end{array}\right.$$

(18)

Due to the large data size, we implemented a strategy that, at each epoch, randomly selects 100% of the initial conditions, 25% of the data from the bottom surface, 5% from the powder surface, 25% from the top surface, and 5% from the internal nodes. This assumption proved acceptable based on the benchmark results. The network was trained for 10,000 epochs. For this part, we consider the three models using the ALR method. The training time for the neural networks is 406 min.

4. Results and Discussions

4.1. Problem with Dirichlet Boundary Conditions

Figure 7 illustrates the loss functions for various activation functions, revealing an interesting competition between them, as some loss functions increase while others decrease. For the Sigmoid activation function, a plateau is quickly reached, with the inlet loss function, ${\mathcal{L}}_{\mathrm{in}}$, being significantly more dominant than the others. This plateau can be attributed to the characteristics of the Sigmoid activation function, which approaches zero for input values less than zero. Although all domain point coordinates are positive, the inlet’s x-position is zero, bringing it closer to the influence of the vanishing part of the Sigmoid function. However, the inlet loss function remains predominant in the graphs of other activation functions as well, but converges more effectively, particularly with the tanh activation function. That being said, this observation should be interpreted with caution, as the rapid convergence of a loss function does not necessarily indicate good predictive performance.

To further analyze the predictions, we plotted the dispersion of prediction errors for the different models in Figure 8. It is clearly visible that the predictions are poor for both the Sigmoid activation function and the PINN2 method. With the PINN1 method, only the tanh function performs well. With our two new methods, SD-PINN and SDFEET-PINN, the results are strong, except for the Sigmoid function, which, in any case, performs poorly across all methods.

These results indicate that our two methods, SD-PINN and SDFEET-PINN, achieve better performance and that the tanh activation function is the most effective. The tanh function was, therefore, retained for all subsequent simulations, as it provides smooth and symmetric gradients (ranging from −1 to 1) that favor the computation of higher-order derivatives required by the physical equations while ensuring stable convergence. This observation is also consistent with many recent studies in the PINN literature.

The temperature fields obtained from all methods were compared to the numerical simulation results, as shown in Figure 9. The initial, middle, and final frames are presented. The predictions appear satisfactory, except for the PINN2 method, which confirms the poor performance observed in the tanh dispersion graphs in Figure 8.

To gain further insights, the absolute error of the thermal field is presented in Figure 10, highlighting significant discrepancies between the predictions of the PINN2 method and the numerical results. For the other methods, SD-PINN, SDFEET-PINN, and PINN1, the maximum absolute error is around $45 °\mathrm{C}$ and is clearly localized at the inlet. This aligns well with the dominance of the inlet loss function, ${\mathcal{L}}_{\mathrm{in}}$, in the loss graphs shown in Figure 7.

Moreover, an absolute error of 45 $°\mathrm{C}$ represents a large deviation at low temperatures, particularly in regions where the temperature is close to 20 $°\mathrm{C}$. This can be explained by the fact that the network was trained using absolute error metrics (RMSE or MSE), while the temperature range under consideration spans from 20 $°\mathrm{C}$ to 1200 $°\mathrm{C}$. This choice is deliberate, since the influence of low-temperature regions on the quantities of interest is negligible. For instance, the alloy microstructure is not affected by deviations on the order of 45 $°\mathrm{C}$ at low temperatures (below 300 $°\mathrm{C}$). Consequently, throughout this work, absolute error measures are consistently adopted, as the networks are trained with absolute error-based losses (RMSE and MSE), which makes them more relevant than relative errors.

4.2. Problem with Dirichlet and Newton Boundary Conditions

As shown in Figure 11, with SD-PINN using a Fixed Learning Rate (FLR), the global loss function, which is dominated by the outlet loss, quickly reaches a plateau. Even with the Adaptive Learning Rate (ALR), the outlet loss function only drops significantly after 4000 epochs. A similar sharp drop is observed for the classical PINN2 method at 2000 epochs, both for ALR and FLR.

For the classical PINN1 method, we observe significant oscillations with very large amplitudes, comparable to the magnitude of the sudden drops seen in other methods. All these aspects indicate substantial convergence difficulties.

In contrast, with the proposed SDFEET-PINN method, neither abrupt drops nor large oscillations are observed, except for a few moderate fluctuations. This behavior is expected, as the SDFEET-PINN approach tends to increase the equation residuals before gradually reducing them.

In the dispersion error graphs shown in Figure 12, for the SD-PINN method with FLR, horizontal points can be observed at the upper extremity of the predictions, confirming the convergence difficulties previously indicated by the plateau in Figure 11. However, with ALR, the predictions are significantly improved.

Once again, PINN2 produces poorer results with both ALR and FLR compared to PINN1, highlighting the considerable influence of the choice of normalization divisor in the classical method. The proposed SDFEET-PINN method delivers accurate predictions, similar to those obtained with the PINN1 method.

When examining the predicted temperature fields of ALR models in Figure 13, all methods appear to produce results consistent with the finite element simulation. The final frame clearly shows that the target temperature of $20 °\mathrm{C}$ at the inlet boundary is successfully achieved. Additionally, the Newton boundary condition applied at the outlet has generated a vertical red band of higher temperatures centered along the x-direction.

These visualizations alone do not allow for a clear comparison between the neural network models. To address this, the absolute error of the temperature field is analyzed in Figure 14, which clearly demonstrates that our two proposed methods, SD-PINN and SDFEET-PINN, outperform the others. However, the plot of the maximum error across all frames in Figure 15 indicates slightly better results for PINN1. In fact, the SD-PINN method with FLR and PINN2 are excluded from the comparison due to their poor performance, particularly PINN2, which exhibits significantly worse results.

All curves increase over time. One might suspect an error that could amplify over time, but this is not the case. Instead, it results from the fact that the Dirichlet boundary conditions and initial conditions are easier for the network to learn compared to the Newton boundary conditions, which are significantly more complex. In the early stages of the simulation, the influence of the boundary conditions is weaker compared to the initial condition. Additionally, in the final frames, the temperature field becomes more complex.

From the two problems with different boundary conditions, SDFEET-PINN performs best, followed by SD-PINN and PINN1. PINN2 is far behind the other three models. Since PINN1 is based on dividing the physical equation by ${\mathcal{C}}_{\mathrm{tran}}$, and ${\mathcal{C}}_{\mathrm{tran}}$ is greater than ${\mathcal{C}}_{\mathrm{cond}}$ in both problems, we can consider PINN1 as a method that normalizes the physical equation by dividing it by the maximum value of its dimensional coefficients. Thus, adopting a normalization strategy using the maximum coefficient value as the divisor allows us to use only PINN1, avoiding the need to manage three classical methods. This is especially useful for the next problem, which will include three terms in the physical equation. In fact, an advection term will be added to the heat equation to test the robustness of the different methods. Hereafter, PINN1 will simply be referred to as PINN.

4.3. Problem with Advection, Dirichlet, and Newton Boundary Conditions

As shown in all graphs of Figure 16, the loss functions are dominated by the physical equation loss throughout all epochs, except for the graph of the SDFEET-PINN method with ALR, where, around epoch 12,000, the outlet loss function takes precedence.

As shown in Figure 17, the dispersion graphs clearly indicate that the SDFEET-PINN method with ALR provides better predictions than all other methods. In Figure 18, when observing the temperature field for models with ALR at the initial, intermediate, and final frames, the performance of the SDFEET-PINN method with the ALR model is clearly confirmed. This method is the only one where the physical equation loss function is not the most dominant. This suggests that the heat equation poses convergence difficulties in other methods. To investigate further, we evaluated the physical equation loss function in detail for all three methods with ALR.

Figure 19 shows that, in all methods, the final value of the conduction term becomes negligible compared to the other terms. The SDFEET-PINN method, due to its technique of increasing weaker terms to match the higher-value terms, initially had very high conduction and transient terms at the beginning of the epochs, allowing it to start from elevated values and descend toward equilibrium. It is observed that the convection term evolves less because it has the largest coefficient, and it is already at its true value from the start. The same evolution of terms is seen in the SDFEET-PINN method with FLR, but since the Learning Rate is fixed ($LR=1 \times {10}^{-4}$) and low at the beginning of the epochs, the curve is slower. Furthermore, the convection term does not coincide with the transient term until the end of the FEET strategy (around epoch 10,000), as seen in the SDFEET-PINN method with ALR. This coincidence appears to be the key difference between the two methods, as once the convection and transient terms coincide in the SDFEET-PINN method with ALR, this state remains stable until the end of the epochs, despite the variation in the values of the two terms. Additionally, in the SDFEET-PINN method with ALR, the conduction term rises after the FEET strategy finishes, reaching its true value, while this increase is not seen in the SDFEET-PINN method with FLR. It seems that this non-coincidence compensates for the lack of increase, as the displacement between the convection and transient terms in the SDFEET-PINN method with FLR appears to match the rise in the conduction term in the SDFEET-PINN method with ALR.

Given the convergence difficulties encountered with several methods, training was extended to 100,000 epochs for approaches using Adaptive Learning Rates (ALRs), and to 200,000 epochs for those with Fixed Learning Rates (FLRs). Additionally, the MSE loss function was used for FLR methods. The standard PINN method, which normalizes by the maximum coefficient, consistently failed to converge. As a result, we also evaluated the PINN1 variant, which normalizes the equation by the transient term coefficient and has shown better performance in previous problems. Training durations were approximately 4 h for 100,000 epochs and 8 h for 200,000 epochs. To focus the comparison on the final predicted temperature fields, Figure 20 presents the last frame of the simulations for various hyperparameter settings. The PINN method did not produce acceptable results in any configuration. PINN1 yielded improved predictions when MSE was used, but remained less accurate than our proposed SD-PINN. The latter provided sharper and more accurate temperature fields, including under ALR with 100,000 epochs. Moreover, the SDFEET-PINN approach demonstrated the best robustness, producing consistent results across all configurations starting from 100,000 epochs, and even from 20,000 epochs under ALR. The only failure case was under 20,000 epochs with MSE, where SD-PINN and, to a lesser extent, PINN1 still performed acceptably. These observations underscore the effectiveness of our proposed approaches across various training settings and highlight the critical influence of the normalization strategy in standard PINNs.

4.4. Application to a Complex 3D Geometry

The dispersion curves of the results, shown in Figure 21, reveal poor predictions from all three models, with the classical PINN method yielding results worse than the SD-PINN and SDFEET-PINN methods. This is further confirmed by the temperature field results shown in Figure 22. The numerical model shows peak temperatures up to $896 °\mathrm{C}$, while the proposed methods reach approximately 730 °C. Although this underestimation indicates that further refinement is needed, both SD-PINN and SDFEET-PINN clearly outperform the classical PINN approach, which only predicts about 400 °C and therefore fails to identify hot-spot regions in the part.

In this subsection, we can conclude that our two new methods yield promising results, especially considering this is the first attempt to address this problem with such a complex geometry. Indeed, the part is meshed using voxel elements, resulting in highly discontinuous surfaces. Moreover, the area that is less well predicted by our models compared to the numerical model consists of elements where many of them have their tops exposed to the boundary condition on the top surface of the part, while their bottoms and sides are exposed to the powder boundary condition. This results in multiple boundary conditions being imposed on the same element faces, with significant differences in their values. To improve accuracy, a higher number of collocation points will be required in these elements, or a more continuous mesh could be used. However, for additive manufacturing, we prefer voxel meshes due to their ease of use in various contexts. Therefore, in the future, we would stick with voxel elements, using many collocation points within these types of elements and prioritizing these collocation points, as some authors have implemented in the literature [17,25]. Our methods also have the advantage of being easily combined with other methods.

5. Conclusions

In this study, we introduced two novel weighting strategies for Physics-Informed Neural Networks (PINNs): the Same Dimensions PINN (SD-PINN) and the Same Dimensions From Equal Equation Terms PINN (SDFEET-PINN), and compared them with the classical PINN approach. The SD-PINN method balances the loss function terms by aligning their order of magnitude and physical dimensions. The SDFEET-PINN method pre-trains the network on a rebalanced governing equation and, after an initial stabilization of the learning process, progressively restores the original formulation, thereby preserving the influence of all terms across different orders of magnitude. After demonstrating that the hyperbolic tangent (tanh) function is the most suitable activation function, we introduced a new Adaptive Learning Rate (ALR) method, which significantly improved the quality of the results. Through the analysis of convergence graphs, dispersion plots, and temperature fields, we gained valuable insights into the convergence issues encountered by the methods and demonstrated how our approach enhances PINN convergence. The SDFEET-PINN method, combined with ALR, proved to be highly effective, converging to the solution for all the tested cases. The second-best performing method was SD-PINN with ALR, followed by the classical PINN method with ALR, also with ALR. Furthermore, we highlighted the importance of selecting the correct divisor for the normalization of the physical equation in the classical PINN method. It should be emphasized that, although the underlying heat transfer equation is physically simple, the combined presence of multiple boundary condition types and complex 3D voxel geometries makes these problems particularly demanding for PINNs. The proposed SD-PINN and SDFEET-PINN methods demonstrate their ability to handle such configurations with improved stability and accuracy compared to classical approaches.

In conclusion, SDFEET-PINN clearly offers a robust and efficient alternative to classical PINNs for addressing complex thermal problems. Its superior convergence behavior and accuracy make it a promising tool for advanced simulations involving varying boundary conditions and complex flow dynamics. Additionally, this method is advantageous because it can be applied to a wide range of existing techniques in the literature, as it focuses solely on the normalization of equations.

Looking forward, this study should be further explored by testing additional hyperparameters, including optimizers, different network dimensions, and even other physical models, such as the Navier–Stokes equations, in order to generalize the approach. Regarding the methods themselves, certain arbitrary choices, such as the use of a coefficient of 0.01 to represent 1% of the residual error terms, and the relationship between the spatiotemporal size of the model and its spatiotemporal increments, $\Delta t$ and $\Delta l$, require further investigation. A future objective will be to systematically explore whether alternative combinations of these parameters could further improve convergence performance. Furthermore, future extensions to multiphysics problems will involve an even larger number of coefficients to normalize, highlighting the relevance of the methodological framework proposed in this study. Further work should also focus on benchmarking the proposed methods using adaptive and weak-form PINN strategies, in order to verify the generalization of the method to various PINN strategies.

The focus will also move to real-time simulations, accounting for the progressive construction of the part in additive manufacturing, a key step toward developing the process digital twin.