Review of Physics-Informed Neural Networks: Challenges in Loss Function Design and Geometric Integration

Abstract

Physics-Informed Neural Networks (PINNs) represent a transformative approach to solving partial differential equation (PDE)-based boundary value problems by embedding physical laws into the learning process, addressing challenges such as non-physical solutions and data scarcity, which are inherent in traditional neural networks. This review analyzes critical challenges in PINN development, focusing on loss function design, geometric information integration, and their application in engineering modeling. We explore advanced strategies for constructing loss functions—including adaptive weighting, energy-based, and variational formulations—that enhance optimization stability and ensure physical consistency across multiscale and multiphysics problems. We emphasize geometry-aware learning through analytical representations—signed distance functions (SDFs), phi-functions, and R-functions—with complementary strengths: SDFs enable precise local boundary enforcement, whereas phi/R capture global multi-body constraints in irregular domains; in practice, hybrid use is effective for engineering problems. We also examine adaptive collocation sampling, domain decomposition, and hard-constraint mechanisms for boundary conditions to improve convergence and accuracy and discuss integration with commercial CAE via hybrid schemes that couple PINNs with classical solvers (e.g., FEM) to boost efficiency and reliability. Finally, we consider emerging paradigms—Physics-Informed Kolmogorov–Arnold Networks (PIKANs) and operator-learning frameworks (DeepONet, Fourier Neural Operator)—and outline open directions in standardized benchmarks, computational scalability, and multiphysics/multi-fidelity modeling for digital twins and design optimization.

1. Introduction

In recent years, physics-informed neural networks (PINNs) have become one of the leading approaches for solving boundary value problems described by partial differential equations (PDEs). This is since this approach, proposed by Raissi et al. [1], largely addressed issues that were inherent in the neural networks applied to such problems before the advent of PINNs. First, solutions obtained using conventional neural networks often did not comply with physical laws, because they did not account for the structure of the differential equations, boundaries, and initial conditions. Therefore, the results could be non-physical, unstable, or unsuitable for interpretation. Second, effective training of conventional neural network models required large volumes of data—numerical or experimental. This limited the application of neural networks to practical problems where obtaining such data is complicated, expensive, or fundamentally restricted.

The core idea of PINNs is to embed physical laws into the neural network training process by representing known differential equations as a component of the loss function. The network simultaneously minimizes errors between predictions or experimental data and the residuals of the governing equations and boundary conditions at selected points. A key prerequisite for the effective application of PINNs is the ability to define loss function components related to PDE residuals and boundary conditions using automatic differentiation available in modern machine learning frameworks (e.g., TensorFlow [2] and PyTorch [3]).

These advantages have sparked intense interest in applying PINNs, and the results of numerous studies have shown that this interest is justified, with PINNs proving effective in modeling various physical processes—fluid and gas dynamics (e.g., reviews by Cai et al. [4], Sharma et al. [5]), structural mechanics (e.g., reviews by Hu et al. [6], Faroughi et al. [7], Herrmann and Kollmannsberger [8]), heat conduction (e.g., reviews by Karniadakis et al. [9], Du and Lu [10]), electromagnetism and optics (e.g., review by Abdelraouf et al. [11]), and more. Cai et al. [4] note that PINNs are particularly effective for solving inverse problems where some data is available from multimodal measurements, for which traditional finite element methods (FEM) are often excessively costly and inefficient [12]. Meanwhile, Sharma et al. [5] point out that, despite significant progress in fluid flow problems, unresolved challenges remain. Specifically, PINN models have been tested only on specific problem configurations and datasets. To evaluate the effectiveness of developed algorithms, they must be validated on standardized datasets established by the community, as is performed for image recognition tasks. Additionally, Sharma et al. [5] suggest that, for robust PINN training, it is advisable to start with simpler problems and progressively train on more complex configurations. They also demonstrated that PINNs are better suited for modeling sequences of states rather than predicting spatiotemporal solutions from initial and boundary conditions. Sharma et al. [5] note that in real problems, PINNs face issues related to the complex geometry of computational domains and multiphysics phenomena. In reviews concerning solid mechanics [6,7], it is noted that, in many cases, PINNs eliminate the need for computationally expensive mesh generation, facilitating work with irregular and moving computational domains. The authors also indicate that PINNs are effective in inverse problems with sparse, unlabeled, and noisy data. However, when assessing the prospects for PINN development, Hu et al. [6] indicate that the problem of weighting PDE residuals in multiscale problems needs to be addressed, and for multiphysics problems, it is advisable to combine the loss function into a single equation. Karniadakis et al. [9] also identify inverse problems as the most promising for PINN applications and emphasize the necessity of domain decomposition for problems with complex geometries. Like the review by Sharma et al. [5], they underscore the urgent need for standardized benchmarks to evaluate various algorithms and the development of new mathematical frameworks for next-generation neural network modeling methods.

Du and Lu [10] explored promising directions for PINN development, focusing on improving thermal management in electronics and battery systems. In addition to the points mentioned above, they highlighted the need to account for variable thermal properties and complex heat transfer mechanisms when modeling intricate problems [13].

Abdelraouf et al. [11] emphasize the transformative impact of PINNs on the design of electromagnetic and nanophotonic systems, focusing on their advantages in rapid optimization, accurate modeling, and maintaining physical consistency with limited data. However, they note that integrating physical laws into these problems creates stiff, nonlinear loss function landscapes, complicating convergence, particularly for multiscale and multidimensional systems. Balancing residuals (physical and data-driven) remains a persistent challenge requiring task-specific tuning, while computational costs limit PINN applications for geometrically complex systems. The authors propose addressing these issues through adaptive PINN architectures with dynamic loss weighting, domain decomposition, and multifidelity modeling. They also stress the need for interdisciplinary integration with existing industrial solutions and the coupling of modeling with experimental validation.

The scope of PINN application for modeling processes of various natures is rapidly developing. The potential of PINN has been demonstrated in a wide range of biomedical applications, including systems biology, systems pharmacology, biomechanics, and epidemiology. By integrating biophysical laws with data-driven learning, PINN offers a powerful foundation for solving complex problems in these fields. The interest in modeling diverse processes using PINN-based algorithms is growing rapidly, as evidenced by the exponential increase in publications. This trend is illustrated in Figure 1, which shows the number of articles indexed in Scopus as of August 2025 containing the term PINN in the Abstract or Keywords. This growth reflects the broad recognition of PINNs’ potential within the scientific and engineering communities.

Figure 2 illustrates the existing applications of such PINNs across various scientific domains [14]. However, this review focuses on the use of PINNs in engineering modeling problems. The successes of applying artificial intelligence (AI) in this field, demonstrated in an increasing number of studies, have prompted leading CAD/CAE system developers (e.g., ANSYS, Autodesk, Siemens, Dassault Systèmes) to begin integrating AI tools, particularly PINNs, into their commercial software packages. Recent reviews by Khanolkar et al. [15], Uulu et al. [16], Montáns et al. [17], and Zhao et al. [18] indicate that AI integration in CAE occurs at three interconnected levels: supporting the workflow (geometry, meshing, pre- and post-processing), simulation and optimization methods (from reduced-order models to PINNs and operator neural networks), and applications in specific domains. However, despite their recognized potential, integrating PINNs with CAE packages faces challenges, including the need for computational resource optimization, ensuring result interpretability, and compatibility with existing CAD system formats.

Review papers indicate that deep learning methods significantly enhance the recovery of complex geometries and automate meshing, simplify the definition of constitutive models, accelerate solutions through surrogates and PINNs, and enable efficient post-processing [16,17,18]. However, limitations in data and training are noted: while PINNs can operate with minimal data, the lack of high-quality validation data in industrial CAE calculations leads to inaccuracies, particularly for complex problems—nonlinear, multiscale, and multiphysics.

It is noted that practical applications may encounter issues with accuracy and stability: in industrial contexts, without pre-training, PINNs often fail to achieve desired accuracy and may significantly underperform traditional solvers in terms of solution time for forward-modeling problems due to difficulties in hyperparameter tuning [18,19]. Applying PINNs in industrial computational packages requires intensive computations to balance data and physical constraint losses, complicating their integration into commercial packages without significant resource optimization [20]. This underscores the need for research into more efficient methods for integrating diverse data types into PINN-based algorithms. When combining PINNs with traditional solvers, issues with interpretability and uncertainty arise: PINNs are difficult to interpret, and assessing result uncertainty is critical for CAE, where errors can have serious consequences [21]. This necessitates additional extensions to ensure result reliability. Finally, integrating PINNs with CAE may pose compatibility issues with existing solvers [22]. Ensuring compatibility with existing CAD system geometry transfer formats remains a challenge.

It should be noted that neural networks, including PINNs, are sometimes viewed as “black boxes” that produce results through a process that is not fully controllable. The conservative nature of the industrial design community complicates the adoption of such approaches, particularly in tasks involving critical systems (e.g., critical infrastructure, nuclear energy, aviation). A promising solution is hybrid approaches, where PINNs are used in conjunction with the FEM, with the final calculation controllably performed by FEM solvers. Recent reviews (e.g., Nath et al. [23], Li et al. [24]) confirm that such combined approaches, when using pre-trained networks, significantly (by orders of magnitude) reduce the time required to obtain modeling results, which is critical for design, optimization, and digital twin construction problems.

Considering the features related to AI integration with engineering modeling packages, there is a need to review and analyze research related to PINNs, focusing on the challenges outlined above. While existing reviews summarize key aspects of PINN development, these issues are not fully addressed.

The review by Toscano et al. [14] provides a broad overview of recent algorithmic developments and covers a wide range of PINN applications across various scientific disciplines. The reviews by Cuomo et al. [25] and Farea et al. [26] focus primarily on PINN methodology and applications in different fields, with less emphasis on algorithmic improvements. The review by Raissi et al. [27] provides a concise overview of PINNs and their extensions, with an example of data-driven equation discovery. Ganga and Uddin [28] focus on discussing algorithmic developments primarily related to thermal management and computational fluid dynamics problems. Lawal et al. [29] conducted a bibliometric analysis of publications related to the development of the PINN concept. Additionally, several of the reviews focus on specific application domains of PINNs [4,5,6,7,8,9,10]. Based on the above results, some of the key problems slowing the implementation of PINNs in engineering modeling practice are [30]:

• balancing error components (loss functions)—the total error to be minimized in most studies is the sum of heterogeneous terms with different scales and physical units, which can lead to significant errors;
• accounting for geometry and complex domain topologies—classical PINNs are built on Euclidean coordinates and do not account for complex or parameterized geometries, limiting their application;
• incorporating boundary conditions—in traditional approaches, boundary conditions are imposed through mean-squared penalties, which are “soft” conditions and lead to poor satisfaction at boundaries. Methods for hard enforcement of boundary conditions are not always universal and require complex analytical forms.

Therefore, the goal of this paper is to systematize the current state of these problems and focus on potential approaches to constructing loss functions, methods for incorporating geometric information, and boundary conditions. Our review aims to outline modern barriers and indicate paths for the further development of PINNs as a reliable, scalable, and accurate tool for solving PDE problems.

2. Research Methodology

2.1. Sources and Coverage

The review draws on three complementary bibliographic sources to balance peer-reviewed depth with preprint recency: Scopus (Elsevier), Web of Science Core Collection (Clarivate), and arXiv (Cornell University preprint server). Scopus and Web of Science provide curated indexing for journals and major conferences across engineering, applied mathematics, and computational science, while arXiv captures rapid advances and emergent methods prior to formal publication. The search targeted methodological work on physics-informed learning with an emphasis on loss design, variational/energy formulations, geometry and boundary encoding (e.g., SDF, phi-/R-functions, TFC), sampling and domain decomposition (e.g., cPINN, XPINN, FBPINN, PDD), and hybridization with classical solvers.

2.2. Time Window

Searches were constrained to 1 January 2019–30 August 2025 to capture the maturation of PINNs and their extensions, while remaining focused on contemporary practices relevant for engineering modeling and CAE integration. Where a line of work originated earlier (e.g., R-functions, operator learning), seminal precursors were consulted selectively to clarify method lineage.

2.3. Query Design and Exact Search Strings

Search strings were constructed to (i) include hyphenation variants (“physics-informed”/“physics informed”), (ii) differentiate method acronyms from homonyms (e.g., exclude “pinning”), and (iii) group families of approaches under common umbrellas (PINN/VPINN/E-PINN; operator learning; KAN/PIKAN). Fielded queries and Boolean logic are listed verbatim below; date filters were applied using each database’s UI.

Scopus:

• PINN family (core):
  TITLE-ABS-KEY(“physics-informed neural network*” OR “physics informed neural network*” OR “PINN*” W/0 (PDE OR “boundary condition*” OR physics OR “governing equation*”))

• Variational/energy:
  TITLE-ABS-KEY(“variational physics-informed neural network*” OR “VPINN*” OR “energy-based PINN*” OR “energy PINN*”)

• Operator learning in physics-informed settings:
  TITLE-ABS-KEY(“physics-informed operator*” OR PINO OR “DeepONet” OR “Fourier Neural Operator” OR “FNO”)

• KAN/PIKAN:
  TITLE-ABS-KEY(“Kolmogorov–Arnold network*” OR “Kolmogorov Arnold network*” OR KAN OR PIKAN OR “physics-informed KAN” OR “physics informed KAN”)

• Geometry and BC encoding:
  TITLE-ABS-KEY((“signed distance function*” OR SDF OR “phi-function*” OR “phi function*” OR “R-function*” OR “R function*” OR “transfinite barycentric coordinate*” OR TFC) AND (PINN OR “physics-informed”))

Notes: (a) W/0 (zero-proximity) keeps acronym “PINN*” tied to physical-model phrases to reduce false hits from “pinning”; (b) additional exclusion terms (e.g., AND NOT pinning) were used during screening if noise remained.

Web of Science:

• PINN family (core):
  TS = (“physics-informed neural network*” OR “physics informed neural network*” OR PINN) AND TS = (PDE OR “boundary condition*” OR physics OR “governing equation*”)

• Variational/energy:
  TS = (“variational physics-informed neural network*” OR VPINN* OR “energy-based PINN*” OR “energy PINN*”)

• Operator learning:
  TS = (“physics-informed operator*” OR PINO OR DeepONet OR “Fourier Neural Operator” OR FNO)

• KAN/PIKAN:
  TS = (“Kolmogorov–Arnold network*” OR “Kolmogorov Arnold network*” OR KAN OR PIKAN) AND TS = (“physics-informed” OR “physics informed”)

• Geometry and BC encoding:
  TS = ((“signed distance function*” OR SDF OR “phi-function*” OR “R-function*” OR “transfinite barycentric coordinate*” OR TFC) AND (PINN OR “physics-informed”))

arXiv:

• Consolidated query across titles and abstracts:
  all:(“physics-informed neural network” OR “variational physics-informed neural network” OR “energy-based PINN” OR “physics-informed operator” OR PINO OR DeepONet OR “Fourier Neural Operator” OR FNO OR “Kolmogorov–Arnold network” OR KAN OR PIKAN) AND submittedDate: [1 January 2019 TO 30 September 2025]

• For geometry-specific streams:
  all:((PINN OR “physics-informed”) AND (“signed distance function” OR SDF OR “phi-function” OR “R-function” OR “transfinite barycentric coordinate” OR TFC)) AND submittedDate: [1 January 2019 TO 30 September 2025]

2.4. Inclusion Criteria

Manuscripts were included when they met the following criteria:

• Presented methodological contributions to physics-informed learning (e.g., adaptive loss weighting; variational/energy formulations; geometry/BC encoding; sampling and domain decomposition; hard-constraint constructions; hybrid FEM–PINN/operator pipelines).
• Provided technical detail sufficient for reuse (derivations, algorithmic steps, loss definitions, or implementation sketches).
• Related directly to PDE-based modeling or boundary-value problems in engineering or applied physics.
• Were journal or major-venue conference papers; preprints were included when they introduced novel methods subsequently adopted or discussed by the community.
• Addressed operator learning or KAN/PIKAN in a physics-informed capacity (e.g., PINO with PDE constraints; KAN variants embedding physics in the objective or architecture).

2.5. Exclusion Criteria

Records were excluded when they met the following criteria:

• Reported pure application case studies with routine PINN usage and no methodological novelty.
• Focused primarily on non-PDE ML or image-only tasks without physical constraints.
• Were editorials, short abstracts, theses, or lacked sufficient methodological detail.
• Were duplicates across sources or minor versions of the same preprint without substantive changes.
• Used the acronym PINN in unrelated contexts (e.g., “pinning” phenomena) despite keyword matches.

2.6. Screening and De-Duplication Workflow

Search results were harvested into a reference manager and screened in two passes. Title/abstract screening removed evidently out-of-scope items and acronym collisions. Full-text screening verified methodological relevance against the inclusion criteria and mapped each work to one or more review axes (loss design; variational/energy; geometry/BC; sampling and decomposition; hybridization). De-duplication relied on DOI/arXiv ID and a normalized title key (lowercased, punctuation-stripped, hyphenation harmonized for “physics-informed”). Backward/forward snowballing from seed papers augmented coverage, with priority given to works that introduced reusable formulations or clarified comparative behavior (e.g., residual- vs. energy-based losses; SDF vs. phi/R for geometry).

2.7. Data Extraction and Synthesis

For each included paper, the extraction template captured the objective formulation (loss components, weighting, variational/energy form), the geometry/BC representation (SDF, phi/R, TFC, hybrids), the training protocol (sampling, domain decomposition, curricula), and the integration pathway (stand-alone PINN vs. hybrid FEM/operator). Emphasis was placed on reconstructing the method rather than aggregating performance numbers: equations, constraints, and ablation-style insights were distilled into comparative narratives (e.g., when energy-based losses stabilize training; when φ-based global constraints pair best with SDF-based local enforcement; when XPINN/FBPINN or PDD decompositions reduce stiffness).

3. Loss Functions for PINNs: Concept and Evolution

PINNs are deep neural networks used to solve problems described by differential equations. The idea is that the neural network not only approximates the solution but also satisfies physical laws embedded in the form of equations in the loss function (Figure 3).

The approach to PINNs proposed by Raissi et al. in [1] is formulated as follows. For the problem of solving a system of partial differential equations in the form

$$u_t+\mathcal{N}\left[u;\lambda \right]=0, x\in \mathrm{\Omega }, t\in \left[0,T\right],$$

(1)

where $u(t, x)$ is the sought solution; $\mathcal{N}$ is the governing differential operator; and $\lambda$ are equation parameters, with initial and boundary conditions

$$u(0, x)=u_0\left(x\right), u(t, x)=g(t, x)\mathrm{on} \mathsf{\partial }\mathrm{\Omega },$$

(2)

the neural network $u_{\theta }\left(t, x\right)$ parameterized by weights $\theta$, approximates the function $u(t, x)$. Since the neural network is differentiable, derivatives can be computed automatically and substituted into the following equation:

$$u{f}_{\theta }\left(t, x\right):= u_t+\mathcal{N}\left[u_{\theta }; \lambda \right].$$

(3)

This is the residual of the governing differential equation, which should be approximated to zero for all $x \in \mathrm{\Omega }$. The total loss function according to Raissi et al. [1] is

$$\mathcal{L}\left(\theta \right)={\mathcal{L}}_b+{\mathcal{L}}_f,$$

(4)

where ${\mathcal{L}}_b$ is the residual of compliance with boundary/initial conditions

$${\mathcal{L}}_b=\left({\displaystyle \frac{1}{N_b}}\right){\sum }_{i=1}^{N_b}{\left|u_{\theta }\left(t_b^i, x_b^i\right)-u^i\right|}^2,$$

(5)

${\mathcal{L}}_f$ is the residual of governing equations at internal points of the domain

$${\mathcal{L}}_f=\left({\displaystyle \frac{1}{N_f}}\right){\sum }_{i=1}^{N_f}{\left|f_{\theta }\left(t_f^i, x_f^i\right)\right|}^2,$$

(6)

$N_b$ is the number of points with boundary conditions (known values); $N_f$ is the number of interior points where PDE error is evaluated.

Although this approach (4)–(6) showed good results, the idea of combining loss components with different physical natures and dimensions seems controversial. This can be mitigated by switching from absolute to relative errors, and for additional regularization, introducing weighting coefficients into the loss function, leading to

$$\mathcal{L}={\omega }_d{\mathcal{L}}_d+{\omega }_f {\mathcal{L}}_f+{\omega }_b {\mathcal{L}}_b,$$

(7)

where weighting coefficients $\omega$ determine the relative importance of each loss term. To select the weighting coefficients in (7), various approaches have been proposed, the main examples of which will be discussed further.

3.1. Methods for Balancing Loss Function Components

The representation of the loss function in the form (7) proved sufficiently effective for simple problems, but, as noted by Krishnapriyan et al. [31], increasing regularization in loss functions makes them more complex to optimize, especially for PDE cases with non-trivial coefficients. The Pareto front analysis for loss function coefficients performed by Rohrhofer et al. [32] showed that multi-objective optimization in PINNs is strongly influenced by the natural form of the Pareto front, determined by the parameters of the governing differential equations and the absolute scale of the studied system. It was noted that the accuracy of the solution for a specific case is usually impossible to determine in advance, and convergence issues remain unresolved.

A logical evolution of the idea of physically reinforced loss functions was the introduction of various algorithms for the adaptive adjustment of weighting coefficients [33]. Liu and Wang [34] proposed assigning weighting coefficients in (7) based on a condition that additionally accounts for the initial conditions error:

$${\omega }_d + {\omega }_f + {\omega }_b+{\omega }_{in}=1,$$

(8)

$${\omega }_i={\displaystyle \frac{{\mathcal{L}}_i}{{\mathcal{L}}_d+{\mathcal{L}}_f+ {\mathcal{L}}_b+{\mathcal{L}}_{in}}},i\in \left\{d,f,b,in\right\}.$$

(9)

The choice of weighting coefficients according to (8), (9) equalizes the contribution of loss function components at each subsequent training iteration based on error calculations from the previous one.

Further development of the idea of adaptive determination of weighting coefficients in the loss function was proposed in the study by Liu and Wang [35]. Instead of equalizing contributions of loss function components, the authors proposed formulating the PINN training problem as a min-max optimization problem, where weighting coefficients are automatically determined as dual variables that minimize the maximum error among loss function components:

$$\underset{\theta }{\min } \underset{\alpha }{\max } \mathcal{L}\left(\theta ,\alpha \right) = {\omega }_i\left(\alpha \right) {\mathcal{L}}_i, i\in \left\{d,f,b,in\right\},$$

(10)

where weighting coefficients are determined by softmax-type functions

$${\omega }_i={\displaystyle \frac{exp\left({\alpha }_i\right)}{exp\left({\alpha }_d\right)+exp\left({\alpha }_f\right)+ exp\left({\alpha }_b\right)+exp\left({\alpha }_{in}\right)}}.$$

(11)

To solve the problem (10), (11) in [35], an algorithm for finding high-order saddle points of non-convex–non-concave loss functions was proposed. On a series of test problems, it was shown that its convergence is significantly faster than the original PINN with an adaptive weighting scheme. To overcome the problem of gradient imbalance in loss function components, Xiang et al. [36] proposed an adaptive approach to balancing them (lbPINN). In this method, weights are dynamically adjusted during training to balance the contribution of each component. Instead of manual weight tuning in the loss function, likelihood maximization based on a Gaussian model is used:

$$p\left(u\left(x\right)\right)=N\left(\widehat{u}\left(x;\theta \right),{\epsilon }^2\right),$$

where for each component, uncertainty ${\epsilon }_i$ is a training parameter and minimizes the loss function, assuming errors have a normal distribution with zero mean and variance ${\epsilon }_i^2$

$$\mathcal{L}\left(\theta ,\epsilon ,N\right)={\displaystyle \frac{1}{2{\epsilon }_f^2}}{\mathcal{L}}_f\left(\theta ,N_f\right)+{\displaystyle \frac{1}{2{\epsilon }_b^2}}{\mathcal{L}}_b\left(\theta ,N_b\right)+{\displaystyle \frac{1}{2{\epsilon }_i^2}}{\mathcal{L}}_{in}\left(\theta ,N_{in}\right)+{\displaystyle \frac{1}{2{\epsilon }_d^2}}{\mathcal{L}}_d\left(\theta ,N_d\right)+\log \left({\epsilon }_f{\epsilon }_b{\epsilon }_{in}{\epsilon }_d\right),$$

(12)

where $\epsilon =\left\{{\epsilon }_f,{\epsilon }_{b,}{\epsilon }_i{,\epsilon }_d\right\}$ defines adaptive weighting coefficients for each loss function component as $\omega =1/{2\epsilon }^2$.

The application of an adaptive approach (12) slightly changes the neural network architecture, which takes the form shown in Figure 4 [37]. Note that the diversity of architecture used in PINNs is detailed in [25,26] and remains beyond the scope of this paper. However, the example in Figure 4 emphasizes that new loss function formulations, changes in geometry accounting methods, and boundary conditions typically necessitate changes in PINN architecture.

The approach proposed by Xiang et al. [36] reduces the risk of domination of the loss function of one component, which is especially important in cases when different physical quantities have different scales. Its advantage is the automation of the balancing process. In subsequent papers, it was demonstrated that this method improves convergence for nonlinear PDEs, such as the Navier–Stokes equations [38]) and in multiphysics problems [37].

Unlike the above approaches, McClenny and Braga-Neto [39] proposed a method for adaptive determination of weighting coefficients in the PINN loss function (SA-PINN), where these coefficients are determined not for the loss function components as a whole, but individually for each training point. Thus, these weights are trainable parameters updated during training along with the neural network weights. The loss function for SA-PINN is written as follows:

$$\mathcal{L}\left(w, {\lambda }_f, {\lambda }_b, {\lambda }_{in}\right)={\mathcal{L}}_s\left(w\right)+{\mathcal{L}}_f\left(w, {\lambda }_f\right)+{\mathcal{L}}_b\left(w, {\lambda }_b\right)+{\mathcal{L}}_{in}\left(w, {\lambda }_{in}\right),$$

(13)

where ${\mathcal{L}}_s\left(w\right)$ are losses associated with sensor data (if available); ${\mathcal{L}}_f\left(w,{\lambda }_f\right)=1/2{\sum }_{i=1}^{N_f}m\left({\lambda }_i^f\right){\left|N_{x,t}\left[u\left(x_i^f,t_i^f;w\right)\right]-f\left(x_i^f,t_i^f\right)\right|}^2$ is the PDE loss; ${\mathcal{L}}_b\left(w,{\lambda }_b\right)=1/2{\sum }_{i=1}^{N_b}m\left({\lambda }_i^b\right){\left|B_{x,t}\left[u\left(x_i^b,t_i^b;w\right)\right]-g\left(x_i^b,t_i^b\right)\right|}^2$ is the boundary condition loss; and ${\mathcal{L}}_{in}\left(w,{\lambda }_{in}\right)=1/2{\sum }_{i=1}^{N_{in}}m\left({\lambda }_i^{in}\right){\left|u\left(x_i^{in},t_i^{in};w\right)-h\left(x_i^0\right)\right|}^2$ is the initial condition loss.

Here, $m\left(\lambda \right)$ is a mask function, which is non-negative, differentiable, and strictly increasing on $[0,\infty )$. It transforms weight values ${\lambda }_i$ into multipliers that amplify or weaken the influence of each point on the loss function. A feature of SA-PINN is that weights ${\lambda }_f$, ${\lambda }_b$, ${\lambda }_{in}$ are determined simultaneously with network weights $w$, but with the opposite goal—network weights are updated using gradient descent to minimize the loss function:

$$w_{k+1}=w_k-{\eta }_k{\nabla }_w\mathcal{L}\left(w_k, {\lambda }_k^f, {\lambda }_k^b, {\lambda }_k^{in}\right).$$

Mask weights (${\lambda }_f$, ${\lambda }_b$, ${\lambda }_{in}$) are updated using gradient ascent to maximize the loss function with respect to these weights:

$${\lambda }_{k+1}^f={\lambda }_k^f+{\rho }_k^f{\nabla }_{{\lambda }_f}\mathcal{L}\left(w_k, {\lambda }_k^f, {\lambda }_k^b, {\lambda }_k^{in}\right).$$

Since weights ${\lambda }_f$, ${\lambda }_b$, ${\lambda }_{in}$ are tied to specific training points, the basic SA-PINN is incompatible with SGD, where points are resampled. To overcome this, Gaussian process regression (GP) is used to create a continuous weight map in the spatiotemporal domain. The soft attention mask in SA-PINN allows the neural network to adapt to the task by dynamically redistributing attention to problematic points. Thanks to trainable weights and a flexible mask function, SA-PINN achieves high accuracy and efficiency, surpassing traditional PINN methods.

The SA-PINN method was further improved. Zhang et al. [40] used a subnetwork that inputs and outputs the weights themselves to find their values for each point within the residual component. Anagnostopoulos et al. [41] investigated weighting of residual points and proposed a residual-based attention strategy for updating point weights during training. Finally, Song et al. [42] proposed a generalized approach that uses an attention mechanism for dynamic weight distribution between loss components (LA-PINN). This was motivated by the fact that with SA-PINN, minimizing errors for some points is significantly harder than for others. That is, there may be a natural error bias at these points that should be accounted for during training. The model proposed by Song et al. [42] learns not only the weight for the training point error but also includes a bias to account for the overall complexity of error minimization. The distinction is the introduction of independent loss attention networks (LAN) for each loss function component. Unlike SA-PINN, a separate neural network is used to determine weights for errors in the training points in each component. These networks generate weights using an attention function, allowing them to focus on complex points that are dynamically updated during each training epoch. Numerical experiments on several benchmark partial differential equations showed that LA-PINN outperforms both PINN and SA-PINN in predictive accuracy, under identical baseline settings and number of training epochs. The predicted $L_2$ error is 1–2 orders of magnitude smaller than other methods (Figure 5).

In addition to improving procedures for determining weighting coefficients, efforts continue to improve the loss functions themselves. For data-based loss functions, this primarily involves introducing components containing higher-order derivatives.

Yu et al. [43] proposed a gradient-enhanced PINN (gPINN). The essence of the proposed approach is the assumption that the exact solution of the partial differential equation is sufficiently smooth that the gradient of the PDE residual $\nabla f\left(x\right)$ exists and that, at the exact solution, the PDE residual and its derivatives vanish ($f\left(x\right)=0$ and $\nabla f\left(x\right)=0$). The loss function in gPINN is given as follows:

$$\mathcal{L}=w_f{\mathcal{L}}_f+ w_b{\mathcal{L}}_b+ w_{in}{\mathcal{L}}_{in}+ {\sum }_{i=1}^d{w}_{g_i}{\mathcal{L}}_{g_i}\left(\theta ; {{\rm T}}_{g_i}\right),$$

(14)

where the derivative loss with respect to $x_i$ is as follows:

$${\mathcal{L}}_{g_i}\left(\theta ; T_{g_i}\right)= 1/\left|T_{g_i}\right|{\sum }_{x\in T_{g_i}}{\left|\mathsf{\partial }f/\mathsf{\partial }{x}_i \right|}^2,$$

$T_{g_i}$ is the set of residual points for the derivative $\mathsf{\partial }f/\mathsf{\partial }{x}_i$.

The effectiveness of gPINN was demonstrated in both forward and inverse test problems [43].

Son et al. [44] developed a similar approach based on Sobolev norms, which includes higher-order derivatives in the loss function to ensure smoothness. This method proved particularly effective for problems with sharp gradients or discontinuities, such as wave phenomena or shock waves, where standard PINNs may yield inaccurate results due to insufficient regularity.

In case of areas of complex shape, their decomposition into simple sub-domains may be applied as in Conservative Physics-Informed Neural Networks (cPINN) [45]. For finding solutions in sub-domains, different neural networks can be used. This makes parallelization of calculations possible, which is quite important from the point of view of achieving computational efficiency. In cPINN the modeling result by neural network for each sub-domain is set as follows:

$$u_{i_{\theta }}\left(x\right)= N_i^L\left(x;\theta \right), \forall i = 1, 2, \dots , N_{sd},$$

where $N_{sd}$ is the total number of sub-domains.

The general solution is obtained as follows:

$$u_{\theta }\left(x\right)={\bigcup }_{i=1}^{N_{sd}}{u}_{i_{\theta }}\left(x\right).$$

The loss function is defined by sub-domains similarly to the base PINN formulation but additionally includes losses on interfaces with corresponding weighting coefficients (Figure 6).

The loss function in this case is written in the form

$$\left.\begin{array}{c}{\mathcal{L}}_i={\omega }_d{\mathcal{L}}_d+{\omega }_f{\mathcal{L}}_f+{\omega }_{int}{\mathcal{L}}_{int}\\ {\mathcal{L}}_{int}={\displaystyle \frac{1}{N_{int}}}\sum _{j=1}^{N_{int}}{\left|f_p\left(x_j,t_j\right)·\mathit{n}-f_{p^{+}}\left(x_j,t_j\right)·\mathit{n}\right|}^2+{\displaystyle \frac{1}{N_{int}}}\sum _{j=1}^{N_{int}}{\left|u_p\left(x_j,t_j\right)-\left\{u\left(x_j,t_j\right)\right\}\right|}^2\end{array}\right\},$$

where $\left\{u\left(x_j,t_j\right)\right\}=u_{avg}=:{\displaystyle \frac{u_p+u_{p^{+}}}{2}}$ is the average value $u$ on the interface; $f_p\left(x_j,t_j\right)·\mathit{n}$ and $f_{p^{+}}\left(x_j,t_j\right)·\mathit{n}$ are normal components of flows on the interface, defined by two different neural networks on sub-domains $p$ and $p^{+}$; and $N_{int}$ is the number of training points on the corresponding interface.

Jagtap and Karniadakis in [46] proposed a more general approach, named eXtended PINNs (XPINNs), which expands the limits of cPINNs. Unlike a cPINN, XPINNs can be extended to any type of PDE. In this case the domain can be decomposed both in space and in time, which is impossible with cPINNs. Thus, XPINNs expand the possibilities of parallelization, thereby effectively reducing the costs of neural network training.

Moseley et al. [47] developed the approach of applying local basis functions for PINNs with finite bases (FBPINN). In FBPINN, neural networks are used for training these basis functions. Numerical experiments have shown that FBPINN is effective at solving both small and large multiscale problems, surpassing standard approaches both in accuracy and in required computational resources.

The problem with the noted approaches is the uncertainty of the principles of decomposition of the computational area into subdomains, although obviously this is the key to their effective application. From this point of view, the use of the progressive domain decomposition method (PDD), proposed by Luo et al. [48], may be promising. The idea of PDD consists of segmentation of the domain based on dynamics of residual losses, with definition of critical sections. Such strategic segmentation allows applying adapted neural networks in defined subdomains, each of which can have different levels of complexity.

During the definition of the loss function, the source of errors can be not only inaccurately balanced components, but also uncertainty of material properties, which are used for computing PDE residuals or boundary conditions. As indicated in many studies, such uncertainty often forms the most substantial contribution to the general model uncertainty (Mangado et al. [49], Sun [50], Berggren et al. [51], etc.). Quantitative estimates show that the error contributed by uncertainty of material properties can reach tens of percent (for example [52]). Considering this, formulation of the loss function in the form (7), (10), (12)–(14) is possible only with accurate definition of material properties, which cannot be ensured in many practical cases of engineering modeling. Proceeding from this, to ensure the accuracy of PINN predictions, it is necessary to account for the uncertainty of material characteristics.

From an analysis of modern research, several strategies for including material properties into PINNs can be distinguished. Firstly, scalar parameters are considered as additional optimization variables. Such a possibility was already considered in the paper by Raissi et al. [1], which served as an impetus for intensive research of PINN applications in inverse problems. Considering the noted problems of uncertainty of material properties, this approach is also used for direct (forward) engineering-modeling problems, e.g., Jo et al. [53], Li et al. [54], etc.

In [53], noisy experimental data were integrated with a wide spectrum of guiding equations and initial and boundary conditions. To solve the problems, the general network architecture was improved for thermal conductivity prediction. In [54] a physics-informed neural network was developed for the problem of diffusion coefficient identification. Sensitivity analysis showed the validity of the model for different combinations of certainty conditions of diffusion flow and concentration gradient. Its architecture was also modified to establish the diffusion coefficient during PINN training (Figure 7). The advantages of such an approach to determining material properties are the simplicity of implementation and faster optimization convergence. It is well suited for tasks with globally constant properties. On the other hand, it is not suitable for spatially or temporally varying parameters and faces the problem of local minima with large numbers of unknown properties.

These shortcomings are largely eliminated if uncertain parameters are approximated through separate subnetworks. One of the first publications where such an approach was used was [55], in which two neural networks were used for determining hydraulic conductivity and head. Further studies by Teloli et al. [56] and Kamali et al. [57] confirmed that the use of additional neural networks enables the model to represent spatially and temporally varying parameters, as well as nonlinear constitutive dependencies, which scale better to complex systems.

Figure 8 presents the scheme of a PINN architecture with several additional neural networks, which was applied in [58] for determining the properties of an inhomogeneous material as a spatial function for assessing variations in material integrity and reliability. At the same time, using several neural networks of different types in a single model requires higher computational costs, imposes additional requirements on the amount of data for stable recovery and calibration of characteristics, and is more sensitive to the choice of hyperparameters.

Finally, interest in hybrid approaches is growing, where PINNs are combined with FEMs or ROMs, which provide simultaneous identification of parameters and acceleration of computations. One of the first works where the effectiveness of applying such hybrid models was shown was [59]. Using a series of examples the authors demonstrated the ability to recover missing coefficients and PDE operators from observations obtained directly during the modeling process. Examples of hybrid model applications were given in [60,61]. They considered applications where such hybrid models would allow for identifying parameters, for example, in temperature forecasting and assessment of 3D printing process parameters. In particular, the scheme of the hybrid S-FEM/PINN approach is given (Figure 9).

Hybrid methods show high computational efficiency for direct and inverse tasks; while PINNs integrate physical laws into their neural networks, providing accurate modeling of complex systems, FEM/ROMs reduce computational complexity while preserving accuracy, and inverse PINNs allow the network to determine the material properties (parallel to task solving, using data without manual calibration [62]. Hybrid models adapt to nonlinear and multiphysics problems, which is an undeniable advantage in engineering modeling problems, but combining PINNs with FEM/ROMs requires careful selection of hyperparameters (for example, network architecture, balancing of residuals), which can be labor-intensive and require expertise. Despite the achieved results, even the latest PINN formulations with data-based components and their derivatives do not eliminate the fundamental problem related to the physical interpretation of such methods. Each loss function component in PINNs, such as ${\mathcal{L}}_f$, ${\mathcal{L}}_b$, ${\mathcal{L}}_{in}$ may correspond to quantities with different physical natures and units of measurement. Direct addition of such quantities in the loss function without proper normalization is physically incorrect.

3.2. Application of Variational and Energy Formulations in PINNs

An alternative is writing the loss function based on variational and energy formulations. The main idea of such an approach consists of transition from the direct use of residuals of differential equations (as in standard PINN) to the minimization of energy functionals or use of the weak forms of equations. The main work in which such a possibility was first considered relative to PINNs was the paper by Kharazmi et al. [63], in which the Variational Physics-Informed Neural Networks (VPINN) approach was proposed (Figure 10).

In Figure 10 the red color represents differential operators in the trial space. Green color represents test functions and their derivatives. The blue color represents variational residuals $R$. The trial functions belong to the neural network space, and test functions can be selected from a separate neural network or other functional spaces, such as polynomials and trigonometric functions. Unlike the classical approach, VPINN uses a weak form based on the Petrov–Galerkin method, leading to an integral component in the following loss function:

$${\mathcal{L}}_{\theta } = {\displaystyle \frac{1}{K}}{\sum }_{i=1}^K{\left|{\int }_{\mathrm{\Omega }}{\phi }_i\left(x\right)\left[\mathcal{N}\left[u_{\theta }\right]\left(x\right)-f\left(x\right)\right]dx\right|}^2+ {\displaystyle \frac{w_{b }}{N_u}}{\sum }_{j=1}^{N_u}{\left|u_{\theta }-g\left(x\right)\right|}^2,$$

(15)

where $\mathcal{N}[\cdot ]$ is the differential operator; $u_{\theta }$ is the approximation of the solution using a neural network with parameters $\theta$; $f\left(x\right)$ is the right-hand side of PDE; ${\phi }_i\left(x\right)$ are basis functions; $K$ is the number of basis functions; $N_u$ is the number of basis functions; and $w_{b }$ is the weighting coefficient.

The effectiveness of the proposed approach was demonstrated in several examples, showing clear advantages of VPINNs over PINNs in terms of accuracy and speed. For shallow networks with one hidden layer, Kharazmi et al. [63] analytically obtained various forms of variational residuals, but for deep networks, computing integrals in (15) requires the application of numerical integration, for which additional definition of quadrature rules is necessary. Research in this direction was conducted by Berrone et al. [64,65], who conducted a posteriori-estimates and analysis of convergence for VPINNs, investigated the role of accuracy of numerical quadrature and choice of test-space and analyzed how quadrature rules of different accuracy and piecewise-polynomial basis functions of different degrees influence the convergence rate of VPINNs. According to the data of studies [64,65] it was in particular shown that for smooth solutions the best strategy for achieving high coefficients of error reduction consists of choosing test functions of the lowest polynomial degree in combination with quadrature formulas of high accuracy.

Kharazmi et al. in [63] used globally defined test functions. Other formulations based on variational forms of the problem were developed. In the VarNet variant, proposed by Khodayi-Mehr and Zavlanos [66], test functions were considered piecewise-linear functions of finite element method form, and in the WAN variant [67], an adversarial structure was proposed, where test functions were defined by a separate network. Later Berrone and Pintore [68] proposed using adaptive sets of test functions (Meshfree Variational PINN (MF-VPINN)). For the generation of the test space they used a posteriori error indicator, and test functions were added only where the error was high. Numerical results showed that with this strategy, the solution accuracy was higher than in aVPINN trained with the same number of test functions.

Kharazmi et al. [69] expanded the VPINN idea through the decomposition of the computational domain (hp-VPINN) using the Petrov–Galerkin method with local weighting functions in the form of high-order polynomials, defined on subdomains. Unlike the original VPINN variant, where global polynomials defined on the entire domain were used as test functions [63], in hp-VPINNs each subdomain has its own basis of test functions. This ensures the localization of integrals during the computation of loss functions (integrals in different subdomains are almost non-interconnected), which leads to fewer correlated gradients and more stable training. On a series of test tasks it is confirmed that this allows the network to obtain high accuracy in complex PDEs without the necessity for a global high-order basis, which reduces numerical instability and computational costs. An additional advantage is the ease of implementation through batch-parallelism or multi-GPU.

The mentioned works implement variational approaches to the formulation of loss function based on methods like Petrov–Galerkin (PG), providing a weak form of PDE by integrating the integral from the residual in parts and using different test and trial functions. Greater physical interpretability is ensured by approaches in which, instead of PDE residuals, an energy functional or potential is introduced into the loss function, which means for neural network training, energy minimization is realized. The advantages of this type of method are understandable through physical interpretation: minimal energy corresponds to a stable state of the system. Additionally, the possibility of working with high orders of derivatives without integration by parts is ensured: derivatives from trial functions have already entered the energy functional. It should also be noted that the stability of optimization is ensured by the fact that the energy functional is positive definite, which facilitates gradient-based optimization.

One of the pioneering and most cited works in which an energy approach to formulation of the loss function was applied was the study by Samaniego et al. [70]. The loss function in it is formulated as general potential energy. In the example of solving a number of computational mechanics problem, the advantages of this energy approach compared to the base variant of PINN were shown: better convergence, smaller number of hyperparameters and efficiency for complex geometries without mesh.

In the already-mentioned study by Cuomo et al. [25] a comprehensive review of PINNs is conducted, with focus on energy-based losses as alternative to PDE-residual-based methods. The authors note that loss functions built on energy functionals have fewer hyperparameters and improve generalization for stochastic PDEs and mechanics but may suffer from “null spaces” without data. The article predicts the prospects of applying hybrid approaches with energy-loss for modeling multiphysics tasks, citing [70] as a basis.

In the study by Bai et al. [71], a modified loss function based on Least Squares Weighted Residual (LSWR) is proposed and compared with energy-based and collocation-based losses in PINNs for solid bodies. The energy-based loss showed good stability for linear tasks, but LSWR surpassed it in nonlinear 3D-cases. The authors emphasize that the energy-based approach facilitates the integration of boundary conditions but is sensitive to data scale.

Energy methods are widely applied to forecast the dynamics of systems described by mechanical laws at long time intervals. In these cases, together with PINNs, Lagrangians [72] and Hamiltonians [73] are both widely used. But the most effective use of energy in a PINN may be for related problems [74]. In these cases interaction in different fields is accounted for by integrating all interdependent components into a single energy functional. Instead of computing local residuals (as in residual-based PINNs), the neural network minimizes this single functional, which makes the approach more stable and effective for multiphysics systems.

In the study by Liu and Wu [75], the energy approach to formulating the loss function developed from the idea of decomposing the computational domain. The key proposed innovation was calculating equation residuals in subdomains of the computational domain through numerical integration. Test functions and integral weighting coefficients were implemented in the structure of convolutional filters. The approach was demonstrated in problems with a large number of subdomains and showed high parallel processing capabilities. In this aspect, the method surpassed hp-VPINN. Subsequently, Chen et al. [76], instead of global calculation of the energy functional, applied domain division with a calculation of the sum of local energy integrals, determined using quadratures (Figure 11). To improve convergence, it was proposed that a preliminary simplified model be used so as not to start from an “empty” network, but to give it a physically conditioned initial state.

In

Table 1. Summary and comparison of key architectural variants of PINNs regarding engineering modeling problems.

Architecture Type	Core Concept	Advantages	Limitations
Basic MLP-PINN	Classical multilayer perceptron architecture; loss includes PDE residuals + initial and boundary conditions	Simplicity of implementation; versatility; no mesh required; automatic differentiation	Imbalance of loss function components (different scales of quantities); low convergence for complex PDE; sensitivity to hyperparameters
MLP-PINN with adaptive weight determination (lbPINN, SA-PINN, LA-PINN, etc.)	Weight coefficients in the loss functions are determined dynamically (minimax optimization, softmax, attention mechanisms, probabilistic models)	Automatic balancing of PDE/BC/IC; reduced risk of any single term dominating; improved convergence for nonlinear PDEs; effective for multiphysics problems	Increase in optimization complexity; need for additional parameters (attention models, masks); increased computational cost
MLP-PINN with domain decomposition (cPINN, XPINN, FBPINN, PDD, etc.)	Division of the area into subdomains; separate networks for each subdomain; additional matching conditions on interfaces	Parallelization of calculations; scalability for complex geometries and multiscale tasks; better accuracy	Choice of decomposition principles not always obvious; matching on interfaces complex; possible numerical instabilities
Variational/energy PINN (VPINN, E-PINN, etc.)	Use of weak form PDE (Petrov–Galerkin) or minimization of energy functional instead of PDE residuals	Better interpretability; fewer hyperparameters; more stable optimization; efficiency for multiphysics problems	Need for numerical integration (quadrature rules); dependence of accuracy on test space; possibility of null spaces
Variational/energy PINN with domain decomposition (hp-VPINN, E-PINN with local integrals, etc.)	Decomposition of the area into subdomains with local test functions or local energy integrals	Fewer correlated gradients; more stable training; effective parallel implementation; high accuracy for complex PDE	Increased implementation complexity; need for correct subdomain localization; additional costs for integration

, summary information is provided that can be used for preliminary assessment of the efficiency of applying the considered PINN architectures for engineering modeling problems.

Separately, we emphasize the usefulness of including material characteristics, which are components of the physical model of the task, into the composition of neural network training parameters [53,54,55,56,57,58,59,60,61]. This is especially relevant for engineering modeling tasks, where it is necessary to account for uncertainty in structural material properties (

Table 2. Typical levels of uncertainty for some properties of structural materials according to ASTM standards.

Material	Property	Typical Uncertainty (COV, %)	Standard
Steel (structural)	Yield strength $\left({\sigma }_y\right)$	≈7–8%	ASTM A370-24. Standard Test Methods and Definitions for Mechanical Testing of Steel Products [77]
Composites (fiber-reinforced polymers)	Tensile strength along fibers $\left({\sigma }_t\right)$	≈10–20%	ASTM D3039/D3039M-14. Standard Test Method for Tensile Properties of Polymer Matrix Composite Materials [78]
Concrete	Compressive strength $\left(f_c\right)$	≈15–25%	ASTM C39/C39M-21. Standard Test Method for Compressive Strength of Cylindrical Concrete Specimens [79]

), as well as the environment acting on the modeled structures.

Since multiphysical, including coupled, tasks are quite often encountered in engineering modeling,

Table 3. Comparison of key PINN variants regarding multiphysics problems.

Architecture Type	Suitability for Multiphysics	Advantages
Basic MLP-PINN	Limited	Simplicity; quick start for simple PDEs
MLP-PINN with adaptive weight determination (lbPINN, SA-PINN, LA-PINN, …)	High	Automatic balancing of losses of different physical nature; improved convergence; capability for nonlinear and coupled PDEs
MLP-PINN with domain decomposition (cPINN, XPINN, FBPINN, PDD, …)	High	Separate networks can be assigned for different physics in different regions; good scalability and parallelism
Variational/energy PINN (VPINN, E-PINN, …)	Very high	A single energy functional naturally integrates different physics; physically consistent formulation
Variational/energy PINN with domain decomposition (hp-VPINN, E-PINN)	Highest	Combine the physical interpretability of energy-based loss with local adaptation; work well in strongly coupled systems

provides a separate comparison of the capabilities of PINNs of different architectures for solving this type of task.

Summarizing the conducted analysis, we note that variational methods use the integral form of PDE, which involves reformulating them into weak formulations and including numerical integration schemes, such as the Petrov–Galerkin or Ritz methods. This requires separate accounting for essential boundary conditions, which are not included in the minimized functional. Applying energy approaches to forming the loss function also requires the correct inclusion of essential and natural boundary conditions either through additional terms or through the requirement of strict compliance with boundary conditions.

Thus, accounting for boundary conditions and geometric information regarding the computational domain are important for the further development of PINN-based methods and require separate consideration.

4. Incorporating Geometry and Boundary Conditions in PINNs

PINNs are a mesh-free method, where the problem geometry (domain $\mathrm{\Omega }$ and boundary $\mathsf{\partial }\mathrm{\Omega }$) is handled through discrete points. Proper handling of geometric information ensures training accuracy, effective incorporation of physical laws (PDE), and boundary conditions (BCs). The sequence of operations with geometric information may vary depending on geometry complexity but generally includes the following steps:

• definition of domain geometry and boundary: the computational domain $\mathrm{\Omega }$ is described, and the boundary $\mathsf{\partial }\mathrm{\Omega }$ is divided into parts for different types of boundaries conditions;

• generation and filtering of collocation points inside the domain and on its boundary, removal from the selection of points that are outside the selection or outside $\mathrm{\Omega }$, division of boundary points by types of boundaries conditions;
• integration of geometry into the loss function, if necessary—construction of functions for strict enforcement of defined types of boundary conditions.

In accordance with this sequence, we will consider the methods used to implement these stages in research related to PINN-based modeling. During the analysis, we will additionally evaluate the possibilities of performing these tasks using tools available in CAD/CAE systems.

4.1. Methods for Incorporating Geometric Information in PINN

The simplest way to determine whether points are located inside the domain or on its boundary can involve various computational geometry algorithms: PIP-type algorithms such as ray casting, winding number [80]), or mesh-based methods [81]. Such methods essentially act as predicate functions, determining whether a point is in the computational domain or outside it. Therefore, in practice, PINN modeling employs analytical methods for incorporating geometric information, which can not only be used in the sampling process but also serve as a tool for constructing the loss function.

The primary such analytical method is the use of Signed Distance Functions (SDF). SDF is defined as the shortest distance from a point $x$ to the domain boundary $\mathsf{\partial }\mathrm{\Omega }$, with a sign indicating whether the point $x$ lies inside ($\phi \left(x\right) < 0$) or outside ($\left(x\right) > 0$) the domain and equals zero on the boundary. Formally, $\phi \left(x\right)= sgn\left(x\right)·{inf}_{y\in \mathsf{\partial }\mathrm{\Omega }}‖x-y‖$, where $sgn\left(x\right)$ is the sign function depending on the point’s location relative to $\mathrm{\Omega }$. Prior to the advent of PINNs, SDFs were already a key tool in geometric modeling, robotics, and computer graphics. This was facilitated by the convenience of describing complex bodies and surfaces, as well as mesh independence. From the SDF definition follows the equality for its ideal (differentiable everywhere) form—the Eikonal equation:

$$\left|\nabla \phi \left(x\right)\right|=1 {x\in \mathbb{R}}^n\backslash \mathsf{\partial }\mathrm{\Omega };, \phi {|}_{\mathsf{\partial }\mathrm{\Omega }}=0.$$

(16)

Numerical methods (Fast Marching, Fast Sweeping, etc.) form the core of classical approaches to computing SDF. An overview of the corresponding concepts is systematized in the papers by Sethian [82,83]. Based on this, several methods have been developed for constructing SDF for domains of complex shapes, differing in accuracy, computational costs, and frequency of usage.

The most popular among them is the analytical method using geometric primitives. It is applied when the form of the computational domain can be created by applying Boolean operations to geometrically simple objects for which SDF exists in analytical form. Such functions are used as building blocks for the SDF of the entire domain using operations such as union ${\phi }_{\cup }\left(x\right) = min\left({\phi }_A\left(x\right), {\phi }_B\left(x\right)\right)$, intersection ${\phi }_{\cap }\left(x\right) = max\left({\phi }_A\left(x\right), {\phi }_B\left(x\right)\right)$, and difference ${\phi }_{\backslash }\left(x\right) = max\left({\phi }_A\left(x\right), {-\phi }_B\left(x\right)\right)$ (e.g., [84]). The method surpasses all others in simplicity, computational costs, accuracy, and frequency of use. The drawback is the limitation to applications where the domain can be constructed from regions with available analytical SDF. Additionally, the $min$ and $max$ functions are not smooth on lines/hypersurfaces where the arguments ${\phi }_A\left(x\right), {\phi }_B\left(x\right)$ are equal. To overcome this limitation in PINN modeling, either smooth approximations like $softmin$ or $softmax$ can be used, or a transition to subgradients can be made [85].

One of the classical methods is a direct distance computation from triangular meshes, where the unsigned distance to each triangle is obtained by projection onto the plane while accounting for regions near vertices, edges, and faces, and the sign is determined using surface normals or angle-weighted pseudonormals, $N = \left({\sum }_i{n}_i{\theta }_i\right)/\left|\right|{\sum }_i{n}_i{\theta }_i\left|\right|$, where $n_i$ are face normals and ${\theta }_i$ are the angles. This approach, detailed in the survey by Jones et al. [86], emphasizes accuracy and efficiency thanks to hierarchical structures such as octrees, which reduce the complexity from $O\left(M\right)$ to $O(log M)$ for $M$ triangles. The method integrates well with modern CAD systems because all of them are compatible with the STL format. STL represents an object’s surface as a set of triangles specified by their vertex coordinates which are, typically, face normal—i.e., everything needed to construct an SDF using this method [87]. Several libraries, such as Open3D or libigl, allow direct STL import and SDF computation for PINNs or other applications. For complex geometries with an excessive number of facets, mesh simplification may be required to reduce the computational cost of the SDF. The method is characterized by high accuracy and, for these reasons, is widely used, but it entails substantial computational expense.

SDF can be found as the solution to the boundary value problem for the Eikonal equation $|\nabla \phi |=1$ with boundary condition $\phi {\mid }_{\mathsf{\partial }\mathrm{\Omega }}=0$. In practice, this equation is solved numerically on a grid using ordered methods: Fast Marching (FMM)—a one-pass algorithm similar to Dijkstra for continuous distances; Fast Sweeping—an iterative method using passes in multiple directions and effective for regular grids. Both classical approaches have well-developed theory and are applied in many CAD/CAE systems [82,83]. Fayolle [84] proposed a method for redefining the distance to an implicit surface. For function $f$, whose zero level defines the geometric surface $S$, SDF was determined by embedding $f$ in the last layer of a deep neural network trained on a loss function derived from the Eikonal equation. These methods are applied less frequently, have accuracy issues in high-curvature zones, and require moderate computational resources.

Neural methods, such as DeepSDF, represent SDFs as continuous functions trained on point samples, where the neural network $f_{\theta }\left(x\right)$ is optimized using the loss function $L\left(f_{\theta }\right(x), s) = \left|clamp\right(f_{\theta }\left(x\right), \delta ) - clamp(s, \delta \left)\right|$, where $clamp(z, \delta ) = max(-\delta , min(\delta , z\left)\right)$ which limits $z$ values within $[-\delta , \delta ]$, $s$ is the true SDF value, and $f_{\theta }\left(x\right)$ is the predicted value, enabling interpolation and form completion for object classes. Park et al. [88] demonstrate this method’s advantage in compression and reconstruction quality over mesh representations, although training requires large datasets and may suffer from overfitting to specific shapes.

One of the limitations of neural approaches is the need for ground-truth signed values or directed surface normals. To address this, Sign-Agnostic Learning (SAL, [89]) emerged, allowing implicit representation training even from “unsigned” data (e.g., simple point clouds), and Neural-Pull [90], which trains the network to “pull” space points to the surface using predicted distances and gradients as steps for query point movement. These approaches significantly expand the dataset for direct training of SDF-parametric models and enable the training of differentiable SDFs without explicit signed labels. Neural network-based methods require very high computational costs during training but operate extremely fast afterward, providing the highest accuracy for complex geometric forms.

Sukumar and Srivastava [91] explored the use of another method in PINN modeling that accounts for computational domain geometry at the analytical level—the R-functions method. The idea of this approach is to use functions that are algebraic but have properties of logical algebra functions in the sense that their sign is fully determined by the sign of their arguments. Such functions ($R$-functions) were first proposed by Rvachev [92] for use in solving PDE problems with variational methods. An $R$-function is a real-valued function $\Phi \left(u_1,u_2,\dots ,u_n\right)$ that has the same sign structure as some Boolean function $B\left({\xi }_1,{\xi }_2,\dots ,{\xi }_n\right)$, ${\xi }_i\in \left\{-1,+1\right\}$. That is, if the Boolean function $B$ is true for a certain set of signs $u_i$, then $\Phi >0$; if false—then $\Phi <0$. Using $R$-functions, analytical functions describing domains of arbitrary geometric shapes can be constructed. For example, for two domains described as ${\mathrm{\Omega }}_1:f_1(x,y)\ge 0$, ${\mathrm{\Omega }}_2:f_2(x,y)\ge 0$, with a complete system of $R$-functions can be defined as follows [93]:

$$\left.\begin{array}{c}\mathrm{intersection} {\mathrm{\Omega }}_1\cap {\mathrm{\Omega }}_2 \mathrm{as} R_{\wedge }\left(f_1,f_2\right)=f_1+f_2-\sqrt{f_1^2+f_2^2},\\ \mathrm{union} {\mathrm{\Omega }}_1\cup {\mathrm{\Omega }}_2 \mathrm{as} R_{\vee }\left(f_1,f_2\right)=f_1+f_2+\sqrt{f_1^2+f_2^2},\\ \mathrm{omplement} {\overline{\mathrm{\Omega }}}_1 \mathrm{as} R_{\neg }\left(f_1\right)=-f_1.\end{array}\right\}$$

(17)

Thus, having a set of geometric primitives $f_i:{\mathbb{R}}^n\to \mathbb{R}\left(i=1,\dots ,k\right)$ with domains ${\mathrm{\Omega }}_i=\{x:f_i(x)\ge 0$} and a Boolean formula $B:{\left\{-1,+1\right\}}^k\to \{-1,+1\}$ describing the target domain $\mathrm{\Omega }$ through operations $\wedge ,\vee ,\neg$ over ${\mathrm{\Omega }}_i$, using $R$-function ${\Phi }_B:{\mathbb{R}}^n\to \mathbb{R}$, a function $\phi \left(x\right)={\Phi }_B\left(f_1\right(x),\dots ,f_k(x\left)\right)$ with which properties can be constructed:

$$\forall x\in {\mathbb{R}}^n: \left\{\begin{array}{c}\phi \left(x\right)>0⟺x\in int \mathrm{\Omega }, \\ \phi \left(x\right)=0⟺x\in \mathsf{\partial }\mathrm{\Omega }, \\ \phi \left(x\right)<0⟺x\in {\mathbb{R}}^n\backslash \overline{\mathrm{\Omega }}.\end{array}\right.$$

(18)

The function $\phi \left(x\right)=0$, constructed from geometric primitives in implicit form using system (17) or its analogs, is analytical everywhere in $\mathrm{\Omega }$, except on curves/surfaces where the arguments are equal to $f_1 = f_2 = 0$, where the derivatives are bounded but vary depending on the direction. In addition to (17), other complete sets of R-functions exist that can be applied for the analytical description of regions with complex shapes [92,94], which possess different properties of smoothness and differentiability:

$$\left.\begin{array}{c}{R}_{{\wedge }_2}\left(f_1,f_2\right)=f_1·f_2/\sqrt{f_1^2+f_2^2}\\ R_{{\vee }_2}\left(f_1,f_2\right)={-f}_1·f_2/\sqrt{f_1^2+f_2^2}\end{array}\right\},$$

(19)

• $C^1$-continuous (first derivative exists) but not smooth in sign-change zones;

$$\left.\begin{array}{c}{R}_{{\wedge }_{\alpha }}\left(f_1,f_2\right)=\left(1/\left(1+\alpha \right)\right)·\left(f_1+f_2-\sqrt{f_1^2+f_2^2-2\alpha f_1{f}_2}\right)\\ R_{{\vee }_{\alpha }}\left(f_1,f_2\right)=\left(1/\left(1+\alpha \right)\right)·\left(f_1+f_2+\sqrt{f_1^2+f_2^2-2\alpha f_1{f}_2}\right)\end{array}\right\},$$

(20)

• for $0<\alpha <1$ is smooth almost everywhere, approximates the distance function without singularities; parameter $\alpha$ controls the “thickness” of medial zones;

$$\left.\begin{array}{c}{R}_{{\wedge }_m}\left(f_1,f_2\right)={\left(f_1^2+f_2^2\right)}^{m/2}·\left(f_1+f_2-\sqrt{f_1^2+f_2^2}\right)\\ R_{{\vee }_m}\left(f_1,f_2\right)={\left(f_1^2+f_2^2\right)}^{m/2}·\left(f_1+f_2+\sqrt{f_1^2+f_2^2}\right)\end{array}\right\},$$

(21)

• m-times differentiable everywhere with vanishing derivatives in singularities.

The systems of functions (17), (19)–(21) are not only used for describing the geometry of the computational domain in the form (18). The main idea of the R-functions method lies in the possibility of creating bundles of functions with uses that identically satisfy arbitrary boundary conditions in PDE problems, termed by Rvachev [92] as solution structures. To this end, the concept of a function $\phi \left(x\right)$ normalized to order $k$, is introduced. According to [92], such a function has additional properties:

$${\displaystyle \frac{{\mathsf{\partial }}^m\phi \left(x\right)}{\mathsf{\partial }{n}^m}}={\delta }_{m1}, \forall x\in \mathsf{\partial }\mathrm{\Omega }, m=\mathrm{1,2},\dots ,k,$$

(22)

where $\mathsf{\partial }/\mathsf{\partial }n$ denotes the derivative in the direction of the outward normal to the boundary, and ${\delta }_{m1}$ is the Kronecker symbol (equals 1 for $m=1$ and 0 for $m\ne 1$).

In [92], it is shown that if $\mathrm{\phi }\left(\mathrm{x}\right)\in C^m\left({\mathbb{R}}^n\right)\to \mathbb{R}$ satisfies conditions ${\left.\mathrm{\phi }\left(\mathrm{x}\right)\right|}_{\mathsf{\partial }\mathrm{\Omega }}=0$, ${\left.\mathsf{\partial }\mathrm{\phi }\left(\mathrm{x}\right)/\mathsf{\partial }n\right|}_{\mathsf{\partial }\mathrm{\Omega }}>0$, then a first-order normalized function can be constructed as follows:

$${\phi }_1\left(\mathrm{x}\right)\equiv \mathrm{\phi }{\left({\mathrm{\phi }}^2+{\left(grad \mathrm{\phi }\right)}^2\right)}^{-1/2}\in C^{m-1}\left({\mathbb{R}}^n\right).$$

(23)

Constructing normalized functions using expression (23) may require additional computational costs, but these can be eliminated if normalized equations for geometric primitives are used for constructing complex geometry. Operations (17), (19)–(21) over them preserve the normalization of the result. For geometric primitives, normalized functions in implicit form can be constructed by simple multiplication by some constant.

In addition, note that if $\mathsf{\partial }\mathrm{\Omega }=\left(\phi =0\right)=0$ is a first-order normalized equation of the boundary $\mathsf{\partial }\mathrm{\Omega }$, and $\mathrm{\Sigma }=(\sigma \ge 0)$ is some region, then

$$\phi 1\equiv \sqrt{{\phi }^2\vee \overline{\sigma }}=0,$$

(24)

where $\vee$ is one of the R-disjunctions from systems (17), (19)–(21) and is a first-order normalized equation of the element $\mathsf{\partial }{\mathrm{\Omega }}_1=\mathsf{\partial }\mathrm{\Omega }\cap \mathrm{\Sigma }$ [92].

Sukumar and Srivastava in [91] applied such an approach to define approximate distance functions (ADF) for boundary segments. Using SDF as the base and a first-order normalized equation of the corresponding region as the trimming function, they obtained functions that satisfied conditions ${\phi }_i\left(x\right)=0⟺x\in \mathsf{\partial }{\mathrm{\Omega }}_i$, and approximately defined the distance to $\mathsf{\partial }\mathrm{\Omega }$ (Figure 12).

Finally, if ${\phi }_1=0,{\phi }_1\in C^m\left(\mathrm{\Omega }\cup \mathsf{\partial }\mathrm{\Omega }\right)$ is a first-order normalized equation of $\mathsf{\partial }\mathrm{\Omega }$, then normalized equations of higher orders $\mathsf{\partial }\mathrm{\Omega }$ are determined by recursive relations:

$$\left.\begin{array}{c}{ \phi }_2\left(x\right)={\phi }_1-{\displaystyle \frac{1}{2!}}{\phi }_1^2{D}_2{\phi }_1\\ \dots \dots \dots \dots \\ {\phi }_n\left(x\right)={\phi }_{n-1}-{\displaystyle \frac{1}{n!}}{\phi }_1^n{D}_n{\phi }_{n-1}\end{array}\right\},$$

(25)

where $D_n\left(·\right)=\sum _{\left|\alpha \right|=m}{\displaystyle \frac{m!}{\alpha !}}\prod _{i=1}^n{\left({\displaystyle \frac{\mathsf{\partial }\phi }{\mathsf{\partial }{x}_i}}\right)}^{{\alpha }_i}{\displaystyle \frac{{\mathsf{\partial }}^n\left(·\right)}{\mathsf{\partial }{x}_1^{{\alpha }_1}\dots \mathsf{\partial }{x}_n^{{\alpha }_n}}}$ is the operator of continuation inside the region $\mathrm{\Omega }$ of the operation of n-fold differentiation along the normal to $\mathsf{\partial }\mathrm{\Omega }$ [92].

An alternative approach to analytically describing complex geometries in PINNs involves the use of phi-functions, originally developed by Stoyan for geometric modeling and object placement problems [95], and further elaborated in numerous studies (e.g., [96,97,98,99,100,101,102]). Unlike SDFs, which encode the distance from a point to the boundary, phi-functions describe the relative positions of two objects through a functional dependent on their parameters, such as distances between surfaces or the degree of overlap. Phi-functions analytically represent placement constraints, including non-overlap, containment, prohibited zones, and allowable distances, for arbitrarily shaped objects that may be connected or disconnected, convex or non-convex, may include holes, cavities, and admit special transformations (e.g., translations/rotations, scaling, stretching/deformation).

For the readers’ convenience, basic definitions of different types of phi-functions are provided below. Let us consider two geometric objects $A\subset {\mathbb{R}}^d$ and $B\subset {\mathbb{R}}^d(d=\mathrm{2,3})$. The position of $A$ ($B$) is defined by a variable motion vector $u_A$ ($u_B$). Phi-functions allow us to distinguish the following three cases: $A$ and $B$ are intersecting, i.e., $A$ and $B$ have common interior points; $A$ and $B$ are in contact, i. e. $A$ and $B$ have only common boundary points; $A$ and $B$ do not intersect, i. e. $A$ and $B$ do not have common points.

Definition 1 

([96]). A continuous and everywhere-defined function phi-function Φ^AB(u_A, u_B) that meets the following properties:

$${\Phi }^{AB}<0 \mathrm{for} \mathit{int}A\cap \mathit{int}B\ne \varnothing ,$$

$${\Phi }^{AB}=0 \mathrm{for} \mathit{int}A\cap \mathit{int}B=\varnothing \mathrm{and} frA\cap frB\ne \varnothing ,$$

$${\Phi }^{AB}>0 \mathrm{for} A\cap B=\varnothing ,$$

where int(.), int(.) denote the topological interior and boundary of geometric object (.).

The inequality ${\Phi }^{AB}(u_A,u_B)\ge 0$ holds the non-overlapping condition, i.e., $\mathit{int}A\cap \mathit{int}B=\varnothing$, while inequality ${\Phi }^{A{B}^{*}}(u_A,u_B)\ge 0$ ensures the containment condition $A\subset B$, i.e., $\mathit{int}A\cap \mathit{int}{B}^{*}=\varnothing$, where $B^{*}={\mathbb{R}}^d\backslash \mathit{int}B.$

While the sign of the phi-function plays a crucial role, its absolute value is not subject to any rigid requirements. If two objects $A$ and $B$ overlap, then $\Phi (u_A,u_B)<0$ and the absolute value $\left|\Phi \right(u_A,u_B\left)\right|$ can just roughly define the extent of overlap. For non-overlapping objects $\Phi (u_A,u_B)>0$, and the value of $\Phi (u_A,u_B)$ may just roughly correspond to the distance between $A$ and $B$.

When objects do not overlap, one can set $\Phi (u_A,u_B)=dist(A,B)$, where

$$dist\left(A,B\right)=\underset{X\in A,\mathit{Y}\in B}{\min }dist(X,Y),$$

denotes the Euclidean distance between two phi-objects.

Let minimal allowable distance $\rho >0$ between objects be given.

Definition 2 

([96]). A phi-function ${\widetilde{ \Phi }}^{AB}(u_A,u_B)$ is called the normalized phi-function for objects A and B if the values of the function coincide with the Euclidean distance between objects A and B when $\mathit{int}A\cap \mathit{int}B=\varnothing$.

The inequality ${\tilde{\Phi }}^{AB}(u_A,u_B)\ge \rho$ provides the distance condition for objects $A$ and $B$.

In many cases the formula for the distance involves radicals, which makes it difficult for local optimization algorithms to use phi-function and its derivatives. In those cases, phi-function should be defined by a simpler formula which holds the allowable distance between the objects. However, if distance conditions must be considered, other forms of phi-functions are needed, because the values of the phi-functions above do not coincide with the Euclidean distances between the corresponding shapes when they do not overlap. To simplify the distance constraints an adjusted phi-function can be applied.

Definition 3 

([96]). A continuous and everywhere-defined function ${\stackrel{⏜}{\Phi }}^{AB}(u_A,u_B)$ based on variables $u_A,u_B$ is called an adjusted phi-function for objects A and B if

$${\stackrel{⏜}{\Phi }}^{AB}>0 \mathrm{for} dist(A,B)>\rho ,$$

$${\stackrel{⏜}{\Phi }}^{AB}=0 \mathrm{for} dist(A,B)=\rho ,$$

$${\stackrel{⏜}{\Phi }}^{AB}<0 \mathrm{for} \mathit{int}A\cap \mathit{int}B<\rho .$$

The inequality ${\stackrel{⏜}{\Phi }}^{AB}(u_A,u_B)\ge 0$ ensures the distance condition, i.e., $dist(A,B)\ge \rho$, while inequality ${\stackrel{⏜}{\Phi }}^{A{B}^{*}}(u_A,u_B)\ge 0$ provides the distance condition $dist(A,B^{*})\ge \rho$, i.e., where $B^{*}={\mathbb{R}}^d\backslash \mathit{int}B$.

For some shapes under motion, an alternative to phi-functions, the use of quasi phi-functions, allows us to simplify descriptions of non-overlapping constraints at the expense of a larger number of variables.

Definition 4 

([96]). A continuous and everywhere-defined function ${{\Phi }^{\prime }}^{AB}(u_A,u_B,u\prime )$ is called a quasi-phi-function for objects $A\left(u_A\right)$ and $B\left(u_B\right)$ if $\underset{u^{\prime }\in {\mathbb{R}}^n}{ max}{{\Phi }^{\prime }}^{AB}(u_A,u_B,u^{\prime })$ is a phi-function for these objects.

The non-overlapping constraint for objects $A$ and $B$ can be described as follows: if ${{\Phi }^{\prime }}^{AB}(u_A,u_B,u^{\prime })\ge 0$ for some $u^{\prime }$, then $\mathit{int}A\cap \mathit{int}B=\varnothing$.

Definition 5 

([96]). A continuous and everywhere-defined function ${{\tilde{\Phi }}^{\prime }}^{AB}(u_A,u_B,u\prime )$ is called a normalized quasi-phi-function for objects $A\left(u_A\right)$ and $B\left(u_B\right)$ if $\underset{u^{\prime }\in {\mathbb{R}}^n}{ max}{{\tilde{\Phi }}^{\prime }}^{AB}(u_A,u_B,u^{\prime })$ is a normalized phi-function for these objects.

The phi-function technique is a versatile tool for describing placement problems as mathematical programming models. It leverages nonlinear programming (NLP) for open dimension problems and mixed-integer nonlinear programming (MINLP) for knapsack problems, representing the solution space as a union of subregions defined by smooth inequalities derived from the phi-function expressions. Constructed using min-max operators, phi-functions are continuous and piecewise smooth, which enables efficient modeling of placement problems while supporting fast, feasible solutions through smart heuristics. The phi-function technique is widely applied to solve different classes of practical problems that arise in nanotechnologies, materials science, aerospace engineering and additive manufacturing [103,104,105,106,107,108].

In the context of PINNs, phi-functions offer flexible representations for complex multi-body geometries because they naturally encode intersection and non-intersection relations. In practice, they can enter the learning objective either as penalization terms that discourage interpenetration and boundary violations or as analytical building blocks for hard-constraint ansätze. However, standard phi-functions are continuous but only piecewise smooth (built via min/max), and they do not provide a local metric like SDF (i.e., no direct distance-based normals), which complicates gradient-based training for tasks requiring Neumann data. This is why a hybrid strategy is recommended: use phi-functions for global placement constraints and SDF for accurate local boundary enforcement and normals. From a learning perspective, this role of phi-based geometric penalties is analogous to recent reinforcement-learning setups for packing, where dense rewards directly encode minimum-distance and non-overlap constraints; see Yamada et al. [109] for an example with PPO agents optimizing the maximum pairwise/boundary distance $\rho$ and explicitly comparing against a phi-function-based baseline.

A hybrid approach combining phi-functions for global geometric constraints with SDF for precise local boundary specification is most promising. This strategy leverages phi-functions’ strengths in handling complex multi-body configurations and optimization tasks while using SDF to ensure accurate boundary condition enforcement.

For instance, in optimization-driven PINN applications, such as shape optimization, phi-functions can serve as a natural functional to enforce global placement constraints while SDF ensures local geometric accuracy. The comparison between SDF and phi-functions in the context of PINNs is summarized in

Table 4. Comparison of SDF and Phi-functions in the Context of PINNs.

Criterion	SDF	Phi-Function
Meaning	Distance from point to boundary (with positive/negative sign).	Continuous function of mutual position of two bodies (intersection, tangency, separation).
Ease of Geometry Specification	Suitable for single objects or smooth boundaries.	Ideal for multi-object systems and complex combinations (intersections, unions).
Local Geometry	Provides surface normal via gradient (∇SDF).	Normals not directly extracted; boundary defined implicitly via Φ = 0.
Smoothness	Smooth and well-differentiable with proper approximation.	Piecewise smooth function. Contains operations (min-max), less suitable for backpropagation.
Applicability in PINNs	Excellent for boundary conditions (Dirichlet/Neumann).	Better for global geometric constraints (non-intersection, object placement).
Interpretation	Metric: “distance to boundary.”	Phi-function: “degree of intersection/ separation”. Normalized phi-function: Euclidean distance between objects.
Flexibility for Complex Domains	Requires specialized methods (e.g., CSG with SDF, implicit surfaces).	Naturally describes combinations and relative objects positions.
Computational Properties	Direct approximation, good training stability.	More complex computations, potential gradient instability.

.

In addition to the theory of R-functions, Sukumar and Srivastava in [91] considered another approach to constructing approximate distance fields based on the theory of mean value potential fields and related to transfinite barycentric coordinates (TFC) [110]. TFC are constructed through harmonic functions or other generalized interpolation schemes that preserve the properties of classical coordinates, and the computational domain is defined through these coordinates as an analytical combination of local primitives. TFC are defined as analytically smooth functions throughout the domain, which practically eliminate the risk of “gradient explosion”. At the same time, TFC can only be applied to regions that can be described by a polygon/polyhedron, limiting their usage possibilities. Information regarding the advantages and disadvantages of the methods described for incorporating geometric information is summarized in

Table 5. Summary and comparison of methods for incorporating geometric information in modeling using PINN.

Method	Key Idea/Essence	Advantages	Disadvantages
PIP algorithms (Ray Casting, Winding Number, mesh-based)	Use predicate functions to test whether a point belongs to the domain/boundary	Simplicity; basic point-membership checks; integrates with sampling	Do not provide smooth information; limited use inside loss functions; ignore curvature
Analytical SDF from geometric primitives	Build complex shapes by combining simple analytic SDFs (min/max)	Simple implementation; high accuracy; low computational cos	Limited to primitives with analytic SDF; non-smoothness at switching surfaces (min/max)
SDF from triangular meshes (STL-based)	Distance computed from CAD mesh facets; sign from (pseudo)normals	CAD/CAE compatibility; high accuracy; library support (Open3D, libigl)	High computational cost; challenging for very large/fine meshes
SDF via Eikonal (Fast Marching, Fast Sweeping)	Numerical solution of $\mid \mathit{\nabla }\phi \mid =1$ with φ = 0 on $\mathsf{\partial }\mathrm{\Omega }$	Smooth fields without gradient blow-up	Limited to polygon/polyhedron-type domains
Neural SDF approaches (DeepSDF, SAL, Neural-Pull)	Train a neural network to approximate a continuous SDF from point samples/unsigned data	Highest flexibility; handles complex shapes; smooth differentiable fields; fast inference after training	High training cost; large datasets needed; potential overfitting
R-functions	Construct implicit geometry via algebraic functions with logical-algebra properties	Universality; analyticity; can form solution structures respecting boundary conditions	Mathematical complexity; limited CAD/CAE integration at present
Phi-functions	Continuous functions encoding the mutual position of two bodies (intersection, tangency, separation); usable as penalties in loss or as analytical building blocks for hard-constraint trial forms	Natural for multi-object configurations and optimization/packing; enforce non-overlap and placement constraints; pair well with SDF in hybrid schemes	No local metric (normals) like SDF; piecewise-smooth due to min/max → can hinder backpropagation; requires smoothing (e.g., R-operators/soft-min)
Transfinite Barycentric Coordinates (TFC)	Mean-value/harmonic-coordinate–based analytic fields; domain defined as an analytical combination of local primitives; smooth everywhere	Analytically smooth (reduces gradient-explosion risk)	Applicable only to polygon/polyhedron-type domains; expensive for complex STL with many faces

.

Considering the features of the approaches described, let us examine the next stage of integrating geometric information into PINN algorithms—the generation of the training point sample.

4.2. Sampling of Collocation Point for Physics-Informed Neural Networks

First, note that the application of neural networks for engineering modeling may be most interesting as a means of radically reducing the time to obtain results [75]. Therefore, practically all leading companies in CAE software are intensively working on implementing neural network technologies in their commercial packages—ANSYS, Autodesk, Siemens, Dassault Systèmes, etc. [111].

Such an application assumes that, either as an add-on to the package or as a user library, a training database for PINNs will be formed for a certain class of problems, allowing sufficiently accurate predictions even for geometric forms not included in the training data. The presence of such an initial approximation allows users to obtain solutions in the shortest time using PINNs.

To form such datasets, it is necessary first to normalize geometric information so that coordinates for all examples are in the range $[0,1]$ or $[-1,1]$. This is usually performed in several steps: forming a sample of training points $\left(x_i,y_i,z_i\right)$; finding the bounding box $\left(x_{min},x_{max},y_{min},y_{max},z_{min},z_{max}\right)$; finding the center $c_x={\displaystyle \frac{x_{max}+x_{min}}{2}}$; shifting the coordinate origin to the determined center; determining the scale $s=\max \left(x_{max}-x_{min},y_{max}-y_{min},z_{max}-z_{min}\right)$; and normalization, for example, to $[-\mathrm{1,1}]-x\prime =2x/s$. Determining the center based on the $min-max$ bounding box approach is dominant in modern research, e.g., Anton et al. [112,113], Valente et al. [114].

The selection of training points is one of the most important stages in building a PINN-based model. The study by Wu et al. [115] is one of the few papers where this issue is systematically covered. The authors considered ten methods for generating training point samples, including six non-adaptive and four adaptive. Among the non-adaptive uniform sampling methods, uniform grid, uniform random sampling, Latin hypercube sampling, Halton sequence, Hammersley sequence, and Sobol sequence were considered (Figure 13).

In most studies performed using PINNs, training points are determined at the very beginning and do not change thereafter. This choice is driven by simplicity of implementation, stability, and predictability of computational costs. Uniform discretization ensures coverage of the entire domain, even in areas with initially small residuals, preventing local overfitting that may occur with adaptive concentration of points in high-residual zones. From the perspective of integration with existing CAE packages, mesh-oriented methods have certain advantages. They allow direct use of existing CAD geometries and mesh and ensure concentration of points in important areas, as the mesh generator already adapts element density to these needs.

Nevertheless, adaptive methods, characterized by dynamic reconstruction of the training point sample, are becoming increasingly popular, as they allow faster reduction in residuals precisely where it most affects solution accuracy and usually lead to faster model convergence (by orders of magnitude for problems considered by Wu et al. [115]) with significantly fewer total training points. This is particularly relevant for problems in domains of complex shapes, with steep gradients and multiscale behavior [116].

The number of studies on improving adaptive generation of training point samples is rapidly growing. Classifying adaptive methods by used metrics, the most used in current research is adaptive distribution based on PDE residuals, where the magnitude $\left|f_{\theta }\left(t_f^i,x_f^i\right)\right|$ at a point is considered an indicator of local model inaccuracy, and points with the largest residuals receive higher priority in regeneration [115,117]. A more advanced approach is DAS-PINNs [118], where the PDE residual is viewed as a probability density function and approximated using a deep generative model. New samples are aligned with the distribution induced by the residual, i.e., more samples are placed in high-residual areas and fewer in low-residual areas.

Lin and Chen [119] extended the residual-based approach to unsteady problems, proposing a causality-driven method. Collocation points selected at each step are released before the next sampling step to avoid cumulative effects and reduce computational costs. The criterion for selecting collocation points at time steps is the largest weighted residual ${\omega }^p{\mathcal{L}}_f\left(t_i,x,\theta \right)$, where $p\ge 0$ is a hyperparameter and

$${\mathcal{L}}_f(t,x,\theta )={\left|{\displaystyle \frac{\mathsf{\partial }{u}_{\theta }}{\mathsf{\partial }t}}\left(t,x\right)+\mathcal{N}\left[u_{\theta }\right]\left(t,x\right)\right|}^2.$$

(26)

For updating $p$ from an initial value $p_0$, Lin and Chen in [119] proposed a time-aligned update scheme (TADU):

$$p_{n+1} = p_n\left[{\beta }_1\mathit{tanh}\left({\beta }_2\left({\displaystyle \frac{t_{ada}}{t_{\omega }}}\right)-1\right)+1\right], n = 0, 1, \dots ,$$

where parameter ${\beta }_1\in \left(0,1\right)$ regulates the maximum variation coefficient for $p$, and ${\beta }_2>0$ controls the difficulty level of achieving the maximum variation coefficient, and $t_{ada}$ indicates the time when the previous batch of adaptive collocation points is concentrated, $t_{\omega }$ for $k\in \left[0.5,1)\right.$ is determined by the model training time as follows:

$$t_{\omega }=\left\{\begin{array}{c}min\left\{t_i:{\omega }_i<k\right\}, {\omega }_{N_t}<k,\\ T_1 , {\omega }_{N_t}\ge k.\end{array}\right.$$

The second most used method for creating training point samples in PINNs is using the PDE residual gradient $‖\nabla \left(f_{\theta }\left(t_f^i,x_f^i\right)\right)‖$. In practice, three working patterns have formed: gradient-driven algorithms—build a probabilistic distribution proportional to ${‖\nabla \left(f_{\theta }\left(t_f^i,x_f^i\right)\right)‖}^{\alpha }$, then shift or regenerate collocation points [115,120]; hybrids with PDE residuals, using weighted combinations of $\left|f_{\theta }\left(t_f^i,x_f^i\right)\right|$ and $‖\nabla \left(f_{\theta }\left(t_f^i,x_f^i\right)\right)‖$ [121] or two-stage sorting [117] to avoid oversaturation of points in areas where the gradient is large but the error is small; anisotropic strategies—orient local “clouds” of new points along the direction of maximum residual growth (using directional derivatives) to better capture thin layers [122,123]. Advantages of gradient-oriented approaches include faster detection of boundary layers, shock waves, and thin structures; better accuracy/sample balance for a fixed number of points; and fewer “blind spots” compared to purely residual methods. However, such approaches have higher computational costs (need to differentiate residuals by inputs, sometimes repeatedly), noisiness and instability of residual gradient estimates at early stages, risk of local overfitting, and loss of global coverage.

Other methods for adaptive generation of training points for PINNs are less commonly used. Among them, those using loss function residuals or energy functionals as metrics [124] can be highlighted. Also noteworthy are algorithms that combine adaptive selection of collocation and experimental points, automatically adjusting the proportion of collocation point types during training [125].

In SDF-PINNs, the domain is represented implicitly through neural SDFs trained on STL files or point clouds, enabling sampling in arbitrary geometries by filtering points based on $\phi \left(x\right)$. Boundary points are adaptively reselected in regions with high residuals to improve convergence for stiff differential equations. These algorithms enhance the efficiency of PINN training by reducing the need for meshes, as demonstrated in fluid flow analysis problems where SDF-weighted sampling reduces weight in discontinuity zones, thereby improving stability and accuracy. Thus, SDFs provide a flexible and efficient tool for handling complex geometries in PINNs, combining geometric accuracy with physical plausibility [87].

Both during the initial generation of training points and during adaptive reconstruction of the sample, SDFs play a key role, implicitly encoding complex domains. For points inside the domain $\mathrm{\Omega }$, a rejection sampling algorithm is used: candidate points are generated uniformly in the bounding box, $\phi \left(x\right)$ is computed, and points where $\phi \left(x\right)<0$ are retained [126]. Evidently, a similar approach can be used when R-functions are employed to describe the domain geometry.

For generating samples on the domain boundary, projection methods are widely applied. This idea was formulated in the early stages of computer graphics development. In particular, in the work by Alexa et al. [127], procedures for projecting arbitrary points onto a surface reconstructed from a point cloud (projection operator) are described and used to define the surface as a set of fixed points, while in [128], tools for increasing or decreasing point density are presented. Calakli and Taubin in [129] provided a clear procedure for obtaining a seamless implicit surface and projecting points onto it. Finally, Ma et al. [130] defined the projection of points onto the surface as a key component for training neural SDFs from sparse point clouds without ground-truth signed distances.

Typically, to form a sample on $\mathsf{\partial }\mathrm{\Omega }$, points are generated near the zero level of the SDF $\phi \left(x\right) < \delta$, after which they are projected onto the boundary using the formula

$$x^{\prime }=x-\phi \left(x\right)· \left({\displaystyle \frac{\nabla \phi \left(x\right)}{\left|\left|\nabla \phi \left(x\right)\right|\right|}}\right),$$

(27)

which ensures their movement in the direction of the normal to the surface by a distance equal to the signed distance. The normal vector $\nabla \phi$ is determined analytically or numerically.

Similarly to the case of generating a training sample in the domain, adaptive procedures are applied on the boundary. For example, for the initial boundary sample $\mathsf{\partial }\mathrm{\Omega }$, more points are generated in regions of high curvature $k=\nabla ·(\nabla \phi (x)/||\nabla \phi (x\left)\right|\left|\right)$ using a curvature metric [131]. The boundary sample obtained in this way can subsequently be adaptively refined using procedures similar to those described earlier. Additionally, in the case of a complex domain shape, neural network-constructed SDFs can be used to form the boundary training point sample [87].

Functions $\phi =0$ constructed using the R-functions method with most operation systems (17), (19)–(21) near the boundary possess the same properties as SDFs, defining the distance to the nearest boundary segment. Therefore, the methodology for using R-functions to generate a training point sample in $\mathrm{\Omega }$ and $\mathsf{\partial }\mathrm{\Omega }$ does not differ from the procedures described above for SDFs. However, such use is not rational. The properties defined by expressions (23)–(25) are used to construct bundles of functions that identically satisfy boundary conditions and, in relation to PINNs, can be used to form a loss function with hard constraint conditions. The specifics of implementing such an approach will be considered further.

Table 6. Comparative analysis of collocation point generation methods for PINNs in engineering modeling problems.

Method of Collocation Point Generation	Disadvantages	Accuracy and Convergence	Integration with CAD/CAE
Non-adaptive methods (uniform grid, random sampling, Latin hypercube, etc.)	Inefficient for complex geometries, may require many points, limited adaptivity, risks of incomplete coverage of gradient zones	Average, stable, but slow convergence	High for grids, average for others (use of bounding box or CAD meshes)
Based on PDE residuals (residual-based, including DAS-PINNs)	Risk of overfitting, noisiness in early stages	High	High (integration with CAE for dynamic reconstruction)
Causality-guided (for unsteady problems)	More complex implementation, dependence on hyperparameters	High for dynamic systems	High (compatible with CAE for time simulations)
Based on PDE residual gradient	High differentiation cost, gradient noisiness, risk of local overfitting, loss of global coverage	High, better for fixed quantity	Average (can integrate with CAD for anisotropic strategies)
Based on loss-function residuals or energy functionals	Less common, depends on loss structure	High for multiphysics problems	High (integration with energy-based CAE methods)
Using SDF or R-functions (rejection sampling, projection methods, hard constraints)	Costs for computing SDF/R-functions, rejection, limitations for non-analytical forms	High, identical satisfaction of BCs for R-functions	High (integration with CAD/CAE for implicit geometry and BCs)

presents summarized data on methods for generating point samples for training models based on PINNs.

From the perspective of integration with CAE systems, non-adaptive mesh-oriented methods have advantages, as they allow direct use of existing geometries and meshes generated in CAD/CAE. Adaptive methods, while more efficient, require additional implementation but can be integrated as post-processing modules.

The next step in incorporating geometric information is the enforcement of boundary conditions, which is crucial for the accuracy of PINN models.

4.3. Enforcement of Boundary Conditions in PINNs

The primary advantage of analytically incorporating geometry is not the simplification of adaptive training point generation but the resolution of the challenge of automatically satisfying boundary conditions. In the context of solving differential equations using neural networks, this approach was first proposed by Lagaris et al. [132]. They suggested representing the solution of a differential equation as the sum of two components. The first component satisfies the initial/boundary conditions and contains no adjustable parameters. The second component is constructed to have no influence on the initial/boundary conditions. For example, by writing

$$u\left(x\right)=g\left(x\right)+\phi \left(x\right)\mathcal{N}(x;\theta ),$$

(28)

where $\phi \left(x\right)=0, x\in \mathsf{\partial }\mathrm{\Omega }$, we ensure automatic satisfaction of Dirichlet boundary conditions $u\left(x\right)=g\left(x\right), x\in \mathsf{\partial }\mathrm{\Omega }$. Here, the neural network component $\mathcal{N}\left(x;\theta \right)$ is responsible for ensuring compliance with the governing equations of the problem. This approach is commonly referred to as a HC-PINN. Examples of its application, as presented in Lagaris et al. [132], were limited to simple domains. This work laid the foundation for hard-constraint modeling with PINNs, although it did not utilize SDFs, focusing instead on analytical distance functions for regular geometries.

Berg and Nyström [133] applied the hard-constraint concept to solve PDEs using SDFs as $\phi \left(x\right)$, but not within the PINN framework. In a test case involving a polygon with over 160,000 vertices, the proposed approach demonstrated superiority over the classical FEM, which failed to solve the test problem.

The modern stage began with the introduction of PINNs in the seminal work by Raissi et al. [1]. While conditions like (28) were not used in this study, they highlight the need for better enforcement of boundary conditions compared to soft-constraint approaches, particularly for inverse problems. Later, Lu et al. [134] proposed hPINNs (hard-constraint PINNs), where the solution was approximated similarly to (28). This eliminated residuals for boundary conditions in the loss function, improving accuracy for problems in elasticity and fluid dynamics. The study demonstrated advantages over soft-constraint approaches for inverse problems but was limited to simple geometries with analytically defined distance functions without utilizing SDFs.

A significant advancement was made by Sukumar and Srivastava [91], who proposed the use of SDFs to enforce hard constraints in PINNs, enabling the handling of arbitrary geometries. This combined PINNs with level-set methods, improving accuracy for irregular domains in mechanics. Subsequently, this approach, associated with changes in neural network architecture (see Figure 14), was adopted by many researchers (e.g., Chen et al. [135]).

In the study by Moseley et al. [47], the hard-constraint approach was applied in conjunction with domain decomposition (FBPINN). This approach introduces additional residuals at interfaces between subdomains. To account for these, Moseley et al. [47] used partially overlapping subdomains. Neural networks approximate the solution for each subdomain such that, at the subdomain center, the network learns the full solution, while in overlapping regions, the solution is defined as the sum of contributions from all overlapping networks. The global solution, constructed as this sum, ensures continuity across all subdomains. This approach allows FBPINN to use a loss function similar to that of PINNs without requiring additional interface terms, unlike other domain decomposition methods.

To enforce interface conditions in domain decomposition, another approach commonly used involves hard constraints. Interface conditions are typically formulated as

$$u^i\left(x\right)=u^j\left(x\right),x\in {\Gamma }_{ij},$$

(29)

$${\displaystyle \frac{{\mathsf{\partial }u}^i\left(x\right)}{\mathsf{\partial }{n}_{ij}}}={\displaystyle \frac{{\mathsf{\partial }u}^j\left(x\right)}{\mathsf{\partial }{n}_{ij}}},x\in {\Gamma }_{ij},$$

(30)

where $u^i$, $u^j$ are solutions in adjacent subdomains, ${\Gamma }_{ij}$ is their shared interface, and $\mathsf{\partial }/\mathsf{\partial }{n}_{ij}$ denotes the derivative in the direction of the outward normal to ${\Gamma }_{ij}$.

Using SDFs, it is straightforward to satisfy Dirichlet-type conditions (29). Meanwhile, condition (30) is incorporated into the loss function as a component when using an energy-based loss formulation. This approach, employed by Lai et al. [136], demonstrates that hard constraints for conditions like (29) significantly simplify the training process and enhance numerical accuracy. The authors note that this approach is simple to implement, scalable to various partial differential equations, and ensures strict enforcement of control constraints.

Berrone et al. [137] applied ADF, as proposed by Sukumar and Srivastava [88], to enforce Dirichlet boundary conditions for a range of problems using both PINNs and VPINNs. The hard-constraint approach was implemented in two variants: with normalized and non-normalized ADF. According to [137], the most effective and accurate approach was based on a variational formulation of the loss function combined with hard constraints using normalized ADF. The non-normalized ADF variant led to suboptimal results and, in some cases, caused convergence failure during training.

Wang et al. [138] applied a similar hard-constraint approach to solid mechanics problems, introducing the Exact Dirichlet Boundary Condition Physics-Informed Neural Network (EPINN). The loss function was formulated based on the principle of minimum work. Figure 15 schematically illustrates the EPINN architecture. A separate neural network was created for each coordinate to accurately reproduce the shape function. Tensor decomposition was used to construct 2D or 3D displacement fields. Hard constraints for displacement vector components were enforced using ADF. For linear elastic problems, the stress field was derived directly from the strain field. For nonlinear problems, pre-trained deep learning constitutive models (e.g., Temporal Convolutional Network (TCN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), or Sequence-to-Sequence models (Seq2Seq)) were used to update stresses.

For forward problems, EPINN achieved a speedup of over 13 times for one-dimensional problems and over 126 times for three-dimensional problems compared to PINNs, with comparable accuracy as assessed against finite element modeling results.

A further development of this approach was made by Tian et al. [139]. This study introduced an extension of EPINN, focusing on an automated machine learning network that integrates Bayesian optimization with a tree-based Parzen estimator (BO-TPE) for hyperparameter optimization (Figure 16). Like the base method, AEPINN [139] employs ADF-based hard constraints to enforce Dirichlet boundary conditions exactly and to impose the principle of minimum work. The core idea is to automate hyperparameter tuning, making the framework more robust and efficient for forward problems in solid mechanics without requiring labeled data.

Hyperparameters considered in AEPINN include neural network architecture parameters (e.g., number of layers, neurons per layer, and activation functions), learning rates, batch sizes, loss weighting coefficients, and optimizer settings. Automating hyperparameter selection reduces user intervention, accelerates model convergence, and improves generalization across diverse geometries.

Results from test problems demonstrate significant improvements in computational efficiency and accuracy. For 2D problems, AEPINN achieves an over 20-fold speedup compared to EPINN and over 200-fold compared to traditional PINNs. In 3D simulations, it provides a fourfold speedup over EPINN and a 400-fold speedup over PINNs, accurately reproducing displacement fields comparable to ABAQUS results.

Notably, in nearly all studies employing the hard-constraint approach, the solution is constructed in the form of (28) for Dirichlet boundary conditions, with SDF or ADF used in the second term to ensure zero value at the boundary. Only a few studies (e.g., Straub et al. [140]) apply hard constraints for Neumann conditions using Fourier feature embeddings: input coordinates are expanded into a Fourier series and fed into the first layer of the neural network, enabling accurate approximation of the solution gradient at the boundary. However, this approach is limited to regular domains (e.g., rectangles with boundaries parallel to the axes) and is less effective for complex geometries without SDF. Sukumar and Srivastava [91] demonstrated how hard constraints can be enforced using ADF and the R-function method. In [91], they provided examples of such solutions for Neumann problems. For Neumann boundary conditions of the form $\mathsf{\partial }u/\mathsf{\partial }n=h$ on $\mathsf{\partial }\Omega$, the solution takes the following form:

$$u = [1 + \phi D_1]\left({\tilde{u}}_1\right) - \phi h + {\phi }^2 {\tilde{u}}_2,$$

(31)

where $D_1$ is defined as in (25), and ${\tilde{u}}_1$ and ${\tilde{u}}_2$ are arbitrary approximation functions defined by separate neural networks. For Robin boundary conditions ${\displaystyle \frac{\mathsf{\partial }u}{\mathsf{\partial }n}}+cu=h$ on $\mathsf{\partial }\Omega$, the solution takes the following form:

$$u = [1 + \phi (c+D_1\left)\right]\left({\tilde{u}}_1\right) - \phi h + {\phi }^2 {\tilde{u}}_2,$$

(32)

Expressions (31) and (32) are specific cases of applying the R-function method for constructing solution structures for boundary value problems [92]. The general sequence for constructing such solutions is as follows:

1.

Construction of normalized equations using one of the complete systems of R-functions (17), (19)–(21) and expressions (23)–(25) to describe the boundary of the computational domain $\mathsf{\partial }\mathrm{\Omega }$ and boundary segments $\mathsf{\partial }{\mathrm{\Omega }}_i$ with different boundary conditions. The order to which $\phi$ and ${\phi }_i$ are normalized is determined by the order of the governing differential equation.

2.

Extension of boundary conditions defined only on the domain boundary to the entire computational domain. In addition to the operator $D_n$(25), which extends n-fold differentiation along the normal to $\mathsf{\partial }\mathrm{\Omega }$ into the domain $\mathrm{\Omega }$ [92], the following operators ${\mathbb{R}}^2$ are used:

$$T_m\left(·\right)={\sum }_{i=0}^m{(-1)}^{m-i}{C}_m^i{\displaystyle \frac{{\mathsf{\partial }}^m(·)}{\mathsf{\partial }{x}_1^{m-i}\mathsf{\partial }{x}_2^i}}{{\left({\displaystyle \frac{\mathsf{\partial }\phi }{\mathsf{\partial }{x}_1}}\right)}^i\left({\displaystyle \frac{\mathsf{\partial }\phi }{\mathsf{\partial }{x}_2}}\right)}^{m-i},$$

(33)

defined everywhere in $\mathrm{\Omega }$, and on $\mathsf{\partial }\mathrm{\Omega }$, it represents the m-th order tangential derivative, and

$$\left.\begin{array}{c}{S}_1=T_1\\ S_2=k{D}_1+T_2\\ \dots \end{array}\right\},$$

(34)

defined everywhere in $\mathrm{\Omega }$, and on $\mathsf{\partial }\mathrm{\Omega }$, it represents the order derivative along the boundary curve.

Using operators $D_m,T_m,S_m$ defined as in (25), (33), (34), mixed derivatives (e.g., ${\displaystyle \frac{{\mathsf{\partial }}^2(·)}{\mathsf{\partial }\nu \mathsf{\partial }\tau }}=D_1{T}_1(·)$) can be extended into the domain. With these operators, boundary conditions of the form

$$B \left(\left(·\right),{\displaystyle \frac{\mathsf{\partial }(·)}{\mathsf{\partial }n}},\dots ,f\right)=0,$$

(35)

are rewritten as

$$B \left(\left(·\right),D_1\left(·\right),\dots ,f\right)=\phi {\Phi }_1,$$

(36)

where $\phi$ is normalized to the order of the equation defining the domain $\mathrm{\Omega }$ shape, and ${\Phi }_1$ is an arbitrary function, which may be a polynomial system or defined by a neural network. Expression (36) is defined everywhere in $\mathrm{\Omega }$, satisfies the governing PDE due to the presence of undetermined components, and corresponds to the boundary conditions (35) on $\mathsf{\partial }\mathrm{\Omega }$, serving as their extension to the computational domain.

3.

Extension of nonhomogeneous boundary conditions to the computational domain. For nonhomogeneous $\mathsf{\partial }\mathrm{\Omega }:\Gamma =\bigcup _{i=1}^N{\Gamma }_{\iota }$ boundary conditions of the form $u\left(x\right)=g_i$, $i=1,\dots ,m$ the function satisfying these conditions can be written as

$$g=g_{i }{w}_{i }, w_{i }={\displaystyle \frac{\prod _{j=1,j\ne i}^M{\phi }_j}{\sum _{k=1}^M\prod _{j=1,j\ne i}^M{\phi }_j}},$$

(37)

where ${\phi }_j$ is defined as in (24). This function is defined throughout the computational domain. Similarly, functions extending boundary conditions (e.g., $\mathsf{\partial }u\left(x\right)/\mathsf{\partial }n=g_i$,$i=1,\dots ,m$) involving derivatives or mixed derivatives of higher orders are constructed.

4.

Construction of the solution structure. For a governing PDE $u\left(x\right)$ of order m, the solution structure, which identically satisfies the boundary conditions and includes arbitrary components, is constructed as

$$u\left(x\right)={\Phi }_2+{\phi }^k{\Phi }_3,$$

(38)

where $\phi \in C^{k+1}\left(\mathrm{\Omega }\cup \mathsf{\partial }\mathrm{\Omega }\right)$ is normalized to order m, and ${\Phi }_2,{\Phi }_3\in C^{k+1}\left(\mathrm{\Omega }\cup \mathsf{\partial }\mathrm{\Omega }\right)$ are arbitrary functions.

Expression (38) is substituted into (36), and, considering the properties of (22), operator operations are performed, and undetermined functions ${\Phi }_i$ with common products of the form ${\phi }^n$ are grouped. The grouped sums or products of undetermined functions are denoted as new undetermined functions ${\Psi }_i$, after which the final solution structure $u\left(x\right)$ is written as a combination of functions ${\Psi }_i$ with products of the form $\phi ,{\phi }^k$, $g_{i }{w}_{i }$, defined by (37) and operators $D_m,T_m,S_m$. This allows the construction of not only structures like (31) and (32) but also more complex structures.

5.

Construction of solution structures for boundary conditions defined by systems of equations. When a system of conditions of the form $B_i=0$ is specified on $\mathsf{\partial }\mathrm{\Omega }$ and expressed as differential equations of different orders $k_1<k_2<\cdots <k_m$, the solution structure can be constructed as follows. First, a bundle of functions $u=W_m\left({\Psi }_{1m},B_m\left({\Psi }_{1m}\right)\right)$ is constructed to satisfy the boundary condition of the highest order $k_m$. Substituting this into the condition $(m-1)$, terms containing the product ${\phi }^{k_m}$ vanish. Then, it is sufficient to choose ${\Psi }_{1m}$ to satisfy the corresponding condition, i.e.,

$$u=W_m\left\{W_{m-1}\left({\Psi }_{1m-1},B_{m-1}\left({\Psi }_{1m-1}\right)\right),B_m\left({W_{m-1}(\Psi }_{1m-1},B_{m-1}\left({\Psi }_{1m-1}\right)\right)\right\}.$$

(39)

By repeating these steps, a structure $u\left(x\right)$ can be constructed to satisfy all boundary conditions on $\mathsf{\partial }\mathrm{\Omega }$ [92].

6.

For cases with multiple boundaries $\mathsf{\partial }\mathrm{\Omega }=\bigcup _{i=1}^N{\mathsf{\partial }\mathrm{\Omega }}_{\iota }$ and systems of equations of the form $B_i=0$ on each ${\mathsf{\partial }\mathrm{\Omega }}_{\iota }$, bundles of functions $u=Q_i\left({\Psi }^i\right)$, ${\Psi }^i=\left({\Psi }_1^i,\dots ,{\Psi }_{m_i}^i\right)$ are constructed to satisfy all conditions on ${\mathsf{\partial }\mathrm{\Omega }}_{\iota }$ with the highest degree $k_i$. The solution structure satisfying the conditions on $\mathsf{\partial }\mathrm{\Omega }$ takes the following form:

$$u={\displaystyle \frac{Q_1\left({\Psi }^1\right){\tau }_1^{k_1+1}+\dots +Q_s\left({\Psi }^s\right){\tau }_s^{k_s+1}}{{\tau }_1^{k_1+1}+\dots +{\tau }_s^{k_s+1}}}, {\tau }_i={\displaystyle \frac{1}{{\phi }_i}}.$$

(40)

Expressions like (38)–(40) enable the construction of functions that satisfy nearly all types of boundary conditions used in engineering modeling with CAE systems, provided that functions $\phi$ with properties (17) and (19) are constructed. In later works by Rvachev and Sheiko [141] and Rvachev and Slesarenko [142], structures for nonlinear boundary conditions in problems of heat conduction, plate theory, and others were also developed. This, in principle, allows us to solve the hard-constraint problem for arbitrary boundary conditions when constructing the loss function for PINNs.

Despite the clear advantages of analytically incorporating geometry, the described approaches have certain limitations that restrict their widespread application. For instance, at junctions of geometric primitives where logical components change dominance, R-functions may lose smoothness and become non-differentiable. This is critical in applications where gradients are used in training, such as in PINNs [94,143]. Collocation points falling in such zones can lead to gradient “explosions,” with corresponding consequences for the training process. Additionally, SDFs, by their initial definition, are not smooth on medial surfaces (equidistant to two or more primitives) where standard algorithms may produce gradient discontinuities, potentially causing instability or failure in PINN training, especially if the network is sensitive to local gradient variations [110]. Furthermore, the use of directional differentiation and the complex form of functions constructed using (39) and (40) degrade the approximation properties of structural formula components, which may lead to convergence issues.

Kraus and Tatsis [87] proposed an approach where the PDE solution is decomposed into two parts: one that satisfies boundary conditions without adjustable parameters and another that incorporates a physics-informed neural network with adjustable parameters (Figure 17). This approach mitigates the aforementioned drawbacks but also undermines the idea of using analytical approaches for engineering calculations with PINNs, which is unlikely to be a promising direction for the development of hard constraints.

Hybrid approaches combining PINNs with hard constraints have also been explored. Sobh et al. [144] proposed a PINN-FEM hybrid approach aimed at more accurately enforcing Dirichlet boundary conditions in problems involving partial differential equations. The core idea is to divide the computational domain into two parts: a near-surface zone, where boundary conditions are directly imposed, uses the finite element method to strictly enforce Dirichlet conditions, while the interior region employs a PINN for solution approximation (Figure 18). Continuity conditions are imposed at the interface between these subdomains to ensure solution and derivative continuity.

Such approaches have several limitations. Applying FEM in the near-surface region requires mesh generation and additional computations, increasing computational cost and complicating implementation in multidimensional problems. The choice of the FEM region size and mesh parameters significantly affects accuracy and stability, requiring careful tuning. Moreover, scaling the method to complex geometries and high-dimensional problems can be challenging, and the lack of rigorous theoretical guarantees for convergence and error estimates limits their application in engineering modeling. A summary of the analysis, focusing on the applicability of these methods in engineering modeling with CAD/CAE systems, is presented in

Table 7. Analysis of methods for integrating boundary conditions into the PINN structure.

Method of Boundary Condition Integration	Key Idea	Advantages	Integration with CAD/CAE
Soft-constraint PINN	Boundary conditions are added to the loss function as penalty terms	Simplicity of implementation; universality	Low (no direct link to CAD geometries)
Hard-constraint PINN	Solution represented as a sum: one part exactly satisfies boundary conditions, the other is approximated by NN	Guaranteed enforcement of Dirichlet conditions; improved accuracy	Medium (CAD analytics can be integrated for simple geometries)
HC-PINN with SDF/ADF	Use of signed/approximate distance functions to enforce conditions on arbitrary geometries	Applicability to complex domains; guaranteed enforcement of Dirichlet, Neumann, and Robin conditions	High (SDF can be generated from STL/CAD meshes)
Domain Decomposition Methods (FBPINN-HC, PINN-FEM)	Decomposition into overlapping subdomains, with interface conditions enforced	Scalability, combining strengths of FEM and PINN	High (CAD → mesh → easy integration with PINN)
Hard-constraint with R-functions	Construction of solution structures that analytically satisfy various boundary conditions	Universality; suitability for complex and combined conditions	Highest (provided R-function construction is automated with CAD data)

.

5. Future Directions and Opportunities

The development of PINNs opens new avenues for integrating machine learning with scientific knowledge. Despite certain challenges, PINNs provide a foundation for advancing engineering modeling systems. In this section, we outline promising avenues for future research, emphasizing innovative opportunities and interdisciplinary approaches that will facilitate the broader practical application of PINNs in integration with CAD/CAE systems.

5.1. Development of Physics-Informed Kolmogorov–Arnold Neural Networks

Physics-informed Kolmogorov–Arnold neural networks (PIKANs) are regarded by many researchers as a promising alternative to conventional PINNs [14,145]. Their key distinction lies in replacing multilayer perceptrons with the Kolmogorov–Arnold Network (KAN) architecture, which is grounded in the superposition theorem. This approach enables more effective approximation of multidimensional nonlinear dependencies and mitigates the spectral bias issue inherent in traditional PINNs. The work by Wang et al. [146] demonstrates that KINN achieves faster convergence and superior accuracy in tasks involving multi-scale features compared to PINNs. Subsequent studies have confirmed the potential for reducing computational costs (SPIKAN, Jacob et al. [147]) through the application of separate KANs for each coordinate, as well as the efficacy of adaptive training for optimizing collocation point selection [148]. The application of PIKAN to multi-material problems [149] has illustrated its capability for naturally accounting for discontinuities in material properties without complex decomposition. Thus, compared to PINNs, PIKANs exhibit greater parametric efficiency, the ability to reproduce multi-scale phenomena, and suitability for tasks involving material heterogeneity. These advantages, if validated across a broader range of problems, could position this method as a primary tool in engineering modeling. However, integrating PIKAN with CAD/CAE systems requires addressing challenges related to incorporating geometric information and boundary conditions for arbitrary cases.

5.2. Operator Learning and Meta-Learning

Operator neural networks represent a dynamic field in machine learning, evolving from foundational architectures such as DeepONet [150] to advanced models like the Fourier Neural Operator (FNO) [151], which enable learning mappings between infinite-dimensional function spaces for solving parametric PDEs. These approaches eliminate dependence on mesh-based discretization, facilitating efficient simulations in scientific computing, and have gained momentum with the emergence of hybrid methods, such as the Physics-Informed Neural Operator (PINO) [152], and transformer-based variants, for example, the General Neural Operator Transformer (GNOT) [153], which integrate spectral and attention mechanisms for improved generalization.

Among contemporary innovations, notable developments include PI-GANO [154], GINO [155], and VINO [156], which extend the capabilities of neural operators in the context of complex geometries and variational formulations. PI-GANO integrates a geometric encoder into the Deep Compositional Operator Network architecture, enabling simultaneous generalization over PDE parameters and variable domains without requiring FEM data, making it efficient for engineering modeling tasks. GINO (Geometry-Informed Neural Operator) employs signed distance function and point-cloud representations for input shapes, combining graph and Fourier operators to solve large-scale 3D PDEs, demonstrating speedups of up to 26,000 times compared to CFD simulators. The Variational Physics-Informed Neural Operator (VINO) introduces a variational approach to integrating physical laws, minimizing the energy formulation of PDEs without the need for labeled data, thereby addressing challenges in efficiently computing derivatives and integrals in weak forms.

Compared to PINNs, operator networks exhibit several advantages. They achieve resolution invariance, enabling predictions without retraining, and demonstrate superior generalization across diverse PDE families due to their operator-theoretic foundation. However, at the current stage of development, operator neural networks cannot provide predictions for arbitrary boundary condition types, being limited to those on which training was conducted. Evidently, the use of hard-constraint methods with R-functions could alleviate such limitations, though research in this direction has yet to be undertaken. Operator networks surpass all other methods in terms of solution speed, potentially making them the primary tool for solving optimization problems. Nevertheless, complete displacement of PINN- or PIKAN-based methods should not be anticipated. These approaches are expected to retain value in data-limited tasks and for non-standard computational models, owing to their capacity for embedding physical laws into neural network architectures.

5.3. Automation of Geometry and Boundary Condition Information Generation in CAE Packages

The efficiency of developing engineering modeling systems integrated with neural network methods can be substantially enhanced by embedding modern geometry description techniques, such as SDFs, ADFs, and R-functions, into CAE environments. Their utilization will enable the representation of geometry in a compact analytical form, providing a basis for automating computational model construction.

A promising direction is the establishment of direct integration modules in CAE packages that automatically generate SDF/ADF/R-functions from CAD system data, for instance, based on the STEP standard. This will facilitate the direct formation of functions describing the domain geometry or its individual segments without additional discretization or manual intervention. This approach will enable the creation of a unified generalized representation of geometric information suitable for both mesh generation and integration with neural networks oriented toward operator learning. A critical component of such integration is the automation of boundary condition descriptions. In contemporary CAE packages, users typically specify boundary conditions through interfaces tied to geometric elements.

However, a promising development is the creation of modules that will form typical solution structures based on R-operations directly during their specification. This will reduce the likelihood of errors, enhance consistency between geometry and the physical problem setup, and lay the groundwork for subsequent utilization of such structures in physics-informed and operator neural networks. Thus, the integration of SDFs, ADFs, and R-functions into CAE environments should be considered a strategic direction for development, ensuring the application of cutting-edge machine learning methods in engineering modeling.

5.4. Development of Hybrid Methods for Engineering Modeling

The implementation of neural network methods must account for the capabilities of existing commercial CAE packages and the fact that, given the investments in their development, such implementation should proceed through modernization of existing software rather than the creation of new systems. Considering achievements in specific aspects of PINN utilization, the most promising approach may involve hybrid methods that combine the advantages of various algorithms. The integration of the finite element method with PINNs or neural operators could unlock new possibilities for engineering modeling [59]. This allows for retaining the accuracy and stability of FEM in critical model regions while employing neural modules for rapid approximation of complex nonlinear or locally varying fields [157]. This could radically reduce computational costs in repeated simulations and enable the integration of multiphysics interactions that are challenging to realize with classical methods.

At the same time, the development of such hybrid schemes poses scientific challenges: harmonizing FEM discretizations with neural approximators, devising stable training strategies, and ensuring result reproducibility [158]. One such approach could be a method that combines the ideas of adaptive computational domain decomposition with hard constraints based on the structural method. In this approach, preliminary domain partitioning could be performed such that each subdomain includes a boundary segment with only one type of boundary condition. This would enable the application of simple structures with sufficient approximation properties in each subdomain, facilitate computational parallelization, and employ adaptation procedures to minimize errors at interfaces, analogous to PDD approaches.

5.5. Standardization and Infrastructure Development

To stimulate the systematic advancement of neural network methods in engineering modeling, it is essential to establish standardized datasets and benchmarks. Specifically, the creation of open databases and standardized test problem sets for evaluating PINN/PIKAN and neural operator algorithms will ensure comparability and reproducibility of results. To enhance the efficiency of developed methods, it is necessary to devise effective algorithms for scaling neural operators on high-performance platforms, leveraging parallel computing and specialized hardware. The integration of neural network technologies with engineering modeling systems requires combining expertise from machine learning, numerical methods, and engineering disciplines through the formation of collaborative research groups and consortia. Finally, the implementation of specialized training programs for engineers and researchers, with an emphasis on neural network methods—including practical courses on their application in engineering modeling tasks—could prove exceedingly beneficial.

6. Conclusions

PINNs offer a robust framework for addressing complex PDE-based problems in engineering modeling, overcoming traditional numerical methods’ limitations by directly integrating physical laws into the learning process. However, their practical implementation faces significant challenges, including formulating consistent loss functions, incorporating complex geometries, and enforcing boundary conditions. The heterogeneity of loss function components, which account for PDE residuals, boundary conditions, and data alignment, introduces varying scales and physical dimensions, complicating optimization and potentially reducing accuracy. Promising solutions include energy-based and variational formulations, adaptive weighting strategies, and advanced sampling techniques to balance these components effectively. The integration of geometric information, critical for engineering applications, is addressed through analytical representations such as SDFs, phi-functions, and R-functions, which enable precise handling of complex and multi-body domains. Hybrid approaches combining these representations enhance robustness, particularly when integrated with CAE systems for automated geometry and boundary condition processing. Emerging methods, such as PIKANs, demonstrate superior parametric efficiency and multiscale modeling capabilities. At the same time, operator learning frameworks, including DeepONet and Fourier Neural Operators, offer resolution-invariant solutions for high-dimensional PDEs. The development of hybrid schemes combining PINNs with FEM and the standardization of benchmarks are critical for ensuring scalability, reproducibility, and industrial applicability. Future research should focus on optimizing computational efficiency, developing adaptive domain decomposition strategies, and establishing interdisciplinary collaborations to advance PINNs as reliable tools for rapid simulation, design optimization, and digital twin development in engineering applications.