Error Estimates and Generalized Trial Constructions for Solving ODEs Using Physics-Informed Neural Networks

Abstract

In this paper, we address the challenge of solving differential equations using physics-informed neural networks (PINNs), an innovative approach that integrates known physical laws into neural network training. The PINN approach involves three main steps: constructing a neural-network-based solution ansatz, defining a suitable loss function, and minimizing this loss via gradient-based optimization. We review two primary PINN formulations: the standard PINN I and an enhanced PINN II. The latter explicitly incorporates initial, final, or boundary conditions. Focusing on first-order differential equations, PINN II methods typically express the approximate solution as $\tilde{u}(x,\theta )=P\left(x\right)+Q\left(x\right)N(x,\theta )$, where $N(x,\theta )$ is the neural network output with parameters $\theta$, and $P\left(x\right)$ and $Q\left(x\right)$ are polynomial functions. We generalize this formulation by replacing the polynomial $Q\left(x\right)$ with a more flexible function $\varphi \left(x\right)$. We demonstrate that this generalized form yields a uniform approximation of the true solution, based on Cybenko’s universal approximation theorem. We further show that the approximation error diminishes as the loss function converges. Numerical experiments validate our theoretical findings and illustrate the advantages of the proposed choice of $\varphi \left(x\right)$. Finally, we outline how this framework can be extended to higher-order or other classes of differential equations.

1. Introduction

The numerical solution of ordinary differential equations (ODEs) underpins much of scientific computation. Classical numerical methods (e.g., Runge–Kutta schemes) can struggle with stiff systems and complex boundary conditions (see [1,2] for a detailed discussion). In recent years, Physics-Informed Neural Networks (PINNs) have emerged as a promising mesh-free alternative that integrates physical laws directly into the training loss [3]. However, standard PINN formulations typically enforce initial or boundary conditions via penalty terms, which may lead to reduced accuracy near boundaries and slow convergence [4]. In a PINN, a neural network $N(x;\theta )$ is trained so that its output nearly satisfies the ODE $\mathcal{F}\left[N\right]=f$, and any specified initial/boundary conditions, across the domain without relying on labeled solution data. One minimizes a loss comprising the mean-squared residual of the differential operator plus terms enforcing the constraints.

While standard PINN formulations typically enforce boundary or initial conditions via penalty terms added to the loss function [5], this soft-constraint approach may lead to suboptimal accuracy near the boundaries, slow convergence, and sensitivity to the choice of penalty weights. Our approach aims to address certain limitations of soft constraints by exactly satisfying the initial/boundary conditions. To overcome these limitations, we adopt a generalized trial solution strategy that embeds the conditions into the structure of the approximation itself. Specifically, we construct trial functions of the form

$$u_{\theta }\left(x\right)=u_0+\varphi \left(x\right)·N_{\theta }\left(x\right),$$

where $\varphi \left(x\right)$ is a smooth function vanishing at the constraint point(s). This ensures that the network output automatically satisfies the prescribed condition(s), thus eliminating the need for explicit penalization. Such hard-constrained formulations are expected to improve training stability and enforce boundary consistency in challenging scenarios.

This approach combines the expressive power of neural networks with physical constraints: by teaching the network the differential law, one can achieve solutions that are data-efficient and physics-consistent. By the universal approximation theorem [6,7], a sufficiently large feedforward network can approximate any continuous function on a compact domain. Hence, there exists in principle a network arbitrarily close to the true solution of a well-posed ODE. However, the classical universal approximation result is non-constructive and does not offer training guidance or error control. This motivates the introduction of structured trial functions in PINNs.

Lagaris et al. [4] proposed using trial solutions of the form

$$\tilde{u}(x,\theta )=P\left(x\right)+Q\left(x\right)N(x,\theta )$$

where $P\left(x\right)$ satisfies the boundary/initial conditions, and $Q\left(x\right)$ vanishes on those boundaries. The trainable network $N(x,\theta )$ fine-tunes the solution while inherently respecting the constraints. This has been shown to accelerate convergence compared to unconstrained networks. Generalizations have introduced basis expansions (polynomial, Fourier, and radial) to enforce smoothness and asymptotic properties. These enhancements remain within the scope of universal approximators [6,7].

In this work, we systematically generalize and analyze the class of hard-constrained trial constructions (Lagaris-type), providing admissibility conditions for $\varphi$ and proving error estimates linking the PINN loss to the true solution error for first-order ODEs.

Related Work

Physics-Informed Neural Networks (PINNs) were formalized by Raissi et al. [3] as a deep learning framework that integrates physical law constraints into the training process. PINNs use automatic differentiation to evaluate the differential equation residuals and include these residuals, along with boundary and initial condition terms, as soft constraints in the loss function during training [3]. By penalizing violations of the governing equations and boundary conditions in the loss, a PINN can learn solutions without any observed data, effectively bridging data-driven learning and physics-based modeling. This mesh-free formulation avoids the need for spatial discretization and can mitigate issues of traditional methods (like meshing or the curse of dimensionality in high-dimensional problems). PINNs have achieved notable success across diverse application areas, including fluid dynamics (e.g., solving Navier–Stokes flow problems) [5], hemodynamics and biomedical flow modeling [8,9], and structural/mechanical system simulations [10]; see also [11] for solar coronal magnetic fields. Their ability to embed physical knowledge leads to improved generalization from limited data, and numerous studies report PINNs outperforming black-box neural networks in data-scarce scenarios. Comprehensive surveys of the field document these wide-ranging successes and the rapid methodological advancements in PINNs [12,13].

Despite their promise, standard PINN formulations have certain limitations. A key issue is that PINNs typically enforce initial and boundary conditions via penalty terms added to the loss, rather than satisfying them exactly [3]. This soft-constraint approach can result in suboptimal accuracy near domain boundaries and can slow down convergence, especially if the penalty weights are not tuned properly [14]. To address this, recent research has revisited the idea of hard constraint trial functions that inherently satisfy the boundary or initial conditions. In the original ANN-based solver of Lagaris et al. [4], the neural network was embedded in a trial solution of the form $u\left(x\right)=P\left(x\right)+Q\left(x\right)N(x,\theta )$, where $P\left(x\right)$ satisfies the boundary conditions and $Q\left(x\right)$ vanishes on the boundaries— thus the network $N(x,\theta )$ only learns the residual part of the solution and automatically respects the constraints. This strategy eliminates the need for boundary penalties and was shown to accelerate convergence compared to unconstrained networks. Following this approach, contemporary PINN works have employed similar hard-constrained trial solutions to improve training stability and accuracy [4,15]. Müller and Zeinhofer [15] analyzed the effect of imposing exact boundary values and proved that enforcing constraints explicitly can change the convergence behavior of residual minimization, leading to more efficient training. In our context of ODEs, we adopt a generalized trial solution strategy (inspired by these works) where the neural network output is composed with a function that vanishes at the boundaries, ensuring the PINN exactly satisfies the initial or boundary conditions from the outset. This approach, also used in some recent PINN extensions [14,15], removes the burden of penalty tuning and improves solution accuracy near the boundaries.

Another active area of research on PINNs is the development of theoretical error analyses and convergence guarantees. Early theoretical studies provided insight into why and when PINNs work. For example, Shin et al. proved the convergence of PINN solutions for certain linear elliptic and parabolic PDEs, under appropriate training conditions [14]. Building on this, Shin et al. derived a priori error estimates for PINNs by analyzing the residual minimization error for linear PDE problems, obtaining bounds on the solution error in terms of the PINN’s training loss [16]. Mishra and Molinaro focused on generalization error bounds for PINNs, showing that for some inverse PDE problems the error between the PINN solution and true solution can be bounded (with high probability) as the network capacity and training set grow, effectively providing learning guarantees for PINNs [17,18]. In the specific context of ODEs, Yoo et al. [19] established rigorous $L^2$ error estimates for PINNs solving initial value problems, proving that the PINN’s solution error is controlled by (and in fact, on the order of) the training loss—thus as the loss is driven to zero, the PINN solution converges to the true solution. This result offers a theoretical foundation that the PINN error will vanish if one can sufficiently minimize the loss [19]. Likewise, in the domain of fluid dynamics, Biswas et al. [20] provided one of the first analyses of PINN-type methods, deriving explicit error estimates and stability analyses for deep learning solutions of Navier–Stokes equations. Their work quantifies how the approximation error depends on network parameters and training iterations for a fixed network size [20]. These theoretical advances (e.g., [14,16,19,20,21]) not only increase confidence in PINNs but also guide the design of improved algorithms.

Beyond these, researchers have proposed various enhancements to improve PINN performance. For instance, adaptive activation functions and architecture changes (including the use of radial basis function networks or input feature embedding based on the Kolmogorov–Arnold theorem) have been explored to help the network train faster or capture complex solution behavior [22]. Domain-decomposition strategies and hybrid algorithms that combine PINNs with traditional solvers have achieved significant speed-ups and accuracy gains in large-scale problems [23]. Some studies integrate PINNs with multi-fidelity or transfer learning techniques to handle multi-scale physics, while others introduce Sobolev training and other modified loss functions to improve convergence in stiff or chaotic systems [24]. A recent trend is to develop open-source tools and libraries to facilitate PINN implementations—for example, NeuroDiffEq provides a Python library for solving ODEs/PDEs with neural networks, lowering the barrier to experimenting with PINN models [25]. With the community rapidly expanding, several extensive review articles have appeared that catalog these developments and outline future directions for PINNs [12,13]. These surveys highlight that while PINNs have achieved impressive results, there remain open challenges such as ensuring training stability, handling complex geometries or multi-physics coupling, and providing stronger theoretical guarantees for general cases.

In summary, the literature shows a clear trajectory from early ANN-based differential equation solvers [4,6,7] to the modern PINN framework [3] and its many variants. PINNs have been applied successfully across scientific and engineering domains, and researchers have introduced various improvements—from hard-constrained trial functions [4,15] and penalty/stability enhancements [26], to advanced training techniques and hybrid models; see also hands-on tutorials [27], to enhance their accuracy and efficiency. At the same time, our understanding of why PINNs work is improving thanks to new convergence and error analyses [14,16,19]. Building on these insights, the present work contributes to this evolving landscape by combining rigorous error estimates with an enhanced PINN formulation that employs generalized trial solutions for ODEs. Our approach aims to ensure that the PINN exactly satisfies the problem’s initial conditions (removing penalty-induced errors) while providing theoretical guarantees on the solution error. This fusion of improved trial function design with provable error bounds addresses some of the key limitations identified in prior works, moving one step closer to reliable and efficient physics-informed neural network solvers for ODEs.

The remainder of this paper is structured as follows: Section 2 introduces the mathematical background. In Section 3 the general formulation of the PINN framework is presented along with the construction of trial solutions tailored for first-order nonlinear ODEs. Section 4 presents the main theoretical results, including universal approximation properties and rigorous error estimates linking the training loss to the approximation error. Section 5 provides numerical experiments illustrating the accuracy and efficiency of the proposed method on benchmark nonlinear problems. Finally, Section 6 concludes the work and outlines future directions, including possible extensions to higher-order equations and inverse problems.

2. Mathematical Background

2.1. Cybenko’s Theorem and Universality of 3-Layer Wide Neural Networks

2.1.1. Cybenko’s Theorem

Let $\Omega \subset {\mathbb{R}}^d$ be compact. Let $\sigma :\mathbb{R}\to \mathbb{R}$. Let $f_{\theta }$ represent width-N 2-layer neural networks

$$f_{\theta }\left(x\right)=\sum _{i=1}^N{u}_i\sigma (a_i^{T}x+b_i),$$

(1)

in this formulation, $a_i\in {\mathbb{R}}^d$ are the weight vectors (often collectively denoted as a weight matrix), $b_i\in \mathbb{R}$ are the bias terms, and $\sigma$ is the activation function applied element-wise. The coefficients $u_i$ are the output weights that linearly combine the activated neurons’ outputs. Together, $\theta =\{a_i,b_i,u_i\}$ denotes the set of all network parameters, where $\theta \in {\mathsf{\Theta }}^{\left(N\right)}$ and

$${\mathsf{\Theta }}^{\left(N\right)}=\{(a_1,\dots ,a_N,b_1,\dots ,b_N,u_1,\dots ,u_N)\mid a_1,\dots ,a_N\in {\mathbb{R}}^d,b_1,\dots ,b_N,u_1,\dots ,u_N\in \mathbb{R}\}.$$

To clarify, ${\mathsf{\Theta }}^{\left(N\right)}\cong {\mathbb{R}}^{dN+2N}$.

Theorem 1 

(Theorem 1 in [28]). Let $\Omega \subset {\mathbb{R}}^d$ be compact. Let $\sigma :\mathbb{R}\to \mathbb{R}$ be a continuous function satisfying

$$\underset{r\to -\infty }{\lim }\sigma \left(r\right)=0,\underset{r\to \infty }{\lim }\sigma \left(r\right)=1.$$

The class of functions

$$\bigcup _{N\in \mathbb{N}}{\left\{f_{\theta }\right\}}_{\theta \in {\mathsf{\Theta }}^{\left(N\right)}}$$

is dense in ${C(\Omega ),\parallel · \parallel }_{\infty }$, i.e.,

$$\mathit{closure}\left(\mathit{span}\left({\left\{\sigma (a^{T}x+b)\right\}}_{a\in {\mathbb{R}}^d,b\in \mathbb{R}}\right)\right)={(C(\Omega ),\parallel · \parallel }_{\infty }).$$

The consequence of this theorem is that for any $f_{★}\in C(\Omega )$ and $\epsilon >0$, there exists a large enough N and network parameter $\theta \in {\mathsf{\Theta }}^{\left(N\right)}$ such that

$$\underset{x\in \Omega }{\sup }|f_{\theta }\left(x\right)-f_{★}\left(x\right)|<\epsilon .$$

Smaller $\epsilon$ will likely require larger N, but this theorem or its proof will not allow us to make any quantitative claims. Moreover, this is an existence result; it does not tell us how to find N and $\theta \in {\mathsf{\Theta }}^{\left(N\right)}$.

2.1.2. Universality of 3-Layer Wide Neural Networks

Consider three-layer neural networks of the form

$$f_{\theta }\left(x\right)=A^{\left(3\right)}{\sigma }_2\left(A^{\left(2\right)}{\sigma }_1\left(A^{\left(1\right)}x+b^{\left(1\right)}\right)+b^{\left(2\right)}\right)+b^{\left(3\right)},$$

(2)

where $N_1,N_2\in \mathbb{N}$, $A^{\left(1\right)}\in {\mathbb{R}}^{N_1\times d}$, $b^{\left(1\right)}\in {\mathbb{R}}^{N_1}$, $A^{\left(2\right)}\in {\mathbb{R}}^{N_2\times N_1}$, $b^{\left(2\right)}\in {\mathbb{R}}^{N_2}$, $A^{\left(3\right)}\in {\mathbb{R}}^{1\times N_2}$, $b^{\left(3\right)}\in {\mathbb{R}}^1$, ${\sigma }_1:\mathbb{R}\to \mathbb{R}$, ${\sigma }_2:\mathbb{R}\to \mathbb{R}$, and ${\sigma }_1$ and ${\sigma }_2$ are applied elementwise.

Universality of wide neural networks of depth 3 or deeper, requires a little bit of additional work to establish.

Theorem 2 

(Theorem 12 in [28]). Let $\Omega \subseteq {\mathbb{R}}^d$ be compact. Assume ${\sigma }_2\in C^{\infty }\left(\mathbb{R}\right)$ and ${\sigma }_1\in C\left(\mathbb{R}\right)$ are non-polynomial. Then the class of 3-layer neural networks of the form (2) are dense in $(C(\Omega ),\parallel ·{\parallel }_{\infty })$.

Theorem 2 provides a universality result for neural networks with three layers (depth-3), under the assumption that the activation functions ${\sigma }_1$ and ${\sigma }_2$ are non-polynomial, with ${\sigma }_2$ being infinitely differentiable. It confirms that such networks can approximate any continuous function defined on a compact domain $\Omega \subseteq {\mathbb{R}}^d$ arbitrarily closely in the uniform norm. This expands the scope of classical universal approximation theorems, which typically focus on depth-2 networks, by demonstrating that adding depth still preserves—if not enhances—the expressive power. Importantly, the result is purely existential: it ensures the existence of suitable network parameters and width but does not offer a constructive approach to identify them in practice.

2.2. Existence, Uniqueness of the ODE Solution

Consider the initial value problem given by the ODE,

$$\left\{\begin{array}{c}{u}^{\prime }\left(t\right)=G(t,u\left(t\right)),\hfill \\ u\left(t_0\right)=u_0,\hfill \end{array}\right.$$

(3)

with $(t_0,u_0)\in {\mathbb{R}}^2$. We assume that G is Lipschitz continuous in its second argument on the rectangle $[t_0-a_1,t_0+a_2]\times [u_0-b_1,u_0+b_2]$. In particular, suppose there exist constants $M_0>0$ and $K>0$ such that

$$\left|G(t,u)\right|\le M_0,\mathrm{for}\mathrm{all}(t,u)\mathrm{in}[t_0-a_1,t_0+a_2]\times [u_0-b_1,u_0+b_2],$$

and

$$|G(t,u_1)-G(t,u_2)|\le K|u_1-u_2|,$$

for all $(t,u_1)$ and $(t,u_2)$ in $[t_0-a_1,t_0+a_2]\times [u_0-b_1,u_0+b_2]$.

Define $a:=\min \{a_1,a_2\}$ and $b:=\min \{b_1,b_2\}$, and let

$$c:=\min \{a,b/M_0\}.$$

Consider the compact time interval $J:=[t_0-c,t_0+c]$. By construction of c, we have $J\subseteq [t_0-a_1,t_0+a_2]$ and also $c\le b/M_0$. The latter implies that any function on J with derivative bounded by $M_0$ stays in the strip $[u_0-b,u_0+b]$. In particular, any solution $u\left(t\right)$ of (3) on J will satisfy $u\left(t\right)\in [u_0-b,u_0+b]$ for all $t\in J$. Thus, a solution on J remains in the region where G is Lipschitz continuous and bounded.

It is a classical result (obtained via Picard iteration; see, e.g., ([1], Chapter 5, Section 4)) that under the above conditions there exists a continuously differentiable function $\phi :J\to \mathbb{R}$ which solves the initial value problem (3) on J. In other words, $\phi \left(t\right)$ satisfies

$${\phi }^{\prime }\left(t\right)=G(t,\phi \left(t\right))\mathrm{for}\mathrm{all}t\in J,\phi \left(t_0\right)=u_0,$$

and $\phi$ is the unique function with these properties. More precisely, if $\psi :I\to \mathbb{R}$ is any other continuously differentiable solution of (3) on some compact interval $I\subseteq [t_0-a_1,t_0+a_2]$ containing $t_0$, then $\psi \left(t\right)=\phi \left(t\right)$ for all $t\in I\cap J$. This establishes the existence and uniqueness of a local solution. Moreover, the solution $\phi \left(t\right)$ depends continuously on the initial data $(t_0,u_0)$, a property often referred to as the stability of the solution with respect to the initial condition.

3. Physics-Informed Neural Networks Method

3.1. Function Approximation via Neural Networks

In this section, we present the core concept of approximating real-valued functions using neural networks within the context of machine learning. A neural network, while biologically inspired by the structure of the human brain, is rigorously defined as a composition of affine transformations interlaced with nonlinear activation functions. Among the various architectures, the three-layer neural network (with two hidden layers) is particularly prominent and widely utilized in practice. This architecture offers a good balance between expressive power and ease of training. As shown by Chester [29], networks with two hidden layers can approximate complex functions more efficiently than single-layer networks, while avoiding the training difficulties of deeper architectures. To approximate a real-valued function on a subset of $\mathbb{R}$, we consider a mapping $N:\mathbb{R}\to \mathbb{R}$ defined as follows:

$$\begin{array}{|c|}\hline N(t,\theta )=A_3\sigma \left(A_2\sigma \left(A_{1}t+b_1\right)+b_2\right)+b_3\\ \hline\end{array},t\in \mathbb{R}.$$

(4)

Here, $A_1\in {\mathbb{R}}^{n_1\times 1}$, $A_2\in {\mathbb{R}}^{n_2\times n_1}$, and $A_3\in {\mathbb{R}}^{1\times n_2}$ are weight matrices, while $b_1\in {\mathbb{R}}^{n_1\times 1}$, $b_2\in {\mathbb{R}}^{n_2\times 1}$, and $b_3\in \mathbb{R}$ are bias vectors. The function $\sigma$ denotes a nonlinear activation function applied element-wise; in this context, we employ the hyperbolic tangent (tanh). The parameter vector $\theta$ encompasses all trainable components: $\theta =\theta (n_1,n_2,A_1,A_2,A_3,b_1,b_2,b_3)$.

The justification for employing neural networks to approximate functions is grounded in a fundamental result from functional analysis known as the universal approximation theorem (see [4,19]). This theorem guarantees that a neural network of sufficient width can uniformly approximate any continuous function on a compact domain. Specifically, given any continuous target function g defined on a compact interval I and any tolerance $\epsilon >0$, there exist integers $n_1,n_2\in \mathbb{N}$ and corresponding matrices $A_1$, $A_2$, $A_3$ and vectors $b_1$, $b_2$, $b_3$ such that the neural network $N(t,\theta )$ satisfies

$$\left|N\right(t,\theta )-g(t\left)\right|<\epsilon ,\mathrm{for}\mathrm{all}t\in I,$$

where $\theta =\theta (A_1,A_2,A_3,b_1,b_2,b_3)$. This result ensures that g can be approximated arbitrarily well by a suitable neural network. The practical challenge then becomes identifying the parameters $\theta$ that yield a satisfactory approximation. By fixing $n_1$ and $n_2$ to sufficiently large values, the task reduces to optimizing the entries of the matrices and vectors comprising $\theta$.

In supervised learning scenarios, the approximation is typically guided by empirical data. Suppose we are given a dataset of the form

$$D={\left\{(t_i,y_i)\right\}}_{1\le i\le n},\mathrm{where}{y}_i=g\left(t_i\right),$$

for some function g and $n\in \mathbb{N}$. The neural network is trained to minimize the discrepancy between its output and the data using the loss function

$$\mathrm{Loss}\left(\theta \right):={\left({\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(N(t_i,\theta )-y_i\right)}^2\right)}^{1/2}.$$

Optimization algorithms such as gradient descent are employed to adjust $\theta$ in a way that minimizes this loss, thereby aligning $N(·,\theta )$ with the underlying function g.

We now turn to the application of the neural network defined in (4) for solving initial value problems associated with ordinary differential equations, as specified in (3). Unlike data-driven tasks, in this context we do not rely on predefined datasets. Instead, the training objective stems from the differential operator itself, encoding the governing physical law. Accordingly, we will define a loss function based on the differential equation in order to guide the neural network towards a solution. The construction of such a loss function will be discussed in the following subsection.

3.2. Two Approaches to Approximating the Solution of (3) Using PINNs

We now focus on approximating the solution of the initial value problem (3) through the use of neural networks within the Physics-Informed Neural Network (PINN) framework.

PINN I: We begin by considering a neural network N of the form introduced in (4). Two distinct loss components are defined: the differential equation loss $L_{de}$ and the initial condition loss $L_{ic}$, given by

$$L_{de}\left(\theta \right):={\left({\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(N^{\prime }(t_i,\theta )-G(t_i,N(t_i,\theta ))\right)}^2\right)}^{1/2},L_{ic}\left(\theta \right):=|N(t_0,\theta )-u_0|.$$

(5)

Here, $S=\{t_1,\dots ,t_n\}$ denotes a set of sample points chosen uniformly at random from the domain where the solution is sought. We opted for the absolute error formulation

$$L_{ic}\left(\theta \right):=|N(t_0,\theta )-u_0|$$

to enforce a strict match at the initial condition. Since this term contributes only once to the overall loss (as opposed to the averaged residuals in $L_{de}\left(\theta \right)$), we found that the absolute error sufficiently penalizes deviations at the initial point without introducing unnecessary complexity.

That said, we acknowledge that using a mean squared error (MSE) formulation, such as

$$L_{ic}\left(\theta \right):={(N(t_0,\theta )-u_0)}^2,$$

is also a valid and commonly used alternative. This choice can yield smoother gradients, which might benefit optimization in certain settings. However, given that $L_{ic}$ is a single scalar term, the practical difference between absolute and squared error is often negligible unless weighted heavily.

The overall loss function is then defined as:

$$L\left(\theta \right):=L_{de}\left(\theta \right)+L_{ic}\left(\theta \right).$$

(6)

The parameters $\theta$ of the neural network are optimized to minimize $L\left(\theta \right)$, typically using gradient descent. It is important to emphasize that the loss function L does not represent the exact error between the true solution and the neural network approximation. However, as we will demonstrate in Section 4, a rigorous mathematical relationship exists between the loss and the actual approximation error.

PINN II: Since $L_{de}$ and $L_{ic}$ may not simultaneously approach zero during training, an alternative strategy involves modifying the trial function itself to enforce the initial condition exactly. Define a new function $N_A$ by

$$N_A(t,\theta ):=u_0+(t-t_0)N(t,\theta ),t\in \mathbb{R},$$

(7)

where N is the neural network given by (4). This construction guarantees that $N_A(t_0,\theta )=u_0$, thus inherently satisfying the initial condition of (3).

Correspondingly, we define the modified loss function $L_A$ based solely on the differential equation residual:

$$L_A\left(\theta \right)={\left({\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(N_A^{\prime }(t_i,\theta )-G(t_i,N_A(t_i,\theta ))\right)}^2\right)}^{1/2},$$

(8)

where the sampling set $S=\{t_1,\dots ,t_n\}$ is chosen as before. Optimization is then performed on $L_A\left(\theta \right)$ to find the best-fit parameters $\theta$ that minimize the residual.

4. Main Results

In general, the PIIN-II approach aims to construct an approximate solution to problem (3) that satisfies the initial, final, or boundary conditions. However, such constructions typically rely exclusively on polynomial-based representations, even when dealing with higher-order differential equations. To extend this framework beyond polynomials, we consider a more general trial function of the form:

$$N_{\varphi }(t,\theta ):=u_0+\varphi \left(t\right)·N(t,\theta ),t\in \mathbb{R},$$

(9)

specifically in the context of first-order ordinary differential equations. The objective is to identify the conditions under which the chosen function $\varphi$ enables our construction to provide a good approximation to the exact solution u of (3). Furthermore, we investigate whether minimizing the associated loss function effectively leads to a reduction in the approximation error between the predicted and exact solutions of (9).

Theorem 3. 

Under assumptions of (3) and (4), and let I be a compact interval in the domain of u with $t_0\in I$. Let ϕ be a function on I only vanishes at $t_0$, differentiable on $t_0$, and ${\varphi }^{\prime }\left(t_0\right)\ne 0$. Let

$$N_{\varphi }(t,\theta ):=u_0+\varphi \left(t\right)·N(t,\theta ),t\in \mathbb{R}.$$

(10)

Then $N_{\varphi }$ is a good uniform approximation for the solution to (3).

Proof of Theorem 3. 

By the existence and uniqueness theorem, the solution u to (3) is continuously differentiable on its domain containing $t_0$. Now define a function v on I as follows:

$$\left\{\begin{array}{cc}v\left(t\right)={\displaystyle \frac{u\left(t\right)-u_0}{\varphi \left(t\right)}}\hfill & \mathrm{if}t\in I\setminus \left\{t_0\right\},\hfill \\ v\left(t_0\right)={\displaystyle \frac{u^{\prime }\left(t_0\right)}{{\varphi }^{\prime }\left(t_0\right)}}.\hfill \end{array}\right.$$

It is clear that v is continuous on $I\setminus \left\{t_0\right\}$. At $t_0$, it is sufficient to observe that, under the assumptions of Theorem

$$v\left(t\right)={\displaystyle \frac{u\left(t\right)-u_0}{\varphi \left(t\right)}}={\displaystyle \frac{u\left(t\right)-u_0}{t-t_0}}\times {\displaystyle \frac{t-t_0}{\varphi \left(t\right)}}$$

it converges to $v\left(t_0\right)$ as t approaches $t_0$.

By the universal approximation theorem for 3-layer neural networks (Theorem 2), given $\epsilon >0$ there exist $n_1,n_2\in \mathbb{N}$, $n_1\times 1$, $n_2\times n_1$, $1\times n_2$ matrices $A_1$, $A_2$ and $A_3$, respectively, and $n_1\times 1$, $n_2\times 1$ and $1\times 1$ matrices $b_1$, $b_2$, $b_3$, respectively, such that if $\theta =\theta (n_1,n_2,A_1,A_2,A_3,b_1,b_2,b_3)$, then

$$|N(t,\theta )-v\left(t\right)|<{\displaystyle \frac{\epsilon }{\alpha }},\mathrm{for}\mathrm{all}t\in I,$$

where $\alpha =\max \{\mid \varphi (t)\mid ;t\in I\}$. Thus, we obtain that

$$u\left(t\right)=u_0+\varphi \left(t\right)·v\left(t\right)\mathrm{for}\mathrm{all}t\in I,$$

and

$$|N_{\varphi }(t,\theta )-u\left(t\right)|=\left|\varphi \left(t\right)·\left(N(t,\theta )-v\left(t\right)\right)\right|\le \alpha ·|N(t,\theta )-v\left(t\right)|<\epsilon \mathrm{for}\mathrm{all}t\in I.$$

Thus, we can regard $N_{\varphi }(·,\theta )$ as a good approximation of u in the perspective of the universal approximation theorem.    □

Example 1. 

By application of Theorem 3, all function below are good uniform approximation for the solution u of the problem (3).

$$N_1(t,\theta ):=y_0+(t-t_0)N(t,\theta ),t\in \mathbb{R},$$

(11)

$$N_2(t,\theta ):=y_0+(1-e^{-(t-t_0)})N(t,\theta ),t\in \mathbb{R},$$

(12)

$$N_3(t,\theta ):=y_0+\sin (t-t_0)N(t,\theta ),t\in \mathbb{R},$$

(13)

$$N_4(t,\theta ):=y_0+\left({\displaystyle \frac{1}{1+e^{-(t-t_0)}}}-{\displaystyle \frac{1}{2}}\right)N(t,\theta ),t\in \mathbb{R},$$

(14)

$$N_5(t,\theta ):=y_0+\left(\ln \left(1+e^{t-t_0}\right)-\ln \left(2\right)\right)N(t,\theta ),t\in \mathbb{R}.$$

(15)

Given an interval $J=[a,b]$, we write $\left|J\right|=b-a$. Theorems 4 and 5 are mathematical results to derive error estimates for PINN II.

Theorem 4. 

Let $(t_0,u_0)\in {\mathbb{R}}^2$, and suppose $G:[t_0-a_1,t_0+a_2]\times [u_0-b_1,u_0+b_2]\to \mathbb{R}$ is continuous in both arguments and Lipschitz continuous in u with Lipschitz constant $\kappa >0$, and $M_0>0$.

Let $u\left(t\right)$ denote the unique solution of the initial value problem (3), existing on an interval $I:=[t_0-c_1,t_0+c_2]$ for some $c_1,c_2>0$.

Let $N_{\varphi }(t;\theta )$ defined under the same conditions of Theorem 3.

Let

$${\mathcal{L}}_{\varphi }\left(\theta \right):={\int }_I|N_{\varphi }^{\prime }(s,\theta )-G(s,N_{\varphi }(s,\theta ))|ds,$$

(16)

Then,

$$|N_{\varphi }(t,\theta )-u\left(t\right)|\le e^{\kappa |t-t_0|}{\mathcal{L}}_{\varphi }\left(\theta \right)\mathit{for}\mathit{all}t\in I.$$

Proof of Theorem 4. 

The proof is based on [25], using the triangle and the Grönwall’s inequalities.

Under the stated assumptions, the error $\tau \left(t\right):=N_{\varphi }(t;\theta )-u\left(t\right)$ satisfies $\tau \left(t_0\right)=0$ and

$${\tau }^{\prime }\left(t\right)=N_{\varphi }^{\prime }(t;\theta )-u^{\prime }\left(t\right)=N_{\varphi }^{\prime }(t;\theta )-G(t,N_{\varphi }(t;\theta ))-\left(u^{\prime }\left(t\right)-G(t,N_{\varphi }(t;\theta ))\right).$$

Using $u^{\prime }\left(t\right)=G(t,u\left(t\right))$ and the Lipschitz continuity of G in its second argument, we obtain the differential inequality

$$\left|{\tau }^{\prime }\left(t\right)\right|=|N_{\varphi }^{\prime }(t;\theta )-G(t,N_{\varphi }(t;\theta ))-\left(G(t,u\left(t\right))-G(t,N_{\varphi }(t;\theta ))\right)|\le \rho \left(t\right)+\kappa \left|\tau \left(t\right)\right|,$$

where

$$\rho (t,\theta )=|N_{\varphi }^{\prime }(t,\theta )-G(t,N_{\varphi }(t,\theta ))|$$

(17)

for all $t\in I$. By Grönwall’s inequality (applied on $[t_0,t]$ or $[t,t_0]$ as appropriate), it follows that

$$\left|\tau \left(t\right)\right|\le e^{\kappa |t-t_0|}{\int }_I\rho (s,\theta )ds=e^{\kappa |t-t_0|}{\mathcal{L}}_{\varphi }\left(\theta \right),$$

for any $t\in I$, where we set ${\mathcal{L}}_{\varphi }\left(\theta \right):={\int }_I\rho (s,\theta )ds$. This ${\mathcal{L}}_{\varphi }\left(\theta \right)$ represents the total residual loss over I.    □

Theorem 5. 

Under the same conditions of Theorems 1 and 2, let ${\left\{{\xi }_i\right\}}_{i=1}^n$ be an independent, identically distributed sample of n random points, each drawn from the uniform distribution on I. Then, for any $t\in I$, the following error estimate holds

$$|N_{\varphi }(t;\theta )-u\left(t\right)|\le \left|I\right|e^{\kappa |t-t_0|}\left(\mathbb{E}{\left[{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(N_{\varphi }^{\prime }({\xi }_i;\theta )-G({\xi }_i,N_{\varphi }({\xi }_i;\theta ))\right)}^2\right]}^{1/2}+{\displaystyle \frac{\nu }{\sqrt{n}}}\right),$$

where $\nu :={\left({\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds-{\left({\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho \left(s\right)ds\right)}^2\right)}^{1/2}\ge 0$ is the standard deviation of the pointwise residual on I, and we define $\rho (t,\theta )$ as in (17), for $t\in I$.

Proof of Theorem 5. 

Consider the n i.i.d. sample points ${\xi }_1,\dots ,{\xi }_n$ uniform on I. Define the random residual values $Y_i:=\rho \left({\xi }_i\right)=|N_{\varphi }^{\prime }({\xi }_i;\theta )-G({\xi }_i,N_{\varphi }({\xi }_i;\theta ))|$ for $1\le i\le n$, and let $\overline{Y}:={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{Y}_i$ be the sample average of the residual magnitudes. By construction, $\mathbb{E}\left[Y_i\right]={\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho \left(s\right)ds={\displaystyle \frac{\mathcal{E}\left(\theta \right)}{\left|I\right|}}$. Denoting $m:=\mathbb{E}\left[\overline{Y}\right]=\mathbb{E}\left[Y_i\right]$ (the true mean residual) and ${\sigma }^2:=\mathrm{Var}\left(Y_i\right)={\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds-{\left({\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho \left(s\right)ds\right)}^2$ (so that $\sigma =\nu$ in the notation of the theorem), standard properties of independent sampling give $\mathrm{Var}\left(\overline{Y}\right)={\sigma }^2/n$. Next, applying the triangle inequality in $L^2(\Omega )$ (with respect to the underlying probability space of the sample), we obtain

$$\left|m\right|=\parallel m{\parallel }_{L^2}\le \parallel \overline{Y}{\parallel }_{L^2}+\parallel \overline{Y}-m{\parallel }_{L^2}=\sqrt{\mathbb{E}\left[{\overline{Y}}^2\right]}+{\displaystyle \frac{\sigma }{\sqrt{n}}}.$$

Using Jensen’s inequality (or the fact $\mathbb{E}\left[{\overline{Y}}^2\right]\le \mathbb{E}\left[Y_1^2\right]$ for $n\ge 1$), we further have $\sqrt{\mathbb{E}\left[{\overline{Y}}^2\right]}\le \sqrt{\mathbb{E}\left[Y_1^2\right]}$. But $\mathbb{E}\left[Y_1^2\right]={\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds$. Combining these estimates yields

$$\mathbb{E}\left[Y_i\right]=m\le {\displaystyle \frac{\sigma }{\sqrt{n}}}+\sqrt{{\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds}.$$

Multiplying through by $\left|I\right|$, we conclude that

$${\mathcal{L}}_{\varphi }\left(\theta \right)={\int }_I\rho \left(s\right)ds\le \left|I\right|\left(\sqrt{{\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds}+{\displaystyle \frac{\sigma }{\sqrt{n}}}\right).$$

Finally, substituting this bound on ${\mathcal{L}}_{\varphi }\left(\theta \right)$ back into the earlier error estimate for $\left|e\right(t\left)\right|$ gives

$$|N_{\varphi }(t;\theta )-u\left(t\right)|\le e^{\kappa |t-t_0|}{\mathcal{L}}_{\varphi }\left(\theta \right)\le \left|I\right|e^{\kappa |t-t_0|}\left(\sqrt{{\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds}+{\displaystyle \frac{\sigma }{\sqrt{n}}}\right),$$

which is exactly the stated inequality (since $\mathbb{E}\left[{\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{(N_{\varphi }^{\prime }({\xi }_i;\theta )-G({\xi }_i,N_{\varphi }({\xi }_i;\theta )))}^2\right]={\displaystyle \frac{1}{\left|I\right|}}{\int }_I\rho {\left(s\right)}^{2}ds$). This completes the proof.    □

This theoretical result illustrates that minimizing the PINN’s training loss directly leads to a small error in the solution. In particular, the bound shows that if the residual $R^{\prime }(t;\theta )-G(t,R(t;\theta ))$ (which is what the loss function penalizes) is small in the mean-square sense, then the network output $R(t;\theta )$ stays uniformly close to the true solution $u\left(t\right)$ on I (up to the stability factor $e^{\kappa |t-t_0|}$ arising from the ODE’s dynamics). In essence, a smaller training loss (residual error) guarantees a smaller actual solution error, corroborating the fundamental principle that PINN training drives the approximate solution towards the correct solution. This kind of error estimate is important as it provides rigorous assurance that the PINN’s learned solution will converge to the true solution as the training loss is minimized (and as $n\to \infty$, the sampling error term $\nu /\sqrt{n}$ becomes negligible). corroborating the fundamental principle that PINN training drives the approximate solution towards the correct solution. This kind of error estimate is important as it provides rigorous assurance that the PINN’s learned solution will converge to the true solution as the training loss is minimized (and as $n\to \infty$, the sampling error term $\nu /\sqrt{n}$ becomes negligible).

5. Experiment Results

In this section, we provide a numerical validation of the derived error estimates by examining the relationship between the approximation error and the training loss functions. This analysis is conducted within the framework of our proposed PINN construction, applied to various differential equations. The experiments are performed using different trial functions, as specified in examples.

The neural network employed in this study follows a fully connected architecture composed of three layers: an input layer, two hidden layers with 16 and 32 neurons, respectively, and a single-node output layer. The activation function used in the hidden layers is the hyperbolic tangent (tanh), which is known for its smoothness and effectiveness in approximating nonlinear functions. The network is trained using the Adam optimizer [30], with a fixed learning rate set to $0.01$. The training process is performed over 1000 epochs, utilizing 10,000 uniformly distributed collocation points within the domain $[0,1]$. The trial solution is constructed to satisfy the initial condition by design, and the loss function is defined as the mean squared residual of the differential equation evaluated at the training points. The training aims to minimize this physics-informed loss, thereby guiding the network to approximate the true solution of the initial value problem.

5.1. Example 1

Consider the following initial value problem:

$${\displaystyle \frac{dy}{dt}}=1.27y(1-y),t\in [0,1],y\left(0\right)=0.67.$$

The exact solution is given by

$$y\left(t\right)={\displaystyle \frac{1}{1+\frac{33}{67}{e}^{-1.27t}}},t\in [0,1].$$

In Figure 1, we illustrate both the approximation error and the training loss obtained during the PINN training process. This visualization helps to assess the consistency between the empirical loss minimization and the actual error behavior.

In this example, we tested five different functions to construct trial solutions, each defined explicitly in

Table 1. Table of several trial solution construction.

Function	Trial Solution Formula	Figure	Max Absolute Error ${\mathit{L}}_{\infty }$
1 *	$y\left(0\right)+tN(t,\theta )$	Figure 1a	$6.62\times {10}^{-5}$
2	$y\left(0\right)+(1-e^{-t})N(t,\theta )$	Figure 1b	$2.42\times {10}^{-5}$
3	$y\left(0\right)+\sin \left(t\right)N(t,\theta )$	Figure 1c	$8.66\times {10}^{-5}$
4	$y\left(0\right)+\left({\displaystyle \frac{1}{1+e^{-\left(t\right)}}}-{\displaystyle \frac{1}{2}}\right)N(t,\theta )$	Figure 1d	$8.78\times {10}^{-5}$
5	$y\left(0\right)+\left(\ln \left(1+e^t\right)-\ln \left(2\right)\right)N(t,\theta )$	Figure 1e	$1.48\times {10}^{-4}$

* The standard polynomial construction.

. Figure 1 presents the training loss and approximation error corresponding to these trial solutions, shown in the same order as in the table.

We now consider two constructions of the trial solution in the form (10). In each example of

Table 2. Table of several trial solution construction.

Function	Trial Solution Formula	Figure	Max Absolute Error ${\mathit{L}}_{\infty }$
1	$y\left(0\right)+t(t-0.2)N(t,\theta )$	Figure 2a	$5.66\times {10}^{-2}$
2	$y\left(0\right)+t^{2}N(t,\theta )$	Figure 2b	$1.07\times {10}^{-2}$

, the function $\varphi$ is deliberately chosen so that it does not satisfy one of the assumptions of Theorem 3. This approach is intended to demonstrate the necessity of the stated conditions. For each case, we provide the plots of both the trial and exact solutions in order to visually compare the results (see Figure 2).

We observe that in Example 1 (the logistic ODE case), all five trial solutions that satisfy the assumptions of Theorem 3 produce highly accurate approximations. In these admissible cases, where $\varphi \left(t_0\right)=0$ and ${\varphi }^{\prime }\left(t_0\right)\ne 0$, the maximum absolute error $L_{\infty }$ remains on the order of ${10}^{-5}$. Table 1 reports the corresponding values, with the smallest and largest errors being $2.42\times {10}^{-5}$ and $1.48\times {10}^{-4}$, respectively. Figure 1a–e confirms this observation, showing that the training loss decreases rapidly and reaches very small values. In contrast, when non-admissible trial functions are employed (as in Table 2), the error increases by at least two orders of magnitude. For example, selecting $\varphi \left(t\right)=t^2$ (which violates ${\varphi }^{\prime }\left(t_0\right)\ne 0$) results in $L_{\infty }\approx 1.07\times {10}^{-2}$, while $\varphi \left(t\right)=(t-t_0)(t-0.2)$ leads to $L_{\infty }\approx 5.66\times {10}^{-2}$. These outcomes are illustrated in Figure 2a,b, where a clear divergence between the predicted and exact solutions is visible. Overall, these results highlight that only the trial solution satisfying the theoretical constraints (a function $\varphi$ vanishing solely at $t_0$ with a nonzero derivative there) yields the near-perfect approximations reported in Table 1.

5.2. Example 2

Consider the following initial value problem:

$${\displaystyle \frac{dy}{dt}}={\displaystyle \frac{4t^3+3t^2+6t+2}{2y+4}},t\in [0,1],y\left(0\right)=0.$$

The exact solution is given by

$$y\left(t\right)=-2+\sqrt{t^4+t^3+3t^2+2t+4},t\in [0,1].$$

The following table presents several distinct constructions as described in Theorem 3. These constructions are illustrated in the accompanying figure which are arranged in

Table 3. Table of several trial solution construction.

Function	Trial Solution Formula	Figure	Max Absolute Error ${\mathit{L}}_{\infty }$
1 *	$y\left(0\right)+(t-t_0)N(t,\theta )$	Figure 3	$3.73\times {10}^{-4}$
2	$y\left(0\right)+\log (1+t-t_0)N(t,\theta )$	Figure 4	$4.09\times {10}^{-4}$
3	$y\left(0\right)+\sin (t-t_0)N(t,\theta )$	Figure 5	$4.06\times {10}^{-4}$
4	$y\left(0\right)+\tan (t-t_0)N(t,\theta )$	Figure 6	$9.10\times {10}^{-4}$
5	$y\left(0\right)+(1-e^{(t-t_0)})N(t,\theta )$	Figure 7	$7.86\times {10}^{-4}$

* The standard polynomial construction.

.

From Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7, we illustrate both the approximation error and the training loss obtained during the PINN training process. In addition the exact and the predicted solutions are traced to give a simple visual comparison.

As in the first example, we give now in

Table 4. Table of several trial solution construction.

Function	Trial Solution Formula	Figure	Max Absolute Error ${\mathit{L}}_{\infty }$
1	$y\left(0\right)+\cos (t-t_0)N(t,\theta )$	Figure 8a	$1.34\times {10}^0$
2	$y\left(0\right)+{(1-\exp (t-t_0))}^{2}N(t,\theta )$	Figure 8b	$3.71\times {10}^{-2}$
3	$y\left(0\right)+(t-t_0)(t-0.5)N(t,\theta )$	Figure 8c	$4.38\times {10}^{-1}$

some choices of the function $\varphi$ that do not satisfy a condition for Theorem 3. We plot the predicted solution in each case with the exact solution to give the comparison.

In Example 2 the same pattern holds. All admissible choices of $\varphi$ (those meeting $\varphi \left(t_0\right)=0$, ${\varphi }^{\prime }\left(t_0\right)\ne 0$) again produce very small errors (on the order of ${10}^{-4}$). Table 3 lists the five valid trial functions and their errors: for instance, $\varphi \left(t\right)=t-t_0$, $\log (1+t-t_0)$, and $\sin (t-t_0)$ yield $L_{\infty }\approx 3.73\times {10}^{-4}$, $4.09\times {10}^{-4}$, and $4.06\times {10}^{-4}$, respectively, while $\varphi \left(t\right)=\tan (t-t_0)$ and $1-e^{-(t-t_0)}$ both remain below ${10}^{-4}$. In each case the predicted solution curve overlaps almost exactly with the exact solution (Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7). In contrast, violating the construction assumptions causes large errors. Table 4 shows that using $\varphi \left(t\right)=\cos (t-t_0)$ (so that $\varphi \left(t_0\right)\ne 0$) gives $L_{\infty }\approx 1.34$, and $\varphi \left(t\right)=(t-t_0)(t-0.5)$ gives $L_{\infty }\approx 4.38\times {10}^{-1}$. These errors (two to three orders of magnitude larger) are evident in Figure 8, where the predicted and exact curves clearly diverge. Thus, as in Example 1, only the trial solutions satisfying the theoretical conditions (Table 3) achieve high accuracy.

Both examples demonstrate that the theoretical requirements on the trial function $\varphi$ are decisive for PINN accuracy. When $\varphi \left(t_0\right)=0$, $\varphi$ is differentiable and ${\varphi }^{\prime }\left(t_0\right)\ne 0$, training converges efficiently with very small errors ($L_{\infty }\sim {10}^{-4}$). In contrast, ignoring these assumptions causes errors to grow by orders of magnitude (often into the ${10}^{-1}$–${10}^0$ range) and yields slow or unstable convergence. These numerical findings (supported by Table 1, Table 2, Table 3 and Table 4 and Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 confirm the paper’s theoretical framework and underscore the critical importance of constructing the trial solution to satisfy the stated conditions.

5.3. Example 3

To probe a more challenging regime, we consider the linear ODE with high-frequency forcing

$${\displaystyle \frac{dy}{dt}}+y=\cos \left(10\pi t\right),t\in [0,1],y\left(0\right)=0.$$

The exact solution is given by:

$$y\left(t\right)={\displaystyle \frac{10\pi \sin \left(10\pi t\right)+\cos \left(10\pi t\right)}{1+100{\pi }^2}}-{\displaystyle \frac{e^{-t}}{1+100{\pi }^2}}$$

In this setting we intentionally restrict the collocation set to 70 points and allow up to $\mathrm{10,000}$ training epochs so as to stress-test convergence under stricter sampling and training budgets. We then evaluate our hard-constrained formulation (PINN II) with several admissible trial functions $\varphi$ (satisfying $\varphi \left(t_0\right)=0$ and ${\varphi }^{\prime }\left(t_0\right)\ne 0$) alongside non-admissible choices that violate these conditions, thereby directly testing the hypothesis of Theorem 3.

Table 5. Table of several trial solution construction.

Function	Trial Solution Formula	Figure	Max Absolute Error ${\mathit{L}}_{\infty }$
1	$y\left(0\right)+(t-t_0)N(t,\theta )$	Figure 9	$2.05\times {10}^{-3}$
2	$y\left(0\right)+\log (1+t-t_0)N(t,\theta )$	Figure 10	$3.09\times {10}^{-4}$
3	$y\left(0\right)+\sin (t-t_0)N(t,\theta )$	Figure 11	$4.76\times {10}^{-4}$
4	$y\left(0\right)+{\displaystyle \frac{1}{1+\exp (-(t-t_0))}}N(t,\theta )$	Figure 12	$2.16\times {10}^{-3}$
5	$y\left(0\right)+(1-e^{(t-t_0)})N(t,\theta )$	Figure 13	$2.84\times {10}^{-3}$
6 *	$y\left(0\right)+\cos (t-t_0)N(t,\theta )$	Figure 14	$3.53\times {10}^0$
7 *	$y\left(0\right)+{(t-t_0)}^{2}N(t,\theta )$	Figure 15	$1.20\times {10}^{-2}$

* In this case, one of the assumptions required by Theorem 3 is not satisfied.

and Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14 and Figure 15 show a clear pattern: admissible PINN II trials reach sub-millipercent accuracy (with $L_{\infty }$ typically in the ${10}^{-4}$–${10}^{-3}$ range) and produce solution curves that closely track the exact response, while non-admissible constructions (e.g., $\varphi \left(t\right)=\cos (t-t_0)$ or squared/exponentially–flattened variants) remain orders of magnitude less accurate within the same budget and exhibit visible deviations in the plots.

These results reinforce the necessity of the admissibility conditions in Theorem 3 and mirror the trends observed in the earlier examples: when $\varphi$ is well-posed, the hard-constrained trial drives faster and steadier error decay; when $\varphi$ violates admissibility, convergence degrades substantially despite identical training protocols.

5.4. Comparison with the Vanilla PINN I Approach

We compare our hard-constrained formulation (PINN II) against the standard soft-constraint approach (PINN I) under a common protocol: identical network $[1;16;32;1]$ and adam-optimizer, the same collocation (70 points for Examples 1 and 2; 100 for Example 4), and an early-stopping target on the test grid ($L_{\infty }\le 7\times {10}^{-5}$ for Example 1 and $L_{\infty }\le 7\times {10}^{-4}$ for Example 2), otherwise a cap of $\mathrm{10,000}$ epochs. Errors and stopping criteria are computed on an independent uniform test grid (e.g., 2001 points on $[0,1]$), disjoint from the collocation points.

For PINN I we sweep the initial-condition penalty weight ${\alpha }_{\mathrm{IC}}\in \{0.1,1,10,100\}$, while for PINN II we test admissible and non-admissible trial functions $\varphi$.

Table 6. Epochs required to reach $L_{\infty }\le 7\times {10}^{-5}$: PINN-I (with various initial-condition penalty weight ${\alpha }_{\mathrm{IC}}$) vs. PINN-II (using different admissible/non-admissible trial function $\varphi$) on Example 1.

Method	Variant	Number of Epoch
PINN I	${\alpha }_{IC}=0.1$	700
PINN I	${\alpha }_{IC}=1$	1300
PINN I	${\alpha }_{IC}=10$	1700
PINN I	${\alpha }_{IC}=100$	3600
PINN II	$\varphi \left(t\right)=t$	1100
PINN II	$\varphi \left(t\right)=\exp \left(t\right)-1$	1200
PINN II	$\varphi \left(t\right)={\displaystyle \frac{1}{1+e^{-t}}}-{\displaystyle \frac{1}{2}}$	600
PINN II	$\varphi \left(t\right)=\ln (1+t)$	500
* PINN II	$\varphi \left(t\right)=\cos \left(t\right)$	10,000
* PINN II	$\varphi \left(t\right)=t^2$	10,000

* Non admissible trial function.

and

Table 7. Epochs required to reach $L_{\infty }\le 7\times {10}^{-4}$: PINN-I (with various initial-condition penalty weight ${\alpha }_{\mathrm{IC}}$) vs. PINN-II (using different admissible/non-admissible trial function $\varphi$) on Example 2.

Method	Variant	Number of Epoch
PINN I	${\alpha }_{IC}=0.1$	1600
PINN I	${\alpha }_{IC}=1$	1100
PINN I	${\alpha }_{IC}=10$	1800
PINN I	${\alpha }_{IC}=100$	6200
PINN II	$\varphi \left(t\right)=t$	700
PINN II	$\varphi \left(t\right)=\exp \left(t\right)-1$	700
PINN II	$\varphi \left(t\right)=\sin \left(t\right)$	800
PINN II	$\varphi \left(t\right)={\displaystyle \frac{1}{1+e^{-t}}}-{\displaystyle \frac{1}{2}}$	800
PINN II	$\varphi \left(t\right)=\ln (1+t)$	1100
* PINN II	$\varphi \left(t\right)=\cos \left(t\right)$	10,000
* PINN II	$\varphi \left(t\right)=t^2$	10,000

* Non admissible trial function.

report the number of epochs required to meet the accuracy target, and Figure 16, Figure 17 and Figure 18 display the corresponding $L_{\infty }$ trajectories.

Across both examples, the data reveal a consistent pattern. In Example 1 (Table 6), admissible PINN II trials achieve the target typically within 500–1200 epochs (e.g., $\varphi \left(t\right)=\log (1+t)$ at 500 and $\varphi \left(t\right)=\sigma \left(t\right)-{\displaystyle \frac{1}{2}}$ at 600), whereas PINN I is sensitive to the penalty weight, ranging from 700 epochs (${\alpha }_{\mathrm{IC}}=0.1$) to 3600 epochs (${\alpha }_{\mathrm{IC}}=100$). In Example 2 (Table 7), admissible PINN II trials again reach the target in 700–1100 epochs, while PINN I requires 1100–6200 epochs depending on ${\alpha }_{\mathrm{IC}}$. Non-admissible choices (e.g., $\varphi \left(t\right)=\cos t$ or $t^2$) systematically fail to meet the threshold within $\mathrm{10,000}$ epochs, underscoring the necessity of the admissibility conditions. The $L_{\infty }$-vs-epoch curves in Figure 16 and Figure 17 corroborate these trends: admissible PINN II exhibits faster and steadier descent to the dashed target line, whereas PINN I shows slower or oscillatory convergence as ${\alpha }_{\mathrm{IC}}$ varies.

Example 4

To probe robustness in a stiff, oscillatory regime, we consider a high-frequency linear ODE and compare the hard-constrained formulation (PINN-II) with the vanilla soft-constraint baseline (PINN-I) (see Figure 18). Anticipating the theory, admissible trial functions in PINN-II track the exact dynamics (see Figure 19) and reach small $L_{\infty }$ errors markedly faster, whereas PINN-I is sensitive to the penalty ${\alpha }_{\mathrm{IC}}$ and non-admissible choices degrade accuracy.

The differential equation is:

$$y^{\prime }+100y=\cos \left(10\pi t\right),y\left(0\right)=0,$$

with the exact solution:

$$y\left(t\right)={\displaystyle \frac{-100e^{-100t}+100\cos \left(10\pi t\right)+10\pi \sin \left(10\pi t\right)}{10000+100{\pi }^2}}$$

6. Conclusions

In conclusion, we have developed a generalized hard-constrained trial solution strategy for physics-informed neural networks (PINNs) and demonstrated its effectiveness through both theoretical analysis and comprehensive experiments. The main theoretical contribution is a general framework for constructing admissible trial functions that exactly satisfy prescribed initial or boundary conditions by design. We proved that these hard-constrained trial functions preserve the universal approximation capability of PINNs, and we established error bounds showing that the solution error (in the $L^2$ sense) is proportional to the PINN training loss under mild assumptions, providing rigorous performance guarantees. This theoretical foundation is reinforced by extensive numerical results: the proposed approach exhibits significantly faster convergence and improved training stability compared to the standard “vanilla” PINN formulation (PINN-I) that uses soft constraints. Across multiple first-order ODE examples—including a challenging stiff oscillatory case—our method consistently achieved high accuracy and robust performance for various choices of trial function $\varphi$. By contrast, the vanilla PINN converged more slowly and showed sensitivity to penalty weight tuning. In direct comparisons, the hard-constrained PINN reached target error thresholds with far fewer epochs and maintained a steadier error decay, highlighting its advantages in both efficiency and error control. We acknowledge that the present study is limited to first-order scalar ODEs, and our theoretical results apply to problems with Lipschitz-continuous right-hand sides. Extensions to non-smooth or weak-solution settings, and systematic studies of other hyperparameters constitute promising directions for future work. Looking ahead, we plan to extend this framework to more complex settings, such as second-order differential equations and partial differential equations, to assess its broader applicability. Additionally, combining the hard-constraint approach with other PINN enhancements (for instance, adaptive loss-weighting schemes or domain decomposition methods) is a promising direction for further improving convergence and tackling even more challenging physics-informed modeling problems.