Differential Transform Method and Neural Network for Solving Variational Calculus Problems

Abstract

The history of variational calculus dates back to the late 17th century when Johann Bernoulli presented his famous problem concerning the brachistochrone curve. Since then, variational calculus has developed intensively as many problems in physics and engineering are described by equations from this branch of mathematical analysis. This paper presents two non-classical, distinct methods for solving such problems. The first method is based on the differential transform method (DTM), which seeks an analytical solution in the form of a certain functional series. The second method, on the other hand, is based on the physics-informed neural network (PINN), where artificial intelligence in the form of a neural network is used to solve the differential equation. In addition to describing both methods, this paper also presents numerical examples along with a comparison of the obtained results.Comparingthe two methods, DTM produced marginally more accurate results than PINNs. While PINNs exhibited slightly higher errors, their performance remained commendable. The key strengths of neural networks are their adaptability and ease of implementation. Both approaches discussed in the article are effective for addressing the examined problems.

1. Introduction

The issues in the calculus of variations have rich applications in many different problems [1]. Using these methods, one can, for example, search for curves connecting fixed points (also considering problems with a moving boundary) and satisfying certain conditions (such as minimal length or the shortest time to travel between them), surfaces meeting specific conditions (such as the surface with the smallest area connecting two fixed circles), problems concerning the propagation of light in inhomogeneous media, isoperimetric problems, geodesic curves, quantum mechanics, hydrodynamics, and many others [2,3,4,5].

The problem of variational calculus can be described in many different ways; this paper focuses on finding the extremum of the following functional:

$$\mathcal{J}\left(y\left(x\right)\right)={\int }_a^{b}F\left(x,y\left(x\right),y^{\prime }\left(x\right),\dots ,y^{\left(n\right)}\left(x\right)\right)dx,$$

(1)

where $y\left(x\right)$ is the sought-after n-times differentiable function defined in the interval $[a,b]$, which satisfies the following boundary conditions:

$$y^{\left(i\right)}\left(a\right)=A_i,y^{\left(i\right)}\left(b\right)=B_i,0\le i\le n-1,$$

(2)

where $a,b\in \mathbb{R}$, $a<b$, $A_i,B_i\in \mathbb{R}$, $0\le i\le n-1$, and F is a certain function of $(n+2)$ variables’ $(n+1)$-times differential, defined in the set $[a,b]\times {\mathbb{R}}^{n+1}$.

For such a formulated problem, a necessary condition for the functional (1) to attain a local extremum for the sought-after function $y\left(x\right)$ under the boundary conditions (2) is that the function F satisfies the following differential equation:

$$\frac{\partial F}{\partial y}-\frac{d}{dx}\left(\frac{\partial F}{\partial y^{\prime }}\right)+\frac{d^2}{d{x}^2}\left(\frac{\partial F}{\partial y^{″}}\right)+\dots +{(-1)}^n\frac{d^n}{d{x}^n}\left(\frac{\partial F}{\partial y^{\left(n\right)}}\right)=0,$$

(3)

(this is the so-called Euler–Poisson equation) with the conditions (2) [6,7].

In the general case, the condition (3) is only a necessary condition for the existence of an extremum of the functional (1). However, in most technical problems, it is also a sufficient condition. Therefore, in this paper, we search for these extrema in this manner.

Equation (3) is a differential equation of the order $2n$ and appears relatively rarely for larger values of n. It is more commonly encountered for $n=1$ or $n=2$. In these cases, it takes the following form:

$$\frac{\partial F}{\partial y}+\sum _{i=1}^n{(-1)}^i\frac{d^i}{d{x}^i}\left(\frac{\partial F}{\partial y^{\left(i\right)}}\right)=\left\{\begin{array}{cc}{\displaystyle \frac{\partial F}{\partial y}-\frac{d}{dx}\left(\frac{\partial F}{\partial y^{\prime }}\right),}\hfill & \mathrm{for}n=1,\hfill \\ {\displaystyle \frac{\partial F}{\partial y}-\frac{d}{dx}\left(\frac{\partial F}{\partial y^{\prime }}\right)+\frac{d^2}{d{x}^2}\left(\frac{\partial F}{\partial y^{″}}\right),}\hfill & \mathrm{for}n=2.\hfill \end{array}\right.$$

In the special case of $n=1$, this equation is known as the Euler equation. Often, there are problems where the function F depends on many unknown functions, e.g., $y_i\left(x\right)$ (and their derivatives). In such cases, the necessary condition for the existence of an extremum of the functional (the Euler–Lagrange equations) reduces to a system of differential equations.

Unfortunately, very often, the differential equations (or their systems) obtained from the corresponding conditions are nonlinear or cannot be expressed in normal form (which is required by many methods for solving such problems, including classical ones like the Runge–Kutta method and less-known ones like the Adomian decomposition method [8,9]). For this reason, we need to seek new methods that can handle such forms of the problem.

Such methods include the differential transformation method (DTM) discussed in Section 2 and the physics-informed neural networks (PIINs) presented in Section 3. PINNs are a type of specifically designed neural network devoted to solving problems involving differential equations that describe physical laws. PINNs integrate knowledge of physical laws, allowing for the more accurate and efficient modeling of physical phenomena compared with traditional neural networks. PINNs can generalize better to unseen conditions because they are guided by underlying physical laws, which remain consistent across different scenarios. This contrasts traditional neural networks that might struggle to extrapolate beyond their training data. Section 4 is devoted to numerical examples illustrating the effectiveness of both methods and comparing them using selected examples.

2. DTM Method

The differential transform method (DTM) was proposed at the end of the 20th century and was quickly recognized and adapted to solve many different problems. Initially, it was used to solve ordinary differential equations, but its applications have expanded to the systems of such equations [10,11,12,13,14,15], partial differential equations [16,17], differential algebraic equations [18], integral equations [19,20], delay differential equations [21,22], fuzzy differential equations [23,24], and fractional differential equations [25,26,27,28]. This method has also been applied to certain important types of equations, such as Schrödinger equations and nonlinear Klein–Gordon equations, and in modeling predator–prey phenomena [29,30,31], among many other applications [32,33,34,35,36].

If a function, f, can be expanded into a Maclaurin series (such a function is called the original), then for this function, we can write the following:

$$f\left(x\right)=\sum _{k=0}^{\infty }\frac{f^{\left(k\right)}\left(0\right)}{k!}{x}^k=\sum _{k=0}^{\infty }F\left(k\right)x^k,$$

Thus, knowing the properties of such a transformation and the transforms of basic functions, we can solve many of the problems mentioned in the previous paragraph.

Below, we present the transforms of selected functions and selected properties of DTM (the included functions and properties were used in the examples presented in Section 4; proofs of these properties, among others, can be found in [16,25]).

Theorem 1. 

Let x be an element of the domain of the considered function, f, and $k\in {\mathbb{Z}}_0$.

If $f\left(x\right)=x^m$, $\mathbb{Z}\ni m\ge 0$, then

$$F\left(k\right)=\delta (k-m)=\left\{\begin{array}{cc}1,\hfill & k=m,\hfill \\ 0,\hfill & k\ne m.\hfill \end{array}\right.$$

(4)

If $f\left(x\right)=e^{ax}$, $a\in \mathbb{R}$, then

$$F\left(k\right)=\frac{a^k}{k!}.$$

(5)

If $f\left(x\right)=sin\left(ax\right)$, $a\in \mathbb{R}$, then

$$F\left(k\right)=\frac{a^k}{k!}sin\frac{k\pi }{2}.$$

(6)

If $f\left(x\right)=cos\left(ax\right)$, $a\in \mathbb{R}$, then

$$F\left(k\right)=\frac{a^k}{k!}cos\frac{k\pi }{2}.$$

(7)

If $f\left(x\right)=u\left(x\right)\pm v\left(x\right)$, then

$$F\left(k\right)=U\left(x\right)\pm V\left(x\right).$$

(8)

If $f\left(x\right)=a·u\left(x\right)$, $a\in \mathbb{R}$, then

$$F\left(k\right)=a·U\left(x\right).$$

(9)

If $f\left(x\right)=x^m·u\left(x\right)$, then

$$F\left(k\right)=\left\{\begin{array}{cc}0,\hfill & k<m,\hfill \\ U(k-m),\hfill & k\ge m.\hfill \end{array}\right.$$

(10)

If $f\left(x\right)=u\left(x\right)·v\left(x\right)$, then

$$F\left(k\right)=\sum _{i=0}^{k}U\left(i\right)V(k-i)=\sum _{i=0}^{k}V\left(i\right)U(k-i).$$

(11)

If $f\left(x\right)=u\left(x\right)·v\left(x\right)·w\left(x\right)$, then

$$F\left(k\right)=\sum _{i=0}^k\sum _{j=0}^{k-i}U\left(i\right)V\left(j\right)W(k-i-j)=\sum _{i=0}^k\sum _{j=0}^{i}U\left(j\right)V(i-j)W(k-i).$$

(12)

If $f\left(x\right)=u^{\left(n\right)}\left(x\right)$, then

$$F\left(k\right)=\frac{(k+n)!}{k!}U(k+n).$$

(13)

3. Physics-Informed Neural Network

Physics-informed neural networks (PINNs) represent a novel approach to solving differential equations (both ordinary and partial) and differential-integral equations, harnessing the power of deep learning. Among the pioneering works on this approach are the papers by Raisi et al. [37] and Karniadakis et al. [38]. PINNs integrate the principles of physics in the form of differential equations with a neural network architecture. This methodology embeds differential equations and initial/boundary conditions into the loss function of the neural network, ensuring that predictions align with the equations describing the system. As a result, PINNs offer a flexible and efficient alternative to traditional numerical methods, which often face limitations related to computational costs and scalability, especially in high-dimensional spaces. Once the model is trained, it can provide solutions for any grid, making this approach advantageous over classical methods like finite difference schemes. Additionally, PINNs excel in handling complex boundary and initial conditions, and they can directly incorporate experimental data into the model. This capability not only enhances solution accuracy but also enables the model to learn and adapt to real-world scenarios where data may be sparse or noisy. Consequently, PINNs have achieved significant success in various applications, including fluid dynamics, materials science, and biological systems [39,40,41].

A PINN architecture consists of classical components of the feedforward and backpropagation of neural networks: hidden layers, nodes per hidden layer, and activation functions. To train a neural network, an optimization method like the adam optimizer is used. The PINN system presented in the article consists of two main modules (see Figure 1): a fully connected neural network and a loss function module that computes the loss as the weighted $L2$ norm of both the governing ODE equation and boundary condition’s residuals.

For the considered ordinary differential equation, the following applies:

$$F(y^{\left(n\right)}\left(x_i\right),\dots ,y^{″}\left(x_i\right),y^{\prime }\left(x_i\right),y\left(x_i\right),x_i)=0,x\in [x_L,x_R],$$

With Dirichlet boundary conditions, the following apply:

$$y\left(x_L\right)=y_L,y\left(x_R\right)=y_R,$$

The loss function takes the following form:

$$\mathcal{L}=\sum _{i=1}^{N}F(y^{\left(n\right)}\left(x_i\right),\dots ,y^{″}\left(x_i\right),y^{\prime }\left(x_i\right),y\left(x_i\right),x_i)+(y\left(x_L\right)-y_L)+(y\left(x_R\right)-y_R),$$

(14)

where N is the number of collocation/training points inside domain $[x_L,x_R]$. After building the neural network and defining the loss function, the neural network is then trained to find the best parameters, ${\mathsf{\Theta }}^{*}$ (weights and biases), by minimizing the loss function, $\mathcal{L}$.

4. Numerical Examples

4.1. Example 1

Let us consider a functional,

$$\mathcal{J}\left(y\left(x\right)\right)={\int }_0^{ln3}\left({(y^{\prime }\left(x\right)-y\left(x\right)+x)}^2-x-y\left(x\right)\right)dx$$

(15)

with boundary conditions,

$$y\left(0\right)=-3,y(ln3)=ln3.$$

(16)

In this case,

$$F(x,y\left(x\right),y^{\prime }\left(x\right))={(y^{\prime }\left(x\right)-y\left(x\right)+x)}^2-x-y\left(x\right),$$

and therefore, according to (3), for $n=1$,

$$\frac{\partial F}{\partial y}-\frac{d}{dx}\left(\frac{\partial F}{\partial y^{\prime }}\right)=-2(y^{\prime }\left(x\right)-y\left(x\right)+x)-1-2(y^{″}\left(x\right)-y^{\prime }\left(x\right)+1),$$

which, after some transformation, gives the following equation:

$$2y^{″}\left(x\right)-y\left(x\right)+2x+3=0.$$

(17)

Taking into account the conditions (16), we therefore have to solve an ordinary second-order differential equation (a linear, non-homogeneous one with constant coefficients), where the exact solution to this equation is the following function:

$$y\left(x\right)=\frac{3}{2}+x-\frac{9}{2}{e}^{-x}.$$

4.1.1. DTM—Example 1

Applying the DTM to Equation (17) (utilizing properties (4), (8), (9), (13)), for $k\ge 0$, we obtain the following:

$$2(k+2)(k+1)Y(k+2)-2Y\left(k\right)+2\delta (k-1)+3\delta (k-0)=0,$$

(18)

whereby from the condition $y\left(0\right)=-3$, we know that $Y\left(0\right)=-3$. Assuming $k=0$ in (18), we obtain the following:

$$2·2Y\left(2\right)-2Y\left(0\right)+2\delta (-1)+3\delta \left(3\right)=0\Rightarrow Y\left(2\right)=-\frac{9}{4}.$$

Notice that neither for $k=0$ nor for $k>0$ are we able to determine the value of $Y\left(1\right)$—this is because the second condition (16) takes the form $y(ln3)=ln3$ (the desired form would be $y^{\prime }\left(0\right)=a\in \mathbb{R})$. Therefore, let us assume $Y\left(1\right)=s\in \mathbb{R}$. Taking $k\ge 1$, we sequentially obtain the following:

$$\begin{array}{c}\hfill \begin{array}{cccccccc}\hfill Y\left(3\right)& =\frac{s-1}{6},\hfill & \hfill Y\left(4\right)& =-\frac{3}{16},\hfill & \hfill Y\left(5\right)& =\frac{s-1}{120},\hfill & \hfill Y\left(6\right)& =-\frac{1}{160},\hfill \\ \hfill Y\left(7\right)& =\frac{s-1}{5040},\hfill & \hfill Y\left(8\right)& =\frac{s-1}{362880},\hfill & \hfill Y\left(9\right)& =-\frac{1}{8960},\hfill & \hfill Y\left(10\right)& =-\frac{1}{806400},\dots \hfill \end{array}\end{array}$$

In the general case, we can note that $Y\left(0\right)=-3$ and $Y\left(1\right)=s$; for $k>1$, we have $Y\left(2\right)=-\frac{9}{4}$ and $Y(i+2)=\frac{Y\left(i\right)}{(i+1)(i+2)}$ for even numbers, i, and $Y\left(3\right)=\frac{s-1}{6}$ and $Y(j+2)=\frac{Y\left(j\right)}{(j+1)(j+2)}$ for odd numbers, j.

Therefore, we can represent the sought-after function, y, as follows:

$$y\left(x\right)=x+(s-1)\sum _{i=1}^{\infty }\frac{x^{2i-1}}{(2i-1)!}-3-\frac{9}{2}\sum _{i=1}^{\infty }\frac{x^{2i}}{\left(2i\right)!},$$

which, after using a known Taylor series, takes the following form:

$$y\left(x\right)=x+(s-1)sinhx+\frac{3}{2}-\frac{9}{2}coshx.$$

There is still an unknown value of s to find. Using the second condition (16), i.e., $y(ln3)=ln3$, we obtain the following:

$$ln3=ln3+(s+1)\frac{4}{3}+\frac{3}{2}-\frac{9}{2}·\frac{5}{3}\Rightarrow s=\frac{11}{2}.$$

This result allows for the final form of the following solution,

$$y=\frac{3}{2}+x-\frac{9}{2}{e}^{-x},$$

which fully coincides with the exact solution. Of course, this is not the standard case—in most instances, we can only find an approximate solution, as we present in the subsequent examples.

4.1.2. PINNs—Example 1

Now, we solve Equation (17) using neural networks. In this case, a fully connected neural network of a depth of 4 with 3 hidden layers and 10 neurons per layer is used. As an optimizer, the Adam method is used with a learning rate of $0.001$. The hyperbolic tangent is assumed as the activation function. Figure 2 shows the obtained approximate solution (left plot, black dots) along with the errors of this approximation (right plot). The obtained solution fits the exact solution quite well. The maximum error is approximately $0.04$.

4.2. Example 2

Consider following functional,

$$\mathcal{J}\left(y\left(x\right)\right)={\int }_0^5\frac{{(1-y\left(x\right))}^2}{{\left(y^{\prime }\left(x\right)\right)}^2}dx$$

(19)

with conditions

$$y\left(0\right)=0,y\left(5\right)=-\frac{3}{4}.$$

(20)

In this case, we have

$$F(x,y\left(x\right),y^{\prime }\left(x\right))=\frac{{(1-y\left(x\right))}^2}{{\left(y^{\prime }\left(x\right)\right)}^2},$$

and therefore we, can, according to (3) for $n=1$, write

$$\frac{\partial F}{\partial y}-\frac{d}{dx}\left(\frac{\partial F}{\partial y^{\prime }}\right)=-\frac{6(y\left(x\right)-1)(y\left(x\right)+1)\left(y{\left(x\right)}^2{y}^{″}\left(x\right)-y^{″}\left(x\right)-2y\left(x\right)y^{\prime }{\left(x\right)}^2\right)}{y^{\prime }{\left(x\right)}^4},$$

which, after some transformation, leads to the equation

$$y{\left(x\right)}^2{y}^{″}\left(x\right)-y^{″}\left(x\right)-2y\left(x\right)y^{\prime }{\left(x\right)}^2=0$$

(21)

with conditions (20) (other solutions, $y\left(x\right)=1$ and $y\left(x\right)=-1$, are neglected because they do not meet the conditions (20)).

4.2.1. DTM—Example 2

Applying the DTM of transformation to Equation (21) (here, we use the properties (8), (9), (12), and (13)), for $k\ge 0$, we obtain

$$\begin{array}{cc}\hfill \sum _{i=0}^k\sum _{j=0}^i& \left((j+1)(j+2)Y(j+2)Y(i-j)Y(k-i)\right)-(k+1)(k+2)Y(k+2)\hfill \\ \hfill & -2\sum _{i=0}^k\sum _{j=0}^i\left((j+1)(i-j+1)Y(j+1)Y(i-j+1)Y(k-i)\right)=0,\hfill \end{array}$$

(22)

and according to the condition, $y\left(0\right)=0$ implies that $Y\left(0\right)=0$.

Similarly to before, taking successive values, $k\ge 0$, in (22), we are not be able to find the value of $Y\left(1\right)$. Therefore, assume $Y\left(1\right)=s\in \mathbb{R}$; as a result, $Y\left(0\right)=0$, $Y\left(1\right)=s$, and

$$\begin{array}{c}\hfill \begin{array}{cccccccccc}\hfill Y\left(2\right)& =0,\hfill & \hfill Y\left(3\right)& =-\frac{s^3}{3},\hfill & \hfill Y\left(4\right)& =0,\hfill & \hfill Y\left(5\right)& =\frac{2s^5}{15},\hfill & \hfill Y\left(6\right)& =0,\hfill \\ \hfill Y\left(7\right)& =-\frac{17s^7}{315},\hfill & \hfill Y\left(8\right)& =0,\hfill & \hfill Y\left(9\right)& =\frac{62s^9}{2835},\hfill & \hfill Y\left(10\right)& =0,\hfill & \hfill Y\left(11\right)& =-\frac{1382s^7}{155,925},\dots \hfill \end{array}\end{array}$$

This time, we are not able to find a function defined by the Taylor series of the obtained terms. Therefore, we must limit ourselves to finding an approximate solution, $y_n\left(x\right)$, of the form

$$y_n\left(x\right)=\sum _{i=0}^{n}Y\left(i\right)x^i,$$

which depends on the unknown value of parameter s. The value of this parameter can be found using the second of the conditions (20), i.e., $y\left(5\right)=-\frac{3}{4}$. Proceeding in this way and taking $n=7$, an approximate solution, $y_7\left(x\right)$, of the following form is obtained:

$$y_7\left(x\right)=sx-\frac{s^3}{3}{x}^3+\frac{2s^5}{15}{x}^5-\frac{17s^7}{315}{x}^7.$$

Solving the equation $y_7\left(5\right)=-\frac{3}{4}$, we unfortunately obtain three real solutions: $s=-0.24919$, $s=-0.20286$, and $s=0.35538$. Therefore, it is necessary to choose the appropriate one. To achieve this, we examine the error as follows:

$${\int }_0^5|y_7{(x,s)}^2{y}_7^{″}(x,s)-y_7^{″}(x,s)-2y_7\left(x\right)y_7^{\prime }{(x,s)}^2|dx.$$

In this case, it turns out that the best value is $s=-0.20286$. For this value, we obtain an approximate solution:

$$y_7\left(x\right)=-0.2063x+0.0028x^3-0.00004x^5+7.63·{10}^{-7}{x}^7.$$

Following a similar approach but taking $n=11$ (again choosing the best value of s), we obtain the following solution:

$$y_{11}\left(x\right)=-0.1955x+0.0025x^3-0.00004x^5+5.88·{10}^{-7}{x}^7-9.11·{10}^{-9}{x}^9+1.41·{10}^{-10}{x}^{11}.$$

In Figure 3, we present the exact solution, $y\left(x\right)$, the approximate solutions, $y_7\left(x\right)$ and $y_{11}\left(x\right)$, and the errors, ${\Delta }_n\left(x\right)$, of these solutions, where

$${\Delta }_n\left(x\right)=|y\left(x\right)-y_n\left(x\right)|,$$

and the exact solution has the following form:

$$y\left(x\right)=\frac{1-7^{x/5}}{1+7^{x/5}}.$$

4.2.2. PINNs—Example 2

The same Equation (21) was also solved using a physics-informed neural network. In this example, a fully connected neural network of a depth of 4 with 3 hidden layers and 10 neurons per layer was used. The Adam optimizer with a learning rate of $0.001$ was employed and the hyperbolic tangent is assumed as the activation function. Inside the domain, 128 training points were considered. Figure 4 depicts the obtained approximate solution (left plot, black dots) along with the errors of this approximation (right plot). The obtained solution is very close to the exact solution. The maximum error does not exceed $0.016$.

4.3. Example 3

In the third example, the following equation is considered,

$$\mathcal{J}\left(y\left(x\right)\right)={\int }_{-1}^1\left(y^{″}\left(x\right)y^{\prime }\left(x\right)e^{-x}sinx+{(y^{\prime }\left(x\right)-y^{″}\left(x\right))}^2-(y\left(x\right)-y^{″}\left(x\right))y^{\prime }\left(x\right)\right)dx$$

(23)

which is supplemented by the following conditions:

$$y(-1)=1,y^{\prime }(-1)=-1,y\left(1\right)=1,y^{\prime }\left(1\right)=-2.$$

(24)

In this case,

$$F(x,y\left(x\right),y^{\prime }\left(x\right))=y^{″}\left(x\right)y^{\prime }\left(x\right)e^{-x}sinx+{(y^{\prime }\left(x\right)-y^{″}\left(x\right))}^2-(y\left(x\right)-y^{″}\left(x\right))y^{\prime }\left(x\right),$$

and therefore, according to (3), for $n=2$, we obtain the following,

$$\frac{\partial F}{\partial y}-\frac{d}{dx}\left(\frac{\partial F}{\partial y^{\prime }}\right)+\frac{d^2}{d{x}^2}\left(\frac{\partial F}{\partial y^{″}}\right)=e^x\left(2y^{\left(4\right)}\left(x\right)e^x+y^{″}\left(x\right)(cosx-sinx-2e^x)-2y^{\prime }\left(x\right)cosx\right),$$

which brings us to the follwing equation,

$$2y^{\left(4\right)}\left(x\right)e^x+y^{″}\left(x\right)(cosx-sinx-2e^x)-2y^{\prime }\left(x\right)cosx=0$$

(25)

with conditions (24).

4.3.1. DTM—Example 3

Applying the DTM of transformation to Equation (25) (we use the properties (5)–(9), (11), and (13)), the following equation is obtained for $k\ge 0$:

$$\begin{array}{cc}\hfill 2\sum _{i=0}^k\frac{(i+1)(i+2)(i+3)(i+4)Y(i+4)}{(k-i)!}& +\sum _{i=0}^k\frac{(i+1)(i+2)Y(i+2)\left(cos\frac{\pi (k-i)}{2}-sin\frac{\pi (k-i)}{2}-2\right)}{(k-i)!}\hfill \\ \hfill & -2\sum _{i=0}^k\frac{(i+1)Y(i+1)cos\frac{\pi (k-i)}{2}}{(k-i)!}=0.\hfill \end{array}$$

(26)

Given successive values of $k\ge 0$ in (26), it is impossible to find the values $Y\left(0\right)$, $Y\left(1\right)$, $Y\left(2\right)$ and $Y\left(3\right)$, so we assume $Y\left(0\right)=s_0$, $Y\left(1\right)=s_1$, $Y\left(2\right)=s_2$ and $Y\left(3\right)=s_3$, $s_i\in \mathbb{R}$, $0\le i\le 3$. Therefore $Y\left(0\right)=s_0$ i $Y\left(1\right)=s_1$, $Y\left(2\right)=s_2$, $Y\left(3\right)=s_3$, and

$$\begin{array}{c}\hfill \begin{array}{cccccc}\hfill Y\left(4\right)& =\frac{s_1+s_2}{24},\hfill & \hfill Y\left(5\right)& =\frac{-s_1+4s_2+3s_3}{120},\hfill & \hfill Y\left(6\right)& =\frac{s_1-11s_2+36s_3}{1440},\hfill \\ \hfill Y\left(7\right)& =\frac{11s_1-12s_2-69s_3}{10,080},\hfill & \hfill Y\left(8\right)& =\frac{-75s_1+109s_2+96s_3}{161,280},\hfill & \hfill Y\left(9\right)& =\frac{33s_1-184s_2+301s_3}{483,840},\dots \hfill \end{array}\end{array}$$

Again, we are unable to find a function with a Taylor series defined by these terms based on the form of the found terms. We must therefore limit ourselves to finding an approximate solution of the form $y_n\left(x\right)$,

$$y_n\left(x\right)=\sum _{i=0}^{n}Y\left(i\right)x^i,$$

which additionally depends on an unknown parameter, $s_i$, $0\le i\le 3$. These parameters are determined using conditions (24) (this time we have a unique set of exactly four constants $s_i$). Proceeding in this way and taking $n=4$, we obtain an approximate solution of the form $y_4\left(x\right)$:

$$y_4\left(x\right)=1.2692+0.75x-0.2885x^2-0.75x^3+0.0192x^4.$$

Proceeding similarly, but taking $n=7$ (again the conditions (24) imply a unique solution to the parameters $s_i$, $0\le$ and $\le 3$), we obtain the following solution:

$$y_7\left(x\right)=1.2398+0.7288x-0.2448x^2-0.7024x^3+0.0202x^4-0.0318x^5-0.0152x^6+0.0053x^7.$$

Figure 5 shows the exact solution $y\left(x\right)$, the approximate solutions, $y_4\left(x\right)$ and $y_7\left(x\right)$, and the errors, ${\Delta }_n\left(x\right)$, of these solutions. In following formula of error,

$${\Delta }_n\left(x\right)=|y\left(x\right)-y_n\left(x\right)|,$$

$y\left(x\right)$ denotes the exact solution obtained form Wolfram Mathematica 14.0 [42,43]. As can be seen in Figure 5, the difference in errors between solutions $y_4\left(x\right)$ and $y_7\left(x\right)$ is significant. The maximum error for $y_7\left(x\right)$ does not exceed $0.004$, whereas for the solution $y_4\left(x\right)$, this error is approximately $0.034$.

4.3.2. PINNs—Example 3

Similar to previous examples, the following network architecture is assumed: 3 hidden layers with 10 neurons per layer. Inside the domain, 128 training points are used. The hyperbolic tangent is assumed as the activation function. The solution obtained after training the model is depicted in Figure 6. The solution using PINNs fits the exact solution quite well, with errors not exceeding $0.04$.

4.4. Comparison of DTM and PINNs Results

In this section, errors obtained from the two methods are compared for examples 2 and 3. In the case of example 1, the DTM method provides the exact solution. Figure 7 presents error plots obtained from both methods, specifically for example 2 (Figure 7a) and example 3 (Figure 7b). As observed in Figure 7, the smallest errors were obtained for DTM with $n=11$ (for example 2) and $n=7$ (for example 3). The errors of the solutions obtained using neural networks are slightly larger but still smaller than those from DTM with $n=7$ (for example 2) and $n=4$ (for example 3). Both methods produced solutions of good quality.

5. Conclusions

The paper presents two different methods for solving variational calculus problems involving differential equations.

The first method, the DTM, is characterized by its flexibility in both the form of the differential equation and the boundary conditions. One of its strengths is its scalability to handle approximate solutions of varying orders, and sometimes it even allows for a prediction of the exact solution based on the form of the coefficients found for $Y\left(k\right)$. However, there are some drawbacks to this method. One notable challenge is the difficulty in automating the process of solving a given problem. It is complex to develop a program that can generate an approximate solution solely based on the form of the differential equation and the boundary conditions. It is often easier to manually construct a discrete counterpart of the differential equation, develop a methodology for finding the unknown coefficients, $Y\left(k\right)$, and then utilize appropriate software tools for further refinement.

The second presented approach—PINNs (physics-informed neural networks)—utilizes neural networks to solve the relevant differential equation. One advantage of this method is its relatively straightforward implementation and flexibility. Once the model for the differential equation is trained, it can provide solutions for different grids (input points) without recalculating the problem each time.

Comparing both methods, it can be observed that slightly more accurate results were obtained using the DTM. The results from PINNs includes slightly larger errors compared with those obtained using the DTM, but they are still of good quality. The advantage of neural networks lies in their flexibility and relatively straightforward implementation. Both methods presented in the article are suitable for solving the considered issues.

In the future, the authors plan to apply PINNs to solve direct and inverse problems in partial differential equations involving fractional-order derivatives.