Machine Learning for Modeling the Singular Multi-Pantograph Equations

In this study, a new approach to basis of intelligent systems and machine learning algorithms is introduced for solving singular multi-pantograph differential equations (SMDEs). For the first time, a type-2 fuzzy logic based approach is formulated to find an approximated solution. The rules of the suggested type-2 fuzzy logic system (T2-FLS) are optimized by the square root cubature Kalman filter (SCKF) such that the proposed fineness function to be minimized. Furthermore, the stability and boundedness of the estimation error is proved by novel approach on basis of Lyapunov theorem. The accuracy and robustness of the suggested algorithm is verified by several statistical examinations. It is shown that the suggested method results in an accurate solution with rapid convergence and a lower computational cost.

Recently, due to the importance of MDEs, the solving of these equations have been frequently considered in the literature and many numerical and analytical methods have been presented [6]. For example, in References [7][8][9], a homotopy approach and power series are developed for solving linear MDEs and coinciding of the estimated solution with the exact solution is investigated. In Reference [10], the spectral tau method is studied and the convergence of the presented approach is investigated by L 2 norm. In Reference [11], by obtaining the fractional integral of Taylor wavelets in the sense of Riemann-Liouville definition, an estimated solution is presented

•
A new numerical method is proposed for solving singular MDEs. • For the first time, a type-2 fuzzy logic based approach is formulated to find an approximated solution. • A new approach on the basis of the Lyapunov theorem is introduced for convergence and stability analysis. • Square root cubature Kalman filter is developed for the optimization of the suggested solver. • Several statistical examinations are presented to demonstrate the accuracy and stability.
The paper organization is as follows. The problem is formulated in Section 2. The suggested T2-FLS is illustrated in Section 3. The learning algorithm is presented in Section 4. The stability is investigated in Section 5. The evaluation indexes are described in Section 6. The simulation results are provided in Section 7, and finally the main outcomes are summarized in Section 8.

Problem Formulation
The suggested solver is designed on the basis of fuzzy systems and SCKF. The general diagram of the suggested solution approach is shown in Figure 1. The problem is described as: where the initial conditions are χ (0) = a 1 andχ (0) = a 2 . F k (t) and G (t) are nonlinear functions.
If there is a singularity in F k (t) and G (t), then both sides of (1) are multiplied by F k (t) and G (t).
The parameters of T2-FLS should be learned such that the estimated solutionχ (t) is to be converged to the exact solution χ (t). Then the estimatedχ (t) satisfies: The cost function is defined as follows: where i = 1, ..., N and N is the number of samples.

T2-FLS Structure
The structure of T2-FLSχ (t) is shown in Figure 2. The details are given as follows: The input t is mapped into time range [0, 1].
The [0, 1] is divided into M section and for each section a Gaussian membership function (MF) with mean m l , l = 1, ..., M and variance ν l is considered. (4) The upper and lower firing rules are computed as: The normalized rule firings (type-reduction by the Nie-Tan approach [39]) are obtained as:

Learning Method
The suggested T2-FLS is optimized through the SCKF. To apply SCKF on learning of T2-FLS such that the cost function (3) to be minimized, the following state-space representation is taken to account: where λ (t) and W (t) are the Gaussian noise with covariance R and Q and zeros mean and ξ is the vector of parameters of T2-FLS that includes rule parameters: The learning algorithm is presented as follows.

Stability and Convergence Analysis
To prove the stability and convergence of the suggested algorithm, the Lyapunov approach [40,41] is used. To apply Lyapunov approach, the following Lyapunov function is defined: Time difference of V, results in: Considering small sample time, eq.dv1 can be simplified as: From (13), From (7), From (29) and Equations (31) and (32), one has: From (25) and (33), one has: From the fact that J m > 0, it is concluded that V (t) − V (t − 1) ≤ 0 and from the Lyapunov theorem, the stability and boundedness of the cost function is derived.

Evaluation Index
To evaluate the accuracy and robustness of the suggested algorithm, the following indexes are defined.
where N is the number of sample times, χ i andχ i are the exact and estimated solutions and RMSE, TIC and VAR are root mean square error, inequality coefficient of Theil index, and variance, respectively.

Simulations
By several statistical analyses, the accuracy of the suggested algorithm is examined.

Example 1.
For the first examination, an SMDE is considered as: where where, α = π. The real solution of (38), is sin (αt). To estimate the solution by the suggested method, the cost function is given as: The time range [0, 1] is divided into 21 sections and then we have 21 rules. The standard division of each MF is considered to be 0.1. The trajectories of the output of T2-FLS (approximated solution), mean of approximated solutions and exact solution are depicted in Figure 3a and the corresponding rule parameters are shown in Figure 3b. The absolute error is depicted in Figure 4a and the values of FIT, RMSE, VAR and TIC are shown in Figure 4b. The statistical analyses for TIC, RMSE, VAR and FIT are given in Figures 5-8. One can observe that the metrics of TIC, RMSE, VAR and FIT are in the favorable level and the trajectory of the approximated solution well tracks the exact solution x(t) = sin(αt).
For the accuracy of the suggested approach to be well seen, the values of interquartile range (IR), median (Med), minimum (Min) and mean of absolute error at each sample time are provided in Table 1.
One can see that the values of mean and IR items are in range of 10 −3 to 10 −2 that indicate an accurate and robust solution.         Example 2. In this Example, the following SMDE is considered: where The cost function is: Similar to Example 1, we have 21 rules. The trajectories of the output of T2-FLS (approximated solution), the mean of approximated solutions and exact solution are depicted in Figure 9a and the corresponding rule parameters are shown in Figure 9b. The absolute error is depicted in Figure 10a and the values of FIT, RMSE, VAR and TIC are shown in Figure 10b. The statistical analysis for TIC, RMSE, VAR and FIT are given in Figures 11-14. One can observe that the metrics of TIC, RMSE, VAR and FIT are in the favorable level and the trajectory of the approximated solution well tracks the exact solution x(t) = exp(t).
Similar to Example 1, in order for the accuracy of the suggested approach to be well seen, the values of interquartile range (IR), median (Med), minimum (Min) and mean of absolute error at each sample time are provided in Table 2. One can see that the values of mean and IR items are in the range of 10 −3 to 10 −2 that indicate an accurate and robust solution.        Example 3. In this section, the performance of the suggested algorithm is compared with other similar techniques in the literature [29]. In Reference [29], a simple NN is learned by genetic algorithm to find a solution for a pantograph system. The following MDE is taken into account [29]: The exact solution of (44) is χ (t) = e t . The number of rules in T2-FLS is considered to be 10, the same as the number of neurons in Reference [29]. The numerical comparison, with the method of Reference [29], is given in Table 3. Although the results are close to each other in this Example, the main advantage of the suggested method is that the results of the suggested method are obtained in only one epoch. However, in the genetic algorithm presented in Reference [29], the learning process of NN is repeated several times until a reasonable result is achieved. The evolutionary based learning techniques are not suitable for online applications because of the high level of computational cost and lack of stability guarantee.

Conclusions
In this paper, a new approach on the basis of fuzzy neural networks and SCKF is introduced for finding a numerical solution for multi-pantograph singular differential equations. The proposed learning method is stable and this property is shown by a new approach on the basis of the Lypunov theorem. Two simulations are provided to demonstrate the efficiency of the designed solver. Several statistical analyses are given to verify the effectiveness of the introduced algorithm such as the analysis of RMSE, Interquartile Range, Theil's Inequality Index and Variance metrics. The metrics of TIC, RMSE, VAR and FIT are shown in the favorable level and the trajectory approximated solution well tracks the exact solution. Also, the performance of the suggested method is compared with the other similar techniques in the literature. It is shown that the proposed technique results in better accuracy despite less computational cost in contrast to the evolutionary based learning genetic algorithm.

Conflicts of Interest:
The authors declare no conflict of interest.