Univariate Neural Network Quantitative (NNQ) Approximation by Symmetrized Operators

Abstract

This paper deals not only with pointwise and uniform convergence but also Y-valued fractional approximation results by univariate symmetrized neural network (SNN) operators on Banach space $\left(Y,∥.∥\right)$. Moreover, our main motivation in this work is to compare the convergence results obtained by classical neural network (NN) operators and symmetric neural network (SNN) operators and try to convert them into numerical examples and graphs through computer programming language Python codes. As a result of this experimental study conducted under the regime of certain parameters, the convergence speed and results of SNN operators are superior to those of classical NN operators.

1. Introduction and Related Work

Pioneering work in the field of machine learning (ML) theory and computational approximation theory are integrating cutting-edge applications into our lives every day (see [1,2,3,4]). With the advancement of artificial intelligence (AI) technology, humanity needs more effective ML models. It is not possible to ignore the contribution of pure mathematics and physics at this point.

Throughout the history of mathematics and physics, the phenomena of “symmetry” and “asymmetry” have been of great importance both in theory and in practice (please, see also [5]). When we have examined the literature, we have come across promising studies on how these phenomena are applied to real-world problems. It would be too assertive to say that the perspectives of different disciplines on symmetry-asymmetry phenomena and the tools and practices used are exactly the same. However, it would not be wrong to say that they are based on similar roots in principle and philosophy.

In [6], together with using ANNs and symplectic NN architectures to solve differential equations according to Noether’s theorem used in mathematical physics, the “physical symmetry” that governs the behavior of physical systems, translational symmetry provides conservation of momentum, and rotational symmetry provides conservation of angular momentum and energy. Mattheakis et al. also exploited the property of being an odd or even function (in the sense of symmetry with respect to the origin/$y-$axis in Euclidean geometry) for the generation of real-world data. In order to obtain better regression estimates with a standard feedforward multilayer perceptron (MLP) that is unaware of symmetry, a hidden layer called a “hub layer” was integrated into the classical MLP architecture. In this way, the desired odd or even function for the regression could be achieved.

In [7], Faroughi et al. consider fundamental physics-based ANNs in three different categories and some applications to solid and fluid mechanics. They especially emphasize that working with sparse data is disadvantageous in terms of the training process under the regime of classical ANNs. When viewed from the philosophy of inserting physical structures into NN architectures, the symmetry applications of Lagrangian neural networks in fundamental physics attract attention.

Lu et al. recommend a new, elegant network called the deep operator network (DeepONet) inspired by the universal approximation theorem (UAT) for operator and the generalized version of UAT for the operator. They also establish that some implicit (for example, integrals, fractional Laplacians) and also explicit operators (like stochastic and deterministic differential equations) are learnable by DeepONet in [8].

In [9], Goswami et al. put under the spotlight a comprehensive literature review, highlighting the usefulness of the graph neural network operators, DeepONet and Fourier neural network operators, and their generalized versions, as well as their applications to the field of computational mechanics.

Again, as an interesting work, in [10], Dong et al. handle concepts of “symmetry”, “anti-symmetry”, “non-symmetry” from the point of view of particle physics. In their data visualization, by employing the Heaviside step function, they use three equations verifying anti-symmetric, symmetric, and fully anti-symmetric cases for a two-dimensional dataset. They also think in their experimental studies that the use of continuous symmetry (like gauge, Lorentz) from field theory will open the door to promising results. Mostly, they study the performance of Equivariant Quantum Neural Networks (EQNNs) with discrete symmetries.

Another work that was the driving force behind the “symmetrization” technique, which was created by Anastassiou (please see [11]) and used in this study, is in [12]. Tahmasebi and Jegelka revealed that it is possible to use “smaller” amounts of data to train artificial neural networks by taking advantage of the “symmetry” phenomenon used in datasets. They have looked at the differential geometric perspective of ways to reduce the complexity of a given dataset in ANNs. More precisely, they have succeeded in extending Weyl’s law, which is used primarily in Spectral Theory, to include symmetry in the evaluation of the complexity of a dataset in machine learning (ML). It is thought that machine learning models that incorporate symmetry will not only make more accurate predictions, but will also fill a serious gap, especially in some disciplines where training data is scarce.

In [13], Na and Park offer architecturally symmetric neural networks (NNs) in which even the weights act in a symmetric manner. Here, the authors particularly draw attention to the “difficulties” of memory space and time spent training the model in the performance process of the ANN models. In order to overcome these difficulties at an optimum level, they highlight the “symmetric” structure of the model with some examples, such as the XOR neural network. The NN structure here is topologically symmetric; in other words, this structure has symmetric weights, connections, and even inputs. Ref. [14] is a comprehensive study of the approximation status of the idea of embedding infinite-dimensional inputs in classical neural network architectures, the approximation potentials of symmetric and antisymmetric networks, and the potentials of learning simple symmetric functions by gradient techniques.

The concept of “permutation symmetries” in multilayer perceptrons (MLPs), which began with the pioneering work of Hecht-Nielsen [15], has spawned enthusiasm among many researchers up to the present day. Albertini et al. [16] generalized this idea and investigated flip sign symmetries in artificial neural networks by considering single functions. Later, Laurent et al. [17] investigated the concept of complexity in Bayesian neural network posteriors. They stated that considering symmetries in NN models yielded promising results and that they gathered considerable insights for future studies.

In [18], Vlačić and Bölcskei solve the open problem posed by Fefferman [19]. Accordingly, they develop a theory of the relationships between NNs with respect to a given function $f:{\mathbb{R}}^m⟶{\mathbb{R}}^n$ by relating $tanh$ and $tanh$-type nonlinear functions and their symmetries. In fact, they point out that the architecture, weights, and biases of feedforward neural networks (FNNs) can be determined with respect to a given nonlinearity g.

In [20], Perin and Deny state that incorporating “symmetries in the sense of transformations with group actions” into the architecture of deep neural networks (DNNs) is promising in enabling ML models to produce more accurate predictions. Thus, they answer the questions of how and under what conditions DNNs can learn symmetries in dataset.

In [21], Hutter considers symmetric and anti-symmetric NNs. He also emphasizes the essential features of an ideal approximation architecture should have. The main emphasis in this work is to prove the existence of an “equivalent MLP” that allows symmetric MLP structures with the “universality” principle.

In this work, we benchmark the performance of the approximation of symmetrized NN operators versus classical NN operator’s performance. Inspired by [12,22], let us consider any function with mirror symmetry in the Euclidean plane (the function that is symmetric about the y-axis—the concept of an “even” function). For instance, let us imagine that we are calculating the integral of such a function over a symmetric region. In this case, since this whole integral region, as is well known, contains two equal subregions, it is sufficient to perform the calculation over one subregion. It is also anticipated that a similar logic can be followed here, and that in ML models, which take symmetry into account, relatively fewer datasets can be used to save data and energy. As pointed out in the study in [23], we observe that the popularity of computer programming languages such as Python, with minset of “number-crunching”, continues to increase. With the motivation we received from here, in our current study, we crowned our numerical examples with the graphics we obtained thanks to the computer programming language Python.

Beyond the above-mentioned papers, in [24], Cantarini and Costarelli consider a research problem related to “simultaneous approximation” on well-known neural network (NN) operators activated by classical logistic functions with interesting Voronovskaja-type results. Again, in [25], density results of deep neural network (DNN) operators have been highlighted. Moreover, here, Costarelli also draws attention to the issue that additional layers of NN operators can play a critical role in the degree of accuracy of the obtained approximations, and thus, increasing the number of hidden layers will improve the approximation. Also, in [26], Costarelli and Spigler study the pointwise and uniform convergence of certain neural network operators. They consider these convergence cases by taking into account both the weights and the number of neurons of the network. They even propose solution methods obtained by using sigmoidal functions in the solutions of integro-differential equations and Volterra integral equations. Turkun and Duman [27] have used regular summability methods to improve the convergence results of NN operators and increase the convergence rate. They also have put forth, with numerical examples and graphs, that they obtained better convergence results than those obtained by classical methods. In [28], Hu et al. have parameterized deep neural network weights to be symmetric. Thanks to these symmetry parameterizations, memory requirements are significantly reduced, and computational efficiency is achieved. These improvements are revolutionary, especially for mobile applications. Ref. [29] deals with an experimental study taking advantage of the “symmetry” property for credit risk modeling. The symmetry feature used in this credit risk modeling offers significant advantages. Some of these can be listed as temporal invariance of risk models, consistent representation of risk factors, and symmetry in financial network structures created by market participants. Peleshchak et al. have studied the classification of mines made of different materials with a neural network structure in [30]. They have shown that in addition to the asymmetry of the neurons in the first and second hidden layers of this neural network structure with respect to the symmetry plane between the hidden layers, the activation functions used also indicated a significant change in the accuracy of the developed neural network model. Even when the symmetry of the number of neurons in the hidden layers is broken, a change in the value of the loss function is observed.

The current study is designed as follows. In Section 1, we provide a gentle introduction and an in-depth literature survey. In Section 2, we give an elegant theoretical background about the symmetrized density function and activation function, which are the backbone of our symmetrized NN operators. We insert this background into the approximation theorems in the subsequent section. Section 3 is devoted to pointwise and uniform approximation results. In Section 4, which is the core part of the paper, a table of convergence rates obtained by employing classical NN operators and symmetrized NN operators, as well as some special functions, is given. Furthermore, these numerical results are interpreted with nice graphics created using the Python symbolic library SymPy and Python numerical computation libraries SciPy and NumPy (see also [31]). In Section 5, the findings are discussed, and ideas that open the doors to the future are suggested. In Appendix A, the Python programming language codes used throughout the article are shared with curious researchers from all fields, especially mathematicians, and also subject experts as “open source” (see [31,32,33,34,35]).

2. Regarding Symmetrized Density Function and Creation of Symmetrized Neural Network (SNN) Operator

In this section, firstly, we consider our activation function as in (1). Moreover, we create the density function ${\aleph }_{\widehat{t},\xi }$ taking advantage of ${\mathbb{k}}_{\widehat{t},\xi }$. Being inspired by [11], and also by a different line of vision compared with other papers like [5,24,26,27,36], our operators shall be derived by a so-called “symmetrized density function” whose technique is given in between (3) and (6).

Now, our activation function to be used here is

$${\mathbb{k}}_{\widehat{t},\xi }\left(\overline{\upsilon }\right):={\displaystyle \frac{1-\widehat{t}{e}^{-2\xi \overline{\upsilon }}}{1+\widehat{t}{e}^{-2\xi \overline{\upsilon }}}},$$

(1)

for $\xi ,\widehat{t}>0,$$\overline{\upsilon }\in \mathbb{R}$. Above $\xi$ is the parameter, and $\widehat{t}$ is the deformation coefficient. For more, read [36,37,38]. We employ the following density function

$${\aleph }_{\widehat{t},\xi }\left(\overline{\upsilon }\right):={\displaystyle \frac{1}{4}}\left({\mathbb{k}}_{\widehat{t},\xi }\left(\overline{\upsilon }+1\right)-{\mathbb{k}}_{\widehat{t},\xi }\left(\overline{\upsilon }-1\right)\right)>0,$$

(2)

$\overline{\upsilon }\in \mathbb{R}$; $\widehat{t},\xi >0$.

For $\forall \overline{\upsilon }\in \mathbb{R};\widehat{t},\xi >0,$ we have that

$${\aleph }_{\widehat{t},\xi }\left(-\overline{\upsilon }\right)={\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\overline{\upsilon }\right),$$

(3)

and

$${\aleph }_{\frac{1}{\widehat{t}},\xi }\left(-\overline{\upsilon }\right)={\aleph }_{\widehat{t},\xi }\left(\overline{\upsilon }\right).$$

(4)

Adding (3) and (4), we obtain that

$${\aleph }_{\widehat{t},\xi }\left(-\overline{\upsilon }\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(-\overline{\upsilon }\right)={\aleph }_{\widehat{t},\xi }\left(\overline{\upsilon }\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\overline{\upsilon }\right),$$

(5)

which is the key to this work.

So that

$$\mathcal{F}\left(\overline{\upsilon }\right):={\displaystyle \frac{{\aleph }_{\widehat{t},\xi }\left(\overline{\upsilon }\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\overline{\upsilon }\right)}{2}}.$$

(6)

$\mathcal{F}$ is symmetric with respect to the $y-$axis. In other words, we have obtained an even function. By (18.18) of [37], for $\xi >0,$ we have that

$${\aleph }_{\widehat{t},\xi }\left({\displaystyle \frac{ln\widehat{t}}{2\xi }}\right)={\displaystyle \frac{tanh\left(\xi \right)}{2}}={\aleph }_{\frac{1}{\widehat{t}},\xi }\left(-{\displaystyle \frac{ln\widehat{t}}{2\xi }}\right),$$

(7)

sharing the same maximum at symmetric points. By Theorem 18.1, p.458 of [37], we have that for all $\overline{\upsilon }\in \mathbb{R};\widehat{t},\xi >0$

$$\sum _{\widehat{i}=-\infty }^{\infty }{\aleph }_{\widehat{t},\xi }\left(\overline{\upsilon }-\widehat{i}\right)=\sum _{\widehat{i}=-\infty }^{\infty }{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\overline{\upsilon }-\widehat{i}\right)=1,$$

(8)

is true. Consequently, we derive that

$$\sum _{\widehat{i}=-\infty }^{\infty }\mathcal{F}\left(\overline{\upsilon }-\widehat{i}\right)=1,$$

(9)

for every $\overline{\upsilon }\in \mathbb{R}.$ By Theorem 18.2, p. 459 of [37], for $\xi ,\widehat{t}>0$, we have that

$${\int }_{-\infty }^{\infty }{\aleph }_{\widehat{t},\xi }\left(\overline{\upsilon }\right)d\overline{\upsilon }={\int }_{-\infty }^{\infty }{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\overline{\upsilon }\right)d\overline{\upsilon }=1,$$

(10)

so that

$${\int }_{-\infty }^{\infty }\mathcal{F}\left(\overline{\upsilon }\right)d\overline{\upsilon }=1,$$

(11)

therefore, $\mathcal{F}\left(\overline{\upsilon }\right)$ is a density function.

By Theorem 18.3, p. 459, of [37], we have

Let $0<\varsigma <1,$ and $\eta \in \mathbb{N}$ with ${\eta }^{1-\varsigma }>2$; $\widehat{t},\xi >0$. Then,

$$\sum _{\begin{array}{c}m=-\infty \\ :\left|\eta \overline{\upsilon }-m\right|\ge {\eta }^{1-\varsigma }\end{array}}^{\infty }{\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)<2max\left\{\widehat{t},{\displaystyle \frac{1}{\widehat{t}}}\right\}{e}^{4\xi }{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}=\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}},$$

(12)

where $\widehat{W}:=2max\left\{\widehat{t},{\displaystyle \frac{1}{\widehat{t}}}\right\}{e}^{4\xi }.$

Similarly, we get that

$$\sum _{\begin{array}{c}m=-\infty \\ :\left|\eta \overline{\upsilon }-m\right|\ge {\eta }^{1-\varsigma }\end{array}}^{\infty }{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)<\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}},$$

(13)

Consequently, we obtain that

$$\sum _{\begin{array}{c}m=-\infty \\ :\left|\eta \overline{\upsilon }-m\right|\ge {\eta }^{1-\varsigma }\end{array}}^{\infty }\mathcal{F}\left(\eta \overline{\upsilon }-m\right)<\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}},$$

(14)

where $\widehat{W}$ is as in (12).

Depict the ceiling and integral part of a real number by $⌈·⌉$ and $⌊·⌋$, respectively.

Theorem 1 

([11]). Let us take ${\xi }_{\widehat{t}}>z_0>0$ with ${\aleph }_{\widehat{t},\xi }\left(z_0\right)={\aleph }_{\widehat{t},\xi }\left(0\right)$ and ${\xi }_{\widehat{t}}>1$ for each $\widehat{t}$, $\xi >0.$ Then

$${\displaystyle \frac{1}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}{\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)}}<max\left\{{\displaystyle \frac{1}{{\aleph }_{\widehat{t},\xi }\left({\xi }_{\widehat{t}}\right)}},{\displaystyle \frac{1}{{\aleph }_{\frac{1}{\widehat{t}},\xi }\left({\xi }_{\frac{1}{\widehat{t}}}\right)}}\right\}=:\overline{A}\left(\widehat{t}\right)$$

(15)

exists such that $⌈\eta a⌉\le ⌊\eta b⌋$ for $\eta \in \mathbb{N}$; $\overline{\upsilon }\in \left[a,b\right]\subset \mathbb{R}$.

Similarly, we consider ${\xi }_{\frac{1}{\widehat{t}}}>z_1>0$ such that ${\aleph }_{\frac{1}{\widehat{t}},\xi }\left(z_1\right)={\aleph }_{\frac{1}{\widehat{t}},\xi }\left(0\right),$ and ${\xi }_{\frac{1}{\widehat{t}}}>1.$

Thus,

$${\displaystyle \frac{1}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)}}<\max \left\{{\displaystyle \frac{1}{{\aleph }_{\frac{1}{\widehat{t}},\xi }\left({\xi }_{\frac{1}{\widehat{t}}}\right)}},{\displaystyle \frac{1}{{\aleph }_{\widehat{t},\xi }\left({\xi }_{\widehat{t}}\right)}}\right\}=:\overline{A}\left(\widehat{t}\right).$$

(16)

Hence,

$$\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)>{\displaystyle \frac{1}{\overline{A}\left(\widehat{t}\right)}},$$

(17)

and

$$\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)>{\displaystyle \frac{1}{\overline{A}\left(\widehat{t}\right)}}.$$

(18)

Consequently, it holds that

$$\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\displaystyle \frac{{\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)}{2}}>{\displaystyle \frac{2}{2\overline{A}\left(\widehat{t}\right)}}={\displaystyle \frac{1}{\overline{A}\left(\widehat{t}\right)}},$$

(19)

so that

$${\displaystyle \frac{1}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}\frac{{\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)}{2}}}<\overline{A}\left(\widehat{t}\right),$$

(20)

that is,

$${\displaystyle \frac{1}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}\mathcal{F}\left(\eta \overline{\upsilon }-m\right)}}<\overline{A}\left(\widehat{t}\right).$$

(21)

Theorem 2 

([11]). Let us pick ${\xi }_{\widehat{t}}>z_0>0$ such that ${\aleph }_{\widehat{t},\xi }\left(z_0\right)={\aleph }_{\widehat{t},\xi }\left(0\right)$; ${\xi }_{\widehat{t}}>1$ for $\widehat{t}$, $\xi >0.$ In addition, let ${\xi }_{\frac{1}{\widehat{t}}}>z_1>0,$ so ${\aleph }_{\frac{1}{\widehat{t}},\xi }\left(z_1\right)={\aleph }_{\frac{1}{\widehat{t}},\xi }\left(0\right)$ is valid for ${\xi }_{\frac{1}{\widehat{t}}}>1.$ Then,

$${\displaystyle \frac{1}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}\mathcal{F}\left(\eta \overline{\upsilon }-m\right)}}<\overline{A}\left(\widehat{t}\right)$$

(22)

is obtained such that $⌈\eta a⌉\le ⌊\eta b⌋$ for $\overline{\upsilon }\in \left[a,b\right]\subset \mathbb{R},\eta \in \mathbb{N}.$

Remark 1. 

(i) By Remark 18.5, p.460 of [37], for at least some ${\overline{\upsilon }}_1\in \left[a,b\right],$ we have that

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\aleph }_{\widehat{t},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)\ne 1,$$

(23)

and for some ${\overline{\upsilon }}_2\in \left[a,b\right],$

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta {\overline{\upsilon }}_2-m\right)\ne 1,$$

(24)

such that $\xi ,\widehat{t}>0.$ Therefore, it holds that

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}\left({\displaystyle \frac{{\aleph }_{\widehat{t},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta {\overline{\upsilon }}_2-m\right)}{2}}\right)\ne 1.$$

(25)

Hence,

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}\left({\displaystyle \frac{{\aleph }_{\widehat{t},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)}{2}}\right)\ne 1$$

(26)

even if

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)=1,$$

(27)

because then

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\displaystyle \frac{{\aleph }_{\widehat{t},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)}{2}}+{\displaystyle \frac{1}{2}}\ne 1,$$

(28)

equivalently

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\displaystyle \frac{{\aleph }_{\widehat{t},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)}{2}}\ne {\displaystyle \frac{1}{2}},$$

(29)

true by

$$\underset{\eta \to +\infty }{lim}\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}{\aleph }_{\widehat{t},\xi }\left(\eta {\overline{\upsilon }}_1-m\right)\ne 1$$

(30)

(ii) For sufficiently large $\eta \in \mathbb{N}$ and $\left[a,b\right]\subset \mathbb{R}$, $⌈\eta a⌉\le ⌊\eta b⌋$ is valid. So, in general, it holds that

$$\sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}\mathcal{F}\left(\eta \overline{\upsilon }-m\right)\le 1,$$

(31)

such that $⌈\eta a⌉\le m\le ⌊\eta b⌋$ if and only if $a\le {\displaystyle \frac{m}{\eta }}\le b$.

Here, $\left(Y,∥·∥\right)$ denotes a Banach space.

Definition 1. 

Y-valued linear “symmetrized neural network (SNN) operator” is defined as below:

$${\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right):={\displaystyle \frac{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}f\left(\frac{m}{\eta }\right)\mathcal{F}\left(\eta \overline{\upsilon }-m\right)}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}\mathcal{F}\left(\eta \overline{\upsilon }-m\right)}}$$

$$={\displaystyle \frac{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}f\left(\frac{m}{\eta }\right)\left({\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)\right)}{{\displaystyle \sum _{m=⌈\eta a⌉}^{⌊\eta b⌋}}\left({\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)+{\aleph }_{\frac{1}{\widehat{t}},\xi }\left(\eta \overline{\upsilon }-m\right)\right),}}$$

(32)

for arbitrary $f\in C\left(\left[a,b\right],Y\right)$ such that $\eta \in \mathbb{N}:⌈\eta a⌉\le ⌊\eta b⌋.$

Definition 2 

([39]). (i) We employ the universal “moduli of continuity” defined as below for $f\in C\left(\left[a,b\right],Y\right)$,

$${\omega }_1\left(f,\eta \right):=\underset{\begin{array}{c}{\overline{\upsilon }}_1,{\overline{\upsilon }}_2\in \left[a,b\right]\\ \left|{\overline{\upsilon }}_1-{\overline{\upsilon }}_2\right|\le \eta \end{array}}{sup}∥f\left({\overline{\upsilon }}_1\right)-f\left({\overline{\upsilon }}_2\right)∥,$$

(33)

such that $\eta >0.$

(ii) (33) is also valid for any $f\in C_{\overline{\upsilon }B}\left(\mathbb{R},Y\right)$, $C_B\left(\mathbb{R},Y\right),$ or $C_{\overline{\upsilon }}\left(\mathbb{R},Y\right),$ where

• $C_{\overline{\upsilon }B}\left(\mathbb{R},Y\right):=\left\{f\mid f:\mathbb{R}⟶Y,boundedandcontinuousuniformly\right\},$
• $C_B\left(\mathbb{R},Y\right):=\left\{f\mid f:\mathbb{R}⟶Y,continuousandbounded\right\}$, and
• $C_{\overline{\upsilon }}\left(\mathbb{R},Y\right):=\left\{f\mid f:\mathbb{R}⟶Y,continuousuniformly\right\},$ respectively.

(iii) $f\in C_{\overline{\upsilon }}\left(\mathbb{R},Y\right)$ or $f\in C\left(\left[a,b\right],Y\right)⟺{\displaystyle \underset{\eta \to 0}{lim}}{\omega }_1\left(f,\eta \right)=0$ as $\eta \downarrow 0.$

Definition 3. 

“Y-valued quasi-interpolation SNN operator” is defined as

$$\overline{{\mathcal{L}}_{\eta }^s}\left(f,\overline{\upsilon }\right):=\sum _{m=-\infty }^{\infty }f\left({\displaystyle \frac{m}{\eta }}\right)\mathcal{F}\left(\eta \overline{\upsilon }-m\right)$$

(34)

for $f\in C_{\overline{\upsilon }B}\left(\mathbb{R},Y\right)$, or $f\in C_B\left(\mathbb{R},Y\right)$ such that $\eta \in \mathbb{N}$, $\overline{\upsilon }\in \mathbb{R}$.

3. Pointwise and Uniform Approximations by SNN Operators

We present a set of Y-valued symmetrized neural network (SNN) approximations to a function given quantitatively.

Theorem 3. 

Let $f\in C\left(\left[a,b\right],Y\right)$, $0<\varsigma <1$, $\eta \in \mathbb{N}:{\eta }^{1-\varsigma }>2$, $\widehat{t},\xi >0$, $\widehat{t}\ne 1$, $\overline{\upsilon }\in \left[a,b\right].$ Then,

(i)

$$∥{\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥\le \overline{A}\left(\widehat{t}\right)\left[{\omega }_1\left(f,{\displaystyle \frac{1}{{\eta }^{\varsigma }}}\right)+2{∥f∥}_{\infty }\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}\right]=:\widehat{\varrho },$$

(35)

where $\widehat{W}$ as in (12),

and

(ii)

$${∥{\mathcal{L}}_{\eta }^s\left(f\right)-f∥}_{\infty }\le \widehat{\varrho }.$$

(36)

We get that ${\displaystyle \underset{\eta \to \infty }{lim}}{\mathcal{L}}_{\eta }^s\left(f\right)=f$, pointwise and uniformly.

Proof. 

Please see Appendix B. □

Next, we give

Theorem 4. 

Let $f\in C_B\left(\mathbb{R},Y\right)$, $0<\varsigma <1$, $\widehat{t}>0$, $\widehat{t}\ne 1$, $\eta \in \mathbb{N}:{\eta }^{1-\varsigma }>2$, $\overline{\upsilon }\in \mathbb{R}.$ Then,

(i)

$$∥{\overline{{\mathcal{L}}^s}}_{\eta }\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥\le {\omega }_1\left(f,{\displaystyle \frac{1}{{\eta }^{\varsigma }}}\right)+2{∥f∥}_{\infty }\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}=:\varrho ,$$

(37)

and

(ii)

$${∥{\overline{{\mathcal{L}}^s}}_{\eta }\left(f\right)-f∥}_{\infty }\le \varrho .$$

(38)

For $f\in C_{\overline{\upsilon }B}\left(\mathbb{R},Y\right)$ we get ${\displaystyle \underset{\eta \to \infty }{lim}}{\overline{{\mathcal{L}}^s}}_{\eta }\left(f\right)=f$, pointwise and uniformly.

Proof. 

Please see Appendix C. □

A high-order approximation follows.

Theorem 5. 

Let $f\in C^N\left(\left[a,b\right],Y\right)$, $\eta ,N\in \mathbb{N}$, $\widehat{t}>0$, $\widehat{t}\ne 1$, $0<\varsigma <1$, $\overline{\upsilon }\in \left[a,b\right]$ and ${\eta }^{1-\varsigma }>2$. Then,

(i)

$$∥{\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥\le \overline{A}\left(\widehat{t}\right)\left\{\sum _{\widehat{j}=1}^N{\displaystyle \frac{∥f^{\left(\widehat{j}\right)}\left(\overline{\upsilon }\right)∥}{\widehat{j}!}}\left[{\displaystyle \frac{1}{{\eta }^{\varsigma \widehat{j}}}}+{\left(b-a\right)}^{\widehat{j}}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}\right]\right.$$

$$\left.+\left[{\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{{\eta }^{\varsigma }}}\right){\displaystyle \frac{1}{{\eta }^{\varsigma N}N!}}+{\displaystyle \frac{2{∥f^{\left(N\right)}∥}_{\infty }{\left(b-a\right)}^N}{N!}}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}\right]\right\},$$

(39)

(ii) assume $f^{\left(\widehat{j}\right)}\left({\overline{\upsilon }}_0\right)=0$, $\widehat{j}=1,...,N,$ for some ${\overline{\upsilon }}_0\in \left[a,b\right]$; thus,

$$∥{\mathcal{L}}_{\eta }^s\left(f,{\overline{\upsilon }}_0\right)-f\left({\overline{\upsilon }}_0\right)∥\le \overline{A}\left(\widehat{t}\right)$$

$$\left\{{\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{{\eta }^{\varsigma }}}\right){\displaystyle \frac{1}{{\eta }^{\varsigma N}N!}}+{\displaystyle \frac{2{∥f^{\left(N\right)}∥}_{\infty }{\left(b-a\right)}^N}{N!}}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}\right\},$$

(40)

and

(iii)

$${∥{\mathcal{L}}_{\eta }^s\left(f\right)-f∥}_{\infty }\le \overline{A}\left(\widehat{t}\right)\left\{\sum _{\widehat{j}=1}^N{\displaystyle \frac{{∥f^{\left(\widehat{j}\right)}∥}_{\infty }}{\widehat{j}!}}\left[{\displaystyle \frac{1}{{\eta }^{\varsigma \widehat{j}}}}+{\left(b-a\right)}^{\widehat{j}}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}\right]\right.$$

$$\left.+\left[{\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{{\eta }^{\varsigma }}}\right){\displaystyle \frac{1}{{\eta }^{\varsigma N}N!}}+2{∥f^{\left(N\right)}∥}_{\infty }{\left(b-a\right)}^N\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\varsigma \right)}}\right]\right\}.$$

(41)

Again, we obtain ${\displaystyle \underset{\eta \to \infty }{lim}}{\mathcal{L}}_{\eta }^s\left(f\right)=f$, pointwise and uniformly.

Proof. 

It is long-winded, and similar to [37]. We would like to bring it to the attention of interested readers. □

All integrals from now on are of Bochner-type.

Definition 4 

([40]). Let $\left[a,b\right]\subset \mathbb{R}$, Y be a Banach space, $\varsigma >0$; $\varkappa =⌈\varsigma ⌉\in \mathbb{N}$; $f:\left[a,b\right]\to Y$; $f^{\left(\varkappa \right)}\in L_1\left(\left[a,b\right],Y\right)$. We call the Caputo–Bochner left fractional derivative of order ς:

$$\left(D_{\ast a}^{\varsigma }f\right)\left(\overline{\upsilon }\right):={\displaystyle \frac{1}{\Gamma \left(\varkappa -\varsigma \right)}}{\int }_a^{\overline{\upsilon }}{\left(\overline{\upsilon }-t\right)}^{\varkappa -\varsigma -1}{f}^{\left(\varkappa \right)}\left(t\right)dt$$

(42)

for all $\overline{\upsilon }\in \left[a,b\right].$

Definition 5 

([40]). Let $\varsigma >0$, $\widehat{t}>0$, $\widehat{t}\ne 1,\left[a,b\right]\subset \mathbb{R}$, Y be a Banach space, $\varkappa :=⌈\varsigma ⌉$. We assume that $f^{\left(\varkappa \right)}\in L_1\left(\left[a,b\right],Y\right)$, where $f:\left[a,b\right]\to Y$. We call the Caputo–Bochner right fractional derivative of order ς:

$$\left(D_{b-}^{\varsigma }f\right)\left(\overline{\upsilon }\right):={\displaystyle \frac{{\left(-1\right)}^{\varkappa }}{\Gamma \left(\varkappa -\varsigma \right)}}{\int }_{\overline{\upsilon }}^b{\left(z-\overline{\upsilon }\right)}^{\varkappa -\varsigma -1}{f}^{\left(\varkappa \right)}\left(z\right)dz$$

(43)

for every $\overline{\upsilon }\in \left[a,b\right].$

A Y-valued fractional approximation follows:

Theorem 6. 

Let $\varsigma >0$, $\widehat{t}>0$, $\widehat{t}\ne 1$, $N=⌈\varsigma ⌉$, $\varsigma \notin \mathbb{N}$, $f\in C^N\left(\left[a,b\right],Y\right)$, $0<\tau <1$, $\overline{\upsilon }\in \left[a,b\right]$, $\eta \in \mathbb{N}:{\eta }^{1-\tau }>2.$ Then, for $\forall \overline{\upsilon }\in \left[a,b\right],$

(i)

$$∥{\mathcal{L}}^s\left(f,\overline{\upsilon }\right)-\sum _{\widehat{j}=1}^{N-1}{\displaystyle \frac{f^{\left(\widehat{j}\right)}\left(\overline{\upsilon }\right)}{\widehat{j}!}}{\mathcal{L}}_{\eta }^s\left({\left(·-\overline{\upsilon }\right)}^{\widehat{j}}\right)\left(\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥$$

$$\le {\displaystyle \frac{\overline{A}\left(\widehat{t}\right)}{\Gamma \left(\varsigma +1\right)}}\left\{{\displaystyle \frac{\left({\omega }_1{\left(D_{\overline{\upsilon }-}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\varsigma \tau }}}\right.$$

$$+\left.\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left({∥D_{\overline{\upsilon }-}^{\varsigma }f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}{\left(\overline{\upsilon }-a\right)}^{\varsigma }+{∥D_{\ast \overline{\upsilon }}^{\varsigma }f∥}_{\infty ,\left[\overline{\upsilon },b\right]}{\left(b-\overline{\upsilon }\right)}^{\varsigma }\right)\right\},$$

(44)

(ii) if $f^{\left(\widehat{j}\right)}\left(\overline{\upsilon }\right)=0$, for $\widehat{j}=1,...,N-1$, we have

$$∥{\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥\le {\displaystyle \frac{\overline{A}\left(\widehat{t}\right)}{\Gamma \left(\varsigma +1\right)}}$$

$$\left\{{\displaystyle \frac{\left({\omega }_1{\left(D_{\overline{\upsilon }-}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\varsigma \tau }}}\right.$$

$$\left.+\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left({∥D_{\overline{\upsilon }-}^{\varsigma }f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}{\left(\overline{\upsilon }-a\right)}^{\varsigma }+{∥D_{\ast \overline{\upsilon }}^{\varsigma }f∥}_{\infty ,\left[\overline{\upsilon },b\right]}{\left(b-\overline{\upsilon }\right)}^{\varsigma }\right)\right\},$$

(45)

(iii)

$$∥{\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥\le \overline{A}\left(\widehat{t}\right)$$

$$\left\{\sum _{\widehat{j}=1}^{N-1}{\displaystyle \frac{∥f^{\left(\widehat{j}\right)}\left(\overline{\upsilon }\right)∥}{\widehat{j}!}}\left\{{\displaystyle \frac{1}{{\eta }^{\tau \widehat{j}}}}+{\left(b-a\right)}^{\widehat{j}}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\right\}\right.$$

$$+{\displaystyle \frac{1}{\Gamma \left(\varsigma +1\right)}}\left\{{\displaystyle \frac{\left({\omega }_1{\left(D_{\overline{\upsilon }-}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\varsigma \tau }}}\right.$$

$$\left.\left.+\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left({∥D_{\overline{\upsilon }-}^{\varsigma }f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}{\left(\overline{\upsilon }-a\right)}^{\varsigma }+{∥D_{\ast \overline{\upsilon }}^{\varsigma }f∥}_{\infty ,\left[\overline{\upsilon },b\right]}{\left(b-\overline{\upsilon }\right)}^{\varsigma }\right)\right\}\right\},$$

(46)

and

(iv)

$${∥{\mathcal{L}}_{\eta }^{s}f-f∥}_{\infty }\le \overline{A}\left(\widehat{t}\right)$$

$$\left\{\sum _{\widehat{j}=1}^{N-1}{\displaystyle \frac{{∥f^{\left(\widehat{j}\right)}∥}_{\infty }}{\widehat{j}!}}\left\{{\displaystyle \frac{1}{{\eta }^{\tau \widehat{j}}}}+{\left(b-a\right)}^{\widehat{j}}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\right\}\right.$$

$$+{\displaystyle \frac{1}{\Gamma \left(\varsigma +1\right)}}\left\{{\displaystyle \frac{\left({\displaystyle \underset{\overline{\upsilon }\in \left[a,b\right]}{sup}}{\omega }_1{\left(D_{\overline{\upsilon }-}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\displaystyle \underset{\overline{\upsilon }\in \left[a,b\right]}{sup}}{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\varsigma \tau }}}\right.$$

$$\left.\left.+\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}{\left(b-a\right)}^{\varsigma }\left(\underset{\overline{\upsilon }\in \left[a,b\right]}{sup}{∥D_{\overline{\upsilon }-}^{\varsigma }f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}+\underset{\overline{\upsilon }\in \left[a,b\right]}{sup}{∥D_{\ast \overline{\upsilon }}^{\varsigma }f∥}_{\infty ,\left[\overline{\upsilon },b\right]}\right)\right\}\right\}.$$

(47)

Above, when $N=1$ the sum ${\sum }_{\widehat{j}=1}^{N-1}·=0.$

We have established the Y-valued fractional approximation, pointwise and uniform quantitatively of ${\mathcal{L}}_{\eta }^s⟶I$, the unit operator, as $\eta ⟶\infty .$

Proof. 

The proof is long and similar to [11], so it is saved for the interested reader. □

Next, we apply Theorem 6 for $N=1.$

Corollary 1. 

Let $0<\varsigma ,\tau <1$, $\widehat{t}>0$, $\widehat{t}\ne 1$, $f\in C^1\left(\left[a,b\right],Y\right)$, $\overline{\upsilon }\in \left[a,b\right]$, $\eta \in \mathbb{N}:{\eta }^{1-\tau }>2.$ Then,

(i)

$$∥{\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥$$

$$\le {\displaystyle \frac{\overline{A}\left(\widehat{t}\right)}{\Gamma \left(\varsigma +1\right)}}\left\{{\displaystyle \frac{\left({\omega }_1{\left(D_{\overline{\upsilon }-}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\varsigma \tau }}}\right.$$

$$\left.+\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left({∥D_{\overline{\upsilon }-}^{\varsigma }f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}{\left(\overline{\upsilon }-a\right)}^{\varsigma }+{∥D_{\ast \overline{\upsilon }}^{\varsigma }f∥}_{\infty ,\left[\overline{\upsilon },b\right]}{\left(b-\overline{\upsilon }\right)}^{\varsigma }\right)\right\},$$

(48)

and

(ii)

$${∥{\mathcal{L}}_{\eta }^{s}f-f∥}_{\infty }\le {\displaystyle \frac{\overline{A}\left(\widehat{t}\right)}{\Gamma \left(\varsigma +1\right)}}$$

$$\left\{{\displaystyle \frac{\left({\displaystyle \underset{\overline{\upsilon }\in \left[a,b\right]}{sup}}{\omega }_1{\left(D_{\overline{\upsilon }-}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\displaystyle \underset{\overline{\upsilon }\in \left[a,b\right]}{sup}}{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\varsigma }f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\varsigma \tau }}}\right.$$

$$\left.+{\left(b-a\right)}^{\varsigma }\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left(\underset{\overline{\upsilon }\in \left[a,b\right]}{sup}{∥D_{\overline{\upsilon }-}^{\varsigma }f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}+\underset{\overline{\upsilon }\in \left[a,b\right]}{sup}{∥D_{\ast \overline{\upsilon }}^{\varsigma }f∥}_{\infty ,\left[\overline{\upsilon },b\right]}\right)\right\}.$$

(49)

When $\varsigma ={\displaystyle \frac{1}{2}}$, we derive

Corollary 2. 

Let $0<\tau <1$, $\widehat{t}>0$, $\widehat{t}\ne 1$, $f\in C^1\left(\left[a,b\right],Y\right)$, $\overline{\upsilon }\in \left[a,b\right]$, $\eta \in \mathbb{N}:{\eta }^{1-\tau }>2.$ Then,

(i)

$$∥{\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)∥$$

$$\le {\displaystyle \frac{2\overline{A}\left(\widehat{t}\right)}{\sqrt{\pi }}}\left\{{\displaystyle \frac{\left({\omega }_1{\left(D_{\overline{\upsilon }-}^{\frac{1}{2}}f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\frac{1}{2}}f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\frac{\tau }{2}}}}\right.$$

$$\left.+\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left({∥D_{\overline{\upsilon }-}^{\frac{1}{2}}f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}\sqrt{\left(\overline{\upsilon }-a\right)}+{∥D_{\ast \overline{\upsilon }}^{\frac{1}{2}}f∥}_{\infty ,\left[\overline{\upsilon },b\right]}\sqrt{\left(b-\overline{\upsilon }\right)}\right)\right\},$$

(50)

and

(ii)

$${∥{\mathcal{L}}_{\eta }^{s}f-f∥}_{\infty }\le {\displaystyle \frac{2\overline{A}\left(\widehat{t}\right)}{\sqrt{\pi }}}$$

$$\left\{{\displaystyle \frac{\left({\displaystyle \underset{\overline{\upsilon }\in \left[a,b\right]}{sup}}{\omega }_1{\left(D_{\overline{\upsilon }-}^{\frac{1}{2}}f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[a,\overline{\upsilon }\right]}+{\displaystyle \underset{\overline{\upsilon }\in \left[a,b\right]}{sup}}{\omega }_1{\left(D_{\ast \overline{\upsilon }}^{\frac{1}{2}}f,\frac{1}{{\eta }^{\tau }}\right)}_{\left[\overline{\upsilon },b\right]}\right)}{{\eta }^{\frac{\tau }{2}}}}\right.$$

$$\left.+\sqrt{\left(b-a\right)}\widehat{W}{e}^{-2\xi {\eta }^{\left(1-\tau \right)}}\left(\underset{\overline{\upsilon }\in \left[a,b\right]}{sup}{∥D_{\overline{\upsilon }-}^{\frac{1}{2}}f∥}_{\infty ,\left[a,\overline{\upsilon }\right]}+\underset{\overline{\upsilon }\in \left[a,b\right]}{sup}{∥D_{\ast \overline{\upsilon }}^{\frac{1}{2}}f∥}_{\infty ,\left[\overline{\upsilon },b\right]}\right)\right\}<\infty .$$

(51)

4. Numerical Approach: Classical Neural Network (CNN) Operators Versus Symmetrized Neural Network (SNN) Operators

To the best of our knowledge, the approximation results and approximation speeds obtained with operators derived from the symmetrized density function with the theoretical framework in this article are different from other studies (for instance, [24,26,27]) in the literature.

In other words, one of the gaps we noticed in the approximation theory literature was the speculative need for improving the convergence results of ANN operators. The elegant “symmetrization technique” presented by Anastassiou helped to close this gap. Below, we present the numerical examples, graphical results, and convergence speeds obtained according to different parameters by using the tools we introduced.

The “difference” between the measures RHS and LHS introduced in detail below are almost closed after a certain number of $\eta$ and with appropriate parameter choices for $\alpha ,\widehat{t},\xi >0$.

The fact that this difference is very close to zero, which is what is desired in approximation theory, is possible with SNN operators compared with the CNN operators, makes the present work interesting. It points to our main contribution to the literature.

Three different test functions are used in the six examples given below. As a result of the approaches made to each of these test functions first with CNN operators and then with SNN operators and the “difference” measurement results calculated using Python 3.9, we see that SNN operators give smaller “difference” values compared with CNNs by looking at the bolded numerical values at the bottom of the six tables. In addition, we support these results, which we have concretely obtained with numerical data, with six graphic visuals.

Notation 1 

([37]). Note that Classical NN operators are defined for $\overline{\upsilon }\in \left[a,b\right]$, $\widehat{t}>0$, $\widehat{t}\ne 1$ as follows:

$${\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right):={\displaystyle \frac{{\displaystyle \sum _{k=⌈\eta a⌉}^{⌊\eta b⌋}}f\left(\frac{m}{\eta }\right){\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)}{{\displaystyle \sum _{k=⌈\eta a⌉}^{⌊\eta b⌋}}{\aleph }_{\widehat{t},\xi }\left(\eta \overline{\upsilon }-m\right)}},$$

(52)

such that $f\in C^1\left(\left[a,b\right],Y\right)$ and $\eta \in \mathbb{N}:⌈\eta a⌉\le ⌊\eta b⌋.$

Notation 2. 

Remember that CNN operators are defined for $\overline{\upsilon }\in \left[a,b\right]$, $\widehat{t}>0$, $\widehat{t}\ne 1$ as in (52). Furthermore, let us introduce the concepts of “Right Hand Side (RHS)”, and “Left Hand Side (LHS)” of the argument in (53) for “pointwise convergence” defined as follows, being inspired by Theorem 3. For any α such that $\alpha >0,$

$$|{\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)|\approx {\displaystyle \frac{1}{{\eta }^{\alpha }}},$$

(53)

• where
• Right Hand Side (RHS)$:={\displaystyle \frac{1}{{\eta }^{\alpha }}}$,
• Left Hand Side (LHS):= $|{\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)|$,
• and,
• Difference := ${\displaystyle \frac{1}{{\eta }^{\alpha }}}-$$|{\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)|$ =: RHS − LHS.

The three estimates, $\mathbf{Difference}$, $\mathbf{RHS}$, and $\mathbf{LHS}$, shall serve as criteria for interpreting the speed of convergence of both classical NN (CNN) operators and symmetrized NN (SNN) operators, and also creating the convergence speed tables employing Python 3.9 programming language.

Example 1. 

For the activation function defined in (1), $\widehat{t}={\displaystyle \frac{1}{4}}$, $\xi ={\displaystyle \frac{1}{2}}$, and the density function in (2), $\overline{\upsilon }\in [a,b]=\left[-\pi ,\pi \right]$, $\eta =5,10,20,50,200$ and the operators ${\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)$ defined in (52), the convergence of the operators ${\mathcal{L}}_5\left(f,\overline{\upsilon }\right)$ (blue), ${\mathcal{L}}_{10}\left(f,\overline{\upsilon }\right)$ (red), ${\mathcal{L}}_{20}\left(f,\overline{\upsilon }\right)$ (fuchsia), ${\mathcal{L}}_{50}\left(f,\overline{\upsilon }\right)$ (aqua), ${\mathcal{L}}_{200}\left(f,\overline{\upsilon }\right)$ (black) to $f\left(x\right)=sinx$ (gold) is displayed in Figure 1.

The convergence speeds of the CNN operators to the test function $f\left(x\right)=sinx$ under the regime of certain parameters are given below in

Table 1. Convergence speeds for CNN operators with different parameter values.

$\mathit{\alpha }$	$\mathit{\eta }$	Right Hand Side (RHS)	Left Hand Side (LHS)	Difference
$0.5$	5	$0.4472135955$	$0.106196835925$	$0.341016759575$
$0.5$	10	$0.316227766017$	$0.076643050878$	$0.239584715139$
$0.5$	20	$0.22360679775$	$0.043869327366$	$0.179737470384$
$0.5$	50	$0.141421356237$	$0.018804729784$	$0.122616626454$
$0.5$	200	$0.070710678119$	$0.004852019867$	$0.065858658252$
$0.5$	300	$0.057735026919$	$0.003245667233$	$0.054489359686$

.

Here, for $\eta =5,10,20,50,200,300$; RHS$:={\displaystyle \frac{1}{{\eta }^{\alpha }}}$, $\alpha ={\displaystyle \frac{1}{2}}$; LHS$:=|{\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)-f\left(\overline{\upsilon }\right)|$, Difference:=RHS − LHS.

Result: We witness that the Difference value decreases for increasing $\eta$ numbers from 5 to 300, which shows us that the approximation is promising after a certain $\eta$ value.

Example 2. 

For the activation function defined in (1); $\widehat{t}={\displaystyle \frac{1}{4}}$, $\xi ={\displaystyle \frac{1}{2}}$, $\alpha =0.5,0.9,1$, and the density function in (6), $\overline{\upsilon }\in [a,b]=\left[-\pi ,\pi \right]$, $\eta =5,10,20,50,200$ and the “symmetrized” operators ${\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)$ defined in (32), the convergence of the operators ${\mathcal{L}}_5^s\left(f,\overline{\upsilon }\right)$ (blue), ${\mathcal{L}}_{10}^s\left(f,\overline{\upsilon }\right)$ (red), ${\mathcal{L}}_{20}^s\left(f,\overline{\upsilon }\right)$ (fuchsia), ${\mathcal{L}}_{50}^s\left(f,\overline{\upsilon }\right)$ (aqua), ${\mathcal{L}}_{200}^s\left(f,\overline{\upsilon }\right)$ (black) to $f\left(x\right)=sinx$ (gold) is displayed in Figure 2.

As seen in Figure 2 and

Table 2. Convergence speeds for SNN operators with different parameter values.

$\mathit{\alpha }$	$\mathit{\eta }$	Right Hand Side (RHS)	Left Hand Side (LHS)	Difference
$0.5$	5	$0.4472135955$	$0.073997483748$	$0.373216111752$
$0.5$	10	$0.316227766017$	$0.019319978172$	$0.296907787845$
$0.5$	20	$0.22360679775$	$0.004883204134$	$0.218723593616$
$0.5$	50	$0.141421356237$	$0.000783723296$	$0.140637632941$
$0.5$	200	$0.070710678119$	4.9009701 $\times {10}^{-5}$	$0.070661668417$
$0.5$	300	$0.057735026919$	2.1782474 $\times {10}^{-5}$	$0.057713244445$
$0.9$	5	$0.234923788618$	$0.073997483748$	$0.16092630487$
$0.9$	10	$0.125892541179$	$0.019319978172$	$0.106572563008$
$0.9$	20	$0.067464142384$	$0.004883204134$	$0.06258093825$
$0.9$	50	$0.029575152733$	$0.000783723296$	$0.028791429436$
$0.9$	200	$0.008493232323$	4.9009701 $\times {10}^{-5}$	$0.008444222622$
$0.9$	300	$0.005896453402$	2.1782474 $\times {10}^{-5}$	$0.005874670927$
1	5	$0.2$	$0.073997483748$	$0.126002516252$
1	10	$0.1$	$0.019319978172$	$0.080680021828$
1	20	$0.05$	$0.004883204134$	$0.045116795866$
1	50	$0.02$	$0.000783723296$	$0.019216276704$
1	200	$0.005$	4.9009701$\times {10}^{-5}$	$0.004950990299$
1	300	$0.003333333333$	2.1782474 $\times {10}^{-5}$	$0.003311550859$

below, it is observed that the approximations by SNN operators clearly give better results than the CNN operators. We monitor this situation with both numerical results and graphical visuals. In other words, they overlap perfectly.

The convergence speeds of the SNN operators to the test function $f\left(x\right)=sinx$ under the regime of certain parameters are given below in Table 2.

Example 3. 

For the activation function defined in (1), $\widehat{t}={\displaystyle \frac{1}{4}}$, $\xi ={\displaystyle \frac{1}{2}}$, $\alpha =0.5,$ and the density function in (2), $\overline{\upsilon }\in [a,b]=\left[-\pi ,\pi \right]$, $\eta =5,10,20,50,200$ and the operators ${\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)$ defined in (52), the convergence of the operators ${\mathcal{L}}_5\left(f,\overline{\upsilon }\right)$ (blue), ${\mathcal{L}}_{10}\left(f,\overline{\upsilon }\right)$ (red), ${\mathcal{L}}_{20}\left(f,\overline{\upsilon }\right)$ (fuchsia), ${\mathcal{L}}_{50}\left(f,\overline{\upsilon }\right)$ (aqua), ${\mathcal{L}}_{200}\left(f,\overline{\upsilon }\right)$ (black) to $f\left(x\right)=sin\left(2\pi x\right)$ (gold) is displayed in Figure 3.

The convergence speeds of the CNN operators to the test function $f\left(x\right)=sin\left(2\pi x\right)$ under the regime of certain parameters are given below in

Table 3. Convergence speeds for CNN operators with different parameter values for $f\left(x\right)=sin\left(2\pi x\right)$.

$\mathit{\alpha }$	$\mathit{\eta }$	Right Hand Side (RHS)	Left Hand Side (LHS)	Difference
$0.5$	5	$0.4472135955$	$1.019590673876$	$-0.572377078376$
$0.5$	10	$0.316227766017$	$0.735035145942$	$-0.418807379925$
$0.5$	20	$0.22360679775$	$0.310398556399$	$-0.086791758649$
$0.5$	50	$0.141421356237$	$0.078890538643$	$0.062530817594$
$0.5$	200	$0.070710678119$	$0.012251715923$	$0.058458962196$
$0.5$	300	$0.057735026919$	$0.007583994866$	$0.050151032053$

.

Example 4. 

For the activation function defined in (1), $\widehat{t}={\displaystyle \frac{1}{4}}$, $\xi ={\displaystyle \frac{1}{2}}$, $\alpha =0.5,0.9,1,$ and the density function in (6), $\overline{\upsilon }\in [a,b]=\left[-\pi ,\pi \right]$; $\eta =5,10,20,50,200,$ and the “symmetrized” operators ${\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)$ defined in (32), the convergence of the operators ${\mathcal{L}}_5^s\left(f,\overline{\upsilon }\right)$ (blue), ${\mathcal{L}}_{10}^s\left(f,\overline{\upsilon }\right)$ (red), ${\mathcal{L}}_{20}^s\left(f,\overline{\upsilon }\right)$ (fuchsia), ${\mathcal{L}}_{50}^s\left(f,\overline{\upsilon }\right)$ (aqua), ${\mathcal{L}}_{200}^s\left(f,\overline{\upsilon }\right)$ (black) to $f\left(x\right)=sin\left(2\pi x\right)$ (gold) is displayed in Figure 4.

The convergence speeds of the SNN operators to the test function $f\left(x\right)=sin\left(2\pi x\right)$ under the regime of certain parameters are given below in

Table 4. Convergence speeds for SNN operators with different parameter values.

$\mathit{\alpha }$	$\mathit{\eta }$	Right Hand Side (RHS)	Left Hand Side (LHS)	Difference
$0.5$	5	$0.4472135955$	$0.99452974412$	$-0.54731614862$
$0.5$	10	$0.316227766017$	$0.646761489603$	$-0.330533723586$
$0.5$	20	$0.22360679775$	$0.232191624821$	$-0.008584827071$
$0.5$	50	$0.141421356237$	$0.041730703041$	$0.099690653196$
$0.5$	200	$0.070710678119$	$0.002665088735$	$0.068045589384$
$0.5$	300	$0.057735026919$	$0.001185437546$	$0.056549589373$
$0.9$	5	$0.234923788618$	$0.99452974412$	$-0.759605955503$
$0.9$	10	$0.125892541179$	$0.646761489603$	$-0.520868948423$
$0.9$	20	$0.067464142384$	$0.232191624821$	$-0.164727482437$
$0.9$	50	$0.029575152733$	$0.041730703041$	$-0.012155550308$
$0.9$	200	$0.008493232323$	$0.002665088735$	$0.005828143588$
$0.9$	300	$0.005896453402$	$0.001185437546$	$0.004711015856$
1	5	$0.2$	$0.99452974412$	$-0.79452974412$
1	10	$0.1$	$0.646761489603$	$-0.546761489603$
1	20	$0.05$	$0.232191624821$	$-0.182191624821$
1	50	$0.02$	$0.041730703041$	$-0.021730703041$
1	200	$0.005$	$0.002665088735$	$0.002334911265$
1	300	$0.003333333333$	$0.001185437546$	$0.002147895788$

.

Example 5. 

For the activation function defined in (1), $\widehat{t}={\displaystyle \frac{1}{4}}$, $\xi ={\displaystyle \frac{1}{2}}$, $\alpha =0.5,$ and density function in (2), $\overline{\upsilon }\in [a,b]=\left[-\pi ,\pi \right]$, $\eta =5,10,20,50,200$ and the operators ${\mathcal{L}}_{\eta }\left(f,\overline{\upsilon }\right)$ defined in (52), the convergence of the operators ${\mathcal{L}}_5\left(f,\overline{\upsilon }\right)$ (blue), ${\mathcal{L}}_{10}\left(f,\overline{\upsilon }\right)$ (red), ${\mathcal{L}}_{20}\left(f,\overline{\upsilon }\right)$ (fuchsia), ${\mathcal{L}}_{50}\left(f,\overline{\upsilon }\right)$ (aqua), ${\mathcal{L}}_{200}\left(f,\overline{\upsilon }\right)$ (black) to $f\left(x\right)=exp\left(x\right)$ (gold) is displayed in Figure 5.

The convergence speeds of the CNN operators to the test function $f\left(x\right)=exp\left(x\right)$ under the regime of certain parameters are given below in

Table 5. Convergence speeds to $exp\left(x\right)$ by CNN operators.

$\mathit{\alpha }$	$\mathit{\eta }$	Right Hand Side (RHS)	Left Hand Side (LHS)	Difference
$0.5$	5	$0.4472135955$	$0.919943701463$	$-0.472730105963$
$0.5$	10	$0.316227766017$	$0.372333139108$	$-0.056105373091$
$0.5$	20	$0.22360679775$	$0.168097690311$	$0.055509107439$
$0.5$	50	$0.141421356237$	$0.06329630096$	$0.078125055277$
$0.5$	200	$0.070710678119$	$0.015355496873$	$0.055355181246$
$0.5$	300	$0.057735026919$	$0.01020291246$	$0.047532114459$

.

Example 6. 

For the activation function defined in (1); $\widehat{t}={\displaystyle \frac{1}{4}}$, $\xi ={\displaystyle \frac{1}{2}}$, $\alpha =0.5,0.9,1,$ and density function in (6), $\overline{\upsilon }\in [a,b]=\left[-\pi ,\pi \right]$, $\eta =5,10,20,50,200,$ and the “symmetrized” operators ${\mathcal{L}}_{\eta }^s\left(f,\overline{\upsilon }\right)$ defined in (32), the convergence of the operators ${\mathcal{L}}_5^s\left(f,\overline{\upsilon }\right)$ (blue), ${\mathcal{L}}_{10}^s\left(f,\overline{\upsilon }\right)$ (red), ${\mathcal{L}}_{20}^s\left(f,\overline{\upsilon }\right)$ (fuchsia), ${\mathcal{L}}_{50}^s\left(f,\overline{\upsilon }\right)$ (aqua), ${\mathcal{L}}_{200}^s\left(f,\overline{\upsilon }\right)$ (black) to $f\left(x\right)=exp\left(x\right)$ (gold) is displayed in Figure 6.

The convergence speeds of the SNN operators to the test function $f\left(x\right)=exp\left(x\right)$ under the regime of certain parameters are given below in

Table 6. Convergence speeds to $exp\left(x\right)$ by SNN operators.

$\mathit{\alpha }$	$\mathit{\eta }$	Right Hand Side (RHS)	Left Hand Side (LHS)	Difference
$0.5$	5	$0.4472135955$	$0.257627908469$	$0.189585687031$
$0.5$	10	$0.316227766017$	$0.06171214644$	$0.254515619577$
$0.5$	20	$0.22360679775$	$0.01525818134$	$0.20834861641$
$0.5$	50	$0.141421356237$	$0.002433781759$	$0.138987574479$
$0.5$	200	$0.070710678119$	$0.000152027539$	$0.070558650579$
$0.5$	300	$0.057735026919$	6.7566605 $\times {10}^{-5}$	$0.057667460314$
$0.9$	5	$0.234923788618$	$0.257627908469$	$-0.022704119852$
$0.9$	10	$0.125892541179$	$0.06171214644$	$0.064180394739$
$0.9$	20	$0.067464142384$	$0.01525818134$	$0.052205961044$
$0.9$	50	$0.029575152733$	$0.002433781759$	$0.027141370974$
$0.9$	200	$0.008493232323$	$0.000152027539$	$0.008341204784$
$0.9$	300	$0.005896453402$	6.7566605 $\times {10}^{-5}$	$0.005828886796$
1	5	$0.2$	$0.257627908469$	$-0.057627908469$
1	10	$0.1$	$0.06171214644$	$0.03828785356$
1	20	$0.05$	$0.01525818134$	$0.03474181866$
1	50	$0.02$	$0.002433781759$	$0.017566218241$
1	200	$0.005$	$0.000152027539$	$0.004847972461$
1	300	$0.003333333333$	6.7566605 $\times {10}^{-5}$	$0.003265766728$

.

As a result of all numerical examples, we conclude that the Difference value decreases for increasing $\eta$ numbers from 5 to 300, which shows us that the approximation is promising after a certain $\eta$ value for especially symmetrized versions of the NN operators.

5. Conclusions and Future Remarks

We believe that this study provides a two-pronged approach as a contribution to the literature. In one aspect, the theoretical background of symmetric univariate neural network operators has been emphasized once again. In another aspect, it has been shown with concrete examples and graphs how these symmetrized neural network operators (SNNs) achieve better approximation results under certain parameter choices compared with the classical ones.

Thanks to the major role that artificial neural networks (ANNs) and computational methods play in the advancement of scientific research, it is of great importance that any new developments in this field are supported by mathematical consistence and robustness for the society.

Furthermore, we think that we have presented the mathematical verification and comparison employing the computer programming language Python 3.9, which is open-source and flexible, to provide evidence of the superiority of symmetric structure. When we examine other studies in the literature, we can say that we have reached a consensus on the effectiveness and game-changing role of symmetrical structures in approximation theory. In addition, our findings anchor the effectiveness of symmetric structures in neural network architectures on the model and the potential for profound transformations in outputs in machine learning.

In a nutshell, we verified the basic theorems of ANN operators in the theory of approximations with numerical examples, compared classical ANN operators with symmetrized ANN accelerated operators, and showed that the symmetric structure, in particular, provides superior results.

We anticipate that this theoretical background will guide the latest applications of future artificial intelligence (AI) subfields such as “Geometric Deep Learning”, “Graph Learning”, “Graph Neural Networks”, “Quantum Neural Networks”, as well as a theory of machine learning (ML), applied analysis, and computational mathematics.

Lastly, there is undoubtedly still a lot to be carried out with other types of activation functions, such as half-hyperbolic tangent functions, sigmoidal functions, ReLu, activated NN operators, etc., in a symmetrized manner. In addition, in Appendix A, we shared the Python 3.9 codes of numerical applications: speed of the approximation of the CNN and SNN operators to certain functions and their 2D space visualizations constructed by using Python 3.9 libraries NumPy 1.21.0, Matplotlib 3.8, and Math.