Neural Network Approximation for Time Splitting Random Functions

Abstract

In this article we present the multivariate approximation of time splitting random functions defined on a box or ${\mathbb{R}}^N,N\in \mathbb{N},$ by neural network operators of quasi-interpolation type. We achieve these approximations by obtaining quantitative-type Jackson inequalities engaging the multivariate modulus of continuity of a related random function or its partial high-order derivatives. We use density functions to define our operators. These derive from the logistic and hyperbolic tangent sigmoid activation functions. Our convergences are both point-wise and uniform. The engaged feed-forward neural networks possess one hidden layer. We finish the article with a great variety of applications.

1. Introduction

The first author of [1,2], see Section 2, Section 3, Section 4 and Section 5, was first to establish neural network approximations of continuous functions with rates by specifically defined neural network operators of Cardaliaguet–Euvrard and ”squashing” types, employing the modulus of continuity of the engaged function or its high-order derivative to produce very tight Jackson-type inequalities. They treated both univariate and multivariate cases. The defining these operators, ”bell-shaped” and ”squashing” functions, are assumed to be of compact support. Furthermore, in [2] they give the Nth order asymptotic expansion for the weak approximation error of these two operators to a special natural class of smooth functions, see Section 4 and Section 5. Continuing on from the the work in [3], the author continued their studies on neural network approximation by introducing and using sigmoidal and hyperbolic tangent-type proper quasi-interpolation operators [4,5,6,7,8], treating both univariate and multivariate cases. Here, we are inspired by the work in [9,10]. In this article we study the quantitative neural network approximations to continuous functions on a box or ${\mathbb{R}}^N,N\in \mathbb{N}$, by linear operators deriving by engaging the logistic and hyperbolic tangent sigmoid activation functions. The degrees of approximations are given with rates by the use of appropriately related moduli of continuity. We finish by providing applications of the work. We have been mostly motivated by:

• Stationary Gaussian processes, represented by

  $$Y_t=cos\left(\lambda t\right){\xi }_1+sin\left(\lambda t\right){\xi }_2,\lambda \in \mathbb{R},$$

  where ${\xi }_1$ and ${\xi }_2$ are random variables, independent from the standard normal distribution, see [11].
• The “Fourier model” of a stationary process, see [12].

The one hidden layer feed-forward neural networks (FNNs) used in this work can be mathematically expressed as networks as

$$M_n\left(x\right)=\sum _{j=0}^n{\lambda }_j{\sigma }^{*}\left(⟨z_j·x⟩+p_j\right),x\in {\mathbb{R}}^m,m\in \mathbb{N},$$

where for $0\le j\le n$, $p_j\in \mathbb{R}$ are thresholds, $z_j\in {\mathbb{R}}^m$ are connecting weights, ${\lambda }_j\in \mathbb{R}$ are coefficients, $⟨z_j·x⟩$ are in the inner product of $z_j$ and x, and ${\sigma }^{*}$ is the activation function in the network. In the vast literature of neural networks, various activation functions are used according to the needs of the study. Ours are very typical, the logistic and hyperbolic activation functions. For further general reading on neural networks one may read [13,14,15]. In [16], author presents a complete study of real-valued approximations by neural network approximations.

2. Background

2.1. Logistic Sigmoid Activation Function

Here, we follow [7], considering a logarithmic-type sigmoidal function

$$s_i\left(x_i\right)=\frac{1}{1+e^{-x_i}},x_i\in \mathbb{R},i=1,...,N;x:=\left(x_1,...,x_N\right)\in {\mathbb{R}}^N.$$

each with the properties ${\displaystyle \underset{x_i\to +\infty }{lim}}{s}_i\left(x_i\right)=1$ and ${\displaystyle \underset{x_i\to -\infty }{lim}}{s}_i\left(x_i\right)=0$, $i=1,...,N.$

These functions play the role of activation functions in the hidden layer of neural networks, with additional applications in biology, demography, etc. [17,18].

As in [3], we consider

$${\mathsf{\Phi }}_i\left(x_i\right):=\frac{1}{2}\left(s_i\left(x_i+1\right)-s_i\left(x_i-1\right)\right),x_i\in \mathbb{R},i=1,\cdots ,N.$$

We notice the following properties:

(i)

${\mathsf{\Phi }}_i\left(x_i\right)>0$, ∀$x_i\in \mathbb{R},$

(ii)

${\sum }_{k_i=-\infty }^{\infty }{\mathsf{\Phi }}_i\left(x_i-k_i\right)=1$, ∀$x_i\in \mathbb{R}$,

(iii)

${\sum }_{k_i=-\infty }^{\infty }{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)=1$, ∀$x_i\in \mathbb{R}$; $n\in \mathbb{N},$

(iv)

${\int }_{-\infty }^{\infty }{\mathsf{\Phi }}_i\left(x_i\right)d{x}_i=1$,

(v)

${\mathsf{\Phi }}_i$ is a density function,

(vi)

${\mathsf{\Phi }}_i$ is even: ${\mathsf{\Phi }}_i\left(-x_i\right)={\mathsf{\Phi }}_i\left(x_i\right)$, $x_i\ge 0$, for $i=1,...,N.$

We see that

$${\mathsf{\Phi }}_i\left(x_i\right)=\left(\frac{e^2-1}{2e^2}\right)\frac{1}{\left(1+e^{x_i-1}\right)\left(1+e^{-x_i-1}\right)},i=1,...,N.$$

(1)

(vii)

${\mathsf{\Phi }}_i$ is decreasing for ${\mathbb{R}}_{+}$, and increasing for ${\mathbb{R}}_{-}$, $i=1,...,N.$

Let $0<\beta <1$, $n\in \mathbb{N}$. Then, as in [8], we obtain

(viii)

$$\sum _{\left\{\begin{array}{c}{k}_i=-\infty \\ :\left|n{x}_i-k_i\right|>n^{1-\beta }\end{array}\right.}^{\infty }{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)=\sum _{\left\{\begin{array}{c}{k}_i=-\infty \\ :\left|n{x}_i-k_i\right|>n^{1-\beta }\end{array}\right.}^{\infty }{\mathsf{\Phi }}_i\left(\left|n{x}_i-k_i\right|\right)$$

$$\le 3.1992e^{-n^{\left(1-\beta \right)}},i=1,...,N.$$

(2)

Denote by $⌈·⌉$ the ceiling of a number, and by $⌊·⌋$ the integral part of a number. Consider $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)\subset {\mathbb{R}}^N$, $N\in \mathbb{N}$ such that $⌈n{a}_i⌉\le ⌊n{b}_i⌋$, $i=1,...,N;$ $a:=\left(a_1,...,a_N\right)$, $b:=\left(b_1,...,b_N\right).$

We obtain

(ix)

$$0<\frac{1}{{\sum }_{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)}<\frac{1}{{\mathsf{\Phi }}_i\left(1\right)}=5.250312578,$$

(3)

∀$x_i\in \left[a_i,b_i\right],$ $i=1,...,N.$

(x)

As in [8], we see that

$$\underset{n\to \infty }{lim}\sum _{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\ne 1,$$

(4)

for at least some $x_i\in \left[a_i,b_i\right]$, $i=1,...,N.$

We will use here

$$\mathsf{\Phi }\left(x_1,...,x_N\right):=\mathsf{\Phi }\left(x\right):=\prod _{i=1}^N{\mathsf{\Phi }}_i\left(x_i\right)=:T_1\left(x\right),x\in {\mathbb{R}}^N.$$

(5)

It has the properties:

(i)’

$\mathsf{\Phi }\left(x\right)>0$, ∀$x\in {\mathbb{R}}^N,$

We see that

$$\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }\mathsf{\Phi }\left(x_1-k_1,x_2-k_2,...,x_N-k_N\right)=$$

$$\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }\prod _{i=1}^N{\mathsf{\Phi }}_i\left(x_i-k_i\right)=\prod _{i=1}^N\left(\sum _{k_i=-\infty }^{\infty }{\mathsf{\Phi }}_i\left(x_i-k_i\right)\right)=1.$$

(6)

That is

(ii)’

$$\sum _{k=-\infty }^{\infty }\mathsf{\Phi }\left(x-k\right):=\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }\mathsf{\Phi }\left(x_1-k_1,...,x_N-k_N\right)=1,$$

(7)

$k:=\left(k_1,...,k_n\right)$, ∀$x\in {\mathbb{R}}^N.$

(iii)’

$$\sum _{k=-\infty }^{\infty }\mathsf{\Phi }\left(nx-k\right):=$$

$$\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }\mathsf{\Phi }\left(n{x}_1-k_1,...,n{x}_N-k_N\right)=1,$$

(8)

∀$x\in {\mathbb{R}}^N;$ $n\in \mathbb{N}$.

(iv)’

$${\int }_{{\mathbb{R}}^N}\mathsf{\Phi }\left(x\right)dx=1,$$

(9)

where $\mathsf{\Phi }$ is a multivariate density function.

Here, ${∥x∥}_{\infty }:=max\left\{\left|x_1\right|,...,\left|x_N\right|\right\}$, $x\in {\mathbb{R}}^N$, and set $\infty :=\left(\infty ,...,\infty \right)$, $-\infty :=\left(-\infty ,...,-\infty \right)$ in the multivariate context, and

$$\begin{array}{ccc}\hfill ⌈na⌉& :& =\left(⌈n{a}_1⌉,...,⌈n{a}_N⌉\right),\hfill \\ \hfill ⌊nb⌋& :& =\left(⌊n{b}_1⌋,...,⌊n{b}_N⌋\right).\hfill \end{array}$$

For $0<\beta <1$ and $n\in \mathbb{N}$, fixed $x\in {\mathbb{R}}^N$ gives

$$\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)=$$

$$\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }\le \frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)+\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right).$$

(10)

In the last two calculations the counting is over a disjoint vector of k, because the condition ${∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}$ implies that there exists at least one $\left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}$, $r\in \left\{1,...,N\right\}.$

We treat

$$\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)=\prod _{i=1}^N\left(\sum _{\left\{\begin{array}{c}{k}_i=⌈n{a}_i⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊n{b}_i⌋}{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right)$$

$$\le \left(\prod _{\begin{array}{c}i=1\\ i\ne r\end{array}}^N\left(\sum _{k_i=-\infty }^{\infty }{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right)\right)·\left(\sum _{\left\{\begin{array}{c}{k}_r=⌈n{a}_r⌉\\ \left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊n{b}_r⌋}{\mathsf{\Phi }}_r\left(n{x}_r-k_r\right)\right)$$

$$=\left(\sum _{\left\{\begin{array}{c}{k}_r=⌈n{a}_r⌉\\ \left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊n{b}_r⌋}{\mathsf{\Phi }}_r\left(n{x}_r-k_r\right)\right)$$

$$\le \sum _{\left\{\begin{array}{c}{k}_r=-\infty \\ \left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}\end{array}\right.}^{\infty }{\mathsf{\Phi }}_r\left(n{x}_r-k_r\right)\stackrel{(\mathrm{by}\left(\mathrm{viii}\right))}{\le }3.1992e^{-n^{\left(1-\beta \right)}}.$$

(11)

We have proven that

(v)’

$$\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)\le 3.1992e^{-n^{\left(1-\beta \right)}},$$

(12)

$0<\beta <1$, $n\in \mathbb{N}$, $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

By (ix), we obtain

$$0<\frac{1}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)}=\frac{1}{{\prod }_{i=1}^N\left({\sum }_{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right)}$$

$$<\frac{1}{{\prod }_{i=1}^N{\mathsf{\Phi }}_i\left(1\right)}={\left(5.250312578\right)}^N.$$

(13)

That is,

(vi)’

it holds

$$0<\frac{1}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)}<{\left(5.250312578\right)}^N,$$

(14)

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $n\in \mathbb{N}$.

It is clear that

(vii)’

$$\sum _{\left\{\begin{array}{c}k=-\infty \\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{\infty }\mathsf{\Phi }\left(nx-k\right)\le 3.1992e^{-n^{\left(1-\beta \right)}},$$

(15)

$0<\beta <1$, $n\in \mathbb{N}$, $x\in {\mathbb{R}}^N.$

From (x), we see that

(viii)’

$$\underset{n\to \infty }{lim}\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)\ne 1$$

(16)

for some $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

Let $f\in C\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$ and $n\in \mathbb{N}$ such that $⌈n{a}_i⌉\le ⌊n{b}_i⌋$, $i=1,...,N.$

We introduce and define the multivariate positive linear neural network operator ($x:=\left(x_1,...,x_N\right)\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$)

$$G_n\left(f,x_1,...,x_N\right):=G_n\left(f,x\right):=\frac{{\sum }_{k=⌈na⌉}^{⌊nb⌋}f\left(\frac{k}{n}\right)\mathsf{\Phi }\left(nx-k\right)}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)}$$

(17)

$$:=\frac{{\sum }_{k_1=⌈n{a}_1⌉}^{⌊n{b}_1⌋}{\sum }_{k_2=⌈n{a}_2⌉}^{⌊n{b}_2⌋}...{\sum }_{k_N=⌈n{a}_N⌉}^{⌊n{b}_N⌋}f\left(\frac{k_1}{n},...,\frac{k_N}{n}\right)\left({\prod }_{i=1}^N{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right)}{{\prod }_{i=1}^N\left({\sum }_{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right)}.$$

For large enough n we obtain $⌈n{a}_i⌉\le ⌊n{b}_i⌋$, $i=1,...,N$. Furthermore, $a_i\le \frac{k_i}{n}\le b_i$, iff $⌈n{a}_i⌉\le k_i\le ⌊n{b}_i⌋$, $i=1,...,N$.

Here, we study the point-wise and uniform convergence of $G_n\left(f\right)$ to f with rates.

For convenience,

$$G_n^{*}\left(f,x\right):=\sum _{k=⌈na⌉}^{⌊nb⌋}f\left(\frac{k}{n}\right)\mathsf{\Phi }\left(nx-k\right)$$

(18)

$$:=\sum _{k_1=⌈n{a}_1⌉}^{⌊n{b}_1⌋}\sum _{k_2=⌈n{a}_2⌉}^{⌊n{b}_2⌋}...\sum _{k_N=⌈n{a}_N⌉}^{⌊n{b}_N⌋}f\left(\frac{k_1}{n},...,\frac{k_N}{n}\right)\left(\prod _{i=1}^N{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right),$$

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

That is

$$G_n\left(f,x\right):=\frac{G_n^{*}\left(f,x\right)}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)},\forall x\in \left(\prod _{i=1}^N\left[a_i,b_i\right]\right),n\in \mathbb{N}.$$

(19)

Hence,

$$G_n\left(f,x\right)-f\left(x\right)=\frac{G_n^{*}\left(f,x\right)-f\left(x\right){\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)}.$$

(20)

Consequently, we derive

$$\left|G_n\left(f,x\right)-f\left(x\right)\right|\le {\left(5.250312578\right)}^N\left|G_n^{*}\left(f,x\right)-f\left(x\right)\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Phi }\left(nx-k\right)\right|,$$

(21)

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

Now we estimate the right-hand side of Equations (21).

For this we need, for $f\in C\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$ the first multivariate modulus of continuity

$${\omega }_1\left(f,h\right):=\underset{\begin{array}{c}x,y\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)\\ {∥x-y∥}_{\infty }\le h\end{array}}{sup}\left|f\left(x\right)-f\left(y\right)\right|,h>0.$$

(22)

Similarly, $f\in C_B\left({\mathbb{R}}^N\right)$ is defined (continuous and bounded functions of ${\mathbb{R}}^N$). We have $\underset{h\to 0}{lim}{\omega }_1\left(f,h\right)=0.$

When $f\in C_B\left({\mathbb{R}}^N\right)$ we define

$${\overline{G}}_n\left(f,x\right):={\overline{G}}_n\left(f,x_1,...,x_N\right):=\sum _{k=-\infty }^{\infty }f\left(\frac{k}{n}\right)\mathsf{\Phi }\left(nx-k\right)$$

(23)

$$:=\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }f\left(\frac{k_1}{n},\frac{k_2}{n},...,\frac{k_N}{n}\right)\left(\prod _{i=1}^N{\mathsf{\Phi }}_i\left(n{x}_i-k_i\right)\right),$$

$n\in \mathbb{N}$, ∀$x\in {\mathbb{R}}^N,$ $N\ge 1$ is and the multivariate quasi-interpolation neural network operator.

Notice here that for large enough $n\in \mathbb{N}$ we obtain

$$e^{-n^{\left(1-\beta \right)}}<n^{-\beta j},j=1,...,m\in \mathbb{N},0<\beta <1.$$

(24)

Thus, given a fixed $A,B>0$, for the linear combination $\left(A{n}^{-\beta j}+B{e}^{-n^{\left(1-\beta \right)}}\right)$ the (dominant) rate of convergence to zero is $n^{-\beta j}$. The closer $\beta$ is to 1 a faster and better rate of convergence to zero is obtained.

Let $f\in C^m\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $m,N\in \mathbb{N}$. Here, $f_{\alpha }$ denotes a partial derivative of f, $\alpha :=\left({\alpha }_1,...,{\alpha }_N\right)$, ${\alpha }_i\in {\mathbb{Z}}^{+}$, $i=1,...,N$, and $\left|\alpha \right|:={\sum }_{i=1}^N{\alpha }_i=l,$ where $l=0,1,...,m$. We also write $f_{\alpha }:=\frac{{\partial }^{\alpha }f}{\partial x^{\alpha }}$ of order l.

We denote

$${\omega }_{1,m}^{max}\left(f_{\alpha },h\right):=\underset{\alpha :\left|\alpha \right|=m}{max}{\omega }_1\left(f_{\alpha },h\right).$$

(25)

Call also

$${∥f_{\alpha }∥}_{\infty ,m}^{max}:=\underset{\left|\alpha \right|=m}{max}\left\{{∥f_{\alpha }∥}_{\infty }\right\},$$

(26)

${∥·∥}_{\infty }$ is the supremum norm.

Next, we present a series of multivariate neural network approximations of a function given with rates.

We first give

Theorem 1. 

Let $f\in C\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right),$ $0<\beta <1,$ $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $n,N\in \mathbb{N}$. Then,

(i) 

$$\left|G_n\left(f,x\right)-f\left(x\right)\right|\le {\left(5.250312578\right)}^N·$$

$$\left\{{\omega }_1\left(f,\frac{1}{n^{\beta }}\right)+\left(6.3984\right){∥f∥}_{\infty }{e}^{-n^{\left(1-\beta \right)}}\right\}=:{\lambda }_1,$$

(27)

(ii) 

$${∥G_n\left(f\right)-f∥}_{\infty }\le {\lambda }_1.$$

(28)

Next we present

Theorem 2 

([7]). Let $f\in C_B\left({\mathbb{R}}^N\right)$, $0<\beta <1$, $x\in {\mathbb{R}}^N$, $n,N\in \mathbb{N}$. Then,

(i) 

$$\left|{\overline{G}}_n\left(f,x\right)-f\left(x\right)\right|\le {\omega }_1\left(f,\frac{1}{n^{\beta }}\right)+\left(6.3984\right){∥f∥}_{\infty }{e}^{-n^{\left(1-\beta \right)}}=:{\lambda }_2,$$

(29)

(ii) 

$${∥{\overline{G}}_n\left(f\right)-f∥}_{\infty }\le {\lambda }_2.$$

(30)

Next we discuss high-order approximations by using the smoothness of f.

We give

Theorem 3 

([7]). Let $f\in C^m\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $0<\beta <1$, $n,m,N\in \mathbb{N}$, $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$. Then,

(i) 

$$\left|G_n\left(f,x\right)-f\left(x\right)-\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{f_{\alpha }\left(x\right)}{{\prod }_{i=1}^N{\alpha }_i!}\right)G_n\left(\prod _{i=1}^N{\left(·-x_i\right)}^{{\alpha }_i},x\right)\right)\right|\le$$

$${\left(5.250312578\right)}^N·\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\right.$$

(31)

$$\left.\left(\frac{\left(6.3984\right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-n^{\left(1-\beta \right)}}\right\},$$

(ii) 

$$\left|G_n\left(f,x\right)-f\left(x\right)\right|\le {\left(5.250312578\right)}^N·$$

$$\left\{\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{\left|f_{\alpha }\left(x\right)\right|}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j}}+\left(\prod _{i=1}^N{\left(b_i-a_i\right)}^{{\alpha }_i}\right)·\left(3.1992\right)e^{-n^{\left(1-\beta \right)}}\right]\right)+\right.$$

(32)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{\left(6.3984\right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-n^{\left(1-\beta \right)}}\right\},$$

(iii) 

$${∥G_n\left(f\right)-f∥}_{\infty }\le {\left(5.250312578\right)}^N·$$

$$\left\{\sum _{j=1}^N\left(\sum _{\left|\alpha \right|=j}\left(\frac{{∥f_{\alpha }∥}_{\infty }}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j}}+\left(\prod _{i=1}^N{\left(b_i-a_i\right)}^{{\alpha }_i}\right)\left(3.1992\right)e^{-n^{\left(1-\beta \right)}}\right]\right)+\right.$$

(33)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{\left(6.3984\right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-n^{\left(1-\beta \right)}}\right\},$$

(iv) 

Assume $f_{\alpha }\left(x_0\right)=0$, for all $\alpha :\left|\alpha \right|=1,...,m;$ $x_0\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$. Then

$$\left|G_n\left(f,x_0\right)-f\left(x_0\right)\right|\le {\left(5.250312578\right)}^N·$$

(34)

$$\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_1^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{\left(6.3984\right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-n^{\left(1-\beta \right)}}\right\},$$

notice in the last equation the extremely high rate of convergence at $n^{-\beta \left(m+1\right)}.$

2.2. Hyperbolic Tangent Sigmoid Activation Function

We now consider the hyperbolic tangent function $tanhx$, $x\in \mathbb{R}:$

$$tanhx:=\frac{e^x-e^{-x}}{e^x+e^{-x}}.$$

It has the properties $tanh0=0,$ $-1<tanhx<1$, ∀$x\in \mathbb{R}$, and $tanh\left(-x\right)=-tanhx$. Furthermore, $tanhx\to 1$ as $x\to \infty$, and $tanhx\to -1$, as $x\to -\infty$, and it is strictly increasing for $\mathbb{R}$.

This function plays the role of an activation function in the hidden layer of neural networks.

We further consider

$$\mathsf{\Psi }\left(x\right):=\frac{1}{4}\left(tanh\left(x+1\right)-tanh\left(x-1\right)\right)>0,\forall x\in \mathbb{R}.$$

We easily see that $\mathsf{\Psi }\left(-x\right)=\mathsf{\Psi }\left(x\right)$, that is $\mathsf{\Psi }$ is even for $\mathbb{R}$. Therefore, $\mathsf{\Psi }$ is differentiable, and thus continuous.

Proposition 1 

([4]). $\mathsf{\Psi }\left(x\right)$ for $x\ge 0$ is strictly decreasing.

Therefore, $\mathsf{\Psi }\left(x\right)$ is strictly increasing for $x\le 0$. In addition, it holds that ${\displaystyle \underset{x\to -\infty }{lim}}\mathsf{\Psi }\left(x\right)=0={\displaystyle \underset{x\to \infty }{lim}}\mathsf{\Psi }\left(x\right).$

In fact, $\mathsf{\Psi }$ has a bell shape with a horizontal asymptote to the x-axis. Therefore, the maximum of $\mathsf{\Psi }$ is zero, $\mathsf{\Psi }\left(0\right)=0.3809297.$

Theorem 4 

([4]). We have ${\sum }_{i=-\infty }^{\infty }\mathsf{\Psi }\left(x-i\right)=1$, ∀$x\in \mathbb{R}$.

Thus,

$$\sum _{i=-\infty }^{\infty }\mathsf{\Psi }\left(nx-i\right)=1,\forall n\in \mathbb{N},\forall x\in \mathbb{R}.$$

(35)

Furthermore, it holds

$$\sum _{i=-\infty }^{\infty }\mathsf{\Psi }\left(x+i\right)=1,\forall x\in \mathbb{R}.$$

(36)

Theorem 5 

([4]). It holds

$${\int }_{-\infty }^{\infty }\mathsf{\Psi }\left(x\right)dx=1.$$

(37)

Therefore, $\mathsf{\Psi }\left(x\right)$ is a density function for $\mathbb{R}$.

Theorem 6 

([4]). Let $0<\alpha <1$ and $n\in \mathbb{N}$. It holds that

$$\sum _{\left\{\begin{array}{c}k=-\infty \\ :\left|nx-k\right|\ge n^{1-\alpha }\end{array}\right.}^{\infty }\mathsf{\Psi }\left(nx-k\right)\le e^4·e^{-2n^{\left(1-\alpha \right)}}.$$

(38)

Denote by $⌊·⌋$ the integral part of the number and by $⌈·⌉$ the ceiling of the number.

Theorem 7 

([4]). Let $x\in \left[a,b\right]\subset \mathbb{R}$ and $n\in \mathbb{N}$ so that $⌈na⌉\le ⌊nb⌋$. It holds that

$$\frac{1}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Psi }\left(nx-k\right)}<\frac{1}{\mathsf{\Psi }\left(1\right)}=4.1488766.$$

(39)

Furthermore, by [4], we obtain

$$\underset{n\to \infty }{lim}\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Psi }\left(nx-k\right)\ne 1,$$

(40)

for at least some $x\in \left[a,b\right]$.

In this article we use

$$\mathsf{\Theta }\left(x_1,...,x_N\right):=\mathsf{\Theta }\left(x\right):=\prod _{i=1}^N\mathsf{\Psi }\left(x_i\right),x=\left(x_1,...,x_N\right)=T_2\left(x\right)\in {\mathbb{R}}^N,N\in \mathbb{N}.$$

(41)

It has the properties:

(i)

$\mathsf{\Theta }\left(x\right)>0$, ∀$x\in {\mathbb{R}}^N,$

(ii)

$$\sum _{k=-\infty }^{\infty }\mathsf{\Theta }\left(x-k\right):=\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }\mathsf{\Theta }\left(x_1-k_1,...,x_N-k_N\right)=1,$$

(42)

where $k:=\left(k_1,...,k_n\right)$, ∀$x\in {\mathbb{R}}^N.$

(iii)

$$\sum _{k=-\infty }^{\infty }\mathsf{\Theta }\left(nx-k\right):=$$

(43)

$$\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }\mathsf{\Theta }\left(n{x}_1-k_1,...,n{x}_N-k_N\right)=1,$$

∀$x\in {\mathbb{R}}^N;$ $n\in \mathbb{N}$.

(iv)

$${\int }_{{\mathbb{R}}^N}\mathsf{\Theta }\left(x\right)dx=1,$$

(44)

where $\mathsf{\Theta }$ is a multivariate density function.

Therefore, we see that

$$\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)=\sum _{k=⌈na⌉}^{⌊nb⌋}\prod _{i=1}^N\mathsf{\Psi }\left(n{x}_i-x_i\right)=$$

(45)

$$\sum _{k_1=⌈n{a}_1⌉}^{⌊n{b}_1⌋}...\sum _{k_N=⌈n{a}_N⌉}^{⌊n{b}_N⌋}\prod _{i=1}^N\mathsf{\Psi }\left(n{x}_i-k_i\right)=\prod _{i=1}^N\left(\sum _{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}\mathsf{\Psi }\left(n{x}_i-k_i\right)\right).$$

For $0<\beta <1$ and $n\in \mathbb{N}$, for a fixed $x\in {\mathbb{R}}^N$, we have

$$\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)=$$

$$\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }\le \frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)+\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right).$$

(46)

In the last two equations the counting is over a disjoint vector sets of k, because the condition ${∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}$ implies that there exists at least one $\left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}$, $r\in \left\{1,...,N\right\}.$

We treat

$$\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)=\prod _{i=1}^N\left(\sum _{\left\{\begin{array}{c}{k}_i=⌈n{a}_i⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊n{b}_i⌋}\mathsf{\Psi }\left(n{x}_i-k_i\right)\right)$$

$$\le \left(\prod _{\begin{array}{c}i=1\\ i\ne r\end{array}}^N\left(\sum _{k_i=-\infty }^{\infty }\mathsf{\Psi }\left(n{x}_i-k_i\right)\right)\right)·\left(\sum _{\left\{\begin{array}{c}{k}_r=⌈n{a}_r⌉\\ \left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊n{b}_r⌋}\mathsf{\Psi }\left(n{x}_r-k_r\right)\right)$$

(47)

$$=\left(\sum _{\left\{\begin{array}{c}{k}_r=⌈n{a}_r⌉\\ \left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊n{b}_r⌋}\mathsf{\Psi }\left(n{x}_r-k_r\right)\right)$$

$$\le \sum _{\left\{\begin{array}{c}{k}_r=-\infty \\ \left|\frac{k_r}{n}-x_r\right|>\frac{1}{n^{\beta }}\end{array}\right.}^{\infty }\mathsf{\Psi }\left(n{x}_r-k_r\right)\stackrel{(\mathrm{by}\mathrm{Theorem}6)}{\le }{e}^4·e^{-2n^{\left(1-\beta \right)}}.$$

We have proven that

(v)

$$\sum _{\left\{\begin{array}{c}k=⌈na⌉\\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)\le e^4·e^{-2n^{\left(1-\beta \right)}},$$

(48)

$0<\beta <1$, $n\in \mathbb{N}$, $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

By Theorem 7, we clearly obtain

$$0<\frac{1}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)}=\frac{1}{{\prod }_{i=1}^N\left({\sum }_{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}\mathsf{\Psi }\left(n{x}_i-k_i\right)\right)}$$

(49)

$$<\frac{1}{{\left(\mathsf{\Psi }\left(1\right)\right)}^N}={\left(4.1488766\right)}^N.$$

That is,

(vi)

it holds that

$$0<\frac{1}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)}<\frac{1}{{\left(\mathsf{\Psi }\left(1\right)\right)}^N}={\left(4.1488766\right)}^N,$$

(50)

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $n\in \mathbb{N}$.

It is also clear that

(vii)

$$\sum _{\left\{\begin{array}{c}k=-\infty \\ {∥\frac{k}{n}-x∥}_{\infty }>\frac{1}{n^{\beta }}\end{array}\right.}^{\infty }\mathsf{\Theta }\left(nx-k\right)\le e^4·e^{-2n^{\left(1-\beta \right)}},$$

(51)

$0<\beta <1$, $n\in \mathbb{N}$, $x\in {\mathbb{R}}^N.$

Furthermore, we obtain

$$\underset{n\to \infty }{lim}\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)\ne 1,$$

(52)

for at least some $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

Let $f\in C\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$ and $n\in \mathbb{N}$ such that $⌈n{a}_i⌉\le ⌊n{b}_i⌋$, $i=1,...,N.$

We introduce and define the multivariate positive linear neural network operator ($x:=\left(x_1,...,x_N\right)\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$)

$$F_n\left(f,x_1,...,x_N\right):=F_n\left(f,x\right):=\frac{{\sum }_{k=⌈na⌉}^{⌊nb⌋}f\left(\frac{k}{n}\right)\mathsf{\Theta }\left(nx-k\right)}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)}$$

(53)

$$:=\frac{{\sum }_{k_1=⌈n{a}_1⌉}^{⌊n{b}_1⌋}{\sum }_{k_2=⌈n{a}_2⌉}^{⌊n{b}_2⌋}...{\sum }_{k_N=⌈n{a}_N⌉}^{⌊n{b}_N⌋}f\left(\frac{k_1}{n},...,\frac{k_N}{n}\right)\left({\prod }_{i=1}^N\mathsf{\Psi }\left(n{x}_i-k_i\right)\right)}{{\prod }_{i=1}^N\left({\sum }_{k_i=⌈n{a}_i⌉}^{⌊n{b}_i⌋}\mathsf{\Psi }\left(n{x}_i-k_i\right)\right)}.$$

For large enough n we always obtain $⌈n{a}_i⌉\le ⌊n{b}_i⌋$, $i=1,...,N$. Furthermore, $a_i\le \frac{k_i}{n}\le b_i$, iff $⌈n{a}_i⌉\le k_i\le ⌊n{b}_i⌋$, $i=1,...,N$.

Here, we study the point-wise and uniform convergence of $F_n\left(f\right)$ to f with rates.

For convenience we call

$$F_n^{*}\left(f,x\right):=\sum _{k=⌈na⌉}^{⌊nb⌋}f\left(\frac{k}{n}\right)\mathsf{\Theta }\left(nx-k\right)$$

(54)

$$:=\sum _{k_1=⌈n{a}_1⌉}^{⌊n{b}_1⌋}\sum _{k_2=⌈n{a}_2⌉}^{⌊n{b}_2⌋}...\sum _{k_N=⌈n{a}_N⌉}^{⌊n{b}_N⌋}f\left(\frac{k_1}{n},...,\frac{k_N}{n}\right)\left(\prod _{i=1}^N\mathsf{\Psi }\left(n{x}_i-k_i\right)\right),$$

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$

That is

$$F_n\left(f,x\right):=\frac{F_n^{*}\left(f,x\right)}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)},$$

(55)

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $n\in \mathbb{N}.$

Hence,

$$F_n\left(f,x\right)-f\left(x\right)=\frac{F_n^{*}\left(f,x\right)-f\left(x\right){\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)}{{\sum }_{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)}.$$

(56)

Consequently, we derive

$$\left|F_n\left(f,x\right)-f\left(x\right)\right|\le {\left(4.1488766\right)}^N\left|F_n^{*}\left(f,x\right)-f\left(x\right)\sum _{k=⌈na⌉}^{⌊nb⌋}\mathsf{\Theta }\left(nx-k\right)\right|,$$

(57)

∀$x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right).$ When $f\in C_B\left({\mathbb{R}}^N\right)$ we define,

$${\overline{F}}_n\left(f,x\right):={\overline{F}}_n\left(f,x_1,...,x_N\right):=\sum _{k=-\infty }^{\infty }f\left(\frac{k}{n}\right)\mathsf{\Theta }\left(nx-k\right):=$$

(58)

$$\sum _{k_1=-\infty }^{\infty }\sum _{k_2=-\infty }^{\infty }...\sum _{k_N=-\infty }^{\infty }f\left(\frac{k_1}{n},\frac{k_2}{n},...,\frac{k_N}{n}\right)\left(\prod _{i=1}^N\mathsf{\Psi }\left(n{x}_i-k_i\right)\right),$$

$n\in \mathbb{N}$, ∀$x\in {\mathbb{R}}^N,$ $N\ge 1$, is the multivariate quasi-interpolation neural network operator. Here, we present a series of multivariate neural network approximations of a function given with rates. We first give

Theorem 8. 

Let $f\in C\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right),$ $0<\beta <1,$ $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $n,N\in \mathbb{N}$. Then,

(i) 

$$\left|F_n\left(f,x\right)-f\left(x\right)\right|\le {\left(4.1488766\right)}^N·$$

$$\left\{{\omega }_1\left(f,\frac{1}{n^{\beta }}\right)+2e^4{∥f∥}_{\infty }{e}^{-2n^{\left(1-\beta \right)}}\right\}=:{\lambda }_1,$$

(59)

(ii) 

$${∥F_n\left(f\right)-f∥}_{\infty }\le {\lambda }_1.$$

(60)

Next, we present

Theorem 9. 

Let $f\in C_B\left({\mathbb{R}}^N\right)$, $0<\beta <1$, $x\in {\mathbb{R}}^N$, $n,N\in \mathbb{N}$. Then,

(i) 

$$\left|{\overline{F}}_n\left(f,x\right)-f\left(x\right)\right|\le {\omega }_1\left(f,\frac{1}{n^{\beta }}\right)+2e^4{∥f∥}_{\infty }{e}^{-2n^{\left(1-\beta \right)}}=:{\lambda }_2,$$

(61)

(ii) 

$${∥{\overline{F}}_n\left(f\right)-f∥}_{\infty }\le {\lambda }_2.$$

(62)

Next, we discuss high-order approximations by using the smoothness of f. We give

Theorem 10. 

Let $f\in C^m\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $0<\beta <1$, $n,m,N\in \mathbb{N}$, $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$. Then,

(i) 

$$\left|F_n\left(f,x\right)-f\left(x\right)-\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{f_{\alpha }\left(x\right)}{{\prod }_{i=1}^N{\alpha }_i!}\right)F_n\left(\prod _{i=1}^N{\left(·-x_i\right)}^{{\alpha }_i},x\right)\right)\right|\le$$

$${\left(4.1488766\right)}^N·\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\right.$$

(63)

$$\left.\left(\frac{2e^4{∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-2n^{\left(1-\beta \right)}}\right\},$$

(ii) 

$$\left|F_n\left(f,x\right)-f\left(x\right)\right|\le {\left(4.1488766\right)}^N·$$

$$\left\{\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{\left|f_{\alpha }\left(x\right)\right|}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j}}+\left(\prod _{i=1}^N{\left(b_i-a_i\right)}^{{\alpha }_i}\right)·e^4{e}^{-2n^{\left(1-\beta \right)}}\right]\right)+\right.$$

(64)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2e^4{∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-2n^{\left(1-\beta \right)}}\right\},$$

(iii) 

$${∥F_n\left(f\right)-f∥}_{\infty }\le {\left(4.1488766\right)}^N·$$

$$\left\{\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{{∥f_{\alpha }∥}_{\infty }}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j}}+\left(\prod _{i=1}^N{\left(b_i-a_i\right)}^{{\alpha }_i}\right)e^4{e}^{-2n^{\left(1-\beta \right)}}\right]\right)+\right.$$

(65)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2e^4{∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-2n^{\left(1-\beta \right)}}\right\},$$

(iv) 

Assume $f_{\alpha }\left(x_0\right)=0$, for all $\alpha :\left|\alpha \right|=1,...,m;$ $x_0\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$. Then

$$\left|F_n\left(f,x_0\right)-f\left(x_0\right)\right|\le {\left(4.1488766\right)}^N·$$

(66)

$$\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_1^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2e^4{∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)e^{-2n^{\left(1-\beta \right)}}\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-\beta \left(m+1\right)}.$

2.3. Combining Section 2.1 and Section 2.2

For the next theorems we call

$$T_1\left(x\right):=\mathsf{\Phi }\left(x\right)$$

$$T_2\left(x\right):=\mathsf{\Theta }\left(x\right),x\in {\mathbb{R}}^{\mathbb{N}},N\in \mathbb{N}.$$

We also set

$${\gamma }_{1,N}:={\left(5.250312578\right)}^N,$$

$${\gamma }_{2,N}:={\left(4.1488766\right)}^N.$$

Furthermore, we set

$${\alpha }_{1,n}\left(\beta \right):=3.1992e^{-n^{\left(1-\beta \right)}},$$

$${\alpha }_{2,n}\left(\beta \right):=e^4{e}^{-2n^{\left(1-\beta \right)}},$$

where $0<\beta <1,n\in \mathbb{N}.$

We define,

$${}_1{L}_n\left(f,x\right):=G_n\left(f,x\right),$$

$${}_2{L}_n\left(f,x\right):=F_n\left(f,x\right),x\in {\mathbb{R}}^{\mathbb{N}},n\in \mathbb{N}$$

and

$${}_1{\overline{L}}_n\left(f,x\right):={\overline{G}}_n\left(f,x\right),$$

$${}_2{\overline{L}}_n\left(f,x\right):={\overline{F}}_n\left(f,x\right),x\in {\mathbb{R}}^{\mathbb{N}},n\in \mathbb{N}.$$

Notice that

$$6.3984e^{-n^{\left(1-\beta \right)}}=2a_{1,n}\left(\beta \right).$$

Theorem 11. 

Let $f\in C\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right),$ $0<\beta <1,$ $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $n,N\in \mathbb{N}$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{L}_n\left(f,x\right)-f\left(x\right)\right|\le {\gamma }_{k,N}$$

$$\left\{{\omega }_1\left(f,\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥f∥}_{\infty }\right\}=:{\lambda }_1,$$

(67)

(ii) 

$${∥{}_k{L}_n\left(f\right)-f∥}_{\infty }\le {\lambda }_1.$$

(68)

Proof. 

From Theorems 1 and 8. □

Next, we present

Theorem 12. 

Let $f\in C_B\left({\mathbb{R}}^N\right)$, $0<\beta <1$, $x\in {\mathbb{R}}^N$, $n,N\in \mathbb{N}$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{\overline{L}}_n\left(f,x\right)-f\left(x\right)\right|\le {\omega }_1\left(f,\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥f∥}_{\infty }=:{\lambda }_2,$$

(69)

(ii) 

$${∥{}_k{\overline{L}}_n\left(f\right)-f∥}_{\infty }\le {\lambda }_2.$$

(70)

Proof. 

From Theorems 2 and 9. □

Next, we discuss high-order approximations by using the smoothness of f.

We give

Theorem 13. 

Let $f\in C^m\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $0<\beta <1$, $n,m,N\in \mathbb{N}$, $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{L}_n\left(f,x\right)-f\left(x\right)-\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{f_{\alpha }\left(x\right)}{{\prod }_{i=1}^N{\alpha }_i!}\right){}_k{L}_n\left(\prod _{i=1}^N{\left(·-x_i\right)}^{{\alpha }_i},x\right)\right)\right|\le$$

$${\gamma }_{k,N}·\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\right.$$

(71)

$$\left.\left(\frac{2{\alpha }_{k,n}{∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

(ii) 

$$\left|{}_k{L}_n\left(f,x\right)-f\left(x\right)\right|\le {\gamma }_{k,N}·$$

$$\left\{\sum _{j=1}^m\left(\sum _{\left|\alpha \right|=j}\left(\frac{\left|f_{\alpha }\left(x\right)\right|}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j}}+\left(\prod _{i=1}^N{\left(b_i-a_i\right)}^{{\alpha }_i}\right)·{\alpha }_{k,n}\left(\beta \right)\right]\right)+\right.$$

(72)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2{\alpha }_{k,n}\left(\beta \right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

(iii) 

$${∥{}_k{L}_n\left(f\right)-f∥}_{\infty }\le {\gamma }_{k,N}·$$

$$\left\{\sum _{j=1}^N\left(\sum _{\left|\alpha \right|=j}\left(\frac{{∥f_{\alpha }∥}_{\infty }}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j}}+\left(\prod _{i=1}^N{\left(b_i-a_i\right)}^{{\alpha }_i}\right){\alpha }_{k,n}\left(\beta \right)\right]\right)+\right.$$

(73)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2{\alpha }_{k,n}\left(\beta \right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

(iv) 

Assume $f_{\alpha }\left(x_0\right)=0$, for all $\alpha :\left|\alpha \right|=1,...,m;$ $x_0\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$. Then,

$$\left|{}_k{L}_n\left(f,x_0\right)-f\left(x_0\right)\right|\le {\gamma }_{k,N}·$$

(74)

$$\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_1^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2{\alpha }_{k,n}\left(\beta \right){∥b-a∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

Notice in the later the extremely high rate of convergence at $n^{-\beta \left(m+1\right)}.$

Proof. 

From Theorems 3 and 10. □

Next, we apply Theorem 13 for $m=1$.

Corollary 1. 

Let $f\in C^1\left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$, $0<\beta <1$, $n,m,N\in \mathbb{N}$, $x\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{L}_n\left(f,x\right)-f\left(x\right)-\sum _{i=1}^N\frac{\partial f}{\partial x_i}{}_k{L}_n\left(\left(·-x_i\right),x\right)\right|\le$$

$${\gamma }_{k,N}·\left\{\frac{N}{n^{\beta }}\underset{i\in \{1,\cdots ,N\}}{max}{\omega }_1\left(\frac{\partial f}{\partial x_i},\frac{1}{n^{\beta }}\right)+\right.$$

(75)

$$\left.\left(2{\alpha }_{k,n}N{∥b-a∥}_{\infty }\underset{i\in \{1,\cdots ,N\}}{max}{∥\frac{\partial f}{\partial x_i}∥}_{\infty }\right)\right\},$$

(ii) 

$$\left|{}_k{L}_n\left(f,x\right)-f\left(x\right)\right|\le {\gamma }_{k,N}·$$

$$\left\{\sum _{i=1}^N\left|\frac{\partial f\left(x\right)}{\partial x_i}\right|\left[\frac{1}{n^{\beta }}+\left(b_i-a_i\right)·{\alpha }_{k,n}\left(\beta \right)\right]+\right.$$

(76)

$$\left.\frac{N}{n^{\beta }}\underset{i\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial f}{\partial x_i},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥b-a∥}_{\infty }\underset{i\in \{1,\dots ,N\}}{max}{∥\frac{\partial f}{\partial x_i}∥}_{\infty }\right)\right\},$$

(iii) 

$${∥{}_k{L}_n\left(f\right)-f∥}_{\infty }\le {\gamma }_{k,N}·$$

$$\left\{\sum _{i=1}^N{\left|\frac{\partial f}{\partial x_i}\right|}_{\infty }\left[\frac{1}{n^{\beta }}+\left(b_i-a_i\right){\alpha }_{k,n}\left(\beta \right)\right]+\right.$$

(77)

$$\left.\frac{N}{n^{\beta }}\underset{i\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial f}{\partial x_i},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥b-a∥}_{\infty }\underset{i\in \{1,\dots ,N\}}{max}{\parallel \frac{\partial f}{\partial x_i}\parallel }_{\infty }\right)\right\}.$$

(iv) 

Assume $\frac{\partial f}{\partial x_i}\left(x_0\right)=0,i=1,\dots ,N$, where $x_0\in \left({\prod }_{i=1}^N\left[a_i,b_i\right]\right)$. Then

$$\left|{}_k{L}_n\left(f,x_0\right)-f\left(x_0\right)\right|\le {\gamma }_{k,N}·$$

(78)

$$\left\{\frac{N}{n^{\beta }}\underset{i\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial f}{\partial x_i},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥b-a∥}_{\infty }\underset{i\in \{1,\dots ,N\}}{max}{∥\frac{\partial f}{\partial x_i}∥}_{\infty }\right)\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-2\beta }.$

3. Multiple Time-Separating Random Functions

Let $\left(\mathsf{\Omega },\mathcal{F},P\right)$ be a probability space, $\omega \in \mathsf{\Omega };Y_1,Y_2,\dots ,Y_m,m\in \mathbb{N},$ be real-valued random variables for $\mathsf{\Omega }$ with finite expectations, and $h_1\left(t\right),h_2\left(t\right),\dots h_m\left(t\right):{\prod }_{j=1}^N{I}_j\to \mathbb{R}$, where $I_j$ are infinite subsets of $\mathbb{R}$ for every $j=1,\dots ,N$. Typically, $I_j$ is an infinite length interval of $\mathbb{R}$, usually $I_j=\mathbb{R}$ or $I_j={\mathbb{R}}_{+},$ for $j=1,\dots ,N$.

Clearly,

$$Y\left(t,\omega \right):=\sum _{i=1}^m{h}_i\left(t\right)Y_i\left(\omega \right),t\in \prod _{j=1}^N{I}_j,$$

(79)

is a common time-separating random function.

We can assume that $h_i\in C^r\left({\prod }_{j=1}^N{I}_j\right),i=1,2,...,m;r\in \mathbb{N}$. Consequently, we have the expectation

$$\left(EY\right)\left(t\right)=\sum _{i=1}^m{h}_i\left(t\right)E{Y}_i\in C\left(\prod _{j=1}^N{I}_j\right)\mathrm{or}{C}^r\left(\prod _{j=1}^N{I}_j\right).$$

(80)

A classical example of a multiple time-separating process is

$$\left(sin\left(\prod _{j=1}^N{t}_j\right)\right)Y_1\left(\omega \right)+\left(cos\left(\prod _{j=1}^N{t}_j\right)\right)Y_2\left(\omega \right),t_j\in I_j,$$

for $j=1,\dots ,N$.

Notice that $\left|sin\left({\displaystyle \prod _{j=1}^N}{t}_j\right)\right|\le 1$ and $\left|cos\left({\displaystyle \prod _{j=1}^N}{t}_j\right)\right|\le 1$.

Another typical example is

$$\left(sinh\left(\prod _{j=1}^N{t}_j\right)\right)Y_1\left(\omega \right)+\left(cosh\left(\prod _{j=1}^N{t}_j\right)\right)Y_2\left(\omega \right),t_j\in I_j,\mathrm{for}j=1,\dots ,N.$$

(81)

In this article we will apply the main results of Section 2.3, to $f\left(t\right)=\left(EY\right)\left(t\right)$. We conclude with several applications.

4. Main Results

We present the following stochastic approximation result.

Theorem 14. 

Let $\left(EY\right)\left(t\right)$ as in (80), $t\in {\prod }_{j=1}^N{I}_j.$ Here $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j},$ for every $j=1,\dots ,N$. Let also $0<\beta <1,$ $n,N\in \mathbb{N}$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{L}_n\left(\left(EY\right),t\right)-\left(EY\right)\left(t\right)\right|\le {\gamma }_{k,N}$$

$$\left\{{\omega }_1\left(\left(EY\right),\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥\left(EY\right)∥}_{\infty }\right\}=:{\lambda }_1,$$

(82)

(ii) 

$${∥{}_k{L}_n\left(EY\right)-\left(EY\right)∥}_{\infty }\le {\lambda }_1.$$

(83)

Proof. 

From Theorem 11. □

The counter part of the previous theorem follows.

Theorem 15. 

Let $\left(EY\right)\left(t\right)$ as in (80), $h_i\in C_B\left({\mathbb{R}}^{\mathbb{N}}\right)$ for every $i=1,\dots ,m$. Let also $0<\beta <1$, $n,N\in \mathbb{N}$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{\overline{L}}_n\left(\left(EY\right),t\right)-\left(EY\right)\left(t\right)\right|\le {\omega }_1\left(\left(EY\right),\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥\left(EY\right)∥}_{\infty }=:{\lambda }_2,$$

(84)

(ii) 

$${∥{}_k{\overline{L}}_n\left(EY\right)-\left(EY\right)∥}_{\infty }\le {\lambda }_2.$$

(85)

Proof. 

From Theorem 12. □

We give

Theorem 16. 

Let $\left(EY\right)\left(t\right)$ as in (80), $\left(EY\right)\left(t\right)\in C^m\left({\displaystyle \prod _{j=1}^N}{I}_j\right),t\in {\prod }_{j=1}^N{I}_j.$ Here, $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j},$ for $j=1,\dots ,N,t_1=\left(t_{1,1},\dots ,t_{1,N}\right),t_2=\left(t_{2,1},\dots ,t_{2,N}\right)$. Let also $0<\beta <1,$ $n,N\in \mathbb{N}$ and $k=1,2$. Then,

(i) 

$$\left|{}_k{L}_n\left(\left(EY\right),t\right)-\left(EY\right)\left(t\right)-\sum _{j^{*}=1}^m\left(\sum _{\left|\alpha \right|=j^{*}}\left(\frac{{\left(EY\right)}_{\alpha }\left(t\right)}{{\prod }_{i=1}^N{\alpha }_i!}\right){}_k{L}_n\left(\prod _{i=1}^N{\left(·-x_i\right)}^{{\alpha }_i},x\right)\right)\right|\le$$

$${\gamma }_{k,N}·\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left({\left(EY\right)}_{\alpha },\frac{1}{n^{\beta }}\right)+\right.$$

(86)

$$\left.\left(\frac{2{\alpha }_{k,n}{∥t_2-t_1∥}_{\infty }^m{∥{\left(EY\right)}_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

(ii) 

$$\left|{}_k{L}_n\left(\left(EY\right),t\right)-\left(EY\right)\left(t\right)\right|\le {\gamma }_{k,N}·$$

$$\left\{\sum _{j^{*}=1}^m\left(\sum _{\left|\alpha \right|=j^{*}}\left(\frac{\left|{\left(EY\right)}_{\alpha }\left(t\right)\right|}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j^{*}}}+\left(\prod _{i=1}^N{\left(t_{2,i}-t_{1,i}\right)}^{{\alpha }_i}\right)·{\alpha }_{k,n}\left(\beta \right)\right]\right)+\right.$$

(87)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left({\left(EY\right)}_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2{\alpha }_{k,n}\left(\beta \right){∥t_2-t_1∥}_{\infty }^m{∥{\left(EY\right)}_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

(iii) 

$${∥{}_k{L}_n\left(\left(EY\right)\right)-\left(EY\right)∥}_{\infty }\le {\gamma }_{k,N}·$$

$$\left\{\sum _{j^{*}=1}^N\left(\sum _{\left|\alpha \right|=j^{*}}\left(\frac{{∥{\left(EY\right)}_{\alpha }∥}_{\infty }}{{\prod }_{i=1}^N{\alpha }_i!}\right)\left[\frac{1}{n^{\beta j^{*}}}+\left(\prod _{i=1}^N{\left(t_{2,i}-t_{1,i}\right)}^{{\alpha }_i}\right){\alpha }_{k,n}\left(\beta \right)\right]\right)+\right.$$

(88)

$$\left.\frac{N^m}{m!n^{m\beta }}{\omega }_{1,m}^{max}\left(f_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2{\alpha }_{k,n}\left(\beta \right){∥t_2-t_1∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

(iv) 

Assume ${\left(EY\right)}_{\alpha }\left(t_0\right)=0$, for all $\alpha :\left|\alpha \right|=1,...,m;$ $x_0\in \left({\prod }_{i=1}^N\left[t_{1,i},t_{2,i}\right]\right)$. Then

$$\left|{}_k{L}_n\left(\left(EY\right),t_0\right)-\left(EY\right)\left(t_0\right)\right|\le {\gamma }_{k,N}·$$

(89)

$$\left\{\frac{N^m}{m!n^{m\beta }}{\omega }_1^{max}\left({\left(EY\right)}_{\alpha },\frac{1}{n^{\beta }}\right)+\left(\frac{2{\alpha }_{k,n}\left(\beta \right){∥t_2-t_1∥}_{\infty }^m{∥f_{\alpha }∥}_{\infty ,m}^{max}{N}^m}{m!}\right)\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-\beta \left(m+1\right)}.$

Proof. 

From Theorem 13. □

Next, we apply Theorem 16 for $m=1$.

Corollary 2. 

Let $\left(EY\right)\left(t\right)$ as in (80), $\left(EY\right)\left(t\right)\in C^1\left({\displaystyle \prod _{j=1}^N}{I}_j\right),t\in {\prod }_{j=1}^N{I}_j.$ Here, $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j}$ for $j=1,\dots ,N,t_1=\left(t_{1,1},\dots ,t_{1,N}\right),t_2=\left(t_{2,1},\dots ,t_{2,N}\right)$. Let also $0<\beta <1$, $n,m,N\in \mathbb{N}$, and $k=1,2$. Then,

(i) 

$$\left|{}_k{L}_n\left(\left(EY\right),t\right)-\left(EY\right)\left(t\right)-\sum _{i=1}^N\frac{\partial \left(EY\right)}{\partial t_i}{}_k{L}_n\left(\left(·-t_i\right),t\right)\right|\le$$

$${\gamma }_{k,N}·\left\{\frac{N}{n^{\beta }}\underset{i\in \{1,\cdots ,N\}}{max}{\omega }_1\left(\frac{\partial \left(EY\right)}{\partial t_i},\frac{1}{n^{\beta }}\right)+\right.$$

(90)

$$\left.\left(2{\alpha }_{k,n}N{∥t_2-t_1∥}_{\infty }\underset{i\in \{1,\cdots ,N\}}{max}{∥\frac{\partial \left(EY\right)}{\partial t_i}∥}_{\infty }\right)\right\},$$

(ii) 

$$\left|{}_k{L}_n\left(\left(EY\right),t\right)-\left(EY\right)\left(t\right)\right|\le {\gamma }_{k,N}·$$

$$\left\{\sum _{i=1}^N\left|\frac{\partial \left(EY\right)\left(t\right)}{\partial t_i}\right|\left[\frac{1}{n^{\beta }}+\left(t_{2,i}-t_{1,i}\right)·{\alpha }_{k,n}\left(\beta \right)\right]+\right.$$

(91)

$$\left.\frac{N}{n^{\beta }}\underset{i\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial \left(EY\right)}{\partial t_i},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥t_2-t_1∥}_{\infty }\underset{i\in \{1,\dots ,N\}}{max}{∥\frac{\partial \left(EY\right)}{\partial t_i}∥}_{\infty }\right)\right\},$$

(iii) 

$${∥{}_k{L}_n\left(\left(EY\right)\right)-\left(EY\right)∥}_{\infty }\le {\gamma }_{k,N}·$$

(92)

$$\left\{\sum _{i=1}^N{\left|\frac{\partial \left(EY\right)}{\partial t_i}\right|}_{\infty }\left[\frac{1}{n^{\beta }}+\left(t_{2,i}-t_{1,i}\right){\alpha }_{k,n}\left(\beta \right)\right]+\right.$$

$$\left.\frac{N}{n^{\beta }}\underset{i\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial \left(EY\right)}{\partial t_i},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥t_2-t_1∥}_{\infty }\underset{i\in \{1,\dots ,N\}}{max}{\parallel \frac{\partial \left(EY\right)}{\partial t_i}\parallel }_{\infty }\right)\right\}.$$

(iv) 

Assume $\frac{\partial \left(EY\right)}{\partial t_i}\left(t_0\right)=0,i=1,\dots ,N$, where $t_0\in \left({\prod }_{i=1}^N\left[t_{1,i},t_{2,i}\right]\right)$. Then

$$\left|{}_k{L}_n\left(\left(EY\right),t_0\right)-\left(EY\right)\left(t_0\right)\right|\le {\gamma }_{k,N}·$$

(93)

$$\left\{\frac{N}{n^{\beta }}\underset{i\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial \left(EY\right)}{\partial t_i},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥t_2-t_1∥}_{\infty }\underset{i\in \{1,\dots ,N\}}{max}{∥\frac{\partial \left(EY\right)}{\partial t_i}∥}_{\infty }\right)\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-2\beta }.$

5. Applications

For applications, we consider $\left(\mathsf{\Omega },F,P\right)$ as a probability space and $Y_0,Y_1$ and $Y_2$ as real-valued random variables of $\mathsf{\Omega }$ with finite expectations. We consider the stochastic processes $Z_i(t,\omega )$ for $i=1,2,\dots ,7,$ where $t=\left(t_1,\dots ,t_N\right)\in {\mathbb{R}}^{\mathbb{N}}$ and $\omega \in \mathsf{\Omega }$ as follows:

$$Z_1(t,\omega )=\left[{\left(\sum _{j=1}^N\left(t_j-t_{j,0}\right)\right)}^{\mu +1}+1\right]Y_0\left(\omega \right),$$

(94)

where $t_0=\left(t_{1,0},\dots ,t_{N,0}\right)\in {\mathbb{R}}^{\mathbb{N}}$ and $\mu \in \mathbb{N}$ are fixed;

$$Z_2(t,\omega )=sin\left(\xi \sum _{j=1}^N{t}_j\right)Y_1\left(\omega \right)+cos\left(\xi \sum _{j=1}^N{t}_j\right)Y_2\left(\omega \right),$$

(95)

where $\xi >0$ is fixed;

$$Z_3(t,\omega )=sinh\left(\mu \sum _{j=1}^N{t}_j\right)Y_1\left(\omega \right)+cosh\left(\mu \sum _{j=1}^N{t}_j\right)Y_2\left(\omega \right),$$

(96)

where $\mu >0$ is fixed;

$$Z_4(t,\omega )=sech\left(\mu \sum _{j=1}^N{t}_j\right)Y_1\left(\omega \right)+tanh\left(\mu \sum _{j=1}^N{t}_j\right)Y_2\left(\omega \right),$$

(97)

where $\mu >0$ is fixed.

Here, $sechx:=\frac{1}{cosh\left({\displaystyle \sum _{j=1}^N}{x}_j\right)}=\frac{2}{exp\left({\displaystyle \sum _{j=1}^N}{x}_j\right)+exp\left(-{\displaystyle \sum _{j=1}^N}{x}_j\right)},x=\left(x_1,\cdots ,x_n\right)\in {\mathbb{R}}^{\mathbb{N}}$.

$$Z_5(t,\omega )=exp\left(-{\ell }_1\prod _{j=1}^N{t}_j\right)Y_1\left(\omega \right)+exp\left(-{\ell }_2\prod _{j=1}^N{t}_j\right)Y_2\left(\omega \right),$$

(98)

where ${\ell }_1,{\ell }_2>0$ are fixed;

$$Z_6(t,\omega )=\frac{1}{1+exp\left(-{\ell }_1{\displaystyle \prod _{j=1}^N}{t}_j\right)}{Y}_1\left(\omega \right)+\frac{1}{1+exp\left(-{\ell }_2{\displaystyle \prod _{j=1}^N}{t}_j\right)}{Y}_2\left(\omega \right),$$

(99)

where ${\ell }_1,{\ell }_2>0$ are fixed;

$$Z_7(t,\omega )=e^{-e^{-{\mu }_1{\displaystyle \prod _{j=1}^N}{t}_j}}{Y}_1\left(\omega \right)+e^{-e^{-{\mu }_2{\displaystyle \prod _{j=1}^N}{t}_j}}{Y}_2\left(\omega \right),$$

(100)

where ${\mu }_1,{\mu }_2>0$ are fixed;

The expectations of $Z_i,i=1,2,...,7$ are

$$\left(E{Z}_1\right)\left(t\right)=\left[{\left(\sum _{j=1}^N\left(t_j-t_{j,0}\right)\right)}^{\mu +1}+1\right]E\left(Y_0\right),$$

(101)

$$\left(E{Z}_2\right)\left(t\right)=sin\left(\xi \sum _{j=1}^N{t}_j\right)E\left(Y_1\right)+cos\left(\xi \sum _{j=1}^N{t}_j\right)E\left(Y_2\right),$$

(102)

$$\left(E{Z}_3\right)\left(t\right)=sinh\left(\mu \sum _{j=1}^N{t}_j\right)E\left(Y_1\right)+cosh\left(\mu \sum _{j=1}^N{t}_j\right)E\left(Y_2\right),$$

(103)

$$\left(E{Z}_4\right)\left(t\right)=sech\left(\mu \sum _{j=1}^N{t}_j\right)E\left(Y_1\right)+tanh\left(\mu \sum _{j=1}^N{t}_j\right)E\left(Y_2\right),$$

(104)

$$\left(E{Z}_5\right)\left(t\right)=exp\left(-{\ell }_1\prod _{j=1}^N{t}_j\right)E\left(Y_1\right)+exp\left(-{\ell }_2\prod _{j=1}^N{t}_j\right)E\left(Y_2\right),$$

(105)

$$\left(E{Z}_6\right)\left(t\right)=\frac{1}{1+exp\left(-{\ell }_1{\displaystyle \prod _{j=1}^N}{t}_j\right)}E\left(Y_1\right)+\frac{1}{1+exp\left(-{\ell }_2{\displaystyle \prod _{j=1}^N}{t}_j\right)}E\left(Y_2\right),$$

(106)

$$\left(E{Z}_7\right)\left(t\right)=e^{-e^{-{\mu }_1{\displaystyle \prod _{j=1}^N}{t}_j}}E\left(Y_1\right)+e^{-e^{-{\mu }_2{\displaystyle \prod _{j=1}^N}{t}_j}}E\left(Y_2\right),$$

(107)

Next, $\left(E{Z}_i\right)\left(t\right)$ and $i=1,2,\dots ,7$ are as defined in relations between (101) and (107), respectively.

We present the results in the following.

Proposition 2. 

Let $t\in {\displaystyle \prod _{j=1}^N}{I}_j.$ Here, $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j},$ for every $j=1,\dots ,N$. Let also $0<\beta <1,$ $n,N\in \mathbb{N}$ and $k=1,2$. Then, for $i=1,2,\dots ,7$

(i) 

$$\left|{}_k{L}_n\left(\left(E{Z}_i\right),t\right)-\left(E{Z}_i\right)\left(t\right)\right|\le {\gamma }_{k,N}$$

$$\left\{{\omega }_1\left(\left(E{Z}_i\right),\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥\left(E{Z}_i\right)∥}_{\infty }\right\}=:{\lambda }_1,$$

(108)

(ii) 

$${∥{}_k{L}_n\left(E{Z}_i\right)-\left(E{Z}_i\right)∥}_{\infty }\le {\lambda }_1.$$

(109)

Proof. 

From Theorem 14. □

Next, we present.

Proposition 3. 

Let $0<\beta <1$, $n,N\in \mathbb{N}$ and $k=1,2$. Then, for $i\in \left\{2,4,6,7\right\},$

(i) 

$$\left|{}_k{\overline{L}}_n\left(\left(E{Z}_i\right),t\right)-\left(E{Z}_i\right)\left(t\right)\right|\le {\omega }_1\left(\left(E{Z}_i\right),\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥\left(E{Z}_i\right)∥}_{\infty }=:{\lambda }_2,$$

(110)

(ii) 

$${∥{}_k{\overline{L}}_n\left(E{Z}_i\right)-\left(E{Z}_i\right)∥}_{\infty }\le {\lambda }_2.$$

(111)

Proof. 

Notice that for every $t\in \mathbb{R}$ we have:

• for $Z_2\left(t,\omega \right)$, $\left|sin\left(\xi {\displaystyle \sum _{i=1}^N}{t}_i\right)\right|\le 1$ and $\left|cos\left(\xi {\displaystyle \sum _{i=1}^N}{t}_i\right)\right|\le 1$,
• for $Z_4\left(t,\omega \right)$, $\left|sech\left(\mu {\displaystyle \sum _{i=1}^N}{t}_i\right)\right|\le 1$ and $\left|tanh\left(\mu {\displaystyle \sum _{i=1}^N}{t}_i\right)\right|\le 1$,
• for $Z_6\left(t,\omega \right)$, $0<\frac{1}{1+exp\left(-{\ell }_1{\displaystyle \prod _{i=1}^N}{t}_i\right)}<1$ and $0<\frac{1}{1+exp\left(-{\ell }_2{\displaystyle \prod _{i=1}^N}{t}_i\right)}<1$,
• for $Z_7\left(t,\omega \right)$, $0<e^{-e^{-{\mu }_1{\displaystyle \prod _{i=1}^N}{t}_i}}<1$ and $0<e^{-e^{-{\mu }_2{\displaystyle \prod _{i=1}^N}{t}_i}}<1$.

Thus, the results from Theorem 16 are □

Proposition 4. 

Let $t\in {\displaystyle \prod _{i=j}^N}{I}_j.$ Here $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j}$ for $j=1,\dots ,N,t_1=\left(t_{1,1},\dots ,t_{1,N}\right),t_2=\left(t_{2,1},\dots ,t_{2,N}\right)$. Let also $0<\beta <1$, $n,m,N\in \mathbb{N}$, and $k=1,2$. Then, for $i=1,\dots ,7$

(i) 

$$\left|{}_k{L}_n\left(\left(E{Z}_i\right),t\right)-\left(E{Z}_i\right)\left(t\right)-{\displaystyle \sum _{j=1}^N}\frac{\partial \left(E{Z}_i\right)}{\partial t_j}{}_k{L}_n\left(\left(·-t_j\right),t\right)\right|\le$$

$${\gamma }_{k,N}·\left\{\frac{N}{n^{\beta }}\underset{j\in \{1,\cdots ,N\}}{max}{\omega }_1\left(\frac{\partial \left(E{Z}_i\right)}{\partial t_j},\frac{1}{n^{\beta }}\right)+\right.$$

(112)

$$\left.\left(2{\alpha }_{k,n}N{∥t_2-t_1∥}_{\infty }\underset{j\in \{1,\cdots ,N\}}{max}{∥\frac{\partial \left(E{Z}_i\right)}{\partial t_j}∥}_{\infty }\right)\right\}.$$

(ii) 

Assume $\frac{\partial \left(E{Z}_i\right)}{\partial t_j}\left(t_0\right)=0,j=1,\dots ,N$, where $t_0\in \left({\displaystyle \prod _{j=1}^N}\left[t_{1,j},t_{2,j}\right]\right)$. Then,

$$\left|{}_k{L}_n\left(\left(E{Z}_i\right),t_0\right)-\left(E{Z}_i\right)\left(t_0\right)\right|\le {\gamma }_{k,N}·$$

(113)

$$\left\{\frac{N}{n^{\beta }}\underset{j\in \{1,\dots ,N\}}{max}{\omega }_1\left(\frac{\partial \left(E{Z}_i\right)}{\partial t_j},\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥t_2-t_1∥}_{\infty }\underset{j\in \{1,\dots ,N\}}{max}{∥\frac{\partial \left(E{Z}_i\right)}{\partial t_j}∥}_{\infty }\right)\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-2\beta }.$

Proof. 

From Corollary 2. □

6. Specific Applications

Let $\left(\mathsf{\Omega },\mathcal{F},P\right)$, where $\mathsf{\Omega }$ is the set of non-negative integers, be a probability space, $Y_{1,1}$ and $Y_{2,1}$ be real-valued random variables on $\mathsf{\Omega }$ following Poisson distributions with parameters ${\lambda }_1$ and ${\lambda }_2\in \left(0,\infty \right)$, respectively.

We consider the stochastic process $W_1(t,\omega ),$ where $t=\left(t_1,\dots ,t_N\right)\in {\mathbb{R}}^{\mathbb{N}}$ and $\omega \in \mathsf{\Omega }$ as follows:

$$W_1(t,\omega )=sin\left(\xi \sum _{j=1}^N{t}_j\right)Y_{1,1}\left(\omega \right)+cos\left(\xi \sum _{j=1}^N{t}_j\right)Y_{2,1}\left(\omega \right),$$

(114)

where $\xi >0$ is fixed.

Since $E\left(Y_{1,1}\right)={\lambda }_1$ and $E\left(Y_{2,1}\right)={\lambda }_2$, the expectations of $Z_{2,1}$ are

$$\left(E{W}_1\right)\left(t\right)=sin\left(\xi \sum _{j=1}^N{t}_j\right){\lambda }_1+cos\left(\xi \sum _{j=1}^N{t}_j\right){\lambda }_2,$$

(115)

Furthermore, for $j^{*}=1,\dots ,N$ we have

$$\frac{\partial \left(E{W}_1\right)\left(t\right)}{\partial t_{j^{*}}}=\xi \left[cos\left(\xi \sum _{j=1}^N{t}_j\right){\lambda }_1-sin\left(\xi \sum _{j=1}^N{t}_j\right){\lambda }_2\right]=:D_1\left(t\right).$$

(116)

Notice that

$$\frac{\partial \left(E{W}_1\right)\left(t\right)}{\partial t_{j^{*}}}=\frac{\partial \left(E{W}_1\right)\left(t\right)}{\partial t_{i^{*}}}=D_1\left(t\right)$$

for every $j^{*},i^{*}=1,\dots ,N.$

For the next we consider $\left(\mathsf{\Omega },\mathcal{F},P\right)$, where $\mathsf{\Omega }=\mathbb{R}$ is a probability space, with $Y_{1,2}$ and $Y_{2,2}$ of $\mathsf{\Omega }$ following Gaussian distributions with expectations ${\widehat{\mu }}_1$ and ${\widehat{\mu }}_2\in \mathbb{R}$, respectively. We consider the stochastic process $W_2(t,\omega ),$ where $t=\left(t_1,\dots ,t_N\right)\in {\mathbb{R}}^{\mathbb{N}}$ and $\omega \in \mathsf{\Omega }$ as follows:

$$W_2(t,\omega )=sech\left(\mu \sum _{j=1}^N{t}_j\right)Y_{1,2}\left(\omega \right)+tanh\left(\mu \sum _{j=1}^N{t}_j\right)Y_{2,2}\left(\omega \right),$$

(117)

where $\mu >0$ is fixed.

Since $E\left(Y_{1,1}\right)={\widehat{\mu }}_1$ and $E\left(Y_{2,1}\right)={\widehat{\mu }}_2$, the expectations of $Z_{2,1}$ are

$$\left(E{W}_2\right)\left(t\right)={\widehat{\mu }}_{1}sech\left(\mu \sum _{j=1}^N{t}_j\right)+{\widehat{\mu }}_{2}tanh\left(\mu \sum _{j=1}^N{t}_j\right),$$

(118)

Furthermore, for $j^{*}=1,\dots ,N$ we have that

$$\frac{\partial \left(E{W}_2\right)\left(t\right)}{\partial t_{j^{*}}}=-\mu \left[{\widehat{\mu }}_{1}sech\left(\mu \sum _{j=1}^N{t}_j\right)tanh\left(\mu \sum _{j=1}^N{t}_j\right)+{\widehat{\mu }}_2\left({tanh}^2\left(\mu \sum _{j=1}^N{t}_j\right)-1\right)\right]=:D_2\left(t\right)$$

(119)

Notice that

$$\frac{\partial \left(E{W}_2\right)\left(t\right)}{\partial t_{j^{*}}}=\frac{\partial \left(E{W}_2\right)\left(t\right)}{\partial t_{i^{*}}}=D_2\left(t\right)$$

for every $j^{*},i^{*}=1,\dots ,N.$

Next, we consider $\left(\mathsf{\Omega },\mathcal{F},P\right)$, where $\mathsf{\Omega }=[0,\infty )$, be a probability space, $Y_{1,3},Y_{2,3}$ be real-valued random variables of $\mathsf{\Omega }$ following Weibull distributions with scale parameters 1 and shape parameters ${\gamma }_1$ and ${\gamma }_2\in \left(0,\infty \right)$, respectively.

We consider the stochastic process $W_3(t,\omega ),$ where $t=\left(t_1,\dots ,t_N\right)\in {\mathbb{R}}^{\mathbb{N}}$ and $\omega \in \mathsf{\Omega }$ as follows:

$$W_3(t,\omega )=\frac{1}{1+exp\left(-{\ell }_1{\displaystyle \prod _{j=1}^N}{t}_j\right)}{Y}_{1,3}\left(\omega \right)+\frac{1}{1+exp\left(-{\ell }_2{\displaystyle \prod _{j=1}^N}{t}_j\right)}{Y}_{2,3}\left(\omega \right),$$

(120)

where ${\ell }_1,{\ell }_2>0$ are fixed;

Since $E\left(Y_{1,3}\right)=\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_1}\right)$ and $E\left(Y_{2,3}\right)=\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_2}\right)$, where $\mathsf{\Gamma }\left(·\right)$ is the Gamma function, The expectations of $Z_{i,3},i=1,2,3,5$, are

$$\left(E{W}_3\right)\left(t\right)=\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_1}\right)\frac{1}{1+exp\left(-{\ell }_1{\displaystyle \prod _{j=1}^N}{t}_j\right)}+\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_2}\right)\frac{1}{1+exp\left(-{\ell }_2{\displaystyle \prod _{j=1}^N}{t}_j\right)},$$

(121)

Furthermore, for $j=1,\dots ,N$ we have that

$$r\frac{\partial \left(E{W}_3\right)\left(t\right)}{\partial t_j}=\left({\displaystyle \prod _{r=1,r\ne j}^N}{t}_r\right)\left[\frac{{\ell }_1\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_1}\right)exp\left(-{\ell }_1{\displaystyle \prod _{r=1}^N}{t}_r\right)}{{\left(1+exp\left(-{\ell }_1{\displaystyle \prod _{r=1}^N}{t}_r\right)\right)}^2}+\frac{{\ell }_2\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_2}\right)exp\left(-{\ell }_2{\displaystyle \prod _{r=1}^N}{t}_r\right)}{{\left(1+exp\left(-{\ell }_2{\displaystyle \prod _{r=1}^N}{t}_r\right)\right)}^2}\right]$$

(122)

Notice that

$$\frac{\partial \left(E{W}_3\right)\left(t\right)}{\partial t_j}=\left({\displaystyle \prod _{i=1,i\ne j}^N}{t}_i\right)D_3\left(t\right)$$

for every $j=1,\dots ,N$, where

$$D_3\left(t\right)=:\frac{{\ell }_1\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_1}\right)exp\left(-{\ell }_1{\displaystyle \prod _{j=1}^N}{t}_j\right)}{{\left(1+exp\left(-{\ell }_1{\displaystyle \prod _{j=1}^N}{t}_j\right)\right)}^2}+\frac{{\ell }_2\mathsf{\Gamma }\left(1+\frac{1}{{\gamma }_2}\right)exp\left(-{\ell }_2{\displaystyle \prod _{j=1}^N}{t}_j\right)}{{\left(1+exp\left(-{\ell }_2{\displaystyle \prod _{j=1}^N}{t}_j\right)\right)}^2}$$

(123)

We present the results in the following.

Proposition 5. 

Let $t\in {\displaystyle \prod _{j=1}^N}{I}_j.$ Here, $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j},$ for every $j=1,\dots ,N$. Let also $0<\beta <1,$ $n,N\in \mathbb{N}$ and $k=1,2$. Then for $i=1,2,3$,

(i) 

$$\left|{}_k{L}_n\left(\left(E{W}_i\right),t\right)-\left(E{W}_i\right)\left(t\right)\right|\le {\gamma }_{k,N}$$

$$\left\{{\omega }_1\left(\left(E{W}_i\right),\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥\left(E{W}_i\right)∥}_{\infty }\right\}=:{\lambda }_1,$$

(124)

(ii) 

$${∥{}_k{L}_n\left(E{W}_i\right)-\left(E{W}_i\right)∥}_{\infty }\le {\lambda }_1.$$

(125)

Proof. 

From Proposition 2. □

Next, we present

Proposition 6. 

Let $0<\beta <1$, $n,N\in \mathbb{N}$ and $k=1,2$. Then for $i=1,2,3$,

(i) 

$$\left|{}_k{\overline{L}}_n\left(\left(E{W}_i\right),t\right)-\left(E{W}_i\right)\left(t\right)\right|\le {\omega }_1\left(\left(E{W}_i\right),\frac{1}{n^{\beta }}\right)+2{\alpha }_{k,n}\left(\beta \right){∥\left(E{W}_i\right)∥}_{\infty }=:{\lambda }_2,$$

(126)

(ii) 

$${∥{}_k{\overline{L}}_n\left(E{W}_i\right)-\left(E{W}_i\right)∥}_{\infty }\le {\lambda }_2.$$

(127)

Proof. 

From Proposition 3. □

Proposition 7. 

Let $t\in {\displaystyle \prod _{i=j}^N}{I}_j.$ Here, $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j}$ for $j=1,\dots ,N,t_1=\left(t_{1,1},\dots ,t_{1,N}\right),t_2=\left(t_{2,1},\dots ,t_{2,N}\right)$. Let also $0<\beta <1$, $n,m,N\in \mathbb{N}$, and $k=1,2$. Then for $i=1,2$,

(i) 

$$\left|{}_k{L}_n\left(\left(E{W}_i\right),t\right)-\left(E{W}_i\right)\left(t\right)-D_i\left(t\right){\displaystyle \sum _{j=1}^N}{}_k{L}_n\left(\left(·-t_j\right),t\right)\right|\le$$

(128)

$${\gamma }_{k,N}·\left\{\frac{N}{n^{\beta }}{\omega }_1\left(D_i\left(t\right),\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}N{∥t_2-t_1∥}_{\infty }{∥D_i\left(t\right)∥}_{\infty }\right)\right\},$$

(ii) 

Assume $D_i\left(t_0\right)=0,$ where $t_0\in \left({\displaystyle \prod _{j=1}^N}\left[t_{1,j},t_{2,j}\right]\right)$. Then,

$$\left|{}_k{L}_n\left(\left(E{W}_i\right),t_0\right)-\left(E{W}_i\right)\left(t_0\right)\right|\le {\gamma }_{k,N}·$$

(129)

$$\left\{\frac{N}{n^{\beta }}{\omega }_1\left(D_i\left(t\right),\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥t_2-t_1∥}_{\infty }{∥D_i\left(t\right)∥}_{\infty }\right)\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-2\beta }.$

Proof. 

From Proposition 4. □

Proposition 8. 

Let $t\in {\displaystyle \prod _{i=j}^N}{I}_j.$ Here $I_j=\left[t_{1,j},t_{2,j}\right],$ where $t_{1,j},t_{2,j}\in \mathbb{R}$ with $t_{1,j}<t_{2,j}$ for $j=1,\dots ,N,t_1=\left(t_{1,1},\dots ,t_{1,N}\right),t_2=\left(t_{2,1},\dots ,t_{2,N}\right)$. Let also $0<\beta <1$, $n,m,N\in \mathbb{N}$, and $k=1,2$. Then

(i) 

$$\left|{}_k{L}_n\left(\left(E{W}_3\right),t\right)-\left(E{W}_3\right)\left(t\right)-D_3\left(t\right){\displaystyle \sum _{j=1}^N}\left({\displaystyle \prod _{i=1,i\ne j}^N}{t}_i\right){}_k{L}_n\left(\left(·-t_j\right),t\right)\right|\le$$

$${\gamma }_{k,N}·\left\{\frac{N}{n^{\beta }}\underset{j\in \{1,\cdots ,N\}}{max}{\omega }_1\left(D_3\left(t\right){\displaystyle \prod _{i=1,i\ne j}^N}{t}_i,\frac{1}{n^{\beta }}\right)+\right.$$

(130)

$$\left.\left(2{\alpha }_{k,n}N{∥t_2-t_1∥}_{\infty }\underset{j\in \{1,\cdots ,N\}}{max}{∥D_3\left(t\right){\displaystyle \prod _{i=1,i\ne j}^N}{t}_i∥}_{\infty }\right)\right\},$$

(ii) 

Assume that there exist $t_0=\left(t_{0,1},t_{0,2},\dots ,t_{0,N}\right)$, where $t_0\in \left({\displaystyle \prod _{j=1}^N}\left[t_{1,j},t_{2,j}\right]\right)$, such that $t_{0,k}=t_{0,l}=0,$ for $k,l\in \left\{1,\dots ,N\right\},$ with $k\ne l$. Then,

$$\left|{}_k{L}_n\left(\left(E{W}_3\right),t_0\right)-\left(E{W}_3\right)\left(t_0\right)\right|\le {\gamma }_{k,N}·$$

(131)

$$\left\{\frac{N}{n^{\beta }}\underset{j\in \{1,\dots ,N\}}{max}{\omega }_1\left(D_3\left(t\right){\displaystyle \prod _{i=1,i\ne j}^N}{t}_j,\frac{1}{n^{\beta }}\right)+\left(2{\alpha }_{k,n}\left(\beta \right)N{∥t_2-t_1∥}_{\infty }\underset{j\in \{1,\dots ,N\}}{max}{∥\frac{\partial \left(E{W}_3\right)}{\partial t_j}∥}_{\infty }\right)\right\},$$

Notice in the latter the extremely high rate of convergence at $n^{-2\beta }.$

Proof. 

From Proposition 4. □

For the latest developments in deep learning see [19].

For work concerning neural network applications in biology, environmental science, and demography, see [20,21].