Quantitative Uniform Approximation by Activated Singular Operators

Abstract

In this article, we study the approximation properties of activated singular integral operators over the real line. We establish their convergence to the unit operator with rates. The kernels here derive from neural network activation functions and their corresponding density functions. The estimates are mostly sharp, and they are pointwise and uniform. The derived inequalities involve the higher order modulus of smoothness.

1. Introduction

The rate of convergence of singular integrals has been studied earlier in [1,2,3,4]. The seminal monograph [5], Ch. 14, is motivated by it and it is used in this work. Here, we consider some activated singular integral operators over $\mathbb{R}$ and we study the degree of approximation to the unit operator with rates over smooth functions. We establish related inequalities involving the higher modulus of smoothness with respect to ${∥·∥}_{\infty }$. The estimates are pointwise and uniform. Most of the time these are optimal, in the sense that the inequalities are attained by basic functions. Our discussed operators are not, in general, positive.

The new thing here is the reverse process from applied mathematics to pure. Our kernels here are formed by density functions from activation functions related to neural networks approximation; see [6,7].

For the history of the topic, we mention the monograph [5] of 2012, which was the first complete source to deal exclusively with the classic theory of the approximation of singular integrals to the identity-unit operator. In it, the authors studied quantitatively the basic approximation properties of the general Picard, Gauss–Weierstrass and Poisson–Cauchy singular integral operators over the real line, which are not positive linear operators. In particular, they studied the rate of convergence of these operators to the unit operator, as well as the related simultaneous approximation. This is given via inequalities and with the use of the higher-order modulus of the smoothness of the high-order derivative of the involved function. Some of these inequalities are proven to be sharp. Also, they studied the global smoothness preservation property of these operators. Furthermore, they gave asymptotic expansions of Voronovskaya type for the error of approximation. They continued with the study of related properties of the general fractional Gauss–Weierstrass and Poisson–Cauchy singular integral operators. These properties were studied with respect to the $L_p$ norm, $1\le p\le \infty$. The case of Lipschitz-type function approximation was studied separately and in detail. Furthermore, they presented the corresponding general approximation theory of general singular integral operators with many applications for the trigonometric singular integral, which had not received much focus until then. Of great interest, and motivating the author, are the articles [8,9,10,11,12]. In contemporary intense mathematical activity with the use of neural networks in solving differential equations, our current work is expected to play a pivotal role, as in the classic case using the earlier versions of singular integrals.

2. Background

Everything in this section comes from [5], Ch. 14. Here, we studied the following smooth general singular integral operators ${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)$ defined as follows. Let $\xi >0$ and let ${\mu }_{\xi }$ be Borel probability measures on $\mathbb{R}$.

For $r\in \mathbb{N}$ and $n\in {\mathbb{Z}}^{+}$, we set

$${\alpha }_j:=\left\{\begin{array}{c}{\left(-1\right)}^{r-j}\left(\begin{array}{c}r\\ j\end{array}\right)j^{-n},j=1,\dots ,r.\hfill \\ 1-{\displaystyle \sum _{j=1}^r}{\left(-1\right)}^{r-j}\left(\begin{array}{c}r\\ j\end{array}\right)j^{-n},j=0,\hfill \end{array}\right.$$

(1)

that is, ${\displaystyle \sum _{j=0}^r}{\alpha }_j=1.$ Let $f:\mathbb{R}\to \mathbb{R}$ be Borel-measurable, and we can define for $x\in \mathbb{R}$ the integral

$${\mathsf{\Theta }}_{r,\xi }\left(f,x\right):={\int }_{-\infty }^{\infty }\left(\sum _{j=0}^r{\alpha }_{j}f\left(x+jt\right)\right)d{\mu }_{\xi }\left(t\right).$$

(2)

We suppose that. ${\mathsf{\Theta }}_{r,\xi }\left(f;x\right)\in \mathbb{R}$ for all $x\in \mathbb{R}$. We also say that

$${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)=\sum _{j=0}^r{\alpha }_j\left({\int }_{-\infty }^{\infty }f\left(x+jt\right)d{\mu }_{\xi }\left(t\right)\right).$$

(3)

We notice that ${\mathsf{\Theta }}_{r,\xi }\left(c,x\right)=c$, c constant, and

$${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)-f\left(x\right)=\sum _{j=0}^r{\alpha }_j\left({\int }_{-\infty }^{\infty }f\left(x+jt\right)-f\left(x\right)\right)d{\mu }_{\xi }\left(t\right).$$

(4)

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$ with the rth modulus of smoothness be finite, i.e.,

$${\omega }_r\left(f^{\left(n\right)},h\right):=\underset{\left|t\right|\le h}{sup}{∥{\mathsf{\Delta }}_t^r{f}^{\left(n\right)}\left(x\right)∥}_{\infty ,x}<\infty ,h>0,$$

(5)

where

$${\mathsf{\Delta }}_t^r{f}^{\left(n\right)}\left(x\right):=\sum _{j=0}^r{\left(-1\right)}^{r-j}\left(\begin{array}{c}r\\ j\end{array}\right)f^{\left(n\right)}\left(x+jt\right),$$

(6)

see [13], p. 44.

We need to introduce

$${\delta }_k:=\sum _{j=0}^r{\alpha }_j{j}^k,k=1,\dots ,n\in \mathbb{N},$$

(7)

and the even function

$${\tilde{G}}_n\left(t\right):={\int }_0^{\left|t\right|}\frac{{\left(\left|t\right|-w\right)}^{n-1}}{\left(n-1\right)!}{\omega }_r\left(f^{\left(n\right)},w\right)dw,n\in \mathbb{N}$$

(8)

with

$${\tilde{G}}_0\left(t\right):={\omega }_r\left(f,\left|t\right|\right),t\in \mathbb{R}.$$

(9)

We mention the following result.

Theorem 1.

The integrals $c_{k,\xi }:={\int }_{-\infty }^{\infty }{t}^{k}d{\mu }_{\xi }\left(t\right)$, $k=1,\dots ,n$ are assumed to be finite. Then,

$$\left|{\mathsf{\Theta }}_{r,\xi }\left(f;x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{c}_{k,\xi }\right|\le {\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)d{\mu }_{\xi }\left(t\right).$$

(10)

Corollary 1.

Assume ${\omega }_r\left(f,\xi \right)<\infty$, $\xi >0$. Then, it holds for $n=0$ that

$$\left|{\mathsf{\Theta }}_{r,\xi }\left(f;x\right)-f\left(x\right)\right|\le {\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)d{\mu }_{\xi }\left(t\right).$$

(11)

Inequality (10) is sharp.

Theorem 2.

Inequality (10) at $x=0$ is attained by $f\left(x\right)=x^{r+n}$, $r,n\in \mathbb{N}$ with $r+n$ even.

Corollary 2.

Inequality (11) is sharp, that is, it is attained at $x=0$ by $f\left(x\right)=x^r$, r even.

Remark 1.

Regarding inequalities (10) and (11), we have the uniform estimates

$${∥{\mathsf{\Theta }}_{r,\xi }\left(f;x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{c}_{k,\xi }∥}_{\infty ,x}\le {\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)d{\mu }_{\xi }\left(t\right),n\in \mathbb{N},$$

(12)

and

$${∥{\mathsf{\Theta }}_{r,\xi }\left(f;x\right)-f\left(x\right)∥}_{\infty ,x}\le {\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)d{\mu }_{\xi }\left(t\right),n=0.$$

(13)

We also need the next result.

Theorem 3.

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$. Set $c_{k,\xi }:={\int }_{-\infty }^{\infty }{t}^{k}d{\mu }_{\xi }\left(t\right)$, $k=1,\dots ,n.$ Assume also that ${\omega }_r\left(f^{\left(n\right)},h\right)<\infty$, ∀$h>0$. It is also supposed that

$${\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^{r}d{\mu }_{\xi }\left(t\right)<\infty .$$

(14)

Then,

$${∥{\mathsf{\Theta }}_{r,\xi }\left(f;x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{c}_{k,\xi }∥}_{\infty ,x}\le$$

$$\frac{{\omega }_r\left(f^{\left(n\right)},\xi \right)}{n!}{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^{r}d{\mu }_{\xi }\left(t\right).$$

(15)

When $n=0$, the sum on the L.H.S. (15) collapses.

Here, L.H.S. means left-hand side.

3. On Activation Functions

3.1. About Richards’s Curve

Here, we follow [7], Chapter 1.

A Richards’s curve is

$$\phi \left(x\right)=\frac{1}{1+e^{-\mu x}};x\in \mathbb{R},\mu >0,$$

(16)

which is strictly increasing on $\mathbb{R}$, and it is a sigmoid function; in particular, this is a generalized logistic function. And it is an activation function in neural networks; see [7], Chapter 1.

It is

$$\underset{x\to +\infty }{lim}\phi \left(x\right)=1\mathrm{and}\underset{x\to -\infty }{lim}\phi \left(x\right)=0.$$

(17)

We consider the function

$$G\left(x\right)=\frac{1}{2}\left(\phi \left(x+1\right)-\phi \left(x-1\right)\right),x\in \mathbb{R},$$

(18)

which is $G\left(x\right)>0$, where all $x\in \mathbb{R}$.

It is

$$\phi \left(0\right)=\frac{1}{2},\phi \left(x\right)=1-\phi \left(-x\right),$$

(19)

and

$$G\left(x\right)=G\left(-x\right),\forall x\in \mathbb{R}.$$

(20)

We also have

$$G\left(0\right)=\frac{e^{\mu }-1}{2\left(e^{\mu }+1\right)}.$$

(21)

We also obtain

$$\underset{x\to +\infty }{lim}G\left(x\right)=\underset{x\to -\infty }{lim}G\left(x\right)=0,$$

(22)

and G is a bell symmetric function with maximum

$$G\left(0\right)=\frac{e^{\mu }-1}{2(e^{\mu }+1)}.$$

(23)

Theorem 4.

It holds that

$$\sum _{i=-\infty }^{\infty }G\left(x-i\right)=1,\forall x\in \mathbb{R}.$$

(24)

Theorem 5.

It holds that

$${\int }_{-\infty }^{\infty }G\left(x\right)dx=1.$$

(25)

Therefore, G is a density function.

Remark 2.

Therefore, we have

$$G\left(x\right)=\frac{1}{2}\left(\phi \left(x+1\right)-\phi \left(x-1\right)\right),\forall x\in \mathbb{R}.$$

(26)

(i)

Let $x\ge 1$. That is, $0\le x-1<x+1$. Applying the mean value theorem, we obtain:

$$G\left(x\right)=\frac{1}{2}2{\phi }^{\prime }\left(\eta \right)={\phi }^{\prime }\left(\eta \right)=\frac{\mu e^{-\mu \eta }}{{\left(1+e^{-\mu \eta }\right)}^2},\mu >0,$$

(27)

where $0\le x-1<\eta <x+1.$

 

  Notice that

$$G\left(x\right)<\mu e^{-\mu \eta }<\mu e^{-\mu \left(x-1\right)},\forall x\ge 1.$$

(28)

(ii)

Now, let $x\le -1$. That is, $x-1<x+1\le 0$. Applying the mean value theorem again, we obtain:

$$G\left(x\right)=\frac{1}{2}2{\phi }^{\prime }\left(\eta \right)={\phi }^{\prime }\left(\eta \right)=\frac{\mu e^{-\mu \eta }}{{\left(1+e^{-\mu \eta }\right)}^2},$$

(29)

where $x-1<\eta <x+1\le 0.$

  Hence, we derive that

$$G\left(x\right)<\mu e^{-\mu \eta }<\mu e^{-\mu \left(x-1\right)},\forall x\le -1.$$

(30)

Consequently, we proved that

$$G\left(x\right)<\mu e^{-\mu \left(x-1\right)},\forall x\in (-\infty ,-1]\cup [1,+\infty )=\mathbb{R}-\left(-1,1\right).$$

(31)

Let $0<\xi \le 1$; it holds that

$$G\left(\frac{x}{\xi }\right)<\mu e^{-\mu \left(\frac{x}{\xi }-1\right)},\forall x\ge \xi ,\mathrm{or}\forall x\le -\xi .$$

(32)

Clearly, according to Theorem 5, we have

$$\frac{1}{\xi }{\int }_{-\infty }^{\infty }G\left(\frac{x}{\xi }\right)dx=1.$$

(33)

Therefore, $\frac{1}{\xi }G\left(\frac{x}{\xi }\right)$ is a density function, and let $d{\mu }_{\xi }\left(x\right):=\frac{1}{\xi }G\left(\frac{x}{\xi }\right)dx$; that is, ${\mu }_{\xi }$ is a Borel probability measure.

We give the following important result.

Theorem 6.

Let $0<\xi \le 1$, and

$$c_{k,\xi }^{*}:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^{k}G\left(\frac{x}{\xi }\right)dx,k=1,\dots ,n\in \mathbb{N}.$$

(34)

Then, $c_{k,\xi }^{*}$ are finite and $c_{k,\xi }^{*}\to 0,$ as $\xi \to 0.$

Proof. 

We can write

$$c_{k,\xi }^{*}=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^{k}G\left(\frac{x}{\xi }\right)dx=$$

$$\frac{1}{\xi }\left[{\int }_{-\infty }^{-\xi }{x}^{k}G\left(\frac{x}{\xi }\right)dx+{\int }_{-\xi }^{\xi }{x}^{k}G\left(\frac{x}{\xi }\right)dx+{\int }_{\xi }^{\infty }{x}^{k}G\left(\frac{x}{\xi }\right)dx\right].$$

(35)

We notice that

$$\left|{\int }_{-\xi }^{\xi }{x}^{k}G\left(\frac{x}{\xi }\right)dx\right|\le {\int }_{-\xi }^{\xi }{\left|x\right|}^{k}G\left(\frac{x}{\xi }\right)dx\le$$

(36)

$${\xi }^k{\int }_{-\xi }^{\xi }G\left(\frac{x}{\xi }\right)dx\le {\xi }^{k+1}{\int }_{-\infty }^{\infty }G\left(\frac{x}{\xi }\right)d\frac{x}{\xi }={\xi }^{k+1}<\infty .$$

Next, we have

$$\left|{\int }_{\xi }^{\infty }{x}^{k}G\left(\frac{x}{\xi }\right)dx\right|\le {\int }_{\xi }^{\infty }{\left|x\right|}^{k}G\left(\frac{x}{\xi }\right)dx={\int }_{\xi }^{\infty }{x}^{k}G\left(\frac{x}{\xi }\right)dx=$$

$${\xi }^{k+1}{\int }_{\xi }^{\infty }{\left(\frac{x}{\xi }\right)}^{k}G\left(\frac{x}{\xi }\right)d\left(\frac{x}{\xi }\right)$$

$$(\xi \le x<\infty \iff 1\le \frac{x}{\xi }=:y<\infty )$$

(37)

$$={\xi }^{k+1}{\int }_1^{\infty }{y}^{k}G\left(y\right)dy\stackrel{\left(28\right)}{\le }\mu {\xi }^{k+1}{\int }_1^{\infty }{y}^k{e}^{-\mu \left(y-1\right)}dy$$

and we have the gamma function $\mathsf{\Gamma }\left(z\right)={\int }_0^{\infty }{t}^{z-1}{e}^{-t}dt$, $z>0$.

$$\le \mu {\xi }^{k+1}{e}^{\mu }{\int }_0^{\infty }{y}^k{e}^{-\mu y}dy={\xi }^{k+1}{e}^{\mu }\frac{\mu }{{\mu }^{k+1}}{\int }_0^{\infty }{\left(\mu y\right)}^k{e}^{-\mu y}d\left(\mu y\right)=$$

$${\mu }^{-k}{e}^{\mu }{\xi }^{k+1}{\int }_0^{\infty }{z}^k{e}^{-z}dz={\mu }^{-k}{e}^{\mu }{\xi }^{k+1}\mathsf{\Gamma }\left(k+1\right).$$

(38)

We have established that

$$\left|{\int }_{\xi }^{\infty }{x}^{k}G\left(\frac{x}{\xi }\right)dx\right|\le {\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}<\infty .$$

(39)

Finally, we observe that

$$\left|{\int }_{-\infty }^{-\xi }{x}^{k}G\left(\frac{x}{\xi }\right)dx\right|={\xi }^{k+1}\left|{\int }_{-\infty }^{-\xi }{\left(\frac{x}{\xi }\right)}^{k}G\left(\frac{x}{\xi }\right)d\frac{x}{\xi }\right|$$

$$\left(-\infty <x\le -\xi \iff -\infty <\frac{x}{\xi }\le -1\right)$$

(40)

$$={\xi }^{k+1}\left|{\int }_{-\infty }^{-1}{z}^{k}G\left(z\right)dz\right|\le {\xi }^{k+1}{\int }_{-\infty }^{-1}{\left|z\right|}^{k}G\left(z\right)dz=$$

$${\xi }^{k+1}\left[-{\int }_{-\infty }^{-1}{\left|-z\right|}^{k}G\left(-z\right)d\left(-z\right)\right]$$

$$\left(-\infty <z\le -1\iff \infty >-z=:y\ge 1\right)$$

$$={\xi }^{k+1}\left[-{\int }_{\infty }^1{\left|y\right|}^{k}G\left(y\right)dy\right]=$$

(41)

$${\xi }^{k+1}\left[{\int }_1^{\infty }{y}^{k}G\left(y\right)dy\right]\le {\xi }^{k+1}\mu \left({\int }_1^{\infty }{y}^k{e}^{-\mu \left(y-1\right)}dy\right)\le$$

$${\xi }^{k+1}\mu \left({\int }_0^{\infty }{y}^k{e}^{-\mu \left(y-1\right)}dy\right)={\xi }^{k+1}\mu e^{\mu }\left({\int }_0^{\infty }{y}^k{e}^{-\mu y}dy\right)=$$

(42)

$${\xi }^{k+1}{e}^{\mu }{\mu }^{-k}\left({\int }_0^{\infty }{\left(\mu y\right)}^k{e}^{-\mu y}d\left(\mu y\right)\right)=$$

$${\xi }^{k+1}{e}^{\mu }{\mu }^{-k}\left({\int }_0^{\infty }{z}^k{e}^{-z}dz\right)={\xi }^{k+1}{e}^{\mu }{\mu }^{-k}\mathsf{\Gamma }\left(k+1\right).$$

Therefore, we have proved that

$$\left|{\int }_{-\infty }^{-\xi }{x}^{k}G\left(\frac{x}{\xi }\right)dx\right|\le {\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}<\infty .$$

(43)

According to (36), (39) and (43), we derive

$$\left|c_{k,\xi }^{*}\right|\le \frac{1}{\xi }\left[{\xi }^{k+1}+2{\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}\right]=$$

$$\left[{\xi }^k+2{\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^k\right]=\left[1+2{\mu }^{-k}{e}^{\mu }k!\right]{\xi }^k<\infty ,$$

(44)

and $c_{k,\xi }^{*}\to 0$, as $\xi \to 0$; $k=1,\dots ,n.$ □

Next, we give the following.

Theorem 7.

It holds that

$${\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^{r}d{\mu }_{\xi }\left(t\right)<\infty ;r,n\in \mathbb{N},$$

(45)

for

$$d{\mu }_{\xi }\left(x\right)=\frac{1}{\xi }G\left(\frac{x}{\xi }\right)dx,0<\xi \le 1.$$

Also, this integral converges to zero, as $\xi \to 0.$

Proof. 

We observe that

$${\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^{r}d{\mu }_{\xi }\left(t\right)\le 2^{r-1}{\int }_{-\infty }^{\infty }{\left|t\right|}^n\left(1+\frac{{\left|t\right|}^r}{{\xi }^r}\right)d{\mu }_{\xi }\left(t\right)=$$

(46)

$$2^{r-1}\left[{\int }_{-\infty }^{\infty }{\left|t\right|}^{n}d{\mu }_{\xi }\left(t\right)+\frac{1}{{\xi }^r}{\int }_{-\infty }^{\infty }{\left|t\right|}^{n+r}d{\mu }_{\xi }\left(t\right)\right].$$

Therefore, we have

$$\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|x\right|}^n{\left(1+\frac{\left|x\right|}{\xi }\right)}^{r}G\left(\frac{x}{\xi }\right)dx\le$$

$$2^{r-1}\left[\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|x\right|}^{n}G\left(\frac{x}{\xi }\right)dx+\frac{1}{{\xi }^r}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|x\right|}^{n+r}G\left(\frac{x}{\xi }\right)dx\right]\stackrel{\left(44\right)}{\le }$$

(47)

$$2^{r-1}\left[\left(1+2{\mu }^{-n}{e}^{\mu }n!\right){\xi }^n+\frac{1}{{\xi }^r}\left(1+2{\mu }^{-\left(n+r\right)}{e}^{\mu }\left(n+r\right)!\right){\xi }^{n+r}\right]=$$

(48)

$$2^{r-1}\left[\left(1+2{\mu }^{-n}{e}^{\mu }n!\right)+\left(1+2{\mu }^{-\left(n+r\right)}{e}^{\mu }\left(n+r\right)!\right)\right]{\xi }^n<\infty ,$$

and it converges to zero, as $\xi \to 0.$ □

3.2. About the q-Deformed and $\lambda$-Parametrized Hyperbolic Tangent Function $g_{q,\lambda }$

We consider the activation function $g_{q,\lambda }$ and study its related properties; all the basics come from [7], Ch. 17.

Let the activation function

$$g_{q,\lambda }\left(x\right)=\frac{e^{\lambda x}-q{e}^{-\lambda x}}{e^{\lambda x}+q{e}^{-\lambda x}},\lambda ,q>0,x\in \mathbb{R}.$$

(49)

It is

$$g_{q,\lambda }\left(0\right)=\frac{1-q}{1+q},$$

and

$$g_{q,\lambda }\left(-x\right)=-g_{\frac{1}{q},\lambda }\left(x\right),\forall x\in \mathbb{R},$$

(50)

with

$$g_{q,\lambda }\left(+\infty \right)=1,g_{q,\lambda }\left(-\infty \right)=-1.$$

We consider the function

$$M_{q,\lambda }\left(x\right):=\frac{1}{4}\left(g_{q,\lambda }\left(x+1\right)-g_{q,\lambda }\left(x-1\right)\right)>0,$$

(51)

∀$x\in \mathbb{R}$, $q,\lambda >0$. We have $M_{q,\lambda }\left(\pm \infty \right)=0$, so that the x-axis is a horizontal asymptote.

It holds that

$$M_{q,\lambda }\left(-x\right)=M_{\frac{1}{q},\lambda }\left(x\right),\forall x\in \mathbb{R},q,\lambda >0,$$

(52)

and

$$M_{\frac{1}{q},\lambda }\left(-x\right)=M_{q,\lambda }\left(x\right),\forall x\in \mathbb{R}.$$

The $M_{q,\lambda }$ maximum is

$$M_{q,\lambda }\left(\frac{lnq}{2\lambda }\right)=\frac{tanh\left(\lambda \right)}{2},\lambda >0.$$

(53)

Theorem 8.

We find that

$$\sum _{i=-\infty }^{\infty }{M}_{q,\lambda }\left(x-i\right)=1,\forall x\in \mathbb{R},\forall \lambda ,q>0.$$

(54)

Theorem 9.

It holds that

$${\int }_{-\infty }^{\infty }{M}_{q,\lambda }\left(x\right)dx=1,\lambda ,q>0.$$

(55)

Therefore, $M_{q,\lambda }$ is a density function on $\mathbb{R};$ $\lambda ,q>0$.

Remark 3.

(i) Let $x\ge 1$. That is, $0\le x-1<x+1$. According to mean value theorem, we obtain

$$M_{q,\lambda }\left(x\right)=\frac{1}{4}\left[g_{q,\lambda }\left(x+1\right)-g_{q,\lambda }\left(x-1\right)\right]=\frac{1}{4}·2·\frac{4q\lambda e^{2\lambda \xi }}{{\left(e^{2\lambda \xi }+q\right)}^2}=\frac{2q\lambda e^{2\lambda \xi }}{{\left(e^{2\lambda \xi }+q\right)}^2},$$

(56)

for some $0\le x-1<\xi <x+1;$ $\lambda ,q>0.$

But $e^{2\lambda \xi }<e^{2\lambda \xi }+q$, and

$$M_{q,\lambda }\left(x\right)<\frac{2q\lambda \left(e^{2\lambda \xi }+q\right)}{{\left(e^{2\lambda \xi }+q\right)}^2}=\frac{2q\lambda }{\left(e^{2\lambda \xi }+q\right)}<\frac{2q\lambda }{\left(e^{2\lambda \left(x-1\right)}+q\right)}<\frac{2q\lambda }{e^{2\lambda \left(x-1\right)}},$$

(57)

$x\ge 1.$

That is,

$$M_{q,\lambda }\left(x\right)<2q\lambda e^{-2\lambda \left(x-1\right)},\forall x\ge 1.$$

(58)

Set $\mu :=2\lambda$; then,

$$M_{q,\lambda }\left(x\right)<q\mu e^{-\mu \left(x-1\right)},\forall x\ge 1.$$

(59)

(ii) Now, let $x\le -1$. That is, $x-1<x+1\le 0$. Again, we have

$$M_{q,\lambda }\left(x\right)<\frac{2q\lambda }{\left(e^{2\lambda \xi }+q\right)},$$

(60)

$x-1<\xi <x+1\le 0;$ $\lambda ,q>0.$

We have

$$e^{2\lambda \left(x-1\right)}<e^{2\lambda \xi }<e^{2\lambda \left(x+1\right)},$$

and

$$e^{2\lambda \left(x-1\right)}+q<e^{2\lambda \xi }+q<e^{2\lambda \left(x+1\right)}+q.$$

(61)

Hence,

$$\frac{1}{e^{2\lambda \xi }+q}<\frac{1}{e^{2\lambda \left(x-1\right)}+q}.$$

(62)

Therefore, it holds that

$$M_{q,\lambda }\left(x\right)<\frac{2q\lambda }{e^{2\lambda \left(x-1\right)}+q}<\frac{2q\lambda }{e^{2\lambda \left(x-1\right)}},x\le -1.$$

(63)

That is,

$$M_{q,\lambda }\left(x\right)<2q\lambda e^{-2\lambda \left(x-1\right)},\forall x\le -1.$$

(64)

Set $\mu :=2\lambda$; then,

$$M_{q,\lambda }\left(x\right)<q\mu e^{-\mu \left(x-1\right)},\forall x\le -1.$$

(65)

We have proved that

$$M_{q,\lambda }\left(x\right)<q\mu e^{-\mu \left(x-1\right)},$$

(66)

∀$x\in (-\infty ,-1]\cup [1,+\infty )=\mathbb{R}-\left(-1,1\right).$

Let $0<\xi \le 1$; it holds that

$$M_{q,\lambda }\left(\frac{x}{\xi }\right)<q\mu e^{-\mu \left(\frac{x}{\xi }-1\right)},\forall x\ge \xi ,\mathit{or}\forall x\le -\xi .$$

(67)

According to Theorem 9, we find that

$$\frac{1}{\xi }{\int }_{-\infty }^{\infty }{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx=1.$$

(68)

Therefore, $\frac{1}{\xi }{M}_{q,\lambda }\left(\frac{x}{\xi }\right)$ is a density function and let

$$d{\mu }_{\xi }\left(x\right):=\frac{1}{\xi }{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx,$$

(69)

that is, ${\mu }_{\xi }$ is a Borel probability measure.

We give the following.

Theorem 10.

Let

$${\overline{c}}_{k,\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx,k=1,\dots ,n\in \mathbb{N}.$$

(70)

Then, ${\overline{c}}_{k,\xi }$ are finite and ${\overline{c}}_{k,\xi }\to 0,$ as $\xi \to 0.$

Proof. 

We can write

$${\overline{c}}_{k,\xi }=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx=$$

$$\frac{1}{\xi }\left[{\int }_{-\infty }^{-\xi }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx+{\int }_{-\xi }^{\xi }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx+{\int }_{\xi }^{\infty }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\right].$$

(71)

We notice that

$$\left|{\int }_{-\xi }^{\xi }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\right|\le {\int }_{-\xi }^{\xi }{\left|x\right|}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\le$$

$${\xi }^k{\int }_{-\xi }^{\xi }{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\le {\xi }^{k+1}{\int }_{-\infty }^{\infty }{M}_{q,\lambda }\left(\frac{x}{\xi }\right)d\frac{x}{\xi }={\xi }^{k+1}<\infty .$$

(72)

Next, we have

$$\left|{\int }_{\xi }^{\infty }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\right|\le {\int }_{\xi }^{\infty }{\left|x\right|}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx={\int }_{\xi }^{\infty }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx=$$

(73)

$${\xi }^{k+1}{\int }_{\xi }^{\infty }{\left(\frac{x}{\xi }\right)}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)d\left(\frac{x}{\xi }\right)$$

$$(\xi \le x<\infty \iff 1\le \frac{x}{\xi }=:y<\infty )$$

$$={\xi }^{k+1}{\int }_1^{\infty }{y}^k{M}_{q,\lambda }\left(y\right)dy\le q\mu {\xi }^{k+1}{\int }_1^{\infty }{y}^k{e}^{-\mu \left(y-1\right)}dy$$

$$\le q\mu {\xi }^{k+1}{e}^{\mu }{\int }_0^{\infty }{y}^k{e}^{-\mu y}dy=q{\xi }^{k+1}{e}^{\mu }{\mu }^{-k}{\int }_0^{\infty }{\left(\mu y\right)}^k{e}^{-\mu y}d\left(\mu y\right)=$$

(74)

$$q{\mu }^{-k}{e}^{\mu }{\xi }^{k+1}{\int }_0^{\infty }{z}^k{e}^{-z}dz=q{\mu }^{-k}{e}^{\mu }{\xi }^{k+1}\mathsf{\Gamma }\left(k+1\right).$$

We have established that

$$\left|{\int }_{\xi }^{\infty }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\right|\le q{\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}<\infty .$$

(75)

Finally, we observe that

$$\left|{\int }_{-\infty }^{-\xi }{x}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)dx\right|={\xi }^{k+1}\left|{\int }_{-\infty }^{-\xi }{\left(\frac{x}{\xi }\right)}^k{M}_{q,\lambda }\left(\frac{x}{\xi }\right)d\frac{x}{\xi }\right|$$

$$\left(-\infty <x\le -\xi \iff -\infty <\frac{x}{\xi }\le -1\right)$$

(76)

$$={\xi }^{k+1}\left|{\int }_{-\infty }^{-1}{z}^k{M}_{q,\lambda }\left(z\right)dz\right|\le {\xi }^{k+1}{\int }_{-\infty }^{-1}{\left|z\right|}^k{M}_{q,\lambda }\left(z\right)dz\stackrel{\left(52\right)}{=}$$

$${\xi }^{k+1}\left[-{\int }_{-\infty }^{-1}{\left|-z\right|}^k{M}_{\frac{1}{q},\lambda }\left(-z\right)d\left(-z\right)\right]$$

$$\left(-\infty <z\le -1\iff \infty >-z=:y\ge 1\right)$$

$$={\xi }^{k+1}\left[-{\int }_{\infty }^1{\left|y\right|}^k{M}_{\frac{1}{q},\lambda }\left(y\right)dy\right]=$$

$${\xi }^{k+1}\left[{\int }_1^{\infty }{y}^k{M}_{\frac{1}{q},\lambda }\left(y\right)dy\right]\stackrel{\left(59\right)}{\le }\frac{1}{q}{\xi }^{k+1}\mu \left({\int }_1^{\infty }{y}^k{e}^{-\mu \left(y-1\right)}dy\right)\le$$

$$\frac{1}{q}{\xi }^{k+1}\mu \left({\int }_0^{\infty }{y}^k{e}^{-\mu \left(y-1\right)}dy\right)=\frac{1}{q}{\xi }^{k+1}\mu e^{\mu }\left({\int }_0^{\infty }{y}^k{e}^{-\mu y}dy\right)=$$

(77)

$$\frac{1}{q}{\xi }^{k+1}{e}^{\mu }{\mu }^{-k}\left({\int }_0^{\infty }{\left(\mu y\right)}^k{e}^{-\mu y}d\left(\mu y\right)\right)=$$

$${\xi }^{k+1}{e}^{\mu }{\mu }^{-k}\frac{1}{q}\left({\int }_0^{\infty }{z}^k{e}^{-z}dz\right)=\frac{1}{q}{\xi }^{k+1}{e}^{\mu }{\mu }^{-k}\mathsf{\Gamma }\left(k+1\right).$$

Therefore, we have proved that

$$\left|{\int }_{-\infty }^{-\xi }{x}^k{M}_{\frac{1}{q},\lambda }\left(\frac{x}{\xi }\right)dx\right|\le \frac{1}{q}{\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}<\infty .$$

(78)

According to (72), (75) and (78), we obtain

$$\left|{\overline{c}}_{k,\xi }\right|\le \frac{1}{\xi }\left[{\xi }^{k+1}+q{\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}+\frac{1}{q}{\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^{k+1}\right]=$$

$$\left[{\xi }^k+\left(q+\frac{1}{q}\right){\mu }^{-k}{e}^{\mu }\mathsf{\Gamma }\left(k+1\right){\xi }^k\right]=\left[1+\left(q+\frac{1}{q}\right){\mu }^{-k}{e}^{\mu }k!\right]{\xi }^k<\infty ,$$

(79)

and ${\overline{c}}_{k,\xi }\to 0$, as $\xi \to 0$; $k=1,\dots ,n.$ □

Furthermore, we present the results.

Theorem 11.

It holds that

$$I_{M,\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{M}_{q,\lambda }\left(\frac{t}{\xi }\right)dt<\infty ;$$

(80)

where $\lambda ,q>0;$ $r,n\in \mathbb{N}$; $0<\xi \le 1.$

Also, $I_{M,\xi }\to 0$, as $\xi \to 0.$

Proof. 

We find that

$$\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{M}_{q,\lambda }\left(\frac{t}{\xi }\right)dt\le$$

(81)

$$\frac{2^{r-1}}{\xi }\left[{\int }_{-\infty }^{\infty }{\left|t\right|}^n\left(1+\frac{{\left|t\right|}^r}{{\xi }^r}\right)M_{q,\lambda }\left(\frac{t}{\xi }\right)dt\right]=$$

$$2^{r-1}\left[\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{M}_{q,\lambda }\left(\frac{t}{\xi }\right)dt+\frac{1}{{\xi }^r}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^{n+r}{M}_{q,\lambda }\left(\frac{t}{\xi }\right)dt\right]\stackrel{\left(79\right)}{\le }$$

$$2^{r-1}\left[\left[1+\left(q+\frac{1}{q}\right){\mu }^{-n}{e}^{\mu }n!\right]{\xi }^n+\frac{1}{{\xi }^r}\left[1+\left(q+\frac{1}{q}\right){\mu }^{-\left(n+r\right)}{e}^{\mu }\left(n+r\right)!\right]{\xi }^{n+r}\right]$$

(82)

$$=2^{r-1}\left[\left[1+\left(q+\frac{1}{q}\right){\mu }^{-n}{e}^{\mu }n!\right]+\left[1+\left(q+\frac{1}{q}\right){\mu }^{-\left(n+r\right)}{e}^{\mu }\left(n+r\right)!\right]\right]{\xi }^n<\infty .$$

and it converges to zero, as $\xi \to 0.$ □

3.3. About the Gudermannian Generated Activation Function

Here, we follow [6], Ch. 2.

Let the related normalized generator sigmoid function be

$$f\left(x\right):=\frac{8}{\pi }{\int }_0^x\frac{1}{e^t+e^{-t}}dt,x\in \mathbb{R},$$

(83)

and the neural network activation function be

$$\psi \left(x\right):=\frac{1}{4}\left(f\left(x+1\right)-f\left(x-1\right)\right)>0,x\in \mathbb{R}.$$

(84)

We mention the following.

Theorem 12.

It holds that

$${\int }_{-\infty }^{\infty }\psi \left(x\right)dx=1.$$

(85)

Therefore, $\psi \left(x\right)$ is a density function.

According to [6], p. 49, we found that

$$\psi \left(x\right)<\frac{2}{\pi cosh\left(x-1\right)},\forall x\ge 1.$$

(86)

But

$$\frac{1}{cosh\left(x-1\right)}=\frac{2}{e^{x-1}+e^{-\left(x-1\right)}}<\frac{2}{e^{x-1}}=2e^{-\left(x-1\right)},$$

(87)

∀$x\in \mathbb{R}$.

Therefore,

$$\psi \left(x\right)<\frac{4}{\pi }{e}^{-\left(x-1\right)}=\frac{4}{\pi }e{e}^{-x},\forall x\ge 1.$$

(88)

So the following,

$$d{\mu }_{\xi }\left(x\right)=\frac{1}{\xi }\psi \left(\frac{x}{\xi }\right)dx,0<\xi \le 1,$$

is the related Borel probability measure.

We give the following results; proofs similar to Theorems 6 and 7 are omitted.

Theorem 13.

Let $0<\xi \le 1$, and

$${\gamma }_{k,\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^k\psi \left(\frac{x}{\xi }\right)dx,k=1,\dots ,n\in \mathbb{N}.$$

(89)

Then, ${\gamma }_{k,\xi }$ are finite and ${\gamma }_{k,\xi }\to 0$, as $\xi \to 0.$

Theorem 14.

It holds that

$$\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r\psi \left(\frac{t}{\xi }\right)dt<\infty ;$$

(90)

$r,n\in \mathbb{N}$; $0<\xi \le 1.$

Also, this integral converges to zero, as $\xi \to 0.$

3.4. About the q-Deformed and $\lambda$-Parametrized Logistic-Type Activation Function

Here, everything comes from [7], Ch. 15.

The activation function now is

$${\phi }_{q,\lambda }\left(x\right):=\frac{1}{1+q{e}^{-\lambda x}},x\in \mathbb{R},$$

(91)

where $q,\lambda >0.$

The density function here will be

$$G_{q,\lambda }\left(x\right):=\frac{1}{2}\left({\phi }_{q,\lambda }\left(x+1\right)-{\phi }_{q,\lambda }\left(x-1\right)\right)>0,x\in \mathbb{R}.$$

(92)

We mention the following.

Theorem 15.

It holds that

$${\int }_{-\infty }^{\infty }{G}_{q,\lambda }\left(x\right)dx=1.$$

(93)

According to [7], p. 373, we find

$$G_{q,\lambda }\left(x\right)<q\lambda e^{-\lambda \left(x-1\right)},\forall x\ge 1.$$

So the following

$$d{\mu }_{\xi }\left(x\right)=\frac{1}{\xi }{G}_{q,\lambda }\left(\frac{x}{\xi }\right)dx,0<\xi \le 1,$$

(94)

is the related Borel probability measure.

We give the following results; proofs similar to Theorems 10 and 11 are omitted.

Theorem 16.

Let

$${\overline{\delta }}_{k,\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^k{G}_{q,\lambda }\left(\frac{x}{\xi }\right)dx,k=1,\dots ,n\in \mathbb{N}.$$

(95)

Then, ${\overline{\delta }}_{k,\xi }$ are finite and ${\overline{\delta }}_{k,\xi }\to 0$, as $\xi \to 0.$

Theorem 17.

It holds that

$$I_{G_{q,\lambda },\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{G}_{q,\lambda }\left(\frac{t}{\xi }\right)dt<\infty ;$$

(96)

where $\lambda ,q>0;$ $r,n\in \mathbb{N}$; $0<\xi \le 1.$

Also, $I_{G_{q,\lambda },\xi }\to 0$, as $\xi \to 0.$

3.5. About the q-Deformed and $\beta$-Parametrized Half-Hyperbolic Tangent Function ${\phi }_{q,\beta }$

Here, everything comes from [7], Ch. 19.

The activation function now is

$${\phi }_{q,\beta }\left(x\right):=\frac{1-q{e}^{-\beta t}}{1+q{e}^{-\beta t}},\forall t\in \mathbb{R},$$

(97)

where $q,\beta >0.$

The corresponding density function will be

$${\mathsf{\Phi }}_{q,\beta }\left(x\right):=\frac{1}{4}\left({\phi }_{q,\beta }\left(x+1\right)-{\phi }_{q,\beta }\left(x-1\right)\right)>0,\forall x\in \mathbb{R}.$$

(98)

Theorem 18.

$${\int }_{-\infty }^{\infty }{\mathsf{\Phi }}_{q,\beta }\left(x\right)dx=1.$$

(99)

According to [7], p. 481, we find that

$${\mathsf{\Phi }}_{q,\beta }\left(x\right)<\beta q{e}^{-\beta \left(x-1\right)},\forall x\ge 1.$$

(100)

Thus, the following

$$d{\mu }_{\xi }\left(x\right)=\frac{1}{\xi }{\mathsf{\Phi }}_{q,\beta }\left(\frac{x}{\xi }\right)dx,0<\xi \le 1,$$

(101)

is the related Borel probability measure.

We state the following results; proofs similar to Theorems 10 and 11 are omitted.

Theorem 19.

Let

$${\epsilon }_{k,\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{x}^k{\mathsf{\Phi }}_{q,\beta }\left(\frac{x}{\xi }\right)dx,k=1,\dots ,n\in \mathbb{N}.$$

(102)

Then, ${\epsilon }_{k,\xi }$ are finite and ${\epsilon }_{k,\xi }\to 0$, as $\xi \to 0.$

Theorem 20.

It holds that

$$I_{{\mathsf{\Phi }}_{q,\beta },\xi }:=\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt<\infty ;$$

(103)

where $q,\beta >0;$ $r,n\in \mathbb{N}$; $0<\xi \le 1.$

Also, $I_{{\mathsf{\Phi }}_{q,\beta },\xi }\to 0$, as $\xi \to 0.$

In Section 2, we generally assumed that ${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)\in \mathbb{R}$, for all $x\in \mathbb{R}$.

The next result proves that this is easily possible.

Theorem 21.

Let $f:\mathbb{R}\to \mathbb{R}$ be a Lipschitz function, that is, there exists $K>0$ such that $\left|f\left(x\right)-f\left(y\right)\right|\le K\left|x-y\right|$, for any $x,y\in \mathbb{R}$. Here, ${\mu }_{\xi }$ is a general Borel probability measure on $\mathbb{R}$ with the property ${\int }_{-\infty }^{\infty }\left|t\right|d{\mu }_{\xi }\left(t\right)$ as finite. Then, ${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)$, defined by (2), is finite, i.e., ${\mathsf{\Theta }}_{r,\xi }\left(f;x\right)\in \mathbb{R}$, $\forall x\in \mathbb{R}.$

Proof. 

We find that

$${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)-f\left(x\right)=\sum _{j=1}^r{\alpha }_j{\int }_{-\infty }^{\infty }\left(f\left(x+jt\right)-f\left(x\right)\right)d{\mu }_{\xi }\left(t\right).$$

(104)

Thus,

$$\left|{\mathsf{\Theta }}_{r,\xi }\left(f,x\right)-f\left(x\right)\right|\le \sum _{j=1}^r\left|{\alpha }_j\right|{\int }_{-\infty }^{\infty }\left|f\left(x+jt\right)-f\left(x\right)\right|d{\mu }_{\xi }\left(t\right)\le$$

$$\left(\sum _{j=1}^r\left|{\alpha }_j\right|j\right)K{\int }_{-\infty }^{\infty }\left|t\right|d{\mu }_{\xi }\left(t\right)=:F_f<\infty .$$

(105)

That is,

$$\left|{\mathsf{\Theta }}_{r,\xi }\left(f,x\right)-f\left(x\right)\right|\le F_f<\infty .$$

(106)

Notice that

$${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)={\mathsf{\Theta }}_{r,\xi }\left(f,x\right)-f\left(x\right)+f\left(x\right),$$

(107)

and

$$\left|{\mathsf{\Theta }}_{r,\xi }\left(f,x\right)\right|\le F_f+\left|f\left(x\right)\right|<\infty ,$$

(108)

proving the claim. □

4. Main Results

Here, we describe the pointwise and uniform approximation properties of the following activated singular operators, which are special cases of ${\mathsf{\Theta }}_{r,\xi }\left(f,x\right)$; see Section 2. Their definitions are based on Section 3. Essentially, we apply our listed results in Section 2.

Definition 1.

Let $f:\mathbb{R}\to \mathbb{R}$ be Borel-measurable and ${\alpha }_j$ as in (1), $x\in \mathbb{R}$, $0<\xi \le 1.$

We call

(1)

$${\mathsf{\Theta }}_{1,r,\xi }\left(f,x\right)=\frac{1}{\xi }{\int }_{-\infty }^{\infty }\left(\sum _{j=1}^r{\alpha }_{j}f\left(x+jt\right)\right)G\left(\frac{t}{\xi }\right)dt,$$

(109)

(2)

$${\mathsf{\Theta }}_{2,r,\xi }\left(f,x\right)=\frac{1}{\xi }{\int }_{-\infty }^{\infty }\left(\sum _{j=1}^r{\alpha }_{j}f\left(x+jt\right)\right)M_{q,\lambda }\left(\frac{t}{\xi }\right)dt,q,\lambda >0,$$

(110)

(3)

$${\mathsf{\Theta }}_{3,r,\xi }\left(f,x\right)=\frac{1}{\xi }{\int }_{-\infty }^{\infty }\left(\sum _{j=1}^r{\alpha }_{j}f\left(x+jt\right)\right)\psi \left(\frac{t}{\xi }\right)dt,$$

(111)

(4)

$${\mathsf{\Theta }}_{4,r,\xi }\left(f,x\right)=\frac{1}{\xi }{\int }_{-\infty }^{\infty }\left(\sum _{j=1}^r{\alpha }_{j}f\left(x+jt\right)\right)G_{q,\lambda }\left(\frac{t}{\xi }\right)dt,q,\lambda >0,$$

(112)

and

(5)

$${\mathsf{\Theta }}_{5,r,\xi }\left(f,x\right)=\frac{1}{\xi }{\int }_{-\infty }^{\infty }\left(\sum _{j=1}^r{\alpha }_{j}f\left(x+jt\right)\right){\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt,q,\beta >0.$$

(113)

We give the following results grouped by operator.

Theorem 22.

It holds that

$$\left|{\mathsf{\Theta }}_{1,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{c}_{k,\xi }^{*}\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)G\left(\frac{t}{\xi }\right)dt.$$

(114)

The last, at $x=0$, is attained by $f\left(x\right)=x^{r+n}$, $r,n\in \mathbb{N}$ with $r+n$ even.

Proof. 

Theorems 1 and 6. □

Corollary 3.

Assume ${\omega }_r\left(f,\xi \right)<\infty$, $0<\xi \le 1.$

Then,

$$\left|{\mathsf{\Theta }}_{1,r,\xi }\left(f,x\right)-f\left(x\right)\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)G\left(\frac{t}{\xi }\right)dt,$$

(115)

which is attained at $x=0$ by $f\left(x\right)=x^r$, where r is even.

Remark 4.

We also have the uniform estimates

$${∥{\mathsf{\Theta }}_{1,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{c}_{k,\xi }^{*}∥}_{\infty ,x}\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)G\left(\frac{t}{\xi }\right)dt,$$

(116)

and

$${∥{\mathsf{\Theta }}_{1,r,\xi }\left(f\right)-f∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)G\left(\frac{t}{\xi }\right)dt.$$

(117)

And the following holds.

Theorem 23.

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$. Assume also that ${\omega }_r\left(f^{\left(n\right)},h\right)<\infty$, $\forall h>0.$

Then,

$${∥{\mathsf{\Theta }}_{1,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{c}_{k,\xi }^{*}∥}_{\infty ,x}\le$$

$$\frac{{\omega }_r\left(f^{\left(n\right)},\xi \right)}{n!}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^{r}G\left(\frac{t}{\xi }\right)dt.$$

(118)

At $n=0$, the sum on the L.H.S. (118) collapses.

Proof. 

Theorems 3 and 7. □

We continue similarly.

Theorem 24.

It holds that

$$\left|{\mathsf{\Theta }}_{2,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\overline{c}}_{k,\xi }\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)M_{q,\lambda }\left(\frac{t}{\xi }\right)dt,$$

(119)

$q,\lambda >0.$

The last, at $x=0$, is attained by $f\left(x\right)=x^{r+n}$, $r,n\in \mathbb{N}$ with $r+n$ even.

Proof. 

Theorems 1 and 10. □

Corollary 4.

Assume that ${\omega }_r\left(f,\xi \right)<\infty$, $0<\xi \le 1.$

Then,

$$\left|{\mathsf{\Theta }}_{2,r,\xi }\left(f,x\right)-f\left(x\right)\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)M_{q,\lambda }\left(\frac{t}{\xi }\right)dt,$$

(120)

which is attained at $x=0$ by $f\left(x\right)=x^r$, where r is even.

Remark 5.

We also have the uniform estimates

$${∥{\mathsf{\Theta }}_{2,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\overline{c}}_{k,\xi }∥}_{\infty ,x}\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)M_{q,\lambda }\left(\frac{t}{\xi }\right)dt,$$

(121)

and

$${∥{\mathsf{\Theta }}_{2,r,\xi }\left(f\right)-f∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)M_{q,\lambda }\left(\frac{t}{\xi }\right)dt.$$

(122)

And the following holds.

Theorem 25.

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$. Assume also that ${\omega }_r\left(f^{\left(n\right)},h\right)<\infty$, ∀$h>0.$ Then,

$${∥{\mathsf{\Theta }}_{2,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\overline{c}}_{k,\xi }∥}_{\infty ,x}\le$$

$$\frac{{\omega }_r\left(f^{\left(n\right)},\xi \right)}{n!}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{M}_{q,\lambda }\left(\frac{t}{\xi }\right)dt.$$

(123)

At $n=0$, the sum on the L.H.S. (123) collapses.

Proof. 

Theorems 3 and 11. □

We continue similarly.

Theorem 26.

It holds that

$$\left|{\mathsf{\Theta }}_{3,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\gamma }_{k,\xi }\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)\psi \left(\frac{t}{\xi }\right)dt.$$

(124)

The last, at $x=0$, is attained by $f\left(x\right)=x^{r+n}$, $r,n\in \mathbb{N}$ with $r+n$ even.

Proof. 

Theorems 1 and 13. □

Corollary 5.

Assume that ${\omega }_r\left(f,\xi \right)<\infty$, $0<\xi \le 1.$ Then,

$$\left|{\mathsf{\Theta }}_{3,r,\xi }\left(f,x\right)-f\left(x\right)\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)\psi \left(\frac{t}{\xi }\right)dt,$$

(125)

which is attained at $x=0$ by $f\left(x\right)=x^r$, where r is even.

Remark 6.

We also have the uniform estimates

$${∥{\mathsf{\Theta }}_{3,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\gamma }_{k,\xi }∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)\psi \left(\frac{t}{\xi }\right)dt,$$

(126)

and

$${∥{\mathsf{\Theta }}_{3,r,\xi }\left(f\right)-f∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)\psi \left(\frac{t}{\xi }\right)dt.$$

(127)

And the following holds.

Theorem 27.

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$. Assume also that ${\omega }_r\left(f^{\left(n\right)},h\right)<\infty$, $\forall h>0.$ Then,

$${∥{\mathsf{\Theta }}_{3,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\gamma }_{k,\xi }∥}_{\infty ,x}\le$$

$$\frac{{\omega }_r\left(f^{\left(n\right)},\xi \right)}{n!}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r\psi \left(\frac{t}{\xi }\right)dt.$$

(128)

At $n=0$, the sum on the L.H.S. (128) collapses.

Proof. 

Theorems 3 and 14. □

We continue on this fashion.

Theorem 28.

It holds that

$$\left|{\mathsf{\Theta }}_{4,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\overline{\delta }}_{k,\xi }\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)G_{q,\lambda }\left(\frac{t}{\xi }\right)dt,$$

(129)

$q,\lambda >0.$

The last, at $x=0$, is attained by $f\left(x\right)=x^{r+n}$, $r,n\in \mathbb{N}$ with $r+n$ even.

Proof. 

Theorems 1 and 16. □

Corollary 6.

Assume that ${\omega }_r\left(f,\xi \right)<\infty$, $0<\xi \le 1.$

Then,

$$\left|{\mathsf{\Theta }}_{4,r,\xi }\left(f,x\right)-f\left(x\right)\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)G_{q,\lambda }\left(\frac{t}{\xi }\right)dt,$$

(130)

which is attained at $x=0$ by $f\left(x\right)=x^r$, where r is even.

Remark 7.

We also have the uniform estimates

$${∥{\mathsf{\Theta }}_{4,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\overline{\delta }}_{k,\xi }∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right)G_{q,\lambda }\left(\frac{t}{\xi }\right)dt,$$

(131)

and

$${∥{\mathsf{\Theta }}_{4,r,\xi }\left(f\right)-f∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right)G_{q,\lambda }\left(\frac{t}{\xi }\right)dt.$$

(132)

And the following holds.

Theorem 29.

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$. Assume also that ${\omega }_r\left(f^{\left(n\right)},h\right)<\infty$, $\forall h>0.$ Then,

$${∥{\mathsf{\Theta }}_{4,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\overline{\delta }}_{k,\xi }∥}_{\infty }\le$$

$$\frac{{\omega }_r\left(f^{\left(n\right)},\xi \right)}{n!}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{G}_{q,\lambda }\left(\frac{t}{\xi }\right)dt.$$

(133)

At $n=0$, the sum on the L.H.S. (133) collapses.

Proof. 

Theorems 3 and 17. □

The last group of results follows.

Theorem 30.

It holds that

$$\left|{\mathsf{\Theta }}_{5,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\epsilon }_{k,\xi }\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right){\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt,$$

(134)

$q,\beta >0.$

The last, at $x=0$, is attained by $f\left(x\right)=x^{r+n}$, $r,n\in \mathbb{N}$ with $r+n$ even.

Proof. 

Theorems 1 and 19. □

Corollary 7.

Assume that ${\omega }_r\left(f,\xi \right)<\infty$, $0<\xi \le 1.$

Then,

$$\left|{\mathsf{\Theta }}_{5,r,\xi }\left(f,x\right)-f\left(x\right)\right|\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right){\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt,$$

(135)

which is attained at $x=0$ by $f\left(x\right)=x^r$, where r is even.

Remark 8.

We also have the uniform estimates

$${∥{\mathsf{\Theta }}_{15r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\epsilon }_{k,\xi }∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\tilde{G}}_n\left(t\right){\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt,$$

(136)

and

$${∥{\mathsf{\Theta }}_{5,r,\xi }\left(f\right)-f∥}_{\infty }\le \frac{1}{\xi }{\int }_{-\infty }^{\infty }{\omega }_r\left(f,\left|t\right|\right){\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt.$$

(137)

And the following holds.

Theorem 31.

Let $f\in C^n\left(\mathbb{R}\right)$, $n\in {\mathbb{Z}}^{+}$. Assume also that ${\omega }_r\left(f^{\left(n\right)},h\right)<\infty$, $\forall h>0.$ Then,

$${∥{\mathsf{\Theta }}_{5,r,\xi }\left(f,x\right)-f\left(x\right)-\sum _{k=1}^n\frac{f^{\left(k\right)}\left(x\right)}{k!}{\delta }_k{\epsilon }_{k,\xi }∥}_{\infty }\le$$

$$\frac{{\omega }_r\left(f^{\left(n\right)},\xi \right)}{n!}\frac{1}{\xi }{\int }_{-\infty }^{\infty }{\left|t\right|}^n{\left(1+\frac{\left|t\right|}{\xi }\right)}^r{\mathsf{\Phi }}_{q,\beta }\left(\frac{t}{\xi }\right)dt.$$

(138)

At $n=0$, the sum on the L.H.S. (138) collapses.

Proof. 

Theorems 3 and 20. □

5. Conclusions

Here, we presented the new idea of moving from the neural networks’ main tools, the activation functions, to singular integral approximation. This is a rare case of employing applied mathematics in theoretical cases.