Composition of Activation Functions and the Reduction to Finite Domain

Abstract

This work takes up the task of the determination of the rate of pointwise and uniform convergences to the unit operator of the “normalized cusp neural network operators”. The cusp is a compact support activation function, which is the composition of two general activation functions having as domain the whole real line. These convergences are given via the modulus of continuity of the engaged function or its derivative in the form of Jackson type inequalities. The composition of activation functions aims to more flexible and powerful neural networks, introducing for the first time the reduction in infinite domains to the one domain of compact support.

1. Introduction

From AI and computer science, we have the following: in essence, composing activation functions in neural networks offers the advantage of potentially tailoring the network’s ability to learn and model complex, non-linear relationships in data. Here is a breakdown of the potential benefits:

• Enhanced Capacity for Complex Modeling:
  • Diversification of Non-linearity: Different activation functions have different characteristics. For example, ReLU introduces sparsity, while Sigmoid squashes values into a range. By composing them, the network potentially can learn a wider variety of non-linear transformations and capture more intricate patterns in the data.
• Improved Training Dynamics:
  • Mitigating Gradient Problems: Activation functions influence gradient flow during training. Using different activation functions can potentially help address issues like vanishing or exploding gradients, which hinder learning in deep networks.
  • Faster Convergence: Certain activation functions, like ReLU, can accelerate the convergence of the training process compared to others like Sigmoid or Tanh. Combining different functions can potentially lead to faster training and competitive performance.
• Enhanced Generalization and Robustness:
  • Better Generalization: By learning richer representations of the data through diverse activation functions, the network’s ability to generalize well to unseen data improves, reducing the risk of overfitting.
  • Increased Robustness: Networks with carefully chosen activation functions can handle variations in input data more effectively, adapting to noise, missing data, or unexpected perturbations.
• Adaptation to Input Characteristics:
  • Handling Diverse Data: Different activation functions can be suited to different data characteristics. For instance, tanh can be useful when dealing with data containing both positive and negative values.
• Potential for Architectural Interpretability:
  • Insight into Learning: By using distinct activation functions, different parts of the network might become responsible for capturing specific features, which can potentially offer insights into how the model learns.

In summary, composing activation functions potentially allows for a more flexible and powerful neural network capable of

• Learning more complex patterns.
• Faster and more stable training.
• Better generalization to new data.
• Greater adaptability to diverse data.

Attention: While composing activation functions can offer benefits, it’s important to choose them judiciously and with consideration for the specific problem at hand, as some combinations might not be beneficial or could even lead to unwanted behaviors like exploding gradients. Empirical testing and validation are crucial when exploring different activation function compositions.

The author was greatly inspired and motivated by [1] and was the pioneer of quantitative neural network approximation, see [2], and since then, he has published numerous of papers and books, e.g., see [3].

In this article, we continue this trend.

In mathematical neural network approximation AMS Mathscinet lists no articles related to composition of activation functions. So this is the first one of its kind.

By using the composition of activation functions, we achieve the first extensive part of this introduction, and most notably, this composition leads to an activation function of compact support, though the initial activation functions had an infinite domain, the whole real line.

Now the resulting activation function is an open cusp of compact support $\left[-1,1\right]$. Our involved activation functions are very general, and the constructed neural network operators resemble the squashing operators in [2,3], and so do the produced quantitative results.

As a result, our produced convergence inequalities look much simpler and nicer.

Of great inspiration are the articles [4,5,6]. References [7,8,9] are foundational. Finally, references [10,11,12,13] represent recent important works.

2. Basics

Let $i=1,2$, and $h_i:\mathbb{R}\to \left[-1,1\right]$ be general sigmoid activation functions, such that they are strictly increasing, $h_i\left(0\right)=0$, $h_i\left(-x\right)=-h_i\left(x\right)$, $x\in \mathbb{R}$, $h_i\left(+\infty \right)=1$, $h_i\left(-\infty \right)=-1$. Also, $h_i$ is strictly convex over $(-\infty ,0]$ and strictly concave over $[0,+\infty )$, with $h_i^{\left(2\right)}\in C\left(\mathbb{R}\right)$.

Clearly, $h_1\circ h_2=h_1{|}_{\left(-1,1\right)}\circ h_2$ is strictly increasing and $\left(h_1\circ h_2\right)\left(0\right)=0$, and

$$h_1\circ h_2\left(-x\right)=h_1\left(h_2\left(-x\right)\right)=h_1\left(-h_2\left(x\right)\right)=-h_1\left(h_2\left(x\right)\right)=-\left(h_1\circ h_2\right)\left(x\right),$$

that is

$$\left(h_1\circ h_2\right)\left(-x\right)=-\left(h_1\circ h_2\right)\left(x\right),\forall x\in \mathbb{R}.$$

Furthermore,

$$\left(h_1\circ h_2\right)\left(+\infty \right)=h_1\left(h_2\left(+\infty \right)\right)=h_1\left(1\right),$$

$$\left(h_1\circ h_2\right)\left(-\infty \right)=h_1\left(h_2\left(-\infty \right)\right)=h_1\left(-1\right).$$

Next, acting over $(-\infty ,0]:$ let $\lambda ,\mu \ge 0:\lambda +\mu =1$. Then, by convexity of $h_2$ there we have

$$h_2\left(\lambda x+\mu y\right)\le \lambda h_2\left(x\right)+\mu h_2\left(y\right),x,y\in {\mathbb{R}}_{-};$$

and

$$h_1\left(h_2\left(\lambda x+\mu y\right)\right)\le h_1\left(\lambda h_2\left(x\right)+\mu h_2\left(y\right)\right)\le \lambda h_1\left(h_2\left(x\right)\right)+\mu h_1\left(h_2\left(y\right)\right),$$

i.e.,

$$\left(h_1\circ h_2\right)\left(\lambda x+\mu y\right)\le \lambda \left(h_1\circ h_2\right)\left(x\right)+\mu \left(h_1\circ h_2\right)\left(y\right),$$

$x,y\in {\mathbb{R}}_{-}.$

So that $h_1\circ h_2$ is convex over $(-\infty ,0].$

Similarly, over $[0,+\infty )$, we get: let $\lambda ,\mu \ge 0:\lambda +\mu =1$. Then, by concavity of $h_2$ there we have

$$h_2\left(\lambda x+\mu y\right)\ge \lambda h_2\left(x\right)+\mu h_2\left(y\right),x,y\in {\mathbb{R}}_{+};$$

and

$$h_1\left(h_2\left(\lambda x+\mu y\right)\right)\ge h_1\left(\lambda h_2\left(x\right)+\mu h_2\left(y\right)\right)\ge \lambda h_1\left(h_2\left(x\right)\right)+\mu h_1\left(h_2\left(y\right)\right).$$

Therefore $h_1\circ h_2$ is concave over $[0,+\infty ).$

Also, it is

$${\left(h_1\left(h_2\left(x\right)\right)\right)}^{″}=h_1^{″}\left(h_2\left(x\right)\right){\left(h_2^{\prime }\left(x\right)\right)}^2+h_1^{\prime }\left(h_2\left(x\right)\right)h_2^{″}\left(x\right)\in C\left(\mathbb{R}\right),x\in \mathbb{R}.$$

So $h_1\circ h_2$ is a sigmoid activation function.

Next we consider the function

$${\psi }_{1,2}\left(x\right):={\displaystyle \frac{1}{4}}\left(h_1\circ h_2\left(x+1\right)-h_1\circ h_2\left(x-1\right)\right)>0,\forall x\in \mathbb{R}.$$

We observe that

$${\psi }_{1,2}\left(-x\right)={\displaystyle \frac{1}{4}}\left(h_1\left(h_2\left(-x+1\right)\right)-h_1\left(h_2\left(-x-1\right)\right)\right)=$$

$${\displaystyle \frac{1}{4}}\left(h_1\left(h_2\left(-\left(x-1\right)\right)\right)-h_1\left(h_2\left(-\left(x+1\right)\right)\right)\right)=$$

$${\displaystyle \frac{1}{4}}\left(h_1\left(-h_2\left(x-1\right)\right)-h_1\left(-h_2\left(x+1\right)\right)\right)=$$

$${\displaystyle \frac{1}{4}}\left(-h_1\left(h_2\left(x-1\right)\right)+h_1\left(h_2\left(x+1\right)\right)\right)=$$

$${\displaystyle \frac{1}{4}}\left(h_1{h}_2\left(x+1\right)-h_1{h}_2\left(x-1\right)\right)={\psi }_{1,2}\left(x\right),$$

that is

$${\psi }_{1,2}\left(-x\right)={\psi }_{1,2}\left(x\right),\forall x\in \mathbb{R}.$$

So ${\psi }_{1,2}$ can serve as a density function in general.

So we have h₂ : ℝ → (−1, 1), h₁|(−1,1) : (−1, 1) → (−1, 1), and the strictly increasing function H := h₁|_(−1,1) ◦ h₂ : ℝ → (−1, 1), with the graph of H containing an arc of finite length, such that H(0) = 0, starting at (−1, h₁(h₂(−1))) and terminating at (1, h₁(h₂(1))). We call this arc also H. In particular H is negative and convex over (−1, 0], and it is positive and concave over [0, 1).

So it has compact support [−1, 1] and it is like a squashing function, see [3], Ch. 1, p. 8.

We will work from now on with |H|, which has as a graph a cusp joining the points (−1, |h₁(h₂(−1))|), (0, 0), (1, h₁(h₂(1))) and with compact support, again, [−1, 1]. The points (−1, |h₁(h₂(−1))|), (1, h₁(h₂(1))) belong to the graph of |H| and (0, 0) too.

Typically H has a steeper slope than of h₂, but it is flatter and closer to the x-axis than h₂ is, e.g. tanh(tanh x) has asymptotes ±0.76, while tanh x has asymptotes ±1, notice that tanh(1) = 0.76. Clearly H has applications in spiking neural networks.

3. Background

Here we consider functions $f:\mathbb{R}\to \mathbb{R}$ to be either continuous and bounded, or uniformly continuous.

The first modulus of continuity is given by

$${\omega }_1\left(f,\delta \right):=\underset{\begin{array}{c}x,y\in \mathbb{R}:\\ \left|x-y\right|\le \delta \end{array}}{\sup }\left|f\left(x\right)-f\left(y\right)\right|,\delta >0.$$

Here we have that ${\omega }_1\left(f,\delta \right)<+\infty$, $\delta >0.$

In this article, we study the pointwise and uniform convergences with rates over the real line, to the unit operator, of the “normalized cusp neural network operators”,

$$\left(A_n\left(f\right)\right)\left(x\right):={\displaystyle \frac{{\displaystyle \sum _{k=-n^2}^{n^2}}f\left(\frac{k}{n}\right)\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=-n^2}^{n^2}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}},$$

(1)

where $0<\alpha <1$ and $x\in \mathbb{R}$, $n\in \mathbb{N}$.

Notice $A_n$ is a positive linear operator with $A_n\left(1\right)=1$.

The terms in the ratio of sums (1) can be non-negative and make sense, iff $0<\left|n^{1-\alpha }\left(x-{\displaystyle \frac{k}{n}}\right)\right|\le 1$, i.e., $0<\left|x-{\displaystyle \frac{k}{n}}\right|\le {\displaystyle \frac{1}{n^{1-\alpha }}},$ iff

$$nx-n^{\alpha }\le x\le nx+n^{\alpha },x\ne {\displaystyle \frac{k}{n}}.$$

(2)

In order to have the desired order of numbers

$$-n^2\le nx-n^{\alpha }\le nx+n^{\alpha }\le n^2,x\ne {\displaystyle \frac{k}{n}},$$

(3)

it is sufficient to assume that

$$n\ge 1+\left|x\right|,x\ne {\displaystyle \frac{k}{n}}.$$

(4)

When $x\in \left[-1,1\right]$, $x\ne {\displaystyle \frac{k}{n}}$, it is enough to assume $n\ge 2$, which implies (3), and $x\ne {\displaystyle \frac{k}{n}}.$

But the unique case $x={\displaystyle \frac{k}{n}}$ contributes nothing and can be ignored.

Thus, without loss of generality, we can always take that $x\ne {\displaystyle \frac{k}{n}}.$

Proposition 1

([2]). Let $a,b\in \mathbb{R}$, $a\le b$. Let $card\left(k\right)$$\left(\ge 0\right)$ be the maximum number of integers contained in $\left[a,b\right]$. Then

$$\max \left(0,\left(b-a\right)-1\right)\le card\left(k\right)\le \left(b-a\right)+1.$$

Note 1.

We would like to establish a lower bound on $card\left(k\right)$ over the interval $\left[nx-n^{\alpha },nx+n^{\alpha }\right]$. By Proposition 1, we get that

$$card\left(k\right)\ge \max \left(2n^{\alpha }-1,0\right).$$

We obtain $card\left(k\right)\ge 1$, if $2n^{\alpha }-1\ge 1$ iff $n\ge 1,$ which is always true.

So to have the desired order $\left(3\right)$ and $card\left(k\right)\ge 1$ over $\left[nx-n^{\alpha },nx+n^{\alpha }\right],$ it is enough to consider

$$n\ge \max \left(1+\left|x\right|,1\right)=1+\left|x\right|.$$

(5)

Also notice that $card\left(k\right)\to +\infty$, as $n\to +\infty .$

Denote $\left[·\right]$ as the integral part of a number and $⌈·⌉$ as its ceiling.

Thus, it is clear that

$$\left(A_n\left(f\right)\right)\left(x\right):={\displaystyle \frac{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}f\left(\frac{k}{n}\right)\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}},$$

(6)

$0<\alpha <1$, and $n\in \mathbb{N}:n\ge 1+\left|x\right|,$ $x\in \mathbb{R}.$

4. Main Results

Next come our first main results.

Theorem 1.

Let $x\in \mathbb{R}$, $n\in \mathbb{N}:n\ge 1+\left|x\right|$, $f:\mathbb{R}\to \mathbb{R}$ is either continuous and bounded, or uniformly continuous. Then

$$\left|A_n\left(f\right)\left(x\right)-f\left(x\right)\right|\le {\omega }_1\left(f,{\displaystyle \frac{1}{n^{1-\alpha }}}\right),$$

(7)

where ${\omega }_1$ is the first modulus of continuity of f. Hence, $\underset{n\to +\infty }{\lim }{A}_n\left(f\right)\left(x\right)=f\left(x\right)$, pointwise, given f is uniformly continuous.

When $n\ge 2$, we obtain

$${∥A_n\left(f\right)-f∥}_{\infty ,\left[-1,1\right]}\le {\omega }_1\left(f,{\displaystyle \frac{1}{n^{1-\alpha }}}\right).$$

(8)

Hence $\underset{n\to +\infty }{\lim }{A}_n\left(f\right)=f$, uniformly over $\left[-1,1\right]$, given f isuni f ormlycontinuous.

Proof. 

$$\left|A_n\left(f\right)\left(x\right)-f\left(x\right)\right|=\left|{\displaystyle \frac{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}f\left(\frac{k}{n}\right)\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}}-f\left(x\right)\right|=$$

$$\left|{\displaystyle \frac{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left(f\left(\frac{k}{n}\right)-f\left(x\right)\right)\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}}\right|\le$$

(9)

$${\displaystyle \frac{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|f\left(\frac{k}{n}\right)-f\left(x\right)\right|\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}}\le$$

$$w_1\left(f,{\displaystyle \frac{1}{n^{1-\alpha }}}\right)\left({\displaystyle \frac{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}}\right)=w_1\left(f,{\displaystyle \frac{1}{n^{1-\alpha }}}\right).$$

□

We continue with our second main result.

Theorem 2.

Let $x\in \mathbb{R}$, $n\in \mathbb{N}:n\ge 1+\left|x\right|$. Let $f\in C^N\left(\mathbb{R}\right)$, $N\in \mathbb{N},$ such that $f^{\left(N\right)}$ is a continuous and bounded function or a uniformly continuous function. Then

$$\left|\left(A_n\left(f\right)\right)\left(x\right)-f\left(x\right)\right|\le \sum _{j=1}^N\left({\displaystyle \frac{\left|f^{\left(j\right)}\left(x\right)\right|}{n^{j\left(1-\alpha \right)}j!}}\right)+{\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}}.$$

(10)

Notice that as $n\to +\infty$, we have that the right-hand side (R.H.S.) (10)$\to 0$; therefore, the left-hand side (L.H.S.) (10)$\to 0,$ i.e., Equation (10) gives us with rates the pointwise convergence of $\left(A_n\left(f\right)\right)\left(x\right)\to f\left(x\right)$, as $n\to +\infty ,$ $x\in \mathbb{R}$.

Proof. 

With Taylor’s formula, we have

$$f\left({\displaystyle \frac{k}{n}}\right)=\sum _{j=0}^{N-1}{\displaystyle \frac{f^{\left(j\right)}\left(x\right)}{j!}}{\left({\displaystyle \frac{k}{n}}-x\right)}^j+{\int }_x^{{\displaystyle \frac{k}{n}}}\left(f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right){\displaystyle \frac{{\left(\frac{k}{n}-t\right)}^{N-1}}{\left(N-1\right)!}}dt.$$

(11)

Call

$$W\left(x\right):=\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}\left|H\left(n^{1-\alpha }\left(x-{\displaystyle \frac{k}{n}}\right)\right)\right|.$$

(12)

Hence

$${\displaystyle \frac{f\left(\frac{k}{n}\right)\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}=\sum _{j=0}^N{\displaystyle \frac{f^{\left(j\right)}\left(x\right)}{j!}}{\left({\displaystyle \frac{k}{n}}-x\right)}^j{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}+$$

$${\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}{\int }_x^{{\displaystyle \frac{k}{n}}}\left(f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right){\displaystyle \frac{{\left(\frac{k}{n}-t\right)}^{N-1}}{\left(N-1\right)!}}dt.$$

Thus

$$\left(A_n\left(f\right)\right)\left(x\right)-f\left(x\right)=\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{f\left(\frac{k}{n}\right)\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}-f\left(x\right)$$

$$=\sum _{j=0}^N{\displaystyle \frac{f^{\left(j\right)}\left(x\right)}{j!}}\left(\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\left({\displaystyle \frac{k}{n}}-x\right)}^j{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}\right)+F,$$

(13)

where

$$F:=\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}{\int }_x^{{\displaystyle \frac{k}{n}}}\left(f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right){\displaystyle \frac{{\left(\frac{k}{n}-t\right)}^{N-1}}{\left(N-1\right)!}}dt.$$

(14)

So that

$$\left|\left(A_n\left(f\right)\right)\left(x\right)-f\left(x\right)\right|\le$$

$$\sum _{j=0}^N{\displaystyle \frac{\left|f^{\left(j\right)}\left(x\right)\right|}{j!}}\left(\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{1}{n^{j\left(1-\alpha \right)}}}{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}\right)+\left|F\right|.$$

And hence

$$\left|\left(A_n\left(f\right)\right)\left(x\right)-f\left(x\right)\right|\le \left(\sum _{j=0}^N{\displaystyle \frac{\left|f^{\left(j\right)}\left(x\right)\right|}{n^{j\left(1-\alpha \right)}j!}}\right)+\left|F\right|.$$

(15)

Next we estimate

$$\left|F\right|=\left|\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}{\int }_x^{{\displaystyle \frac{k}{n}}}\left(f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right){\displaystyle \frac{{\left(\frac{k}{n}-t\right)}^{N-1}}{\left(N-1\right)!}}dt\right|\le$$

$$\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}\left|{\int }_x^{{\displaystyle \frac{k}{n}}}\left(f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right){\displaystyle \frac{{\left(\frac{k}{n}-t\right)}^{N-1}}{\left(N-1\right)!}}dt\right|\le$$

(16)

$$\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}\rho \le \left(\ast \right),$$

where

$$\rho :=\left|{\int }_x^{{\displaystyle \frac{k}{n}}}\left|f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right|{\displaystyle \frac{{\left|\frac{k}{n}-t\right|}^{N-1}}{\left(N-1\right)!}}dt\right|$$

(17)

and

$$\left(\ast \right)\le \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{W\left(x\right)}}\psi =\psi ,$$

(18)

where

$$\psi :={\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{n!n^{N\left(1-\alpha \right)}}}.$$

(19)

The last part of inequality (18) comes from the following:

(i) Let $x\le {\displaystyle \frac{k}{n}},$ then

$$\rho ={\int }_x^{{\displaystyle \frac{k}{n}}}\left|f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right|{\displaystyle \frac{{\left|\frac{k}{n}-t\right|}^{N-1}}{\left(N-1\right)!}}dt\le$$

$${\int }_x^{{\displaystyle \frac{k}{n}}}{\omega }_1\left(f^{\left(N\right)},\left|t-x\right|\right){\displaystyle \frac{{\left|\frac{k}{n}-t\right|}^{N-1}}{\left(N-1\right)!}}dt\le$$

$${\omega }_1\left(f^{\left(N\right)},\left|x-{\displaystyle \frac{k}{n}}\right|\right){\int }_x^{{\displaystyle \frac{k}{n}}}{\displaystyle \frac{{\left(\frac{k}{n}-t\right)}^{N-1}}{\left(N-1\right)!}}dt\le$$

$${\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{{\left(\frac{k}{n}-x\right)}^N}{N!}}\le {\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}};$$

i.e., when $x\le {\displaystyle \frac{k}{n}}$ we get

$$\rho \le {\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}}.$$

(20)

(ii) Let $x\ge {\displaystyle \frac{k}{n}}$, then

$$\rho =\left|{\int }_{{\displaystyle \frac{k}{n}}}^x\left|f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right|{\displaystyle \frac{{\left|t-\frac{k}{n}\right|}^{N-1}}{\left(N-1\right)!}}dt\right|=$$

$${\int }_{{\displaystyle \frac{k}{n}}}^x\left|f^{\left(N\right)}\left(t\right)-f^{\left(N\right)}\left(x\right)\right|{\displaystyle \frac{{\left(t-\frac{k}{n}\right)}^{N-1}}{\left(N-1\right)!}}dt\le$$

$${\int }_{{\displaystyle \frac{k}{n}}}^x{\omega }_1\left(f^{\left(N\right)},\left|t-x\right|\right){\displaystyle \frac{{\left(t-\frac{k}{n}\right)}^{N-1}}{\left(N-1\right)!}}dt\le$$

$${\omega }_1\left(f^{\left(N\right)},\left|x-{\displaystyle \frac{k}{n}}\right|\right){\int }_{{\displaystyle \frac{k}{n}}}^x{\displaystyle \frac{{\left(t-\frac{k}{n}\right)}^{N-1}}{\left(N-1\right)!}}dt=$$

$${\omega }_1\left(f^{\left(N\right)},\left|x-{\displaystyle \frac{k}{n}}\right|\right){\displaystyle \frac{{\left(x-\frac{k}{n}\right)}^N}{N!}}\le {\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}}=\psi .$$

Thus, we get in both cases

$$\rho \le {\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}}.$$

(21)

Therefore, from (16), (18), (19) and (21), we obtain

$$\left|F\right|\le {\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}}.$$

(22)

At the end from (15) and (22), we derive inequality (10). □

Corollary 1

(to Theorem 2). It holds

$$\left|\left(A_n\left(f\right)\right)\left(x\right)-f\left(x\right)-\right.$$

$$\left.\sum _{j=1}^N{\displaystyle \frac{f^{\left(j\right)}\left(x\right)}{j!}}\left(\sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}{\left({\displaystyle \frac{k}{n}}-x\right)}^j{\displaystyle \frac{\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}{{\displaystyle \sum _{k=⌈nx-n^{\alpha }⌉}^{\left[nx+n^{\alpha }\right]}}\left|H\left(n^{1-\alpha }\left(x-\frac{k}{n}\right)\right)\right|}}\right)\right|$$

$$\le {\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{1}{N!n^{N\left(1-\alpha \right)}}}.$$

(23)

Proof. 

By (13) and (22). □

Corollary 2

(to Theorem 1). Let $x\in \left[-\varphi ,\varphi \right]$, $\varphi >0$, and $n\in \mathbb{N}$ be such that $n\ge 1+\varphi$, $0<\alpha <1$. Consider $p\ge 1$. Then

$${∥A_n\left(f\right)-f∥}_{p,\left[-\varphi ,\varphi \right]}\le {\omega }_1\left(f,{\displaystyle \frac{1}{n^{1-\alpha }}}\right)2^{{\displaystyle \frac{1}{p}}}{\varphi }^{{\displaystyle \frac{1}{p}}},$$

(24)

By (24), we derive the $L_p$ convergence of $A_n\left(f\right)$ to f with rates, given f is uniformly continuous.

We finish with

Corollary 3

(to Theorem 2). In the assumptions of Theorem 2 and Corollary 2 we have

$${∥A_n\left(f\right)-f∥}_{p,\left[-\varphi ,\varphi \right]}\le \sum _{j=1}^N{\displaystyle \frac{{∥f^{\left(j\right)}∥}_{p,\left[-\varphi ,\varphi \right]}}{n^{j\left(1-\alpha \right)}j!}}+$$

$${\omega }_1\left(f^{\left(N\right)},{\displaystyle \frac{1}{n^{1-\alpha }}}\right){\displaystyle \frac{2^{\frac{1}{p}}{\varphi }^{\frac{1}{p}}}{N!n^{N\left(1-\alpha \right)}}},N\in \mathbb{N}.$$

(25)

By (25) we derive again the $L_p$ convergence of $A_n\left(f\right)$ to f with rates.

Proof. 

Inequality (25) comes by integration of (10) over $\left[-\varphi ,\varphi \right]$ and the triangle and homogeneity properties of the $L_p$ norm. □