Next Article in Journal
Leveraging Bayesian Quadrature for Accurate and Fast Credit Valuation Adjustment Calculations
Previous Article in Journal
A Hybrid Forecasting System Based on Comprehensive Feature Selection and Intelligent Optimization for Stock Price Index Forecasting
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multivariate Smooth Symmetrized and Perturbed Hyperbolic Tangent Neural Network Approximation over Infinite Domains

by
George A. Anastassiou
Department of Mathematical Sciences, University of Memphis, Memphis, TN 38152, USA
Mathematics 2024, 12(23), 3777; https://doi.org/10.3390/math12233777
Submission received: 20 November 2024 / Revised: 27 November 2024 / Accepted: 28 November 2024 / Published: 29 November 2024

Abstract

:
In this article, we study the multivariate quantitative smooth approximation under differentiation of functions. The approximators here are multivariate neural network operators activated by the symmetrized and perturbed hyperbolic tangent activation function. All domains used here are infinite. The multivariate neural network operators are of quasi-interpolation type: the basic type, the Kantorovich type, and the quadrature type. We give pointwise and uniform multivariate approximations with rates. We finish with illustrations.

1. Introduction

The author of [1,2] (see Chapters 2–5) was the first to establish neural network approximation to continuous functions with rates by very specific neural network operators of Cardaliaguet–Euvrard and “squashing” types, by using the modulus of continuity of the engaged function or its high order derivative, and producing very tight Jackson type inequalities. He treats both the univariate and multivariate cases. These operators “bell-shaped” and “squashing” functions are assumed to be of compact support.
Further, the author, inspired by [3], continued his studies on neural network approximation by introducing and using the proper quasi-interpolation operators of sigmoidal and hyperbolic tangent types which resulted in the complete monographs [4,5], by treating both the univariate and multivariate cases. He included also the corresponding fractional cases.
The author here performs symmetrized and perturbed hyperbolic tangent activated multivariate neural network approximation to differentiated functions from R N into R , N N .
We present real multivariate quasi-interpolation quantitative approximations. We derive very tight Jackson type multivariate inequalities.
Real feed-forward neural networks (FNNs) with one hidden layer, the ones we use here are mathematically expressed by
N n x = j = 0 n c j σ α j · x + b j , x R s , s N ,
where for 0 j n , b j R are the thresholds, α j R s are the connection weights, c j R are the coefficients, α j · x is the inner product of α j and x, and σ is the activation function of the network. About neural networks in general read [6,7,8].
Recent developments in neural network approximation are [9,10,11,12,13,14,15,16,17,18].

2. Basics

Initially we follow [5], pp. 455–460.
Our perturbed hyperbolic tangent activation function is
g q , λ x : = e λ x q e λ x e λ x + q e λ x , λ , q > 0 , x R .
Above λ is the parameter and q is the deformation coefficient.
For more read Chapter 18 of [5]: “q-deformed and λ -Parametrized Hyperbolic Tangent based Banach space Valued Ordinary and Fractional Neural Network Approximation”.
Chapters 17 and 18 of [5] motivate our current work.
The proposed “symmetrization method” aims to use half data feed to our multivariate neural networks.
We will employ the following density function
M q . λ x : = 1 4 g q , λ x + 1 g q , λ x 1 > 0 ,
x R ; q , λ > 0 .
We have that
M q , λ x = M 1 q , λ x , x R ; q , λ > 0 ,
and
M 1 q , λ x = M q , λ x , x R ; q , λ > 0 .
Adding (3) and (4) we obtain
M q , λ x + M 1 q , λ x = M q , λ x + M 1 q , λ x ,
a key to this work.
So that
Φ x : = M q , λ x + M 1 q , λ x 2
is an even function, symmetric with respect to the y-axis.
By (18.18) of [5], we have
M q , λ ln q 2 λ = tanh λ 2 , and M 1 q , λ ln q 2 λ = tanh λ 2 , λ > 0 .
sharing the same maximum at symmetric points.
By Theorem 18.1, p. 458 of [5], we have that
i = M q , λ x i = 1 , x R , λ , q > 0 , and i = M 1 q , λ x i = 1 , x R , λ , q > 0 .
Consequently, we derive that
i = Φ x i = 1 , x R .
By Theorem 18.2, p. 459 of [5], we have that
M q , λ x d x = 1 , and M 1 q , λ x d x = 1 ,
so that
Φ x d x = 1 ,
therefore Φ is a density function.
By Theorem 18.3, p. 459 of [5], we have:
Let 0 < α < 1 , and n N with n 1 α > 2 ; q , λ > 0 . Then
k = : n x k n 1 α M q , λ n x k < 2 max q , 1 q e 4 λ e 2 λ n 1 α = T e 2 λ n 1 α ,
where T : = 2 max q , 1 q e 4 λ .
Similarly, we get that
k = M 1 q , λ n x k < T e 2 λ n 1 α .
Consequently we obtain that
k = : n x k n 1 α Φ n x k < T e 2 λ n 1 α ,
where T : = 2 max q , 1 q e 4 λ .
Remark 1. 
We introduce
Z x 1 , , x N : = Z x : = i = 1 N Φ x i , x = x 1 , , x N R N , N N .
It has the properties:
(i)
Z x > 0 , x R N ,
(ii)
k = Z x k : = k 1 = k 2 = k N = Z x 1 k 1 , , x N k N = 1 ,
where k : = k 1 , , k n Z N , x R N , hence
(iii)
k = Z n x k = 1 ,
x R N ;   n N , and
(iv)
R N Z x d x = 1 ,
that is Z is a multivariate density function.
Here, denote x : = max x 1 , , x N , x R N , also set : = , , , : = , , upon the multivariate context,
(v)
k = k n x > 1 n β Z n x k < ( 14 ) T e 2 λ n 1 β ,
where T : = 2 max q , 1 q e 4 λ ,   0 < β < 1 ,   n N : n 1 β > 2 , x R N .
Theorem 1. 
Let 0 < β < 1 , and n N with n 1 β > 2 . It holds
k = k Z N : n x k n 1 β n x k m Z n x k <
2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
Proof. 
The condition k Z N : n x k n 1 β , implies that there exists at least one k r Z : n x r k r n 1 β , where r 1 , , N .
Indeed, it is
k Z N : n x k n 1 β
r = 1 N Z N 1 k r Z : n x r k r n 1 β
Z N 1 k r * Z : n x r * k r * n 1 β ,
for some r * 1 , , N .
Let x 1 , that is 0 x 1 < x + 1 . Using the mean value theorem we have that
M q . λ x = 1 4 g q , λ x + 1 g q , λ x 1 = 2 q λ e 2 λ ξ e 2 λ ξ + q 2 = : ψ ,
for some 0 x 1 < ξ < x + 1 .
Hence
ψ < 2 q λ e 2 λ ξ + q e 2 λ ξ + q 2 = 2 q λ e 2 λ ξ + q < 2 q λ e 2 λ x 1 + q < 2 q λ e 2 λ x 1 , x 1 .
That is
M q . λ x < 2 q λ e 2 λ e 2 λ x , x 1 .
Similarly, it holds
M 1 q . λ x < 2 q λ e 2 λ e 2 λ x , x 1 ,
and hence
Φ x < q + 1 q λ e 2 λ e 2 λ x , x 1 .
We also have that ( x 0 )
e x = k = 0 x k k ! > x m m ! , m N .
Hence it is x m < e x m ! , so that x m 2 m < e x 2 m ! and
x m < 2 m e x 2 m ! , x 0 , m N .
Next, we observe that
k = : k Z N : n x k n 1 β n x k m Z n x k
k r * = : k r * Z : n x r * k r * n 1 β n x r * k r * m Φ n x r * k r * < ( 26 )
( set μ : = q + 1 q λ e 2 λ )
μ 2 λ m k r * = : k r * Z : n x r * k r * n 1 β 2 λ n x r * k r * m e 2 λ n x r * k r * < ( 28 )
2 m m ! μ 2 λ m k r * = : k r * Z : n x r * k r * n 1 β e λ n x r * k r * e 2 λ n x r * k r * =
2 m m ! μ 2 λ m k r * = : k r * Z : n x r * k r * n 1 β e λ n x r * k r *
2 m + 1 λ m ! μ 2 λ m n 1 β 1 e λ x d λ x = y : = λ x 2 m + 1 λ m ! μ 2 λ m n 1 β 1 e y d y =
2 m + 1 λ m ! μ 2 λ m e λ n 1 β 1 = 2 m + 1 m ! λ 2 m λ m q + 1 q λ e 2 λ e λ n 1 β 1 =
2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
We have proved that
k = : k Z N : n x k n 1 β n x k m Z n x k
2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
We need
Definition 1. 
The modulus of continuity here is defined by
ω 1 f , δ : = sup x , y R N : x y < δ f x f y , δ > 0 ,
where f : R N R is a bounded and continuous function, denoted by f C B R N , N N . Similarly ω 1 is defined for f C U R N (uniformly continuous functions). We have that f C U R N , iff ω 1 f , δ 0 as δ 0 .
Notation 1. 
Let f C m R N , m , N N . Here, f α denotes a partial derivative of f, α : = α 1 , , α N , α i Z + , i = 1 , , N , and α = i = 1 N α i = l , where l = 0 , 1 , , m . We write also f α : = α f x α and we say it is of order l.
We denote
ω 1 , m max f α , h : = max α : α = m ω 1 f α , h , h > 0 .
Call also
f α , m max : = max α = m f α ,
where · is the supremum norm.
Under differentiation the speed of convergence of our neural network operators improves a lot, that is the aim of this work.
Next, we describe our neural network operators.
Definition 2. 
Let f C B R N , we define
A n f , x : = A n f , x 1 , , x N : = k = f k n Z n x k : =
k 1 = k 2 = k N = f k 1 n , k 2 n , , k N n i = 1 N Φ n x i k i ,
n N , x R N , N N , the multivariate quasi-interpolation neural network operator.
Also for f C B R N we define the multivariate Kantorovich type neural network operator
K n f , x : = K n f , x 1 , , x N : = k = n N k n k + 1 n f t d t Z n x k : =
k 1 = k 2 = k N = n N k 1 n k 1 + 1 n k 2 n k 2 + 1 n k N n k N + 1 n f t 1 , , t N d t 1 d t N
i = 1 N Φ n x i k i ,
n N , x R N .
Again for f C B R N ,   N N , we define the multivariate neural network operator of quadrature type Q n f , x , n N , as follows. Let θ = θ 1 , , θ N N N , r = r 1 , , r N Z + N , w r = w r 1 r 2 r N 0 , such that
r = 0 θ w r = r 1 = 0 θ 1 r 2 = 0 θ 2 r N = 0 θ N w r 1 r 2 r N = 1 ; k Z N ,
and
δ n k f : = δ n , k 1 , k 2 , , k N f : = r = 0 θ w r f k n + r n θ : =
r 1 = 0 θ 1 r 2 = 0 θ 2 r N = 0 θ N w r 1 r 2 r N f k 1 n + r 1 n θ 1 , k 2 n + r 2 n θ 2 , , k N n + r N n θ N ,
where r θ : = r 1 θ 1 , r 2 θ 2 , , r N θ N .
We put
Q n f , x : = Q n f , x 1 , , x N : = k = δ n k f Z n x k : =
k 1 = k 2 = k N = δ n , k 1 , k 2 , , k N f i = 1 N Φ n x i k i , x R N .
Remark 2. 
We notice that
k n k + 1 n f t d t = k 1 n k 1 + 1 n k 2 n k 2 + 1 n k N n k N + 1 n f t 1 , t 2 , , t N d t 1 d t 2 d t N =
0 1 n 0 1 n 0 1 n f t 1 + k 1 n , t 2 + k 2 n , , t N + k N n d t 1 d t N
= 0 1 n f t + k n d t .
Thus, it holds
K n f , x = k = n N 0 1 n f t + k n d t Z n x k .
Motivation here comes from the following.
Here, X , · γ is an arbitrary Banach space.
Theorem 2.
([5], p. 58). Let f C B R N , X (continuous and bounded X-valued functions from R N ), 0 < β < 1 , μ > 0 , x R N , N , n N with n 1 β > 2 , ω 1 is defined similarly as in Definition 1. Then
(i)
A n f , x f x γ ω 1 f , 1 n β + 4 f γ e μ n 1 β 2 = : λ 1 * n ,
(ii)
A n f f γ λ 1 * n .
Given that f C U R N , X C B R N , X , we obtain lim n A n f = f , uniformly.
The speed of convergence above is 1 n β .
The space C U R N , X denotes the uniformly continuous functions. The operator A n f , x is defined similarly to (36).

3. Main Results

Next, we study the approximation properties of A n , K n , and Q n neural network operators.
Theorem 3. 
Let 0 < β < 1 , n N : n 1 β > 2 , x R N , f C m R N , m , N N , with f α C B R N , for all α : = α 1 , , α N , α i Z + ,   i = 1 , , N , and α = i = 1 N α i = l , where l = 0 , 1 , , m . Then
(1)
A n f , x f x j = 1 m α : α = j 1 i = 1 N α i ! f α x A n i = 1 N · x i α i x
N m m ! n m β ω 1 , m max f α , 1 n β +
4 f , m max N m n m λ m q + 1 q e 2 λ e λ n 1 β 1 = : Φ 1 * ,
(2) assume that f α x = 0 , for all α : α = j , j = 1 , , m , we have
A n f , x f x Φ 1 * ,
with the high speed of convergence n β m + 1 ,
(3)
A n f , x f x
j = 1 N α : α = j 1 i = 1 N α i ! f α x
1 n β j + 1 n j 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 + Φ 1 * ,
and
(4)
A n f f
j = 1 N α : α = j 1 i = 1 N α i ! f α 1 n β j + 1 n j 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 + Φ 1 * .
We have that A n f f , as n , pointwise and uniformly.
Proof. 
Consider g z t : = f x 0 + t z x 0 , t 0 ; x 0 , z R N . Then
g z j t = i = 1 N z i x 0 i x i j f x 01 + t z 1 x 01 , , x 0 N + t z N x 0 N ,
for all j = 0 , 1 , , m .
We have the multivariate Taylor’s formula
f z 1 , , z N = g z 1 = j = 0 m g z j 0 j ! +
1 m 1 ! 0 1 1 θ m 1 g z m θ g z m 0 d θ .
Notice g z 0 = f z 0 . Also for j = 0 , 1 , , m , we have
g z j 0 = α : = α 1 , , α N , α i Z + i = 1 , , N , α : = i = 1 N α i = j j ! i = 1 N α i !
i = 1 N z i x 0 i α i f α x 0 .
Furthermore
g z m θ = α : = α 1 , , α N , α i Z + i = 1 , , N , α : = i = 1 N α i = m m ! i = 1 N α i !
i = 1 N z i x 0 i α i f α x 0 + θ z x 0 ,
0 θ 1 .
So, we treat f C m R N .
Thus, we have for x R N ,   x = x 1 , , x N , k i Z , i = 1 , , N ,   k : = k 1 , , k N ;   n N , that
f k 1 n , , k N n f x =
j = 1 m α : = α 1 , , α N , α i Z + i = 1 , , N , α : = i = 1 N α i = j 1 i = 1 N α i !
i = 1 N k i n x i α i f α x + R ,
where
R = m 0 1 1 θ m 1 α : = α 1 , , α N , α i Z + i = 1 , , N , α : = i = 1 N α i = m 1 i = 1 N α i !
i = 1 N k i n x i α i f α x + θ k n x f α x d θ .
We see that
R m 0 1 1 θ m 1 α = m 1 i = 1 N α i !
i = 1 N k i n x i α i f α x + θ k n x f α x d θ
m 0 1 1 θ m 1 α = m 1 i = 1 N α i ! i = 1 N k i n x i α i
ω 1 f α , θ k n x d θ .
Notice here that
k n x 1 n β , iff k i n x i 1 n β , i = 1 , , N .
We assume (55) and we further see that
m ω 1 , m max f α , 1 n β 0 1 1 θ m 1 α = m 1 i = 1 N α i ! i = 1 N 1 n β α i d θ
= ω 1 , m max f α , 1 n β m ! n m β α = m m ! i = 1 N α i ! =
ω 1 , m max f α , 1 n β m ! n m β N m .
Conclusion: When k n x 1 n β , we proved that
R N m m ! n m β ω 1 , m max f α , 1 n β .
By (53) we have that
R m 0 1 1 θ m 1 α : = α 1 , , α N , α i Z + i = 1 , , N , α = m 1 i = 1 N α i !
i = 1 N k i n x i α i 2 f α d θ =
2 α = m 1 i = 1 N α i ! i = 1 N k i n x i α i f α
2 k n x m f α , m max m ! α = m m ! i = 1 N α i ! =
2 k n x m f α , m max N m m ! .
We proved in general that
R 2 k n x m f α , m max N m m ! .
Next we see that, let
θ n : = k = R Z n x k ,
then
θ n k = R Z n x k =
k = : k Z N : n x k < n 1 β R Z n x k + k = : k Z N : n x k n 1 β R Z n x k
by { ( 57 ) ,   ( 59 ) } N m m ! n m β ω 1 , m max f α , 1 n β +
2 f α , m max N m m ! k = : k Z N : n x k n 1 β k n x m Z n x k
N m m ! n m β ω 1 , m max f α , 1 n β +
2 f α , m max N m n m m ! k = : k Z N : n x k n 1 β n x k m Z n x k < ( 20 )
N m m ! n m β ω 1 , m max f α , 1 n β +
2 f α , m max N m n m m ! 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 =
N m m ! n m β ω 1 , m max f α , 1 n β +
4 f α , m max N m n m λ m q + 1 q e 2 λ e λ n 1 β 1 .
We have proved that
θ n N m m ! n m β ω 1 , m max f α , 1 n β +
4 f α , m max N m n m λ m q + 1 q e 2 λ e λ n 1 β 1 .
By (17) and (52), we have
A n f , x f x = k = f k n Z n x k f x =
j = 1 m α : α = j 1 i = 1 N α i ! f α x A n i = 1 N · x i α i x + θ n .
Consequently, it holds
A n f , x f x j = 1 m α : α = j 1 i = 1 N α i ! f α x A n i = 1 N · x i α i x
N m m ! n m β ω 1 , m max f α , 1 n β +
4 f α , m max N m n m λ m q + 1 q e 2 λ e λ n 1 β 1 .
Next, we estimate
A n i = 1 N · x i α i x =
k = Z n x k i = 1 N k i n x i α i
k = Z n x k i = 1 N k i n x i α i =
k = : k n x < 1 n β Z n x k i = 1 N k i n x i α i +
k = : k n x 1 n β Z n x k i = 1 N k i n x i α i
1 n β j + 1 n j k = : k n x 1 n β Z n x k i = 1 N n x i k i α i
1 n β j + 1 n j k = : n x k n 1 β n x k j Z n x k ( 20 )
1 n β j + 1 n j 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 .
We have proved that
A n i = 1 N · x i α i x
1 n β j + 1 n j 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 ,
for j = 1 , , m .
The proof of the theorem is now completed. □
We continue with the following result.
Theorem 4. 
Assume all the conditions are the same as in Theorem 3. Then
(1)
K n f , x f x j = 1 m α : α = j 1 i = 1 N α i ! f α x K n i = 1 N · x i α i x
N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β +
2 N m f α , m max n m m ! T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 = : Φ 2 * ,
(2) assume that f α x = 0 , for all α : α = j , j = 1 , , m , we have
K n f , x f x Φ 2 * ,
with the high speed of convergence 1 n + 1 n β m + 1 ,
(3)
K n f , x f x j = 1 N α : α = j 1 i = 1 N α i ! f α x
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 + Φ 2 * ,
and
(4)
K n f f j = 1 N α : α = j 1 i = 1 N α i ! f α
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 + Φ 2 * .
We have that K n f f , as n , pointwise and uniformly.
Proof. 
It holds that
f t + k n f x j = 1 m α : α = j 1 i = 1 N α i ! i = 1 N t i + k i n x i α i f α x = R ,
where
R : = m 0 1 1 θ m 1 α : α = m 1 i = 1 N α i !
i = 1 N t i + k i n x i α i f α x + θ t + k n x f α x d θ .
We see that
R m 0 1 1 θ m 1 α = m 1 i = 1 N α i !
i = 1 N t i + k i n x i α i f α x + θ t + k n x f α x d θ
m 0 1 1 θ m 1 α = m 1 i = 1 N α i ! i = 1 N t i + k i n x i α i
ω 1 f α , θ t + k n x d θ .
Notice that
k n x 1 n β , iff k i n x i 1 n β , i = 1 , , N .
Let k n x 1 n β .
Here, we consider 0 t i 1 n , i = 1 , , N .
We further see that
m ω 1 , m max f α , 1 n + 1 n β 0 1 1 θ m 1
α = m 1 i = 1 N α i ! i = 1 N 1 n + 1 n β α i d θ =
ω 1 , m max f α , 1 n + 1 n β m ! 1 n + 1 n β m α = m m ! i = 1 N α i ! =
ω 1 , m max f α , 1 n + 1 n β m ! 1 n + 1 n β m N m .
Conclusion: When k n x 1 n β , we proved that
R N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β .
By (73) we have that
R m 0 1 1 θ m 1 α : α = m 1 i = 1 N α i !
i = 1 N t i + k i n x i α i 2 f α d θ
m 0 1 1 θ m 1 α : α = m 1 i = 1 N α i !
i = 1 N 1 n + k i n x i α i 2 f α d θ =
α : α = m 1 i = 1 N α i ! i = 1 N 1 n + k i n x i α i 2 f α
2 k n x + 1 n m f α , m max m ! N m .
We proved in general that
R 2 k n x + 1 n m f α , m max N m m ! .
Next, we see that
n N 0 , 1 n N f t + k n d t f x j = 1 m α : α = j 1 i = 1 N α i ! f α x
n N 0 , 1 n N i = 1 N t i + k i n x i α i d t = n N 0 , 1 n N R d t .
So, when k n x 1 n β , we get
n N 0 , 1 n N R d t N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β .
And, in general it holds
n N 0 , 1 n N R d t
2 k n x + 1 n m f α , m max N m m ! .
Furthermore, it holds
K n f , x f x j = 1 N α : α = j 1 i = 1 N α i ! f α x K n i = 1 N · x i α i x =
k = n N 0 , 1 n N R d t Z n x k .
Call
U n * : = k = n N 0 , 1 n N R d t Z n x k .
When k n x < 1 n β , we clearly obtain that
U n * | k n x < 1 n β N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β .
And, it is
U n * | k n x 1 n β
2 f α , m max N m m ! k = : k n x 1 n β k n x + 1 n m Z n x k =
2 f α , m max N m n m m ! k = : n x k n 1 β 1 + n x k m Z n x k
2 m f α , m max N m n m m ! k = : n x k n 1 β 1 + n x k m Z n x k =
2 m f α , m max N m n m m !
k = : n x k n 1 β Z n x k + k = : n x k n 1 β n x k m Z n x k
(by (19) and (20))
2 m f α , m max N m n m m ! T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
That is
U n * | k n x 1 n β 2 m f α , m max N m n m m !
T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
Consequently it holds
U n * U n * | k n x < 1 n β + U n * | k n x 1 n β
N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β +
2 m f α , m max N m n m m ! T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
Finally, we estimate
K n i = 1 N · x i α i x =
k = n N 0 , 1 n N i = 1 N t i + k i n x i α i d t Z n x k
k = n N 0 , 1 n N i = 1 N t i + k i n x i α i d t Z n x k =
k = : x k n < 1 n β n N 0 , 1 n N i = 1 N t i + k i n x i α i d t Z n x k +
k = : x k n 1 n β n N 0 , 1 n N i = 1 N t i + k i n x i α i d t Z n x k
k = : x k n < 1 n β i = 1 N 1 n + 1 n β α i Z n x k +
1 n j k = : n x k n 1 β i = 1 N 1 + n x i k i α i Z n x k
1 n + 1 n β j + 1 n j k = : n x k n 1 β 1 + n x k j Z n x k
1 n + 1 n β j + 2 j 1 n j k = : n x k n 1 β 1 + n x k j Z n x k =
1 n + 1 n β j + 2 j 1 n j k = : n x k n 1 β Z n x k +
k = : n x k n 1 β n x k j Z n x k ( by   ( 19 ) ,   ( 20 ) )
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 .
We have found that
K n i = 1 N · x i α i x
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 ,
for j = 1 , , m .
The proof of the theorem now it is completed. □
Theorem 5. 
Assume all the conditions are the same as in Theorem 3. Then
(1)
Q n f , x f x j = 1 m α : α = j 1 i = 1 N α i ! f α x Q n i = 1 N · x i α i x
N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β +
2 N m f α , m max n m m ! T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 = : Φ 2 * ,
(2) assume that f α x = 0 , for all α : α = j , j = 1 , , m , we have
Q n f , x f x Φ 2 * ,
with the high speed of convergence 1 n + 1 n β m + 1 ,
(3)
Q n f , x f x j = 1 N α : α = j 1 i = 1 N α i ! f α x
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 + Φ 2 * ,
and
(4)
Q n f f j = 1 N α : α = j 1 i = 1 N α i ! f α
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 + Φ 2 * .
We have that Q n f f , as n , pointwise and uniformly.
Proof. 
We have that
f k n + r n θ f x j = 1 m α : α = j 1 i = 1 N α i ! i = 1 N k i n + r i n θ i x i α i f α x = R ,
where
R : = m 0 1 1 θ m 1 α : α = m 1 i = 1 N α i !
i = 1 N k i n + r i n θ i x i α i f α x + θ k n + r n θ x f α x d θ .
We see that
R m 0 1 1 θ m 1 α = m 1 i = 1 N α i !
i = 1 N k i n + r i n θ i x i α i f α x + θ k n + r n θ x f α x d θ
m 0 1 1 θ m 1 α = m 1 i = 1 N α i ! i = 1 N k i n + r i n θ i x i α i
ω 1 f α , θ k n + r n θ x d θ .
Notice that
k n x 1 n β , iff k i n x i 1 n β , i = 1 , , N .
We further see that
m ω 1 , m max f α , 1 n + 1 n β 0 1 1 θ m 1
α = m 1 i = 1 N α i ! i = 1 N 1 n + 1 n β α i d θ =
ω 1 , m max f α , 1 n + 1 n β m ! 1 n + 1 n β m α = m m ! i = 1 N α i ! =
ω 1 , m max f α , 1 n + 1 n β m ! 1 n + 1 n β m N m .
Conclusion: When k n x 1 n β , we proved that
R N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β .
By (99), we obtain that
R m 0 1 1 θ m 1 α : α = m 1 i = 1 N α i !
i = 1 N k n x i + 1 n α i 2 f α d θ =
α : α = m 1 i = 1 N α i ! i = 1 N k i n x i + 1 n α i 2 f α
2 k n x + 1 n m f α , m max m ! α = m m ! i = 1 N α i ! =
2 k n x + 1 n m f α , m max m ! N m .
We have extablished in general
R 2 k n x + 1 n m f α , m max N m m ! .
Next, we observe that
r = 0 θ w r f k n + r n θ f x j = 1 m α : α = j 1 i = 1 N α i ! f α x
r = 0 θ w r i = 1 N k i n + r i n θ i x i α i = r = 0 θ w r R .
So, when k n x 1 n β , we get
r = 0 θ w r R r = 0 θ w r R N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β .
And, it holds in general
r = 0 θ w r R 2 k n x + 1 n m f α , m max N m m ! .
Furthermore, it holds
Q n f , x f x j = 1 N α : α = j 1 i = 1 N α i ! f α x Q n i = 1 N · x i α i x =
k = r = 0 θ w r R Z n x k .
Call
E n * : = k = r = 0 θ w r R Z n x k .
We derive
E n * | k n x < 1 n β N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β .
And, furthermore we get that
E n * | k n x 1 n β 2 f α , m max N m m ! k = k n x + 1 n m Z n x u .
As in the proof of Theorem 4, we obtain
E n * | k n x 1 n β 2 m f α , m max N m n m m !
T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
Consequently it holds
E n * E n * | k n x < 1 n β + E n * | k n x 1 n β
N m m ! 1 n + 1 n β m ω 1 , m max f α , 1 n + 1 n β +
2 m f α , m max N m n m m ! T e 2 λ n 1 β + 2 m ! λ m q + 1 q e 2 λ e λ n 1 β 1 .
Finally, we estimate
Q n i = 1 N · x i α i x =
k = r = 0 θ w r i = 1 N k i n + r i n θ i x i α i Z n x k
k = r = 0 θ w r i = 1 N k i n x i + 1 n α i Z n x k =
k = i = 1 N k i n x i + 1 n α i Z n x k
k = : x k n < 1 n β i = 1 N k i n x i + 1 n α i Z n x k +
k = : x k n 1 n β i = 1 N k i n x i + 1 n α i Z n x k
k = : x k n < 1 n β i = 1 N 1 n β + 1 n α i Z n x k +
1 n j k = : n x k n 1 β i = 1 N 1 + n x i k i α i Z n x k
( as in Theorem 4 )
1 n + 1 n β j + 2 j 1 n j T e 2 λ n 1 β + 2 j ! λ j q + 1 q e 2 λ e λ n 1 β 1 ,
for j = 1 , , m .
The theorem is proved. □

4. Illustrations for m = 1

We present
Corollary 1. 
Let 0 < β < 1 , n N : n 1 β > 2 , x R N , f C 1 R N , N N , with f α C B R N , for all α : = α 1 , , α N , α i Z + ,   i = 1 , , N , and α = i = 1 N α i = l , where l = 0 , 1 . Then
(1)
A n f , x f x α : α = 1 1 i = 1 N α i ! f α x A n i = 1 N · x i α i x
N n β ω 1 , 1 max f α , 1 n β + 4 f α , 1 max N n λ q + 1 q e 2 λ e λ n 1 β 1 = : Φ 1 * ¯ ,
(2) assume that f α x = 0 , for all α : α = 1 , we have
A n f , x f x Φ 1 * ¯ ,
at the high speed of convergence n 2 β ,
(3)
A n f , x f x α : α = 1 1 i = 1 N α i ! f α x
1 n β + 2 n λ q + 1 q e 2 λ e λ n 1 β 1 + Φ 1 * ¯ ,
(4)
A n f f
α : α = 1 1 i = 1 N α i ! f α 1 n β + 2 n λ q + 1 q e 2 λ e λ n 1 β 1 + Φ 1 * ¯ .
We have that A n f f , as n , pointwise and uniformly.
Proof. 
By Theorem 3. □
We finish with
Corollary 2. 
Assume all the conditions are the same as in Corollary 1. Then
(1)
K n f , x f x α : α = 1 1 i = 1 N α i ! f α x K n i = 1 N · x i α i x Q n f , x f x α : α = 1 1 i = 1 N α i ! f α x Q n i = 1 N · x i α i x
N 1 n + 1 n β ω 1 , 1 max f α , 1 n + 1 n β +
2 N f α , 1 max n T e 2 λ n 1 β + 2 λ q + 1 q e 2 λ e λ n 1 β 1 = : Φ 2 * ¯ ,
(2) assume that f α x = 0 , for all α : α = 1 , we have
K n f , x f x Q n f , x f x Φ 2 * ¯ ,
with the high speed of convergence 1 n + 1 n β 2 ,
(3)
K n f , x f x Q n f , x f x α : α = 1 1 i = 1 N α i ! f α x
1 n + 1 n β + 1 n T e 2 λ n 1 β + 2 λ q + 1 q e 2 λ e λ n 1 β 1 + Φ 2 * ¯ ,
and
(4)
K n f f Q n f f α : α = 1 1 i = 1 N α i ! f α
1 n + 1 n β + 1 n T e 2 λ n 1 β + 2 λ q + 1 q e 2 λ e λ n 1 β 1 + Φ 2 * ¯ .
We have that K n f f , Q n f f , as n , pointwise and uniformly.
Proof. 
By Theorems 4 and 5. □

5. Conclusions

Here, we presented multivariate neural network approximation over infinite domains under differentiation of functions. The activation function was the symmetrized and perturbed hyperbolic tangent function. The rates of convergence were higher and the data needed to feed the neural network were half due to symmetry.

Funding

This research received no external funding.

Data Availability Statement

No new data were created or analyzed in this study.

Conflicts of Interest

The author declares no conflicts of interest.

References

  1. Anastassiou, G.A. Rate of convergence of some neural network operators to the unit-univariate case. J. Math. Anal. Appl. 1997, 212, 237–262. [Google Scholar] [CrossRef]
  2. Anastassiou, G.A. Quantitative Approximations; Chapman & Hall/CRC: Boca Raton, NY, USA, 2001. [Google Scholar]
  3. Chen, Z.; Cao, F. The approximation operators with sigmoidal functions. Comput. Math. Appl. 2009, 58, 758–765. [Google Scholar] [CrossRef]
  4. Anastassiou, G.A. Inteligent Systems II: Complete Approximation by Neural Network Operators; Springer: Heidelberg, Germany; New York, NY, USA, 2016. [Google Scholar]
  5. Anastassiou, G.A. Parametrized, Deformed and General Neural Networks; Springer: Heidelberg, Germany; New York, NY, USA, 2023. [Google Scholar]
  6. Haykin, S. Neural Networks: A Comprehensive Foundation, 2nd ed.; Prentice Hall: New York, NY, USA, 1998. [Google Scholar]
  7. McCulloch, W.; Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 7, 115–133. [Google Scholar] [CrossRef]
  8. Mitchell, T.M. Machine Learning; WCB-McGraw-Hill: New York, NY, USA, 1997. [Google Scholar]
  9. Yu, D.; Cao, F. Construction and approximation rate for feedforward neural network operators with sigmoidal functions. J. Comput. Appl. Math. 2025, 453, 116150. [Google Scholar] [CrossRef]
  10. Cen, S.; Jin, B.; Quan, Q.; Zhou, Z. Hybrid neural-network FEM approximation of diffusion coefficient in elliptic and parabolic problems. IMA J. Numer. Anal. 2024, 44, 3059–3093. [Google Scholar] [CrossRef]
  11. Coroianu, L.; Costarelli, D.; Natale, M.; Pantiş, A. The approximation capabilities of Durrmeyer-type neural network operators. J. Appl. Math. Comput. 2024, 70, 4581–4599. [Google Scholar] [CrossRef]
  12. Warin, X. The GroupMax neural network approximation of convex functions. IEEE Trans. Neural Netw. Learn. Syst. 2024, 35, 11608–11612. [Google Scholar] [CrossRef] [PubMed]
  13. Fabra, A.; Guasch, O.; Baiges, J.; Codina, R. Approximation of acoustic black holes with finite element mixed formulations and artificial neural network correction terms. Finite Elem. Anal. Des. 2024, 241, 104236. [Google Scholar] [CrossRef]
  14. Grohs, P.; Voigtlaender, F. Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces. Found. Comput. Math. 2024, 24, 1085–1143. [Google Scholar] [CrossRef]
  15. Basteri, A.; Trevisan, D. Quantitative Gaussian approximation of randomly initialized deep neural networks. Mach. Learn. 2024, 113, 6373–6393. [Google Scholar] [CrossRef]
  16. De Ryck, T.; Mishra, S. Error analysis for deep neural network approximations of parametric hyperbolic conservation laws. Math. Comp. 2024, 93, 2643–2677. [Google Scholar] [CrossRef]
  17. Liu, J.; Zhang, B.; Lai, Y.; Fang, L. Hull form optimization research based on multi-precision back-propagation neural network approximation model. Int. J. Numer. Methods Fluid 2024, 96, 1445–1460. [Google Scholar] [CrossRef]
  18. Yoo, J.; Kim, J.; Gim, M.; Lee, H. Error estimates of physics-informed neural networks for initial value problems. J. Korean Soc. Ind. Appl. Math. 2024, 28, 33–58. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Anastassiou, G.A. Multivariate Smooth Symmetrized and Perturbed Hyperbolic Tangent Neural Network Approximation over Infinite Domains. Mathematics 2024, 12, 3777. https://doi.org/10.3390/math12233777

AMA Style

Anastassiou GA. Multivariate Smooth Symmetrized and Perturbed Hyperbolic Tangent Neural Network Approximation over Infinite Domains. Mathematics. 2024; 12(23):3777. https://doi.org/10.3390/math12233777

Chicago/Turabian Style

Anastassiou, George A. 2024. "Multivariate Smooth Symmetrized and Perturbed Hyperbolic Tangent Neural Network Approximation over Infinite Domains" Mathematics 12, no. 23: 3777. https://doi.org/10.3390/math12233777

APA Style

Anastassiou, G. A. (2024). Multivariate Smooth Symmetrized and Perturbed Hyperbolic Tangent Neural Network Approximation over Infinite Domains. Mathematics, 12(23), 3777. https://doi.org/10.3390/math12233777

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop