Abstract
In this article, for the first time, the univariate symmetrized and perturbed hyperbolic tangent activated convolution-type operators of three kinds are introduced. Their approximation properties are presented, i.e., the quantitative convergence to the unit operator via the modulus of continuity. It follows the global smoothness preservation of these operators. The related iterated approximation as well as the simultaneous approximation and their combinations, are also extensively presented. Including differentiability and fractional differentiability into our research produced higher rates of approximation. Simultaneous global smoothness preservation is also examined.
Keywords:
symmetrized and perturbed hyperbolic tangent; convolution-type operator; Caputo fractional derivative; quantitative approximation; global smoothness preservation; simultaneous approximation; iterated approximation MSC:
26A33; 41A17; 41A25; 41A35; 47A58
1. Organization
In Section 2, we give the preliminaries of our theory. In Section 3 are the basics, the introduction of our activated symmetrized and perturbed hyperbolic tangent convolution-type operators with properties. In Section 4 are the main approximation results. We also include the global smoothness preservation by our operators there. We further study the differentiation of these operators, as well as introducing their iterates and giving their basic properties. Next, we present the convergence of our operators under differentiability and Caputo fractional differentiability, achieving higher rates of approximation. It follows the simultaneous differential approximation and simultaneous global smoothness preservation in detail, as well as the iterated approximation. We finish with the combination of the simultaneous and iterated approximations.
We are motivated and inspired by [1,2,3,4,5,6,7,8,9,10,11,12].
2. About -Deformed and -Parameterized Hyperbolic Tangent Function
Here, all of this initial background comes from Chapter 18 of [1].
We use (see (1)), exhibit that it is a sigmoid function, and we will present several of its properties related to the approximation by neural network operators.
So, let us consider the hyperbolic tangent activation function
We have that
We notice also that
That is,
and
hence,
It is
i.e.,
Furthermore,
i.e.,
We find that
therefore, is strictly increasing.
Next, we obtain ()
We observe that
So, in case of , we have that is strictly concave up, with .
And, in case of , we have that is strictly concave down.
Clearly, is a shifted sigmoid function with , and , (a semi-odd function).
By and , we consider the function
; . Notice that , so the x-axis is horizontal asymptote.
We have that
Thus,
a deformed symmetry.
Next, we have that
Let , then and (by being strictly concave up for ), i.e., . Hence, is strictly increasing over .
Now, let ; then, , and , i.e., .
Therefore, is strictly decreasing over .
Let us next consider . We have that
By .
By .
Clearly, by (13) we see that , for .
More precisely, is concave down over , and strictly concave down over .
Consequently, has a bell-type shape over .
Of course, it holds that .
At , we have
Thus,
That is, is the only critical number of over . Hence, at , achieves its global maximum, which is
Conclusion: The maximum value of is
We mention the following.
Theorem 1
([1], Ch. 18, p. 458). We have that
Also, the following holds.
Theorem 2
([1], Ch. 18, p. 459). It holds that
Thus, is a density function on ; .
Similarly, we see that
thus, is a density function.
Furthermore, we observe the symmetry
Furthermore,
is a new density function over , i.e.,
Clearly, then,
3. Basic
We give
Definition 1.
Let (continuous and bounded functions on ), . We define the following basic activated hyperbolic tangent perturbed convolution-type operators,
In this work, we examine the quantitative convergence of to the unit operator.
We study similarly the activated Kantorovich-type operators,
where , , ∀, and the activated Quadrature operators
where , ; , , ∀.
An essential property follows.
Theorem 3.
Let , . Then,
Proof.
We have that , ∀.
Let , i.e., . Applying the mean value theorem, we obtain
where .
That is,
Similarly, it holds that
Hence, we see that
Consider the set
We have that
The claim is proven. □
The first modulus of continuity here is
4. Main Results
We present the following approximation results.
Theorem 4.
Let , , , . Then,
and
So, for (where the uniformly continuous functions on ), we see that , pointwise and uniformly.
Proof.
Call
and
That is, .
We have that
□
Theorem 5.
Let , , , . Then,
and
For , we have , pointwise and uniformly.
Theorem 6.
Let , , , . Then,
and
For , we have , pointwise and uniformly.
We need
Proposition 1.
It holds that (), and
Proof.
We can write
□
Next we describe the global smoothness preservation property of our activated operators.
Theorem 7.
Here, . Then,
If , then .
Remark 1.
Clearly, (54) is attained by identity map .
We have
and
We also see that
So, for fixed , we have
Of course, .
Proof.
We notice that (),
and
We make the following remark.
Remark 2.
Let be fixed. Assume that , with , for .
We have that
By repeatedly applying Leibnitz’s rule, we obtain
Clearly, it is valid that
and
∀.
So, all of our results in this work can be written in the simultaneous approximation context; see Remark 7 and Theorems 16–19.
We make the following remark about iterated convolution.
Remark 3.
We have that
where .
Let , as , and
We have that
Furthermore, it holds that
by dominated convergence theorem, because we have that
and is integrable over , ∀.
Hence, .
Furthermore, it holds that
i.e.,
So, is a bounded positive linear operator.
Clearly it holds that
And, for , we obtain
so the contraction property valid, and is a bounded linear operator.
Remark 4.
Let . We observe that
Then,
Therefore,
Now, let , and , as above.
Consequently, it holds, as in Chapter 2 of [1], that
Next, we have
Remark 5.
.
Let , as , and
This is true by the bounded convergence theorem, and we see that
and
and is finite interval. Thus,
Therefore, it holds that
Further,
as , ∀.
Furthermore, we have
with being integrable over .
Therefore, by the dominated convergence theorem,
Hence, is bounded and continuous in .
Remark 6.
Next, we observe the following. Let , and
Let , as . Then,
The last comes by the dominated convergence theorem,
and
and
as , ∀.
Furthermore, it holds that
in which the function is integrable over .
Therefore,
Hence, is bounded and continuous in .
See the related Theorems 20 and 21 later.
Next, we greatly improve the speed of convergence of our activated operators by using the differentiation of functions.
First, we treat the basic ones.
Theorem 10.
Let , ; , , , with . Then,
(i)
(ii) If , , we have
(iii)
(iv)
Proof.
We have that
Then,
Hence,
Thus, it holds that
Call
Call
Let .
(i) The case where . Then,
(ii) The case where . Then,
Therefore, it holds that
when .
Consequently, we see that
Next, we treat
Let ; then,
If , then
Consequently, we see that
Furthermore, we obtain
(here, it is )
We have proved that
Finally, we have that
Let and . Then,
as .
The theorem now is proven. □
We continue with the activated Kantorovich operators.
Theorem 11.
Let , ; , , , with . Then,
(i)
(ii) If , , we have
(iii)
(iv)
as .
Proof.
We have that
and
Hence,
Furthermore, it holds that
where
Call
Let , ().
(i) The case where . Then,
Therefore, it holds that
(ii) The case where . Then,
(by )
So, when , we have proved that
Consequently, we see that
Next, we treat
We also have that
(let )
So, when , we obtain
If , then
Consequently, we see that
Furthermore, we obtain
(here, it is )
as .
We have proved that
Finally, we have that
We also observe that ()
The theorem is proven. □
We continue with the activated Quadrature operators.
Theorem 12.
Let , ; , , , with . Then,
(i)
(ii) If , , we have
(iii)
(iv)
as .
Proof.
We have that
and
Furthermore, it holds that
and it is
where
Let
Let .
(i) The case where . Then,
(ii) The case where . Then,
So, when , we see that
Consequently, we have that
Next, we treat
Let , then
Let , then
Consequently, we see that
(see also (127)).
The quantity
is estimated as in Theorem 11.
The theorem is proven. □
We need the following.
Definition 2.
A function is absolutely continuous over , iff is absolutely continuous, for every . We write , iff (absolutely continuous functions over ), .
Definition 3.
Let , ( is the ceiling of the number), . We call left Caputo fractional derivative ([13,14,15], pp. 49–52), the function
∀, , where Γ is the gamma function.
Notice that and exists almost everywhere on , ∀.
We set , ∀.
We need the following.
Lemma 1
(see also [16]). Let , , , and . Then, for any .
Definition 4
(see also [14,17,18]). Let , , . The right Caputo fractional derivative of order is given by
∀, . We set .
Notice that and exists almost everywhere on , ∀.
Lemma 2
(see also [16]). Let , , , . Then, , for any .
We assume that
We mention the following.
Proposition 2
(see also [16]). Let , , . Then, is continuous in , .
Also we have the following.
Proposition 3
(see also [16]). Let , , . Then, is continuous in , .
We further mention the following.
Proposition 4
(see also [16]). Let , , , and let . Then, is continuous in .
Proposition 5
(see also [16]). Let , , , , and let . Then, is continuous in .
Proposition 6
(see also [16]). Let , , ; . Then, , are jointly continuous functions in from .
Here comes our first main fractional result.
Theorem 13.
Let , , , , , , , . Assume also that both , .
Then,
(i)
(ii) Given that , , we have
(iii)
(iv)
Above, when , the sum .
As we see here, we obtain fractional pointwise and uniform convergence with rates of the unit operator, as .
Proof.
Let . We have that .
From [13], p. 54, we obtain the left Caputo fractional Taylor formula that
for all .
Also, from [17], using the right Caputo fractional Taylor formula, we obtain
for all .
Hence,
for all , and
for all .
Therefore, we have
and
Consequently, it holds that
where
∀.
Denote by
∀, and
∀.
That is,
Let first .
Call
and
Let us assume that ; then,
Hence,
Let us assume that ; then,
Hence,
Consequently, we see that
and
Furthermore, we have that
where , and
where .
Next, we see that
We have proved that
Next, we see that
(as before)
Therefore, it holds that
Consequently, it holds that
We need , iff , iff .
(i) In the case of (i.e., ), then , and . So, for large enough , we can have .
(ii) If (i.e., ), then and . So, for any , we have that .
See also the assumption.
We have that
and
∀.
Therefore, it holds that
, and
.
Thus, it is reasonable to assume that , .
Consequently, it holds that
The theorem is now proven. □
We continue with the following results.
Theorem 14.
Let , , , , , , , . Assume also that both , .
Then,
(i)
(ii) Given that , , we have
(iii)
(iv)
Above, when , the sum .
So, we obtain the pointwise and uniform convergence with rates of , as .
Proof.
Let . We have that .
We can write
for all .
Also it holds that
for all .
Hence, we see
for all , iff .
Also, it holds that
for all , iff .
Therefore, we obtain
and
Let .
Call
and
We have that
and
Consequently, we see that
and
We continue as follows:
and
We have that
So, it holds that
Similarly, we obtain
At the end, we see that
The theorem is proven. □
We also present the following.
Theorem 15.
Let , , , , , , , . Assume also that both , .
Then,
(i)
(ii) Given that , , we have
(iii)
(iv)
Above, when , the sum .
So, we obtain the pointwise and uniform convergence with rates of , as .
Proof.
Let . We have that .
We can write
for all , .
Also, it holds that
for all , .
Hence,
and
Furthermore, it holds that
and
Let .
Call
and
Then,
Therefore, it holds that
Furthermore, wee see that
That is,
Consequently, we see that
and
Furthermore, it holds that
and
The rest of the proof is similar to Theorem 14. As such, it is omitted. □
Next, we discuss briefly simultaneous approximation.
Remark 7.
Let be fixed. Assume that , with , for . We derive from Remark 2 that
for all .
We give the following.
Theorem 16.
Let , , , , for . Then,
(I)
and
for all ;
(II)
and
for ;
(III)
and
for .
Proof.
Based on Theorems 4–6. □
We continue with simultaneous global smoothness preservation.
Theorem 17.
Here, , for . Then,
(I)
and if , then ;
(II)
and if , then ;
(III)
and if , then .
Proof.
By Theorems 7–9. □
The related simultaneous differentiation results follow.
Theorem 18.
Let , ; , ; and , , with . Then,
(I)
(II)
(III)
Proof.
By Theorems 10–12. □
We continue with simultaneous fractional results.
Theorem 19.
Let , , , ; , , , , . Assume also that both , .
Then,
(I)
(II)
(III)
Proof.
By Theorems 13–15. □
In the final part of this work we present results related to activated iterated approximation. This is a continuation of Remarks 4–6.
Theorem 20.
Let , , , , . Then,
(I)
(II)
(III)
So, the speed of convergence of , , to unit I is not worse than the speed of convergence of , , to I.
Proof.
By Theorems 4–6 and (75). □
We continue with the following.
Theorem 21.
Let ; , with , ; ; . Then,
(I)
(II)
(III)
Clearly, we notice that the speed of convergence to the unit operator of the above activated multiply iterated operators is not worse the speed of operators , , to the unit, respectively.
Proof.
By Theorems 4–6 and (77). □
We finish our work with simultaneous iterations.
Remark 8.
Let be fixed. Assume that , with , for ; . Then, by (75), we obtain
Similarly, we derive that
and
for .
Similarly, we see that
for , and
for .
To keep our work short, we stop here.
5. Conclusions
Here, we presented a new idea for going from neural networks’ main tools, the activation functions, to convolution integrals approximation. This is a rare case of employing applied mathematics to theoretical ones.
Funding
This research received no external funding.
Data Availability Statement
No new data were created or analyzed in this study.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Anastassiou, G.A. Parametrized, Deformed and General Neural Networks; Springer: Heidelberg, Germany; New York, NY, USA, 2023. [Google Scholar]
- Anastassiou, G.A.; Gal, S.G. Approximation Theory; Birkhäuser: Boston, MA, USA; Basel, Switzerland; Berlin, Germany, 2000. [Google Scholar]
- Wang, Y.; Wang, T.; Liam, H. The series expansions and blow-up time estimation for the solutions of convolution Volterra-Hammerstein integral equations. Numer. Algorithms 2024, 95, 637–663. [Google Scholar] [CrossRef]
- Askhabov, S.N. A system of Inhomogeneous integral equations of convolution type with power nonlinearity. Syb. Math. J. 2023, 64, 691–698. [Google Scholar] [CrossRef]
- Wang, M.; Dai, X.; Yu, Y.; Xiao, A. Fast θ-Maruyama scheme for stochastic Volterra integral equations of convolution type: Mean-square stability and strong convergence analysis. Comput. Appl. Math. 2023, 42, 36. [Google Scholar] [CrossRef]
- Angeloni, L.; Merentes, N.J.; Valera-López, M.A. Convolution integral operators in variable bounded variation spaces. Mediterr. J. Math. 2023, 20, 20. [Google Scholar] [CrossRef]
- Askhabov, S.N. System of inhomogeneous integral equations of convolution type with power nonlinearity. Vladikavkazskii Mat. Zhurnal 2022, 24, 5–14. (In Russian) [Google Scholar] [CrossRef]
- Khachatryan, K.A.; Petrosyan, A.S. Alternating bounded solutions of a class of nonlinear two-dimensional convolution-type integral equations. Trans. Moscow Math. Soc. 2021, 82, 259–271. [Google Scholar] [CrossRef]
- Askhabov, S.N. Nonlinear convolution type integral equations in complex spaces. Ufa Math. J. 2021, 13, 17–30. [Google Scholar] [CrossRef]
- Safaryan, M.H. On generalizations of Fatou’s theorem in Lp for convolution integrals with general kernels. J. Geom. Anal. 2021, 31, 3280–3299. [Google Scholar] [CrossRef]
- Karchevskii, A.L. Solution of the convolution type Volterra integral equations of the first kind by the quadtrature-sum method. Sib. Zh. Ind. Mat. 2020, 23, 40–52. Translation in J. Appl. Ind. Math. 2020, 14, 503–512. (In Russian) [Google Scholar] [CrossRef]
- Karapetrović, B.; Mashreghi, J. Hadamard convolution and area integral means in Bergman spaces. Results Math. 2020, 75, 11. [Google Scholar] [CrossRef]
- Diethelm, K. The Analysis of Fractional Differential Equations; Lecture Notes in Mathematics 2004; Springer: Berlin/Heidelberg, Germany, 2010. [Google Scholar]
- Frederico, G.S.; Torres, D.F.M. Fractional Optimal Control in the sense of Caputo and the fractional Noether’s theorem. Int. Math. Forum 2008, 3, 479–493. [Google Scholar]
- Samko, S.G.; Kilbas, A.A.; Marichev, O.I. Fractional Integrals and Derivatives, Theory and Applications; Gordon and Breach: Amsterdam, The Netherlands, 1993; [English translation from the Russian, Integrals and Derivatives of Fractional Order and Some of Their Applications (Nauka i Tekhnika, Minsk, 1987)]. [Google Scholar]
- Anastassiou, G.A. Fractional Korovkin theory. Chaos Solitons Fractals 2009, 42, 2080–2094. [Google Scholar] [CrossRef]
- Anastassiou, G.A. On Right Fractional Calculus. Chaos Solitons Fractals 2009, 42, 365–376. [Google Scholar] [CrossRef]
- El-Sayed, A.M.A.; Gaber, M. On the finite Caputo and finite Riesz derivatives. Electron. J. Theor. Phys. 2006, 3, 81–95. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).