Abstract
A reproducing kernel Hilbert space (RKHS) approximation problem arising from learning theory is investigated. Some K-functionals and moduli of smoothness with respect to RKHSs are defined with Fourier–Bessel series and Fourier–Bessel transforms, respectively. Their equivalent relation is shown, with which the upper bound estimate for the best RKHS approximation is provided. The convergence rate is bounded with the defined modulus of smoothness, which shows that the RKHS approximation can attain the same approximation ability as that of the Fourier–Bessel series and Fourier–Bessel transform. In particular, it is shown that for a RKHS produced by the Bessel operator, the convergence rate sums up to the bound of a corresponding convolution operator approximation. The investigations show some new applications of Bessel functions. The results obtained can be used to bound the approximation error in learning theory.
1. Introduction
The error analysis in learning theory shows that the learning rate of the kernel regularized regression depends upon the approximation ability of the kernel function spaces (see, for example, [1,2,3]).
Let X be a complete metric space and be a Borel measure on X. Denoted by , the Hilbert space consisting of (real) square integrable functions with the inner product
Suppose that is continuous, symmetric and strictly positive definite, i.e., for any given integers are positive definite matrices for given finite sets Assume that , i.e.,
Then the linear operator defined by
is positive, and its range lies in . Take to be the linear operator on satisfying and , the inverse of . Additionally, define . Then is a reproducing kernel Hilbert space associated with , i.e., (see [1,4,5,6,7]),
where the inner product is induced by a norm defined as
i.e.,
One of the targets of learning theory is to find an unknown function from the random observations drawn i.i.d. (identically and independently distributed) according to a unknown probability defined on (see [1,6]). A usual algorithm to realize this aim is to solve the following kernel regularized optimization problem:
where is taken as the hypothesis space, is a parameter which balances the relationship between the empirical error term and the penalty term . Let be the regression function. Then is the least-squares-best predictor (see Section 9.4 in Section 9 of [8]), i.e.,
It is known that the convergence analysis of model (5) sums up to bound the convergence rate for error , which depends upon the decay of the best approximation defined as (see e.g., [1,2,6])
as
Formula (6) deals with a decay rate which depends upon the approximation property of . Many mathematicians have performed investigations on it. For example, D. X. Zhou gives the decay of (6) with the RKHS interpolation theory (see [2,3]). P.X. Ye gives the decay using convolution operators in the Euclidean space (see [9]). H.W. Sun gives a decay for (6) with the help of operator theory in a Hilbert space (see [10]). It is known that the Fourier–Bessel series is a good approximation tool and has been studied by many mathematicians (see for example, [11,12,13,14,15,16]). Additionally, we found that approximation by RBF networks of Delsarte translates was studied by some mathematicians. The essence of RBF is summed up as the approximation of Fourier–Bessel transforms (see, for example, [17,18,19,20]). So it is of interest for us to conduct investigations on the decay of with both the Fourier–Bessel series and the Fourier–Bessel transforms.
Let and be given real numbers, and denote the space of all measurable real functions on such that
where The normalized Bessel function of the first kind and order is
where
is the Bessel function of first kind and order , and is the Gamma function.
For , the usual Fourier–Bessel transform is defined as
In the present paper, some investigations on the decay of in the case that are constructed with and are provided. Some K-functional and moduli of smoothness are defined with the help of the semigroup of operators, and their equivalences are shown, with which the error for the decay is bounded. The results obtained are two kinds of upper bound estimates associated with Fourier–Bessel series and Fourier–Bessel transforms, respectively.
The paper is organized as follows. In Section 2, some notions and results of the Fourier–Bessel series and Fourier–Bessel transforms are provided, with which two kinds of RKHSs are constructed; the corresponding best RKHS approximation problem in these setting is restated. Some K-functionals and moduli of smoothness associated with Fourier–Bessel series and Fourier–Bessel transforms are provided, and their equivalence is shown, with which some upper bounds for the best approximation are shown in Section 3 and Section 4, respectively. All the proofs for the propositions, the theorems and lemmas are given in Section 5. Some further analysis for the results of the present paper are given in Section 6, from which one can see the value of writing this manuscript. A general proposition for the strong equivalence of K-functionals and moduli of smoothness is listed in the Appendix A.
2. Preliminaries
Let be the positive zeros of arranged in increasing order. It is well known that form a complete orthogonal system in (see, for example, [12,16,21]), i.e.,
Take . Then
forms an orthonormal basis of and for any , there holds Fourier–Bessel series
where and
Lemma 1.
We have the following results:
- (i)
- Let . Then
- (ii)
- The generalized translation operator on defined ashas the expansion ofwhere and
- (iii)
- The zeros satisfy
Proof.
See it from Section 5. □
Inequality (13) is a theoretical basis for defining the moduli of smoothness with translation operators .
Let be the set of given positive real sequences such that the right side of the series
has uniform convergence for all . It therefore is a Mercer kernel. Then
Take
Then it is easy to verify that , and
is a RKHS in associating with reproducing kernel and an inner product defined as
Since
we have
Equality (6) becomes
as
Let be the class of even -functions on . Denoted by , the space of even -functions on R which are rapidly decreasing together with all their derivatives, i.e.,
where is the set of natural numbers.
Let denote the space of even -functions on R with support in and
Additionally, define the generalized translation operator on as
and define a convolution on by
For the Bessel operators
we have (see p. 12 or p. 177 of [22])
and therefore
Moreover, we have the following lemma.
Lemma 2.
There hold the following:
- (i)
- is dense in ;
- (ii)
- Both and are dense in and
- (iii)
- If , then and ;
- (iv)
- is a topological isomorphism from to itself and
- (v)
- There holdandIt follows
- (vi)
- If , then
- (vii)
- Let or . Then
- (viii)
- There hold the following relations
Proposition 2.1 of [23] shows that if satisfies and , then
defines a Mercer kernel on . We give an assumption
Assumption I.
Letsatisfyand for anythere is a real numbersuch that
We point here that the functionssatisfying Assumption 1 are existent, and give two examples.
Example 1.
For the function defined by
satisfies and for (see Problem 5. VIII 2 in Section 5.VIII Problems of [22]).
Example 2.
For the function defined by
satisfies and for (see Problem 5. VIII 1 in Section 5.VIII Problems of [22]).
Define
with norm
Define an inner product on as
It is known that is a reproducing kernel of (see [24]), i.e.,
We have
Defi
ne for a given real number an operator as
Then it is easy to show that ,
and
In this case, the decay (6) becomes
for
If , then we define the corresponding RKHS
and for , there holds
We have by (34) that
for
3. An Upper Bound Estimate with Fourier–Bessel Series
Then we have the following Proposition 1 whose proofs can be found from Section 5.
Proposition 1.
There holds an equivalent relation
Proof.
See it from Section 5. □
Theorem 1.
There is a constant such that
if
Proof.
See it from Section 5. □
Taking into (15), we have a kernel
It follows that
which shows that and
and
where
We have two corollaries.
Corollary 1.
For any , there holds
Corollary 2.
For any , there holds
4. An Upper Bound Estimate with the Fourier–Bessel Transform
To bound , we define a K-functional and a modulus respectively corresponding to as
and
where
The K-functional and the modulus are equivalent, i.e., we have the following proposition.
Proposition 2.
Let satisfy Assumption 1. Then there holds the equivalence
We now give an upper bound estimate for (34).
Theorem 2.
Under the conditions of Proposition 2, there is a constant such that
if
For we define a K-functional on as
Define a modulus of smoothness as
where
Then we have the following two corollaries.
Corollary 3.
There holds the equivalent relation
Corollary 4.
There is a constant such that
We give further computations for . By Example 1, we know , which, together with (21), gives
which with (42) shows that
Take (43) into (42). Then
(44) shows that the decay of is controlled by the approximation order of convolution operator for .
For we define
Then
Define a K-functional on as
Define a modulus of smoothness as
where
Then we have the following two corollaries.
Corollary 5.
There holds
Corollary 6.
There is a constant such that
We know by (49) that the decay of is controlled by the approximation order of the convolution operator for
5. Proofs
Proof of Lemma 1.
Proof of Lemma 2.
Proof of (i). See Proposition 2.III.1 in P51 of [22].
Proof of (ii). See Corollary 4.III.2 in P104 and Corollary 4.III.3 in P105 of [22].
Proof of (iii). See Theorem 5.III.1 in P127 and Proposition 5.II.4 in P129 of [22].
Proof of (iv). See Theorem 5.III.1 in P127 and (5.III.3) in P128 of [22].
Proof of (v). See Proposition 5.II.2 in P120 of [22] and (4.III.10) in Proposition 4.III.4 of [22].
Proof of (vi). See Theorem 5.II.2 in P126 of [22].
Proof of (vii). See (5.III.5) and (5.III.6) in Proposition 5.III.2 in P128,(5.V.2) in P139 of [22], and Proposition 2.2 in [25].
Proof of Proposition 1.
We show it with the help of Proposition A1 in the Appendix A.
It follows
Proof of Theorem 1.
Because defining
we have for any that
and
Since the arbitrariness of , we have
Take . Then and
Take . Then . By the definition of , we have (39). □
Proof of Proposition 2.
It is easy to see that
It follows by (26) again that
By the same method, we have
6. Further Discussions
We now give some comments on the results obtained in the present paper.
A more general problem arising from learning theory is to bound the decay rate of the function (see [2])
where is a Banach space and is a dense subspace with for
It is known that the approximation ability of a function class is determined by the smoothness of its functions. So the decay of is influenced by the smoothness of the functions in
Smale and Zhou (see [2]) give the first estimate for the decay of (61) in the case that , which is a particular Besov space (in fact, it is the interpolation space of B and H). This work is improved in [9]. For (the Sobolev space, see [2] for the definition) and the reproducing kernel Hilbert space , Zhou gives an estimate as (see [3])
if , where is the Gaussian kernels
The tools used is the RKHS function interpolation.
It is known that the most commonly used tool in approximation theory is the K-functional. The most helpful relation is the strong equivalent relation between a K-functional and a corresponding modulus of smoothness (see, for example, [26]). The most commonly used quantity for describing the approximation ability of a function class is the Jackson inequality expressed with a K-functional or a modulus of smoothness (see also [26]). As far as we know from the literature, no Jackson inequality has been established for the decay of (6). There is little description for the smoothness of a RKHS. Recent research shows that any RKHS has some smoothness; it can be considered from the view of fractional derivative and orthogonal series and show that the well-known K-functional ([27])
is equivalent to a modulus of smoothness, where X is chosen as some compact sets, for example, and . It is valuable for us to extend these results to the RKHS defined on a noncompact set. The set X used in the present paper is , which is a noncompact set and has essential properties different from those of a compact set (see, for example, [5]). Moreover, it is the first time that a Jackson inequality is established to describe the decay (6). A advantage of this manuscript is the use of the Bessel series and Bessel transforms, which transforms the RKHS approximation problem into the classical Bessel–Fourier approximation problem and gives the decay rate with Bessel–Fourier approximation skills.
The Jackson inequalities in Theorem 1 and Theorem 2 show that the RKHSs constructed with Bessel series and Bessel transforms have the same approximation as that of the Bessel series and Bessel transforms.
The moduli of smoothness defined in this manuscript are one-order moduli. It is a valuable problem for us to define higher-order moduli of smoothness and show the Jackson inequality to describe the decay of (6).
Author Contributions
Formal analysis, B.S.; Investigation, S.W.; Writing—original draft, M.T. All authors have read and agreed to the published version of the manuscript.
Funding
This work is supported partially by the NSF (Project No. 61877039), the NSFC/RGC Joint Research Scheme (Project No. 12061160462 and N-CityU102/20) of China, the NSF (Project No. LY19F020013) of Zhejiang Province, the Science and Technology Project in Jiangxi Province Department of Education (Project No. GJJ211334).
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
Not applicable.
Conflicts of Interest
The authors declare no conflict of interest.
Appendix A
It is known that the moduli of smoothness defined by a semi-group of operators have the same properties as those of the usual moduli of smoothness defined by the difference of the function (see Chapter Two of [28]) and have been used to describe the degree of approximation in approximation theory (see, for example, [27,29,30,31,32]). We restate here a proposition for a general strong equivalent relation.
Let be a normed linear space, be a strongly continuous semi-group of operators satisfying
and
The infinitesimal generator E is given by
whenever the limit exists. is the domain of E. Then we have the following proposition.
References
- Cucker, F.; Smale, S. On the mathematical foundations of learning. Bull. Amer. Math. Soc. 2001, 39, 1–49. [Google Scholar] [CrossRef] [Scilit]
- Smale, S.; Zhou, D.X. Estimating the approximation error in learning theory. Anal. Appl. 2003, 1, 17–41. [Google Scholar] [CrossRef] [Scilit]
- Zhou, D.X. Density problem and approximation error in learning theory. Abstr. Appl. Anal. 2013, 715683. [Google Scholar] [CrossRef] [Scilit]
- Aronszajn, N. Theory of reproducing kernels. Trans. Amer. Math. Soc. 1950, 68, 337–404. [Google Scholar] [CrossRef]
- Sun, H.W. Mercer theorem for RKHS on noncompact sets. J. Complex. 2005, 21, 337–349. [Google Scholar] [CrossRef] [Scilit]
- Cucker, F.; Zhou, D.X. Learning Theory: An Approximation Theory Viewpoint; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
- Ferreiar, J.C.; Menegatto, V.A. Reproducing kernel Hilbert spaces associated with kernels on topological spaces. Funct. Anal. Appl. 2012, 46, 89–91. [Google Scholar] [CrossRef] [Scilit]
- Williams, D. Probability with Martingales; Cambridge University Press: Cambridge, UK, 1990. [Google Scholar]
- Ye, P.X. Some Approximation Problems in Learning Theory, Post-Doctoral Research Work Report; Chinese Academy of Sciences: Bejing, China, 2003. [Google Scholar]
- Sun, H.W. Behavior of a functional in learning theory. Front. Math. China 2007, 2, 455–465. [Google Scholar] [CrossRef] [Scilit]
- Abilov, V.A.; Abilova, F.V. Approximation of functions by Fourier-Bessel sums. Izv. Vyssh. Uchebn. Zaved. Math. 2001, 8, 3–9. [Google Scholar]
- Abilov, V.A.; Abilova, F.V.; Kerimov, M.K. Some issues concerning approximation of functions by Fourier-Bessel sums. Comput. Math. Math. Phy. 2013, 53, 867–873. [Google Scholar] [CrossRef] [Scilit]
- Abilov, V.A.; Abilova, F.V.; Kerimov, M.K. Sharp estimates for the convergence rate of Fourier-Bessel series. Comput. Math. Math. Phy. 2015, 55, 907–916. [Google Scholar] [CrossRef] [Scilit]
- Abilov, V.A.; Abilova, F.V.; Kerimov, M.K. On sharp estimates of the convergence of double Fourier-Bessel series. Comput. Math. Math. Phy. 2017, 57, 1735–1740. [Google Scholar] [CrossRef] [Scilit]
- Abilov, V.A.; Kerimov, M.K. Some estimates for the error in mixed Fourier-Bessel expansions of functions of two variables. Comput. Math. Math. Phy. 2006, 46, 1465–1486. [Google Scholar] [CrossRef] [Scilit]
- Hochstadt, H. The mean convergence of Fourier-Bessel series. SIAM Rev. 1967, 9, 211–218. [Google Scholar] [CrossRef] [Scilit]
- Arteaga, C.; Marrero, I. Universal approximation by radial basis function networks of Delsarte translates. Neural Netw. 2013, 46, 299–305. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Arteaga, C.; Marrero, I. Approximation in weighted p-mean by RBF networks of Delsarte translates. J. Math. Anal. Appl. 2014, 414, 450–460. [Google Scholar] [CrossRef] [Scilit]
- Dai, F.; Wang, H.P. Interpolation by weighted Paley-Wiener spaces associated with the Dunkl transform. J. Math. Anal. Appl. 2012, 390, 556–572. [Google Scholar] [CrossRef] [Scilit]
- Marrero, I. Radial basisi function neural networks of Hankel translates as universal approximation. Anal. Appl. 2019, 17, 897–930. [Google Scholar] [CrossRef] [Scilit]
- Vladimirov, V.S. Equations of Matheamtical Physics; Marcel Dekker: New York, NY, USA, 1971. [Google Scholar]
- Triméche, K. Generalized Harmonic Analysis and Wavelet Packets; Gordon and Breach Science Publishers: Singapore, 2001. [Google Scholar]
- Sheng, B.H. The weighted norm for some Mercer kernel matrices. Acta Math. Sci. 2013, 33A, 6–15. (In Chinese) [Google Scholar]
- Sheng, B.H.; Zuo, L. Error analysis of the kernel regularized regression based on refined convex losses and RKBSs. Int. J. Wavelets Multiresolut. Inform. Process 2021, 19, 2150012. [Google Scholar] [CrossRef] [Scilit]
- Quadih, S.E.; Daher, R. Estimates for the generalized Fourier-Bessel transform in the space . Internat. J. Math. Model Comput. 2016, 6, 269–275. [Google Scholar]
- Ditzian, Z.; Totik, V. Moduli of Smoothness; Springer: New York, NY, USA, 1987. [Google Scholar]
- Sheng, B.H.; Wang, J.L. On the K-functional in learning theory. Anal. Appl. 2020, 18, 423–446. [Google Scholar] [CrossRef] [Scilit]
- Butzer, P.L.; Berens, H. Semi-Group of Operators and Approximation; Springer: New York, NY, USA, 1967. [Google Scholar]
- Dai, F.; Ditzian, Z. Strong converse inequality for Poisson sums. Proc. Amer. Math. Soc. 2005, 133, 2609–2611. [Google Scholar] [CrossRef] [Scilit]
- Dai, F.; Ditzian, Z. Cesaro summability and Marchaud inequality. Constr. Approx. 2007, 25, 73–88. [Google Scholar] [CrossRef] [Scilit]
- Ditzian, Z. New moduli of smoothness on the unit ball and other domains, introduction and main properties. Constr. Approx. 2014, 40, 1–36. [Google Scholar] [CrossRef] [Scilit]
- Ditzian, Z. New moduli of smoothness on the unit ball, applications and computability. J. Approx. Theory 2014, 180, 49–76. [Google Scholar] [CrossRef] [Scilit]
- Ditzian, Z.; Ivanov, K.G. Strong converse inequalities. J. Anal. Math. 1993, 61, 61–111. [Google Scholar] [CrossRef] [Scilit]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).