Next Article in Journal
Time Automorphisms on C*-Algebras
Previous Article in Journal
Topological Integer Additive Set-Sequential Graphs
Article

Reproducing Kernel Hilbert Space vs. Frame Estimates

1
Department of Mathematics, The University of Iowa, 14 MacLean Hall, Iowa City, IA 52242, USA
2
Department of Mathematics and Statistics, Southern Illinois University Edwardsville, Box 1653, Edwardsville, IL 62026, USA
*
Author to whom correspondence should be addressed.
Academic Editor: Lokenath Debnath
Mathematics 2015, 3(3), 615-625; https://doi.org/10.3390/math3030615
Received: 21 May 2015 / Revised: 3 July 2015 / Accepted: 3 July 2015 / Published: 8 July 2015

Abstract

We consider conditions on a given system F of vectors in Hilbert space H, forming a frame, which turn H into a reproducing kernel Hilbert space. It is assumed that the vectors in F are functions on some set Ω . We then identify conditions on these functions which automatically give H the structure of a reproducing kernel Hilbert space of functions on Ω. We further give an explicit formula for the kernel, and for the corresponding isometric isomorphism. Applications are given to Hilbert spaces associated to families of Gaussian processes.
Keywords: Hilbert space; frames; reproducing kernel; Karhunen-Loève Hilbert space; frames; reproducing kernel; Karhunen-Loève

1. Introduction

A reproducing kernel Hilbert space (RKHS) is a Hilbert space H of functions on a set, say Ω, with the property that f ( t ) is continuous in f with respect to the norm in H . There is then an associated kernel. It is called reproducing because it reproduces the function values for f in H . Reproducing kernels and their RKHSs arise as inverses of elliptic PDOs, as covariance kernels of stochastic processes, in the study of integral equations, in statistical learning theory, empirical risk minimization, as potential kernels, and as kernels reproducing classes of analytic functions, and in the study of fractals, to mention only some of the current applications. They were first introduced in the beginning of the 20ties century by Stanisaw Zaremba and James Mercer, Gábor Szegö, Stefan Bergman, and Salomon Bochner. The subject was given a global and systematic presentation by Nachman Aronszajn in the early 1950s. The literature is by now vast, and we refer to the following items from the literature, and the papers cited there [1,4,7,12,15,16]. Our aim in the present paper is to point out an intriguing use of reproducing kernels in the study of frames in Hilbert space.

2. An Explicit Isomorphism

Let H be a separable Hilbert space, and let { φ n } n N be a system of vectors in H . Then we shall study relations of H as a reproducing kernel Hilbert space (RKHS) subject to properties imposed on the system { φ n } n N . An RKHS is a Hilbert space H of functions on some set Ω such that for all t Ω , there is a (unique) K t H with f ( t ) = K t , f H , for all t Ω , for all f H . In the theorem below we study what systems of functions
φ n H { functions on some set Ω }
yield RKHSs; i.e., if { φ n } n N satisfies (1), what additional conditions are required to guarantee that H is an RKHS?
Given { φ n } n N H , we shall introduce the Gramian G = ( φ i , φ j H ) considered as an × - matrix.
Under mild restrictions on { φ n } n N , it turns out that G defines an unbounded (generally) selfadjoint linear operator
l 2 G l 2
( G ( c j ) ) k = j N φ k , φ j H c j
Let F denote finitely supported sequence with (2) defined on all finitely supported sequence ( c j ) F in l 2 , i.e., ( c j ) F if and only if there exists n Z + such that c j = 0 , for all j n ; but note that n depends the sequences. Denoting δ j the canonical basis in l 2 , δ j ( j ) = δ i , j , note F = s p a n { δ j | j N } .
Further, note that the RHS in (2) is well defined when j φ k , φ j H 2 < , for all k N .
  • Remark 1. 1. Since the statement of our result entails G 1 (inverse Gramian, it is actually a pseudo-inverse, see below), we will add a few comments about this point. It is known to specialists in frame theory, but to make the presentation self-contained we add a lemma giving equivalent properties and conditions for G 1 .
2.
The Gramian of a frame is invertible if and only if the frame is a Riesz basis.
3.
By definition a frame refers to a fixed Hilbert space, say H , and then a system of vectors in H satisfying an axiomatic a priori estimate, see (11). But because of applications to stochastic processes (see Section 3) it is natural to consider frames in some specified Hilbert space which in fact consists of function on some set, say Ω (given at the outset).
4.
Our concern here is to make the connection between the initial Hilbert space H (with frame vectors) and an associated reproducing kernel Hilbert space of functions on Ω. An application to the study of Gaussian processes is motivating our approach.
Theorem 1. Suppose H , { φ n } are given. Assume that
  • (a) each φ n is a function on Ω where Ω is a given set
  • (b) { φ n } is a frame in H , see (10) and (11), and that
  • (c) { φ n ( t ) } l 2 , for all t Ω
then H is a reproducing kernel Hilbert space (RKHS) with kernel
K G ( s , t ) = l ( s ) , G 1 l ( t ) 2 = l ( s ) * G 1 l ( t )
where l ( t ) = { φ n ( t ) } l 2 , and where G is the Gramian of G = ( φ n , φ m H ) . Moreover, G defines selfadjoint operator in l 2 with dense domain, and we get an isometric isomorphism
H R K T G H
T G ( n c n φ n ) = T * c where T is the frame operator.
Proof of Theorem 1. Overview: Since { φ n } H is a frame, the Gramian G m n : = φ m , φ n H , is an × matrix defining a bounded operator l 2 l 2 , invertible with ( G 1 ) m n such that
k = 1 ( G 1 ) m k φ k , φ n H = δ m , n
and the reproducing kernel of H is m n φ m ( s ) ¯ G m n 1 φ n ( t ) = l ( s ) , G 1 l ( t ) 2  ☐
Proof of Theorem 1. (details) By (4) and Lemma 10, the frame operators T and T * are as follows: Given H , { φ n } , set
T : H l 2 T * : l 2 H
to be the two linear operators
T f = ( φ n , f H )
and adjoint T * as follows:
T * c = n c n φ n
Lemma 2. We have
T f , c l 2 = f , T * c H , and T * T f = φ n , f φ n , f H , c l 2
( T T * c ) n = ( G c ) n = m G n m c m , c l 2
Do the real case first, then it is easy to extend to complex valued functions.
Note that T T * is an operator in l 2 , i.e.,
l 2 T T * l 2
It has a matrix-representation as follows
( T T * ) i , j = δ i , T T * δ j l 2
Lemma 3. We have
( T T * ) i , j = G i , j = φ i , φ j H , ( i , j ) N × N
Proof of Lemma 3. By (8), we have
( T T * ) i , j = δ i , T T * δ j l 2 = T * δ i , T * δ j H = φ i , φ j H = G i , j
which is the desired conclusion (9).  ☐
Both T * T and T T * are self-adjoint: If B i , i = 1 , 2 are the constants from the frame estimates, then:
B 1 c 2 2 T * c H 2 B 2 c 2 2 c l 2 , and
B 1 f H 2 T f l 2 2 B 2 f H 2 f H
equivalently
B 1 f H 2 n | φ n , f H | 2 B 2 f H 2
Set
K ( s , t ) = n = 1 φ n ( s ) * φ n ( t ) = l ( s ) * l ( t ) = l ( s ) , l ( t ) 2
We have
B 1 I l 2 T T * B 2 I l 2 , and
B 1 I H T * T B 2 I H
If B 1 = B 2 = 1 , then we say that { φ n } n N is a Parseval frame.
For the theory of frames and some of their applications, see e.g., [5,6] and the papers cited there.
By the polar-decomposition theorems, see e.g., [11] we conclude that there is a unitary isomorphism u : H l 2 such that T = u ( T * T ) 1 / 2 = ( T T * ) 1 / 2 u ; and so in particular, the two s.a. operators T * T and T T * are unitarily equivalent.
Definition 4.
l ( t ) = ( φ n ( t ) ) l 2
Therefore ( T * T ) 1 / 2 is well defined H H . Now (6) holds if and only if
f = ( T * T ) 1 / 2 φ , f ( T * T ) 1 / 2 φ n
or equivalently:
f = ψ n , f H ψ n
where
ψ n : = ( T * T ) 1 / 2 φ n
Here we used that T * T is a selfadjoint operator in H , and it has a positive spectral lower bound; where { φ j } j N is assumed to be a frame.  ☐
Lemma 5. There is an operator L : H H (the Lax-Milgram operator) such that
n = 1 f , φ n H φ n L g H = f , g H
holds for all f H .
Proof of Lemma 5. We shall apply the Lax-Milgram lemma [11], p. 57 to the sesquilinear form
B ( f , g ) = n = 1 f , φ n H φ n , g H , f , g H
Since { φ n } n = 1 is given to be a frame in H , then our frame-bounds B 1 > 0 and B 2 < such that (11) holds. Introducing B from (17) this into
B 1 f H 2 B ( f , f ) B 2 f H 2 , f H
The existence of the operator L as stated in (16) now follows from the Lax-Milgram lemma.  ☐
Corollary 6. Let H , { φ n } , T, T * be as in Lemma 8; and let L be the Lax-Milgram operator; then L = ( T * T ) 1 .
Lemma 7. The kernel K G ( · , · ) on Ω × Ω from (3) is well-defined and positive definite.
Proof of Lemma 7. We must show that all the finite double summations
i j c i ¯ c j K G ( t i , t j )
are 0 , whenever ( c i ) is a finite system of coefficients, and ( t i ) is a finite sample of points in Ω. Now fix ( c i ) and ( t i ) as specified, and, for n N , set
F n : = i c i φ n ( t i )
then we have the following:
i j c i ¯ c j K G ( t i , t j ) = i j c i ¯ c j l ( t i ) , G 1 l ( t j ) l 2 = i j c i ¯ c j m n φ m ( t i ) ¯ G m , n 1 φ n ( t j ) = m n F m ¯ G m , n 1 F n 0 .
Lemma 8. We have the following:
ψ n ( t ) = ( G 1 / 2 φ ) n ( t ) = m = 1 ( G n m 1 / 2 φ m ) ( t ) = G 1 / 2 l ( t ) n
and these functions are in the RKHS of the kernel K G from (3).
Proof of Lemma 8. Begin with (the frame identity):
( T * T ) φ n = b y ( 6 ) m = 1 φ m , φ n φ m = ( G l ) n , n Z + , where l = φ 1 φ 2
if and only if
( T * T ) l ( t ) = G ( l ( t ) )
Now approximate x with polynomials (Weierstrass), and we get
( T * T ) 1 / 2 l ( t ) = G 1 / 2 l ( t )
Recall, ψ n = ( T * T ) 1 / 2 φ n . ψ n ( t ) = ( G 1 / 2 l ( t ) ) n . Now rewrite (14) as
f ( t ) = n = 1 ψ n , f H ψ n ( t ) = n = 1 G 1 / 2 φ n , f H ( G 1 / 2 φ ) n ( t ) = K t G , f
where
K t G = n = 1 G 1 / 2 φ n ( · ) ( G 1 / 2 φ n ) ( t ) = b y ( 21 ) K G ( s , t ) = n = 1 ( G 1 / 2 φ n ) ( s ) ( G 1 / 2 φ n ) ( t ) = G 1 / 2 l ( s ) , G 1 / 2 l ( t ) 2 = l ( s ) , G 1 l ( t ) 2
For the complex case, the result still holds, mutatis mutandis; one only needs to add the complex conjugations.  ☐
Note that (22) is the reproducing property.
Corollary 9. The function ( ψ n ( t ) ) from (19) in Lemma 8 satisfy
n N ψ n ( s ) ¯ ψ n ( t ) = K G ( s , t ) , ( s , t ) Ω × Ω .
Proof of Corollary 9.
L H S ( 23 ) = G 1 / 2 l ( s ) , G 1 / 2 l ( t ) 2 = l ( s ) , ( G 1 / 2 ) 2 l ( t ) 2 = l ( s ) , G 1 ( l ( t ) ) , 2 = K G ( s , t ) .
Lemma 10. The following isometric property holds:
n = 1 c n φ n ( · ) H 2 = c m c n φ n , φ m H = c T G c = c , G c 2 = c , T T * c 2 = T * c H 2 , T * c H , c l 2
where T and T * are the frame operators H T * T l 2 , i.e., T f = ( φ n , f H ) n l 2
Corollary 11. The Lax operator L satisfies L f : = n ( T * 1 f ) n φ n ( · ) , for all f H and it is isometric H H .
The following partial converse holds to Corollary 9.
Let H R K ( Ω ) be a reproducing kernel Hilbert space associated with a fixed kernel
K : Ω × Ω C
then a system
{ ψ n ( · ) } n N H R K ( Ω )
is a Parseval-frame in H R K ( Ω ) if and only if
n N ψ n ( s ) ψ n ( t ) ¯ = K ( s , t )
holds for all ( s , t ) Ω × Ω ; with absolute convergence in (25).
Proof of Lemma 10. The non-trivial part. Suppose a system of functions ( φ n ( · ) ) H R K ( Ω ) is given, and that it is a Parseval frame in H R K ( Ω ) . Then for all f H R K ( Ω ) , we have
f R K 2 = n ψ n , f R K 2 .
Apply this to f t = K ( t , · ) H R K ( Ω ) ; then by polarization of (26), we get for all ( s , t ) Ω × Ω ,
K ( s , t ) = K s , K t R K = n K s , ψ n R K ψ n , K t R K = b y ( 26 ) n ψ n ( s ) ψ n ( t ) ¯ by the reproducing property in RK
which is the desired conclusion in (25).  ☐
Example 1. In the theorem, we assume that the given Hilbert space H has a frame { φ n } H consisting of functions on a set Ω. So this entails a lower, and an upper frame bound, i.e., 0 < B 1 B 2 < .
The following example shows that the conclusion in the theorem is false if there is not a positive lower frame-bound.
Set H = L 2 ( 0 , 1 ) , Ω = ( 0 , 1 ) the open unit-inbound, and φ n ( t ) = t n , n { 0 } N = N 0 . In this case, the Gramian
G n m = 0 1 x n + m d x = 1 n + m + 1
is the × Hilbert matrix, see ([10,13,14]). In this case it is known that there is an upper frame bound B 2 = π , i.e.,
n = 0 0 1 f ( x ) x n d x 2 π 0 1 | f ( x ) | 2 d x
in fact, for the operator-norm, we have
G l 2 l 2 = π
but there is not a lower frame bound. Moreover, G define a selfadjoint operator in l 2 ( N 0 ) with spectrum [ 0 , π ] = the closed interval. This implies that there cannot be a positive lower frame-bound.
Moreover, it is immediate by inspection that H = L 2 ( 0 , 1 ) is not a RKHS.

3. Frames and Gaussian Processes

In [2] and [3], it was shown that for every positive Borel measure σ on R such that
R d σ ( u ) 1 + u 2 <
there is a unique (up to measure isomorphism) Gaussian proess X as follows:
(i)
X = X φ is indexed by the Schwartz-space S = S ( R ) of function φ on R , i.e., φ S φ C , and for all N , M N we have
m a x m M s u p x R x N x m φ ( x ) <
with E ( X φ ) = 0 , and E ( X φ 2 ) = R | φ ^ | 2 d σ for all φ S .
(ii)
Let Ω : = S = the dual = the Schwartz space of all tempered distribution, then X φ is defined on S , by
X φ ( w ) = w ( φ ) , φ S , w S
It is real valued Gaussian random variable.
(iii)
[2,3] there is a unique measure P = P σ on S such that
(a)
X φ is Gaussian for all φ S ,
(b)
E ( X φ ) = 0 , and
E σ ( e i X φ ) = S e i X φ d P σ = e 1 2 φ ^ σ 2 = e 1 2 R | φ ^ ( u ) | 2 d σ ( u )
where φ ^ = the Fourier transform,
φ ^ ( u ) = R e i x u φ ( x ) d x
Theorem 12. Let { f n } n N be a real-valued frame in L 2 ( R , σ ) ( = { f on R , such that f σ 2 : = R | f ( u ) | 2 d σ ( u ) < } ) with frame bounds a, b such that 0 < a b < , so
a f σ 2 n N R f ( u ) f n ( u ) d σ ( u ) 2 b f σ 2
Let { B n } n N be a system of i.i.d. (independent identically distributed) N ( 0 , 1 ) Gaussian random variables on ( Ω , F , P σ ) , and set
Y φ ( · ) = n N f n , φ ^ L 2 ( σ ) B n ( · ) , (Karhunen-Loève)
then
a E σ | X φ | 2 E σ | Y φ | 2 b E σ | X φ | 2
Proof of Theorem 12. Using the i.i.d. N ( 0 , 1 ) property of { B n } n N , we get
E σ | Y φ | 2 = n N | f n , φ ^ L 2 ( σ ) | 2
The desired conclusion (34) now follows from (32) combined with
E σ | X φ | 2 = φ ^ L 2 ( σ ) 2 , see [ 2 , 3 ]
while (36) is immediate from (30).  ☐
Corollary 13. The property for { Y φ } φ S in (33) agrees with { X φ } φ S if and only if { f n } n N is a Parseval frame in L 2 ( σ ) .
Proof of Corollary 13. This follows from the Karhunen-Loève theorem; see [2,3,8,9].  ☐

Acknowledgments

The authors are please to acknowledge helpful discussions, both recent and not so recent, with John Benedetto, Ilwoo Cho, D. Dutkay, Keri Kornelson, Kathy Merrill, P. Muhly, Judy Packer, Erin Pearse, Steen Pedersen, Gabriel Picioroaga, Karen Shuman.

Author Contributions

Palle E. T. Jorgensen and Myung-Sin Song discussed together and wrote the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Arqub, O.A.; Al-Smadi, M.; Shawagfeh, N. Solving Fredholm integro-differential equations using reproducing kernel Hilbert space method. Appl. Math. Comput. 2013, 219, 8938–8948. [Google Scholar] [CrossRef]
  2. Alpay, D.; Jorgensen, P. Stochastic processes induced by singular operators. Numer. Funct. Anal. Optim. 2012, 33, 708–735. [Google Scholar] [CrossRef]
  3. Alpay, D.; Jorgensen, P.; Levanony, D. A class of Gaussian processes with fractional spectral measures. J. Funct. Anal. 2011, 261, 507–541. [Google Scholar] [CrossRef]
  4. Aronszajn, N. Theory of reproducing kernels. Trans. Am. Math. Soc. 1950, 68, 337–404. [Google Scholar] [CrossRef]
  5. Casazza, P.G.; Christensen, O. Frames containing a Riesz basis and preservation of this property under perturbations. SIAM J. Math. Anal. 1998, 29, 266–278, (electronic). [Google Scholar] [CrossRef]
  6. Casazza, P.G.; Kutyniok, G. Frames of Subspaces. In Wavelets, Frames and Operator Theory; Amer. Math. Soc.: Providence, RI, USA, 2004; Volume 345, pp. 87–113. [Google Scholar]
  7. Han, D.; Wu, J.; Larson, D.; Li, P.; Mohapatra, R.N. Dilation of dual frame pairs in Hilbert C*-modules. Results Math. 2013, 63, 241–250. [Google Scholar] [CrossRef]
  8. Jorgensen, P.E.T.; Song, M.-S. Entropy encoding, hilbert space, and karhunen-loève transforms. J. Math. Phys. 2007, 48. [Google Scholar] [CrossRef]
  9. Jorgensen, P.E.T.; Song, M.-S. An Extension of Wiener Integration with the Use of Operator Theory. J. Math. Phys. 2009, 50. [Google Scholar] [CrossRef]
  10. Kato, T. On the Hilbert matrix. Proc. Amer. Math. Soc. 1957, 8, 73–81. [Google Scholar] [CrossRef]
  11. Peter, D.L. Functional Analysis; Pure and Applied Mathematics (New York) Wiley-Interscience (John Wiley & Sons): New York, NY, USA, 2002. [Google Scholar]
  12. Nosedal-Sanchez, A.; Storlie, C.B.; Lee, T.C.M.; Christensen, R. Reproducing kernel Hilbert spaces for penalized regression: A tutorial. Am. Stat. 2012, 66, 50–60. [Google Scholar] [CrossRef]
  13. Rosenblum, M. On the Hilbert matrix. II. Proc. Am. Math. Soc. 1958, 9, 581–585. [Google Scholar]
  14. Taussky, O. A remark concerning the characteristic roots of the finite segments of the Hilbert matrix. Quart. J. Math. Oxford Ser. 1949, 20, 80–83. [Google Scholar] [CrossRef]
  15. Watanabe, S. Jacobi polynomials and associated reproducing kernel Hilbert spaces. J. Math. Anal. Appl. 2012, 389, 108–118. [Google Scholar] [CrossRef]
  16. Akira, Y. Inequalities for gram matrices and their applications to reproducing kernel Hilbert spaces. Taiwan. J. Math. 2013, 17, 427–430. [Google Scholar]
Back to TopTop