1. Introduction
Let
f be expressible in a domain
D in the form
where
and
is some convenient function class. This type of presentation is the
sampling series (of
f), since
f is restored by its values sampled in the discrete subset
by (in general) an infinite linear combination. In the ’classical case’, when
and
we arrive at the so-called WKS (Whittaker–Kotel’nikov–Shannon) sampling theorem, which reads
when the input function
f satisfies certain suitable assumptions. The convergence is absolute and uniform with respect to
x, when it belongs to any compact from
. In (
2) is also usual the notation
, while
, originating back to the
sinus cardinalis terminology introduced by Woodward in 1952.
Independently of the mathematicians’ works, the Formula (
2) was obtained by the young Soviet electrical engineer Kotel’nikov [
1], who pointed out its great importance for communication theory (16 years before) Shannon, who deals with the problems that occur in certain communication system [
2,
3]. For this reason, the above mentioned Formula (
2) in Russian literature is called
Kote’lnikov formula and in Western literature
Shannon formula. That controversy in nomenclature “Kotel’nikov versus Shannon” was precisely treated by Kolmogorov in 1956 in his MIT talk [
4].
At the end of the 1940s the prerequisites for extending the sampling theorem to stochastic weakly stationary processes were developed—Shannon’s paper, the spectral representation formula, Karhunen–Cramér theorem [
5] (p. 156). Therefore, except Belyaev’s oversampling paper [
6] all other mentioned results follow the described research direction used by the Frenchmen Ville [
7,
8,
9] and Oswald [
10], then Americans Lloyd [
11] and Parzen [
12] and/or the Russian (in that time Soviet) mathematician Belyaev. At this point we have to point out that it is conventionally said that Balakrishnan in 1957 was the inventor of stochastic sampling theorem. It is interesting to quote that both Parzen and Balakrishnan have submitted their manuscripts in 1956; Parzen’s technical report [
12] (never published in a journal) was submitted at December 22, while Balakrishnan’s manuscript [
13] has been submitted to the IRE Transactions of Information Theory
a month earlier, precisely at November 23. Both articles discuss the
mean square restoration of bandlimited weak sense stationary stochastic processes. This can confirm Balakrishnan’s priority.
Despite these facts Ville and Oswald gave introductory results to the sampling reconstruction procedure for bandlimited stochastic processes years earlier, see [
14]. In his article [
6] Belyaev emphasized that “...
a rigorous proof of the Formula (2) for the sample functions of random processes with bounded spectra has not been given...” before his work for both the mean-square (and
a fortiori almost sure) sampling reconstruction formulae [
6] (p. 411). The above referenced authors didn’t cite each other, which we can clearly see by using into account the excellent Lloyd paper [
11] obviously unknown by Belyaev. For further information concerning sampling theory development, see among others the cornerstone papers [B], the famous Jerri’s ‘1977–sampling–paper’ [
15], the overview of the Soviet sampling research results which are definitely devoted to a greater extent to stochastic signals than the Eastern (mainly USSR) investigations by Khurgin and Yakovlev [
16], Higgins ’5 short stories’ [
17] and Unser’s [
18]. Finally, for the less known almost sure sense convergence approach in stochastic signals reconstruction one can consult the book chapter [
19] with the exhaustive references list therein.
2. Paul Butzer’s Stochastic Sampling Theoretical Research
In the following we expose Paul Butzer’s contribution and his excellent results to the topic which gives deep insight into the background and mathematical formulation and generalizations of the Formula (
1) when the input signal is a stochastic process.
In his RWTH Aachen, Paul Butzer was a leading person in Analysis area, teaching, conducting research, supervising PhD students in Aachen (according to the Mathematics Genealogy database he has 38 PhD students in 35 years).
Moreover by the MathSciNet, among Approximation theory, Fourier analysis, Harmonic analysis, Functional analysis, Special functions (the famous Butzer-Flocke-Hauss Omega function), Integral transforms (specially Fourier and Mellin transforms), mentioning only a part of his interest areas, Paul Butzer authored and co–authored 47 titles in which “sampling” is mentioned. His co-authors’ list contains the following names: R. L. Stens (22), G. Schmeisser (7), W. Engels (5), J. R. Higgins (5), W. Splettstösser (5), C. Bardaro (4), P. J. S. G. Ferreira (4), G. Vinti (4), M. H. Annaby (3), A. Fischer (3), G. Hinsen (3), I. Mantellini (3), S. Ries (3), G. Schöttler (3), A. I. Zayed (2), M. M. Dodson (1), A. Gessinger (1), M. Hauss (1), S. Jansche (1), O. Lange (1) and U. Scheben (1). This tremendously wide opus cannot be presented in ‘finite/real time’, so we concentrate to few of his papers with results regarding stochastic signals restoration, which are:
[A] Butzer, P.L. A survey of the Whittaker-Shannon sampling theorem and some of its extensions. J. Math. Res. Exposition 1983, 3, 185–212.
[B] Butzer, P.L., Splettstösser, W. Stens, R.L. The sampling theorem and linear prediction in signal analysis. Jahresber. Deutsch. Math.-Verein. 1988, 90, 1–70.
Leading the research in the RWTH Aachen, Butzer has also switched to a classical question of probability theory and linear stochastic processes, precisely to the Central Limit Therem (CLT) in which he realized the following titles in cooperation with Ursula Gather:
[C] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. I. General theorems on rates of convergence. Math. Methods Appl. Sci. 1979, 1, 241–264.
[D] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. II. Models of the general random noise and pulse train processes. Math. Methods Appl. Sci. 1979, 1, 346–353.
Finally, the prediction of signals from the whole ‘past history’, that is, the known past samples, was treated by Butzer and his most frequent co-author, colleague, PhD student and friend Rudolf (Rolf) Leonard Stens, which is represented out of excellent memoir [B], by
[E] Butzer, P.L., Stens R.L. Prediction of nonbandlimited signals from past samples in terms of splines of low degree. Math. Nachr. 1987, 132, 115–130.
In [C, D] not necessarily wide sense stationary processes (a connection to the so–called
Piranašvili processes, introduced in the Piranašvili’s 1967 cornerstone sampling/interpolation paper [
20] could act as a further interesting research goal) are treated from the Central Limit Theorem point of view, exploiting Honda’s concept of linear stochastic processes [
21].
The article [E] discusses the signals prediction from past samples, in which the similar approach is exploited as in the sampling reconstruction procedures. It is worth mentioning that the simple implementation of the classical prediction procedure (generalized in [E]) to the weakly stationary stochastic processes was published in the conference talk [
22].
3. The Results
My attention to the WKS sampling theorem has been drawn in the mid 1980s by Professor Mátyás Arató during my ten months long research grant spent in the Department of Probability and Statistics of Eötvös Loránd University, Budapest. (Professor Arató was a PhD student of A.N. Kolmogorov. Ergo, he called (
2) precisely Kotel’nikov formula). At that time, I knew about this result reading Yaglom’s ‘small book’ [
23] (p. 204) but become interested in the sampling theorems for stochastic processes after this suggestion, publishing my first sampling paper [
24]. Collecting the research material for this topic, I have found, among others, Paul Butzer’s titles regarding sampling theorems. I have met Paul Butzer in 1987 in Niš, Serbia at the
Third Numerical Methods and Approximation Theory Conference—NMAT’87, organized by Gradimir N. Milovanović, where Butzer delivered a lecture composed from results published in [E]. Writing him soon after that conference concerning the possibility for a postdoc position in Aachen, I have been very pleased with his few pages long response letter in which he explained that this would be not possible since his several duties either in teaching, research or other numerous obligations with postdoc fellows and PhD students. However, he kindly sent me a big package of his own papers covering the area including [A, B, E]. With this gift, I began to collect sampling theorems titles and have a strong insight into stochastic sampling results from Balakrishnan’s and Parzen’s papers to new results (for instance, in the so–called
exponential sampling) published nowadays.
First, we recall few definitions and results (listed in
boldface). Let
. The
modulus of continuity with respect to the difference order
r is defined by
and the related Lipschitz class of order
having Lipschitz constant
L by
The Bernstein function class
consists of exponentially bounded functions of type
, which restriction to
belongs to
. The characterization of
by the Paley–Wiener theorem asserts that an
has an extension to
as an element of
if the associated Fourier transform pair
vanishes a.e. outside of
, that is, it is
bandlimited to this interval:
In explaining (even in a heuristic manner!) these results, now, in a stochastic framework, we need the following definitions. The stochastic process defined on a standard probability space is an –measurable function, that is becomes a random variable for any fixed , while is a deterministic function called trajectory of X. The process belongs to the class of all finite second moment random variables . This class possesses a Hilbert space structure with scalar product defined by the expectation operator , that is and with the norm equipped.
The expectation function
, while the correlation function
When
is a constant and
, the process is wide sense (Hinčin) stationary (WSS) and we have that
;
is the variance of the process, with the property
.
The
Karhunen–Cramér theorem, which characterization theorem can also be called
generalized harmonizability theorem [
25,
26,
27], reads as follows [
5] (p. 156).
Assume that the correlation function of a stochastic process possesses a double integral representation in the formwhere is a positive definite kernel of finite variation andThen has a spectral representation andwhere is a stochastic measure satisfyinghere stands for the appropriate σ-field.Conversely, if the stochastic process is representable in the form (4) in which the spectral measure satisfies Equation (5) and is of bounded variation, then there holds the spectral representation (3) of the correlation function . In the case when
is a Fourier kernel
and
, i.e., it is concentrated on the diagonal of
, the process
becomes WSS. Then, the stochastic process
X and the related correlation function
have the following spectral representations
respectively. Here,
, for all
. Moreover, if
the process is
non-bandlimited, while for
the stochastic process is
bandlimited to the bandwidth
w. Obviously, these formulae readily follow from the spectral representations (
4) and (
3), having in mind Display (
5).
Next, the stochastic process
is continuous and respectively differentiable at
with a first derivative
, both in the mean–square sense used, when
Higher order mean–square derivatives
, used in the following exposition, can be defined iteratively.
Finally, let
, that is let
X a finite second–order moment stochastic process. The
modulus of continuity for stochastic signal X is defined by
Obviously, for WSS processes this definition is t-free. The related Lipschitz class is defined analogously.
Also, we introduce the
convolution of a process with a (deterministic) function
that turns out to be the (stochastic) integral
where the validity of this representation holds almost surely (with probability 1). For
with
,
, we have the estimate (in the mean–sequre sense)
Here is
, where
denotes the
de la Vallée Poussin kernel [B] (p. 8, Eq. (2.3)), [
28]
Finally, let us recall that the WKS sampling theorem for the bandlimited stochastic process
, having a bandwidth
reads
in accordance with the WKS Formula (
2) valid for deterministic signals. However, when we consider the convergence of its truncated to
partial–sums–sequence we arrive at the following result.
Theorem 1 ([B] (p. 57, Theorem 6.11.))
. The bandlimited WSS
process X has . Then Theorem 2 ([B] (p. 57, Theorem 6.12.))
. Let X be an r time differentiable WSS
process with , . If , , , and , then Closing the definitions, we introduce
generalized sampling expansion [B] (pp. 24–25) when the sinc kernel is replaced with another suitable kernel
:
Consider [B] (p. 25, Theorem 4.1). When
for all
, and
Then
- (a)
there holds true , for all , when ;
- (b)
defines a family of bounded linear operators from
into itself, so that
The generalized sampling series expansion associated with the kernel
we define in a similar way as in (
6) for the input function
f, applying the operator
, i.e.,
Theorem 3 ([B] (p. 58, Theorem 6.14))
. Let satisfies (6) and (7), . Then there holdsuniformly in . Let us define the absolute moment of
of the order
r [B] (p. 24]
Theorem 4 ([B] (p. 59, Theorem 6.16))
. Let , for certain . If the momentsthen - (c)
for r–fold differentiable WSS
process X, we have - (d)
moreover, there exists an absolute constant , such that for WSS
process there holds
When
coincides with the classical sinc–kernel and the input WSS process is non-bandlimited, we have the well–known Brown’s aliasing error upper bound [
29]
giving a result related to (
8) for arbitrary finite
. The similar fashion result for the non-bandlimited differentiable homogeneous random fields has been established by the author, compare [
30] (p. 128, Theorem). During the Fourier Analysis and Applications conference, held at York University in 1993 organized by M. Dodson and M. Beaty, Paul Butzer has drawn the author’s attention to the fact that Brown’s results, and, at the same time his generalization of Brown’s upper bound (
9) to non–bandlimited random fields, are Jackson–type inequalities.
Ending this exposure, we have to remark that Theorems 3, 4 are more condensed than Theorems 1 and 2 due to the stochastic nature of the input signals, concerning to the spectral representations either of the stochastic input processes or their correlation functions.
Finally, the generalized sampling expansion operator gives a highly elastic tool in generating very different sampling results manipulating with this operator in several directions: derivative sampling, approximative sampling (considering non-bandlimited input signals), multi-channel sampling etc. The modulus of continuity formulation of the earned results, compare (d) in Theorem 4, gives an excellent insight into analysis background of probabilistic setting of the aliasing approximation error’s magnitude in function of the bandwidth of the input stochastic process by its generalized sampling expansion series.