All articles published by MDPI are made immediately available worldwide under an open access license. No special
permission is required to reuse all or part of the article published by MDPI, including figures and tables. For
articles published under an open access Creative Common CC BY license, any part of the article may be reused without
permission provided that the original article is clearly cited. For more information, please refer to
Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature
Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for
future research directions and describes possible research applications.
Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive
positive feedback from the reviewers.
Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world.
Editors select a small number of articles recently published in the journal that they believe will be particularly
interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the
most exciting work published in the various research areas of the journal.
The sampling reconstruction theory is one of the great areas of the analysis in which Paul Leo Butzer earned longstanding and excellent theoretical results. Thus, we are forced either by earlier exhaustive presentations of his research activity and/or the highly voluminous material to restrict ourselves to a more narrow and precise sub-area in consideration; we discuss here, giving deeper insight, Paul Butzer’s sampling theoretical work with special attention concerning sampling stochastic signals.
where and is some convenient function class. This type of presentation is the sampling series (of f), since f is restored by its values sampled in the discrete subset by (in general) an infinite linear combination. In the ’classical case’, when and we arrive at the so-called WKS (Whittaker–Kotel’nikov–Shannon) sampling theorem, which reads
when the input function f satisfies certain suitable assumptions. The convergence is absolute and uniform with respect to x, when it belongs to any compact from . In (2) is also usual the notation , while , originating back to the sinus cardinalis terminology introduced by Woodward in 1952.
Independently of the mathematicians’ works, the Formula (2) was obtained by the young Soviet electrical engineer Kotel’nikov , who pointed out its great importance for communication theory (16 years before) Shannon, who deals with the problems that occur in certain communication system [2,3]. For this reason, the above mentioned Formula (2) in Russian literature is called Kote’lnikov formula and in Western literature Shannon formula. That controversy in nomenclature “Kotel’nikov versus Shannon” was precisely treated by Kolmogorov in 1956 in his MIT talk .
At the end of the 1940s the prerequisites for extending the sampling theorem to stochastic weakly stationary processes were developed—Shannon’s paper, the spectral representation formula, Karhunen–Cramér theorem  (p. 156). Therefore, except Belyaev’s oversampling paper  all other mentioned results follow the described research direction used by the Frenchmen Ville [7,8,9] and Oswald , then Americans Lloyd  and Parzen  and/or the Russian (in that time Soviet) mathematician Belyaev. At this point we have to point out that it is conventionally said that Balakrishnan in 1957 was the inventor of stochastic sampling theorem. It is interesting to quote that both Parzen and Balakrishnan have submitted their manuscripts in 1956; Parzen’s technical report  (never published in a journal) was submitted at December 22, while Balakrishnan’s manuscript  has been submitted to the IRE Transactions of Information Theory a month earlier, precisely at November 23. Both articles discuss the mean square restoration of bandlimited weak sense stationary stochastic processes. This can confirm Balakrishnan’s priority.
Despite these facts Ville and Oswald gave introductory results to the sampling reconstruction procedure for bandlimited stochastic processes years earlier, see . In his article  Belyaev emphasized that “... a rigorous proof of the Formula (2) for the sample functions of random processes with bounded spectra has not been given...” before his work for both the mean-square (and a fortiori almost sure) sampling reconstruction formulae  (p. 411). The above referenced authors didn’t cite each other, which we can clearly see by using into account the excellent Lloyd paper  obviously unknown by Belyaev. For further information concerning sampling theory development, see among others the cornerstone papers [B], the famous Jerri’s ‘1977–sampling–paper’ , the overview of the Soviet sampling research results which are definitely devoted to a greater extent to stochastic signals than the Eastern (mainly USSR) investigations by Khurgin and Yakovlev , Higgins ’5 short stories’  and Unser’s . Finally, for the less known almost sure sense convergence approach in stochastic signals reconstruction one can consult the book chapter  with the exhaustive references list therein.
2. Paul Butzer’s Stochastic Sampling Theoretical Research
In the following we expose Paul Butzer’s contribution and his excellent results to the topic which gives deep insight into the background and mathematical formulation and generalizations of the Formula (1) when the input signal is a stochastic process.
In his RWTH Aachen, Paul Butzer was a leading person in Analysis area, teaching, conducting research, supervising PhD students in Aachen (according to the Mathematics Genealogy database he has 38 PhD students in 35 years).
Moreover by the MathSciNet, among Approximation theory, Fourier analysis, Harmonic analysis, Functional analysis, Special functions (the famous Butzer-Flocke-Hauss Omega function), Integral transforms (specially Fourier and Mellin transforms), mentioning only a part of his interest areas, Paul Butzer authored and co–authored 47 titles in which “sampling” is mentioned. His co-authors’ list contains the following names: R. L. Stens (22), G. Schmeisser (7), W. Engels (5), J. R. Higgins (5), W. Splettstösser (5), C. Bardaro (4), P. J. S. G. Ferreira (4), G. Vinti (4), M. H. Annaby (3), A. Fischer (3), G. Hinsen (3), I. Mantellini (3), S. Ries (3), G. Schöttler (3), A. I. Zayed (2), M. M. Dodson (1), A. Gessinger (1), M. Hauss (1), S. Jansche (1), O. Lange (1) and U. Scheben (1). This tremendously wide opus cannot be presented in ‘finite/real time’, so we concentrate to few of his papers with results regarding stochastic signals restoration, which are:
[A] Butzer, P.L. A survey of the Whittaker-Shannon sampling theorem and some of its extensions. J. Math. Res. Exposition1983, 3, 185–212.
[B] Butzer, P.L., Splettstösser, W. Stens, R.L. The sampling theorem and linear prediction in signal analysis. Jahresber. Deutsch. Math.-Verein.1988, 90, 1–70.
Leading the research in the RWTH Aachen, Butzer has also switched to a classical question of probability theory and linear stochastic processes, precisely to the Central Limit Therem (CLT) in which he realized the following titles in cooperation with Ursula Gather:
[C] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. I. General theorems on rates of convergence. Math. Methods Appl. Sci.1979, 1, 241–264.
[D] Butzer, P.L., Gather U. Asymptotic expansions for central limit theorems for general linear stochastic processes. II. Models of the general random noise and pulse train processes. Math. Methods Appl. Sci.1979, 1, 346–353.
Finally, the prediction of signals from the whole ‘past history’, that is, the known past samples, was treated by Butzer and his most frequent co-author, colleague, PhD student and friend Rudolf (Rolf) Leonard Stens, which is represented out of excellent memoir [B], by
[E] Butzer, P.L., Stens R.L. Prediction of nonbandlimited signals from past samples in terms of splines of low degree. Math. Nachr.1987, 132, 115–130.
In [C, D] not necessarily wide sense stationary processes (a connection to the so–called Piranašvili processes, introduced in the Piranašvili’s 1967 cornerstone sampling/interpolation paper  could act as a further interesting research goal) are treated from the Central Limit Theorem point of view, exploiting Honda’s concept of linear stochastic processes .
The article [E] discusses the signals prediction from past samples, in which the similar approach is exploited as in the sampling reconstruction procedures. It is worth mentioning that the simple implementation of the classical prediction procedure (generalized in [E]) to the weakly stationary stochastic processes was published in the conference talk .
3. The Results
My attention to the WKS sampling theorem has been drawn in the mid 1980s by Professor Mátyás Arató during my ten months long research grant spent in the Department of Probability and Statistics of Eötvös Loránd University, Budapest. (Professor Arató was a PhD student of A.N. Kolmogorov. Ergo, he called (2) precisely Kotel’nikov formula). At that time, I knew about this result reading Yaglom’s ‘small book’  (p. 204) but become interested in the sampling theorems for stochastic processes after this suggestion, publishing my first sampling paper . Collecting the research material for this topic, I have found, among others, Paul Butzer’s titles regarding sampling theorems. I have met Paul Butzer in 1987 in Niš, Serbia at the Third Numerical Methods and Approximation Theory Conference—NMAT’87, organized by Gradimir N. Milovanović, where Butzer delivered a lecture composed from results published in [E]. Writing him soon after that conference concerning the possibility for a postdoc position in Aachen, I have been very pleased with his few pages long response letter in which he explained that this would be not possible since his several duties either in teaching, research or other numerous obligations with postdoc fellows and PhD students. However, he kindly sent me a big package of his own papers covering the area including [A, B, E]. With this gift, I began to collect sampling theorems titles and have a strong insight into stochastic sampling results from Balakrishnan’s and Parzen’s papers to new results (for instance, in the so–called exponential sampling) published nowadays.
First, we recall few definitions and results (listed in boldface). Let . The modulus of continuity with respect to the difference order r is defined by
and the related Lipschitz class of order having Lipschitz constant L by
The Bernstein function class consists of exponentially bounded functions of type , which restriction to belongs to . The characterization of by the Paley–Wiener theorem asserts that an has an extension to as an element of if the associated Fourier transform pair vanishes a.e. outside of , that is, it is bandlimited to this interval:
In explaining (even in a heuristic manner!) these results, now, in a stochastic framework, we need the following definitions. The stochastic process defined on a standard probability space is an –measurable function, that is becomes a random variable for any fixed , while is a deterministic function called trajectory of X. The process belongs to the class of all finite second moment random variables . This class possesses a Hilbert space structure with scalar product defined by the expectation operator , that is and with the norm equipped.
The expectation function , while the correlation function
When is a constant and , the process is wide sense (Hinčin) stationary (WSS) and we have that ; is the variance of the process, with the property .
The Karhunen–Cramér theorem, which characterization theorem can also be called generalized harmonizability theorem [25,26,27], reads as follows  (p. 156). Assume that the correlation function of a stochastic process possesses a double integral representation in the form
where is a positive definite kernel of finite variation and
Then has a spectral representation and
where is a stochastic measure satisfying
here stands for the appropriate σ-field.
Conversely, if the stochastic process is representable in the form (4) in which the spectral measure satisfies Equation (5) and is of bounded variation, then there holds the spectral representation (3) of the correlation function .
In the case when is a Fourier kernel and , i.e., it is concentrated on the diagonal of , the process becomes WSS. Then, the stochastic process X and the related correlation function have the following spectral representations
respectively. Here, , for all . Moreover, if the process is non-bandlimited, while for the stochastic process is bandlimited to the bandwidth w. Obviously, these formulae readily follow from the spectral representations (4) and (3), having in mind Display (5).
Next, the stochastic process is continuous and respectively differentiable at with a first derivative , both in the mean–square sense used, when
Higher order mean–square derivatives , used in the following exposition, can be defined iteratively.
Finally, let , that is let X a finite second–order moment stochastic process. The modulus of continuity for stochastic signal X is defined by
Obviously, for WSS processes this definition is t-free. The related Lipschitz class is defined analogously.
Also, we introduce the convolution of a process with a (deterministic) function that turns out to be the (stochastic) integral
where the validity of this representation holds almost surely (with probability 1). For with , , we have the estimate (in the mean–sequre sense)
Here is , where denotes the de la Vallée Poussin kernel [B] (p. 8, Eq. (2.3)), 
Finally, let us recall that the WKS sampling theorem for the bandlimited stochastic process , having a bandwidth reads
in accordance with the WKS Formula (2) valid for deterministic signals. However, when we consider the convergence of its truncated to partial–sums–sequence we arrive at the following result.
Theorem1 ([B] (p. 57, Theorem 6.11.)).
The bandlimited WSS process X has . Then
Theorem2 ([B] (p. 57, Theorem 6.12.)).
Let X be an r time differentiable WSS process with , . If , , , and , then
Closing the definitions, we introduce generalized sampling expansion [B] (pp. 24–25) when the sinc kernel is replaced with another suitable kernel :
Consider [B] (p. 25, Theorem 4.1). When for all , and
there holds true , for all , when ;
defines a family of bounded linear operators from into itself, so that
The generalized sampling series expansion associated with the kernel we define in a similar way as in (6) for the input function f, applying the operator , i.e.,
Let us define the absolute moment of of the order r [B] (p. 24]
Theorem4 ([B] (p. 59, Theorem 6.16)).
Let , for certain . If the moments
for r–fold differentiable WSS process X, we have
moreover, there exists an absolute constant , such that for WSS process there holds
When coincides with the classical sinc–kernel and the input WSS process is non-bandlimited, we have the well–known Brown’s aliasing error upper bound 
giving a result related to (8) for arbitrary finite . The similar fashion result for the non-bandlimited differentiable homogeneous random fields has been established by the author, compare  (p. 128, Theorem). During the Fourier Analysis and Applications conference, held at York University in 1993 organized by M. Dodson and M. Beaty, Paul Butzer has drawn the author’s attention to the fact that Brown’s results, and, at the same time his generalization of Brown’s upper bound (9) to non–bandlimited random fields, are Jackson–type inequalities.
Ending this exposure, we have to remark that Theorems 3, 4 are more condensed than Theorems 1 and 2 due to the stochastic nature of the input signals, concerning to the spectral representations either of the stochastic input processes or their correlation functions.
Finally, the generalized sampling expansion operator gives a highly elastic tool in generating very different sampling results manipulating with this operator in several directions: derivative sampling, approximative sampling (considering non-bandlimited input signals), multi-channel sampling etc. The modulus of continuity formulation of the earned results, compare (d) in Theorem 4, gives an excellent insight into analysis background of probabilistic setting of the aliasing approximation error’s magnitude in function of the bandwidth of the input stochastic process by its generalized sampling expansion series.
It is very interesting, therefore it should be emphasized, that Butzer’s stochastic processes sampling results do not take into account the Karhunen–Cramér theorem. However, his research project was realized in the following way: take the generalized sampling expansion of the signal and find its correlation function; (ii) studying the resulting truncated series; now, of the correlation function, the authors describe the final behavior of this double sum. Actually, Butzer and co-authors follow the similar derivation steps as the great probabilists Kari Karhunen and Harald Cramér when they establish the above mentioned, so-called Karhunen–Cramér theorem. Karhunen’s and Cramér’s proving method was based on the isometric isomorphy between the so-called Hilbert space of the stochastic process , , say, which is the in medio closure of the linear span of the set , and the H–space
being the class of all complex valued functions integrable with respect to the spectral measure .
On the other side, the excellent real specialist for stochastic processes in the ‘Aachener Schule’ was Wolfgang Splettstösser; it was an interesting coincidence that three great sampling theoretical specialists Butzer, Splettstösser and Stens have been close not only in their research interests, but also geographically, or this wasn’t the case? We could think in an opposite direction: when they were in the same research team, the Aachen Sampling Project began? Anyway, we have the tremendous research record (consult the exhaustive references list in [B] and !) in the topic presented here, and stochastic counterpart results consist due to such beautiful formulae.
This research received no external funding.
The author is highly grateful to the organizers of the Harmonic Analysis and Applications - Strobl’18 conference in Strobl, Austria, held June 4-8, 2018, to invent and organize a special section ’Sampling Theory after Paul Butzer’ to honor the 90th birthday of Paul Leo Butzer. Thanking to this initiative the author has the excellent opportunity to expose his homage talk entitled On generalized derivative sampling series expansion in which several parts of this work were presented. In addition, the author is indebted to all three anonymous referees for their careful reading, constructive suggestions and reorganization requests of the first draft of the manuscript. By their kind guiding process, this survey note encompasses its final desired form.
Conflicts of Interest
The author declares no conflict of interest.
Kotel’nikov, V.A. On the carrying capacity of the “ether” and wire in telecommunications. In Material for the First USSR, All-Union Conference and Questions of Communications; Izdanie Red. Upr. Svyazi RKKA: Moscow, Russia, 1933; pp. 1–19, Reprint in Benedetto, J.J., Ferreira, P.J.S.G. (Eds.) Modern Sampling Theory: Mathematics and Applications; Birkhäuser, Boston, MA, USA, 2000. [Google Scholar]
Shannon, C.E. A mathematical theory of communication. Bell Sys. Tech. J.1948, 27, 623–657. [Google Scholar] [CrossRef]
Pogány, T. Almost sure sampling restoration of bandlimited stochastic signals. In Sampling Theory in Fourier and Signal Analysis: Advanced Topics; Higgins, J.R., Stens, R.L., Eds.; Oxford University Press: Oxford, UK, 1999; pp. 203–232, 284–286. [Google Scholar]
Piranašvili, Z.A. The problem of interpolation of random processes. Teor. Verojatnost. Primenen.1967, 12, 708–717. (In Russian) [Google Scholar]
Honda, I. On the rate of the central limit theorem for a class of linear processes. Rep. Statist. Appl. Res. Jpn. Sci. Eng.1979, 26, 18–28. [Google Scholar]
Pogány, T. On the perfect linear prediction of bandlimited stationary stochastic processes. In Proceedings of the KOI’91; Neralić, L., Ed.; Ekonomski Fakultet: Zagreb, Croatia, 1991; pp. 259–264. [Google Scholar]
Yaglom, A.M. An Introduction to the Theory of Stationary Random Functions, Revised English ed.; Silverman, R.A., Ed.; Prentice-Hall, Inc.: Englewood Cliffs, NJ, USA, 1962. [Google Scholar]
Pogány, T. An approach to the sampling theorem for continuous time processes. Aust. J. Statist.1989, 31, 427–432. [Google Scholar] [CrossRef]
Karhunen, K. Über lineare Methoden in der Wahrscheinlichkeitsrechnung. Ann. Acad. Sci. Fennicae Ser. A1947, 37, 3–79. [Google Scholar]
Cramér, H. A contribution to the theory of stochastic processes. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability; University of California Press: Berkeley/Los Angeles, CA, USA, 1951; pp. 329–339. [Google Scholar]
Yaglom, A.M. Correlation Theory of Stationary and Related Random Functions I. Basic Results; Springer Series in Statistics; Springer: New York, NY, USA, 1987. [Google Scholar]
de la Vallée Poussin, C. Sur la meilleure approximation des fonctions d’une variable réelle par des expressions d’ordre donné. C. R. Acad. Sci. Paris1918, 166, 799–802. [Google Scholar]
Brown, J.R., Jr. On mean-square aliasing error bound in cardinal sampling expansion of random processes. IEEE Trans. Inf. Theory1978, IT-24, 254–256. [Google Scholar] [CrossRef]
Pogány, T. On the aliasing error upper bound for homogeneous random fields. Singal Process.1993, 33, 127–129. [Google Scholar] [CrossRef]
Görlich, E.; Krieg, A.; Nessel, R.J.; Stens, R.L.; Paul, L. Butzer–Five years as professor emeritus. Results Math.1998, 34, 20–31. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely
those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or
the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas,
methods, instructions or products referred to in the content.