On the Rate-Distortion Function of Sampled Cyclostationary Gaussian Processes

Man-made communications signals are typically modelled as continuous-time (CT) wide-sense cyclostationary (WSCS) processes. As modern processing is digital, it is applied to discrete-time (DT) processes obtained by sampling the CT processes. When sampling is applied to a CT WSCS process, the statistics of the resulting DT process depends on the relationship between the sampling interval and the period of the statistics of the CT process: When these two parameters have a common integer factor, then the DT process is WSCS. This situation is referred to as synchronous sampling. When this is not the case, which is referred to as asynchronous sampling, the resulting DT process is wide-sense almost cyclostationary (WSACS). The sampled CT processes are commonly encoded using a source code to facilitate storage or transmission over wireless networks, e.g., using compress-and-forward relaying. In this work, we study the fundamental tradeoff between rate and distortion for source codes applied to sampled CT WSCS processes, characterized via the rate-distortion function (RDF). We note that while RDF characterization for the case of synchronous sampling directly follows from classic information-theoretic tools utilizing ergodicity and the law of large numbers, when sampling is asynchronous, the resulting process is not information stable. In such cases, the commonly used information-theoretic tools are inapplicable to RDF analysis, which poses a major challenge. Using the information-spectrum framework, we show that the RDF for asynchronous sampling in the low distortion regime can be expressed as the limit superior of a sequence of RDFs in which each element corresponds to the RDF of a synchronously sampled WSCS process (yet their limit is not guaranteed to exist). The resulting characterization allows us to introduce novel insights on the relationship between sampling synchronization and the RDF. For example, we demonstrate that, differently from stationary processes, small differences in the sampling rate and the sampling time offset can notably affect the RDF of sampled CT WSCS processes.


I. INTRODUCTION
Man-made signals are typically generated using a repetitive procedure, which takes place at fixed intervals.The resulting signals are thus commonly modeled as continuous-time (CT) random processes exhibiting periodic statistical properties [1]- [3], which are referred to as wide-sense cyclostationary (WSCS) processes.In digital communications, where the transmitted waveforms commonly obey the WSCS model [3], the received CT signal is first sampled to obtain a discrete-time (DT) received signal.
In the event that the sampling interval is commensurate with the period of the statistics of the CT WSCS signal, cyclostationarity is preserved in DT [3,Sec. 3.9].In this work, we refer to this situation as synchronous sampling.However, it is practically common to encounter scenarios in which the sampling rate at the receiver and symbol rate of the received CT WSCS process are incommensurate, which is referred to as asynchronous sampling.The resulting sampled process in such cases is a DT wide-sense almost cyclostationary (WSACS) stochastic process [3,Sec. 3.9].
This research aims at investigating lossy source coding for asynchronously sampled CT WSCS processes.In the source coding problem, every sequence of information symbols from the source is mapped into a sequence of code symbols, referred to as codewords, taken from a predefined codebook.In lossy source coding, the source sequence is recovered up to a predefined distortion constraint, within an arbitrary small tolerance of error.The figure-of-merit for lossy source coding is the rate-distortion function (RDF) which characterizes the minimum number of bits per symbol required to compress the source sequence such that it can be reconstructed at the decoder within the specified maximal distortion [4].For an independent and identically distributed (IID) random source process, the RDF can be expressed as the minimum mutual information between the source variable and the reconstruction variable, such that for the corresponding conditional distribution of the reconstruction symbol given the source symbol, the distortion constraint is satisfied [5,Ch. 10].The source coding problem has been further studied in multiple different scenarios, including the reconstruction of a single source at multiple destinations [6] and the reconstruction of multiple correlated stationary Gaussian sources at a single destination [7]- [9].
For stationary source processes, ergodicity theory and the asymptotic equipartition property (AEP) [5,Ch. 3] were applied for characterizing the RDF for different scenarios [10,Ch. 9], [4, Sec.I], [11].However, as in a broad range of applications, including digital communication networks, most CT signals are WSCS, the sampling operation results in a DT source signal whose statistics depends on the relationship between the sampling rate and the period of the statistics of the source signal.When sampling is synchronous, the resulting DT source signal is WSCS [3,Sec. 3.9].The RDF for lossy compression of DT WSCS Gaussian sources with memory was studied in [12].This used the fact that any WSCS signal can be transformed into a set of stationary subprocess [2]; thereby facilitating the application of information-theoretic results obtained for multivariate stationary sources to the derivation of the RDF; Nonetheless, in many digital communications scenarios, the sampling rate and the symbol rate of the CT WSCS process are not related in any way, and are possibly incommensurate, resulting in a sampled process which is a DT WSACS stochastic process [3,Sec. 3.9].Such situations can occur as a result of the a-priori determined values of the sampling interval and the symbol duration of the WSCS source signal, as well as due to sampling clock jitters resulting from hardware impairments.A comprehensive review of trends and applications for almost cyclostationary signals can be found in [13].Despite their apparent frequent occurrences, the RDF for lossy compression of WSACS sources was not characterized, which is the motivation for the current research.A major challenge associated with characterizing fundamental limits for asynchronously sampled WSCS processes stems from the fact that the resulting processes are not information stable, in the sense that its conditional distribution is not ergodic [14, Page X], [15], [16].
As a result, the standard information-theoretic tools cannot be employed, making the characterization of the RDF a very challenging problem.
Our recent study in [17] on channel coding reveals that for the case of additive CT WSCS Gaussian noise, capacity varies significantly with sampling rates, whether the Nyquist criterion is satisfied or not.
In particular, it was observed that the capacity can change dramatically with minor variations in the sampling rate, causing it to switch from synchronous sampling to asynchronous sampling.This is in direct contrast to the results obtained for wide-sense stationary noise for which the capacity remains unchanged for any sampling rate above the Nyquist rate [18].A natural fundamental question that arises from this result is how the RDF of a sampled Gaussian source process varies with the sampling rate.As a motivating example, one may consider compress-and-forward (CF) relaying, where the relay samples at a rate which can be incommensurate with the symbol rate of the incoming communications signal.
In this work, we employ the information spectrum framework [14] in characterizing the RDF of asynchronously sampled memoryless Gaussian WSCS processes, as this framework is applicable to the information-theoretic analysis of non information-stable processes [14,Page VII].We further note that while rate characterizations obtained using information spectrum tools and its associated quantities may be difficult to evaluate [14,Remark 1.7.3],here we obtain a numerically computable characterization of the RDF.In particular, we focus on the mean squared error (MSE) distortion measure in the low distortion regime, namely, source codes for which the average MSE of the difference between the source and the reproduction process is not larger than the minimal source variance.The results of this research lead to accurate modelling of signal compression in current and future digital communications systems.Furthermore, we utilize our characterization of the RDF RDF for a sampled CT WSCS Gaussian source with different sampling rates and sampling time offsets.We demonstrate that, differently from stationary signals, when applying a lossy source code a sampled WSCS process, the achievable rate-distortion tradeoff can be significantly affected by minor variations in the sampling time offset and the sampling rate.Our results thus allow identifying the sampling rate and sampling time offsets which minimize the RDF in systems involving asynchronously sampled WSCS processes.
The rest of this work is organised as follows: Section II provides a scientific background on cyclostationary processes, and on rate-distortion analysis of DT WSCS Gaussian sources.Section III presents the problem formulation and auxiliary results, and Section IV details the main result of RDF characterization for sampled WSCS Gaussian process.Numerical examples and discussions are addressed in Section V, and Section VI concludes the paper.

II. PRELIMINARIES AND BACKGROUND
In the following we review the main tools and framework used in this work: In Subsection II-A, we detail the notations.In Subsection II-B we review the basics of cyclostationary processes and the statistical properties of a DT process resulting from sampling a CT WSCS process.In Subsection II-C, we recall some preliminaries in rate-distortion theory, and present the RDF for a DT WSCS Gaussian source process.This background creates a premise for the statement of the main result provided in Section IV of this paper.

A. Notations
In this paper, random vectors are denoted by boldface uppercase letters, e.g., X; boldface lowercase letters denote deterministic column vectors, e.g., x.Scalar RVs and deterministic values are denoted via standard uppercase and lowercase fonts respectively, e.g., X and x.Scalar random processes are denoted with X(t), t ∈ R for CT and with X[n], n ∈ Z for DT.Uppercase Sans-Serif fonts represent matrices, e.g., A, and the element at the i th row and the l th column of A is denoted with (A) i,l .We

B. Wide-Sense Cyclostationary Random Processes
Here, we review some preliminaries in the theory of cyclostationarity.We begin by recalling the definition of wide-sense cyclostationary processes: Definition 1 (Wide-sense cyclostationary processes [2, Sec.17.2]).A scalar stochastic process {S(t)} t∈T , where T is either discrete (T = Z) or continuous (T = R) is called WSCS if both its first-order and its second-order moments are periodic with respect to t ∈ T with some period N p ∈ T .
WSCS signal are thus random processes whose first and second-order moments are periodic functions.
To define WSACS signals, we first recall the definition of almost-periodic functions: Definition 2 (Almost-periodic-function [19]).A function x(t), t ∈ T where T is either discrete (T = Z) or continuous (T = R), is called an almost-periodic function if for every ǫ > 0 there exists a number l(ǫ) > 0 with the property that any interval in T of length l(ǫ) contains a τ , such that Definition 3 (Wide-sense almost-cyclostationary processes [2, Def.17.2]).A scalar stochastic process {S(t)} t∈T where T is either discrete (T = Z) or continuous (T = R), is called WSACS if its first and its second order moments are almost-periodic functions with respect to t ∈ T .
3.2].The example above demonstrates that the statistical properties of sampled WSCS signals depend on the sampling rate and the sampling time offset, implying that the RDF of such processes should also depend on these quantities, as we demonstrate in the sequel.

C. The Rate-Distortion Function for DT WSCS Processes
In this subsection we review the source coding problem and the existing results on the RDF of WSCS processes.We begin by recalling the definition of a source coding scheme, see, e.g., [26, ch. 3], [5, Ch.10]: Definition 4 (Source coding scheme).A source coding scheme with blocklength l consists of: 1) An encoder f S which maps a block of l source samples The encoder-decoder pair is referred to as an (R, l) source code, where R is the rate of the code in bits per source sample, defined as: The RDF characterizes the minimal average number of bits per source sample, denoted R(D), that can be used to encode a source process such that it can be reconstructed from its encoded representation with a recovery distortion not larger than D > 0 [5, Sec.10.2].In the current work, we use the MSE distortion measure, which measures the cost of decoding a source symbol S into Ŝ via d(S, Ŝ) = S − Ŝ

2
. The distortion for a sequence of source samples S (l) decoded into a reproduction sequence Ŝ(l) is given by and the average distortion in decoding a random source sequence S (l) into a random reproduction sequence Ŝ(l) is defined as: where the expectation in ( 2) is taken with respect to the joint probability distributions on the source Definition 6.The rate-distortion function R(D) is defined as the infimum of all achievable rates R for a given maximum allowed distortion D.
Def. 5 defines a rate-distortion pair to as that achievable using source codes with any sufficiently large blocklength.In the following lemma, which is required to characterize the RDF of DT WSCS signals, we state that it is sufficient to consider only source codes whose blocklength is an integer multiple of some fixed integer: Lemma 1.Consider the process{S[i]} i∈N with a finite and bounded variance.For a given maximum allowed distortion D, the optimal reproduction process { Ŝ[i]} i∈N is also the optimal reproduction process when restricted to using source codes whose blocklengths are integer multiples of some fixed positive integer r.
Proof.The proof of the lemma is detailed in Appendix A.
This lemma facilitates switching between multivariate and scalar representations of the source and the reproduction processes.
The RDF obviously depends on the distribution of the source {S[i]} i∈N .Thus, modifying the source yields a different RDF.However, when a source is scaled by some positive constant, the RDF of the scaled process with the MSE criterion can be inferred from that of the original process, as stated in the following theorem: where where D m min σ 2 m , θ , and θ is defined such that Proof.Applying Equations (6a) and (6b) to our specific case of a memoryless WSCS source, we obtain equations (7a) and (7b) as follows: First note that the corresponding DCD components for a zero-mean memoryless WSCS process are also zero-mean and memoryless; hence the PSD matrix for the multivariate process S (Np) [i] is a diagonal matrix, whose eigenvalues are the constant diagonal elements such that the m th diagonal element is equal to the variance σ 2 m : λ m e j2πf = σ 2 m .Now, writing Eqn.(6a) for this case we obtain: Since Dm it follows that (8) coincides with (7a).Next, expressing Eqn.(6b) for the memoryless source process, we obtain: proving Eqn.(7b).Now, from Lemma 1, we conclude that the RDF for compression of source sequences whose blocklength is an integer multiple of N p is the same as the RDF for compressing source sequences whose blocklength is arbitrary.We recall that from [5,Ch. 10.3.3] it follows that for the zero-mean memoryless Gaussian DCD vector source process S (Np) [i] the optimal reproduction process which achieves the RDF is an N p × 1 memoryless process whose covariance matrix is diagonal with non-identically distributed elements.From [2], we can apply the inverse DCD to obtain a WSCS process.Hence, from Lemma 1 we can conclude that the optimal reproduction process for the DT WSCS Gaussian source is a DT WSCS Gaussian process.

III. PROBLEM FORMULATION AND AUXILIARY RESULTS
Our objective is to characterize the RDF for compression of asynchronously sampled CT WSCS Gaussian sources when the sampling interval is larger than the memory of the source.In particular, we focus on the minimal rate required to achieve a high fidelity reproduction, representing the RDF curve for distortion values not larger than the variance of the source.Such characterization of the RDF for asynchronous sampling is essential for comprehending the relationship between the minimal required number of bits and the sampling rate at a given distortion.Our analysis constitutes an important step towards constructing joint source-channel coding schemes in scenarios in which the symbol rate of the transmitter is not necessarily synchronized with the sampling rate of the source to be transmitted.Such scenarios arise, for example, when recording a communications signal for storage or processing, or in compress-and-forward relaying [26,Ch. 16.7], [27] in which the relay compresses the sampled received signal, which is then forwarded to the assisted receiver.As the relay operates with its own sampling clock, which need not necessarily be synchronized with the symbol rate of the assisted transmitter, sampling at the relay may result in a DT WSACS source signal.In the following we first characterize the sampled source model in Subsection III-A.Then, as a preliminary step to our characterization the RDF for asynchronously sampled CT WSCS Gaussian processes stated in Section IV, we recall in Subsection III-B the definitions of some information-spectrum quantities used in this study.Finally, in Subsection III-C, we recall an auxiliary result relating the information spectrum quantities of a collection of sequences of RVs to the information spectrum quantities of its limit sequence of RVs.This result will be applied in the derivation of the RDF with asynchronous sampling.

A. Source Model
Consider a real CT, zero-mean WSCS Gaussian random process S c (t) with period T ps .Let the variance function of S c (t) be defined as σ 2 Sc (t) E S 2 c (t) , and assume it is both upper bounded and lower bounded away from zero, and that it is continuous in t ∈ R. Let τ m > 0 denote the maximal correlation we have that σ 2 Sc (t) = σ 2 Sc (t + T ps ), ∀t ∈ R. Let S c (t) be sampled uniformly with the sampling interval p+ǫ .In this work, as in [17], we assume that the duration of temporal correlation of the CT signal is shorter than the sampling interval T s , namely, τ m < T s .Consequently, the DT Gaussian process S ǫ [i] is a memoryless zero-mean Gaussian process and its autocorrelation function is given by: While we do not explicitly account for sampling time offsets in our definition of the sampled process It can be noted from ( 10) that if ǫ is a rational number, i.e., ∃u, v ∈ N , u and v are relatively prime, such For this class of processes, the RDF can be obtained from [12, Thm.1] as stated in Corollary 1.On the other hand, if ǫ is an irrational number, then sampling becomes asynchronous and leads to a WSACS process whose RDF has not been characterized to date.

B. Definitions of Relevant Information Spectrum Quantities
Conventional information theoretic tools for characterizing RDFs are based on an underlying ergodicity of the source.Consequently, these techniques cannot be applied to characterize the RDF of of asynchronously sampled WSCS processes.To tackle this challenge, we use information spectrum methods.
The information spectrum framework [14] can be utilized to obtain general formulas for rate limits for any arbitrary class of processes.The resulting expressions do are not restricted to specific statistical models of the considered processes, and in particular, do not require information stability or stationarity.
In the following, we recall the definitions of several information-spectrum quantities used in this study, see also [14,Def. 1.3.1-1.3.2]:Definition 7. The limit-inferior in probability of a sequence of real RVs {Z k } k∈N is defined as Hence, α 0 is the largest real number satisfying that ∀ α < α 0 and ∀µ > 0 there exists k 0 (µ, α) ∈ N such that Pr(Z k < α) < µ, ∀k > k 0 (µ, α).
Definition 8.The limit-superior in probability of a sequence of real RVs {Z k } k∈N is defined as Hence, β 0 is the smallest real number satisfying that ∀ β > β 0 and ∀µ > 0, there exists The notion of uniform integrability of a sequence of RVs is a basic property in probability [28, Ch.
12], which is not directly related to information spectrum methods.However, since it plays an important role in the information spectrum characterization of RDFs, we include its statement in the following definition: Definition 9 (Uniform Integrability [28, Def.12.1], [14, Eqn.(5.3. 2)]).The sequence of real-valued The aforementioned quantities allow characterizing the RDF of arbitrary sources.Consider a general source process {S[i]} ∞ i=1 (stationary or non-stationary) taking values from the source alphabet S[i] ∈ S and a reproduction process { Ŝ[i]} ∞ i=1 with values from the reproduction alphabet Ŝ[i] ∈ Ŝ.It follows from [14,Sec. 5.5] that for a distortion measure which satisfies the uniform integrability criterion, i.e., that there exists a deterministic sequence {r[i]} ∞ i=1 such that the sequence of RVs {d S (k) , r (k) } ∞ k=1 satisfies Def. 9 [14, Pg. 336], then the RDF is expressed as [14, Eqn.(5.4.2)]: where , and Ī S (k) : Ŝ(k) represents the limit superior in probability of the mutual information rate of S (k) and Ŝ(k) , given by: In order to use the RDF characterization in ( 14), the distortion measure must satisfy the uniform integrability criterion.For the considered class of sources detailed in Subsection III-A, the MSE distortion satisfies this criterion, as stated in the following lemma: Lemma 2. For any real memoryless zero-mean Gaussian source max for all i ∈ N , the MSE distortion satisfies the uniform integrability criterion.
Proof.Set the deterministic sequence {r[i]} ∞ i=1 to be the all-zero sequence.Under this setting and the MSE distortion, it holds that d To prove the lemma, we show that the sequence of RVs d S (k) , r (k) ∞ k=1 has a bounded ℓ 2 norm, which proves that it is uniformly integrable by [28,Cor. 12.8].The where Since, as detailed in Subsection III-A, we focus in the following on memoryless zero-mean Gaussian sources, Lemma 2 implies that the RDF of the source can be characterized using (14).However, ( 14) is in general difficult to evaluate, and thus does not lead to a meaningful understanding of how the RDF of sampled WSCS sources behaves, motivating our analysis in Section IV.

C. Information Spectrum Limits
The following theorem originally stated in [17, Thm.1], presents a fundamental result which is directly useful for the derivation of the RDF: which explicitly considers only (17a).

A. Main Result
Using the information spectrum based characterization of the RDF ( 14) combined with the characterization of the limit of a sequence of information spectrum quantities in Theorem 3, we now analyze the RDF of asynchronously sampled WSCS processes.Our analysis is based on formulating a sequence of synchronously sampled WSCS processes, whose RDF is given in Corollary 1.Then, we show that the RDF of the asynchronously sampled process can be obtained as the limit superior of the computable RDFs of the sequence of synchronously sampled processes.We begin by letting ǫ n ⌊n•ǫ⌋ n for n ∈ N and defining a Gaussian source process S n [i] = S c i•Tps p+ǫn .From the discussion in Subsection III-A (see also [17, Sec.II.C]), it follows that since ǫ n is rational, S n [i] is a WSCS process and its period is given by p n = p • n + ⌊n • ǫ⌋.Accordingly, the periodic correlation function of S n [i] can be obtained similarly to (10) as: Due to cyclostationarity of where for D ≤ and is given by: It also follows from [5,Sec. 10.3.3] that for the IID Gaussian multivariate process whose entries are independent and distributed via S (pn) ), m ∈ {1, 2, . . ., p n }, the optimal reconstruction vector process Ŝ(pn) n [i] and the corresponding noise vector process W (pn) a multivariate Gaussian distribution: We are now ready to state our main result, which is the RDF of the asynchronously sampled DT source i=1 can be obtained as the limit: where R n (D) is defined Prop. 1.
Proof.The detailed proof is provided in Appendix D. Here, we give a brief outline: The derivation of the RDF with asynchronous sampling follows three steps: First, we note that sampling rate T s (n) = Tps p+ǫn used to obtain the sequence of DT WSCS sources {S n [i]} i∈N ,n∈N asymptotically approaches the sampling interval for irrational ǫ given by T s = Tps p+ǫ as n → ∞.We define a sequence of rational numbers ǫ n s.t.ǫ n → ǫ as n → ∞; Building upon this insight, we prove that the RDF with T s can be stated as a double limit where the outer limit is with respect to the blocklength and the inner limit is with respect to ǫ n .
Lastly, we use Theorem 3 to show that the limits can be exchanged, obtaining a limit of expressions which are computable.

Remark 1. Theorem 4 focuses on the low distortion regime, defined as the values of
. We note that since every element of the vector W

B. Discussion and Relationship with Capacity Derivation in [17]
Theorem 4 provides a meaningful and computable characterization for the RDF of sampled WSCS signals.We note that the proof of the main theorem uses some of the steps used in our recent study on the capacity of memoryless channels with sampled CT WSCS Gaussian noise [17].It should be emphasized, however, that there are several fundamental differences between the two studies, which require the introduction of new treatments and derivations original to the current work.First, it is important to note that in the study on capacity, a physical channel model exists, and therefore the conditional PDF of the output signal given the input signal can be characterized explicitly for both synchronous sampling and asynchronous sampling for every input distribution.For the current study of the RDF we note that the relationship (21), commonly referred to as the backward channel [30], [5,Ch.10.3.2],characterizes the relationship between the source process and the optimal reproduction process, and hence is valid only for synchronous sampling and the optimal reproduction process.Consequently, in the RDF analysis the limiting relationship (21) as n → ∞ is not even known to exist and, in fact, we can show it exists under a rather strict condition on the distortion (namely, the condition D < min 0≤t≤Tps σ 2 Sc (t) stated in Theorem 4).In particular, to prove the statement in Theorem 4, we had to show that from the backward channel (21), we can define an asymptotic relationship, as n → ∞, which corresponds to the asynchronously sampled source process, denoted by S ǫ [i], and relates S ǫ [i] with its optimal reconstruction process Ŝǫ [i].This is done by showing that the PDFs for the reproduction process Ŝn [i] and noise process W n [i] from (21), each converge uniformly as n → ∞ to a respective limiting PDF, which has to be defined as well.This enabled us to relate the RDFs for the synchronous sampling and for the asynchronous sampling cases using Theorem 3, eventually leading to (22).Accordingly, in our detailed proof of Theorem 4 given in Appendix D, Lemmas D.4 and D.6 as well as a significant part of Lemma D.2 are largely new, addressing the special aspects of the proof arising from the fundamental differences between current setup and the setup in [17], while the derivations of Lemmas D-A and D.5 follow similarly to [  Similarly In the following, we model the time varying variance of the WSCS source σ 2 Sc (t) to be a linear periodic function of Π tdc,trf (t).To that aim, we define a time offset between the first sample and the rise start time of the periodic continuous pulse function; we denote the time offset by φ ∈ [0, 1).This corresponds to the sampling time offset normalized to the period T ps .The variance of S c (t) is periodic function with period T ps which is given by with period of T ps = 5 µsecs.

A. Convergence of R n (D) in n
From Theorem 4 it follows that if the distortion satisfies D < min 0≤t≤Tps σ 2 Sc (t), the RDF of the asynchronously sampled CT WSCS Gaussian process is given by the limit superior of the sequence {R n (D)} n∈N ; where R n (D) is obtained via Corollary 1.In this subsection, we study the sequence of RDFs {R n (D)} n∈N as n increases.For this evaluation setup, we fixed the distortion constraint at D = 0.18 and set ǫ = π 7 and p = 2. Let the variance of the CT WSCS Gaussian source process σ 2 Sc (t) be modelled by Eq. ( 24) for two sampling time offsets φ = {0, 1  16 }.For each offset φ, four duty cycle values were considered: t dc = [20, 45, 75, 98]%.For each n we obtain the synchronous sampling mismatch ǫ n ⌊n•ǫ⌋ n , which approaches ǫ as n → ∞, where n ∈ N .Since ǫ n is a rational number, corresponding to a sampling period of T s (n) = Tps p+ǫn , then for each n, the resulting DT process is WSCS with the period p n = p • n + ⌊n • ǫ⌋ and its RDF follows from Corollary 1.
Figures 2 and 3 depict R n (D) for n ∈ [1, 500] with the specified duty cycles and sampling time offsets, where in Fig. 2 there is no sampling time offset, i.e., φ = 0, and in Fig. 3 the sampling time offset is set to φ = 1  16 .We observe that in both figures the RDF values are higher for higher t dc .This can be explained by noting that for higher t dc values, the resulting time averaged variance of the DT source process increases, hence, a higher number of bits per sample is required to encode the source process maintaining the same distortion value.Also, in all configurations, R n (D) varies significantly for smaller values of n.Comparing Figures 2 and 3, we see that the pattern of these variations depends on the sampling time offset φ.For example, when t dc = 45% at n ∈ [4,15], then for φ = 0 the RDF varies in the range [1.032, 1.143] bits per sample, while for φ = 1  16 the RDF varies in the range [1.071, 1.237] bits per sample.However, as we increase n above 230, the variations in R n (D) become smaller and are less dependent on the sampling time offset, and the resulting values of R n (D) are approximately in the same range in both Figures 2 and 3 for n ≥ 230.This behaviour can be explained by noting that as n varies, the period p n also varies and hence the statistics of the DT variance differs over its respective period.This consequently affects the resulting RDF (especially for small periods).As n increases ǫ n approaches the asynchronous sampling mismatch ǫ and the period p n takes a sufficiently large value such that the samples of the DT variance over the period are similarly distributed irrespective of the value of φ; leading to a negligible variation in the RDF as seen in the above figures.

B. The Variation of the RDF with the Sampling Rate
Next, we observe the dependence of the RDF for the sampled memoryless WSCS Gaussian process on the value of the sampling interval T s .For this setup, we fix the distortion constraint to D = 0.18 and set the duty cycle in the source process (24) to t dc = [45, 75]%.Figures 4-5 demonstrate the numerically evaluated values for R n (D) at sampling intervals in the range 2 < Tps Ts < 4 with the sampling time offsets φ = 0 and φ = 1 16 , respectively.A very important insight which arises from the figures is that the sequence of RDFs R n (D) is not convergent; hence, for example, one cannot approach the RDF for Tps Ts = 2.5 by simply taking rational values of Tps Ts which approach 2.5.This verifies that the RDF for asynchronous sampling cannot be obtained by straightforward application of previous results, and indeed,  the entire analysis carried in the manuscript is necessary for the desired characterization.We observe in Figures 4-5 that when Tps Ts has a fractional part with a relatively small integer denominator, the variations in the RDF are significant, and the variations depend on the sampling time offset.However, when Tps Ts approaches an irrational number, the period of the sampled variance function becomes very long, and consequently, the RDF is approximately constant and independent of the sampling time offset.As an example, consider Tps Ts = 2.5 and t dc = 75%: For sampling time offset φ = 0 the RDF takes a value of 1.469 bits per sample, as shown in Figure 4 while for the offset of φ = 1  16 the RDF peaks to 1.934 bits per sample as we see in Figure 5. On the other hand, when approaching asynchronous sampling, the RDF takes an approximately constant value 1.85 bits per sample for all the considered values of Tps Ts and this value is invariant to the offsets of φ.This follows since when the denominator of the fractional part of Tps Ts increases, then the DT period of the resulting sampled variance, p n , increases and practically captures the entire set of values of the CT variance regardless of the sampling time offset.In a similar manner as with the study on capacity in [17], we conjecture that since asynchronous sampling captures the entire set of values of the CT variance, the respective RDF represents the RDF of the analog source, which does not depend on the specific sampling rate and offset.Figures 4-5 demonstrate how slight variations in the sampling rate can result in significant changes in the RDF.For instance, at φ = 0 we notice in Figure 4 that when the sampling rate switches from T s = 2.25 • T ps to T s = 2.26 • T ps , i.e., the sampling rate switches from being synchronous to being nearly asynchronous, and the RDF changes from 1.624 bits per channel use to 1.859 bits per sample for t dc = 75%; also, we observe in Figure 5 for t dc = 45%, that when the sampling rate switches from T s = 2.5 • T ps to T s = 2.51 • T ps , i.e., the sampling rate also switches from being synchronous to being nearly asynchronous, and the RDF changes from 1.005 bits per source sample to 1.154 bits per source sample.
Lastly, Figures 6-7   the sampling time offsets of 0 and 1 16 respectively.At each φ, the result is evaluated at three different values of synchronization mismatch ǫ.For this setup, we fix t dc = 75%, p = 2 and ǫ ∈ {0.5, 5π 32 , 0.6}.The only mismatch value that refers to the asynchronous sampling case is ǫ = 5π 32 and its corresponding sampling interval is approximately µsecs, which is a negligible variation from the sampling intervals corresponding to ǫ ∈ {0.5, 0.6}, which are 2.000 µsecs and 1.923 µsecs, respectively.Observing both figures, we see that the RDF may vary significantly for very slight variation in the sampling rate.For instance, as shown in Figure 6 for φ = 0, at D = 0.18, a slight change in the synchronization mismatch from ǫ = 5π 32 (i.e., T s ≈ 2.007µsecs) to ǫ = 0.5 (i.e., T s = 2.000µsecs) results to approximately 20% decrease in the RDF.For φ = 1  16 the same change in the sampling synchronization mismatch at D = 0.18 results to a rise in the RDF by roughly 4%.These results demonstrate the unique and counter-intuitive characteristics of the RDF of sampled WSCS signals which arise from our derivation.

VI. CONCLUSIONS
In this work the RDF of a sampled CT WSCS Gaussian source process was characterized for scenarios in which the resulting DT process is memoryless.This characterization shows the relationship between the sampling rate and the minimal number of bits required for compression at a given distortion.For cases in which the sampling rate is synchronized with the period of the statistics of the source process, the resulting DT process is WSCS and standard information theoretic framework can be used for deriving its RDF.
For asynchronous sampling, information stability does not hold, and hence we resort to the information spectrum framework to obtain a characterization.To that aim we derived a relationship between some relevant information spectrum quantities for uniformly convergent sequences of RVs.This relationship was further applied to characterize the RDF of an asynchronously sampled CT WSCS Gaussian source process as the limit superior of a sequence of RDFs, each corresponding to the synchronous sampling of the CT WSCS Gaussian process.The results were derived in the low distortion regime, i.e., under the condition that the distortion constraint D is less than the minimum variance of the source, and for sampling intervals which are larger than the correlation length of the CT process.Our numerical examples give rise to nonintuitive insights which follow from the derivations.In particular, the numerical evaluation demonstrates that the RDF for a sampled CT WSCS Gaussian source can change dramatically with minor variations in sampling rate and sampling time offset.In particular, when the sampling rate switches from being synchronous to being asynchronous and vice versa, the RDF may change considerably as the statistical model of the source switches between WSCS to WSACS.The resulting analysis enables determining the sampling system parameters in order to facilitate accurate and efficient source coding of acquired CT signals.

APPENDIX A PROOF OF LEMMA 1
Proof.To prove that the minimum achievable rate at a given maximum distortion for a code with arbitrary blocklength can be achieved by considering only codes whose blocklength is an integer multiple of r, we apply the following approach: We first show that every rate-distortion pair achievable when restricted to using source codes whose blocklength is an integer multiple of r is also achievable when using arbitrary blocklenghts; We then prove that every achievable rate-distortion pair is also achievable when restricted to using codes whose blocklength is an integer multiple of r.Combining these two assertions proves that the rate-distortion function of the source {S[i]} i∈N can be obtained when restricting the blocklengths to be an integer multiple of r.Consequently, a reproduction signal { Ŝ[i]} i∈N which achieves the minimal rate for a given D under the restriction to use only blocklengths which are an integer multiple of r is also the reproduction signal achieving the minimal rate without this restriction, and vice versa, thus proving the lemma.
To prove the first assertion, consider a rate-distortion pair (R, D) which is achievable when using codes whose blocklength is an integer multiple of r.It thus follows directly from Def. 5 that for every η > 0, We now show that we can construct a code with an arbitrary blocklength l = b • r + j where 0 < j < r (i.e., the blocklength l is not an integer multiple of r) satisfying The rate Consequently, any rate-distortion pair achievable with codes whose blocklength is an integer multiple of r can be achieved by codes with arbitrary blocklengths.
Next, we prove that any achievable rate-distortion pair (R, D) can be achieved by codes whose blocklength is an integer multiple of r.To that aim, we fix η > 0. By Def. 5, it holds that there exists a code of blocklength l satisfying (3)-(4).To show that (R, D) is achievable using codes whose blocklength is an integer multiple of r, we assume here that l is not an integer multiple of r, hence, there exist some positive integers b and j such that j < r and l = b • r + j.We denote this code by The average distortion can now be computed as follows: and again ∃b > b o such that The rate R (b+1)•r can be expressed as follows: It follows that R (b+1)•r ≤ R + η for any arbitrary η by selecting a sufficiently large b.This proves that every rate-distortion pair achievable with arbitrary blocklengths (e.g., l = b • r + j, j < r) is also achievable when considering source codes whose blocklength is an integer multiple of r (e.g., l = b • r).
This concludes the proof.

APPENDIX B PROOF OF THEOREM 1
Recall that α ∈ R ++ .To prove the theorem, we fix a rate-distortion pair (R, D) that is achievable for the source {S[i]} i∈N .By Def. 5 this implies that for all η > 0 there exists l 0 (η) ∈ N such that for all l > l 0 (η) there exists a source code C l with rate R l ≤ R+η and MSE distortion D l = E 1 l S (l) − Ŝ(l) 2 ≤ D+η.Next, we use the code C l to define the source code C (α) l , which operates in the following manner: The encoder first scales its input block by 1/α.Then, the block is encoded using the source code C l .Finally, the selected codeword is scaled by α.Since the C (α) l has the same number of codewords and the same blocklength as C l , it follows that its rate, denote R l , it holds that its reproduction vector when applied to α • S (l) is equal to the output of C l applied to S (l) scaled by α, i.e., α • Ŝ(l) .Consequently, the MSE of C (α) l when applied to the source It thus follows that for all η > 0 there exists l0 (η) = l 0 min(η, α 2 η) such that for all l > l0 (η) there exists a code C In this appendix, we prove (17b) by applying a similar approach as used for proving (17a) in [17, Appendix A].We first note that Def. 8 can also be written as follows: For the equality (a), we note that the set of probabilities {Pr (Z k > β)} k∈N is non-negative and bounded in [0, 1]; hence, for any β ∈ R for which lim sup k→∞ Pr (Z k > β) = 0, it also holds from [31, Thm.
3.17] that the limit of any subsequence of {Pr (Z k > β)} k∈N is also 0, since non-negativity of the probability implies lim inf where (a) follows from [31,Example. 3.18(c)].This implies lim k→∞ Pr (Z k > β) exists and is equal to 0.
Consequently, for every subsequence 0 < k 1 < k 2 < . . .such that lim l→∞ Fkl,n (β) exists for any n > n 0 (η), it follows from [31,Thm. 7.11] 1 that, as the convergence over k is uniform, the limits over n and l are interchangeable: The existence of such a convergent subsequence is guaranteed by the Bolzano-Weierstrass theorem From the properties of the limit inferior [31,Thm. 3.17] 3 it follows that there exists a subsequence = lim ≥ lim 1 [31,Thm. 7.11]: Suppose fn → f uniformly in a set E in a metric space.Let x be a limit point of E, and suppose that lim t→x fn(t) = An, (n = 1, 2, 3, . ..).Then An converges, and lim Hence, we obtain that and from the right-hand side of (C.6) we have that To facilitate our proof we will need uniform convergence in k ∈ N , of p S (k) k) , respectively.To that aim, we will make the following scaling assumption w.l.o.g.: Assumption D.1.The variance of the source and the allowed distortion are scaled by some factor α 2 such that Note that this assumption has no effect on the generality of the RDF for multivariate stationary processes detailed in [5, Sec. 10.
Uniformly with Respect to k ∈ N In order to prove the uniform convergence in distribution, S ǫ , uniformly with respect to k ∈ N , we first prove, in Lemma D.1, that as n → ∞ the sequence of PDFs of S with their respective diagonal correlation matrices expressed below: Now we note that since σ 2 Sc (t) is uniformly continuous, then by the definition of a uniformly continuous function, for each i ∈ N , the limit in (D.4) implies that ; Hence, we can state the following lemma: Proof.The proof of the lemma directly follows from the steps in the proof of [17, n and S (k) ǫ also have independent entries, for the synchronously and for the asynchronously sampled case respectively.The proof for uniform convergence in k ∈ N from [17,Lemma B.1] also applies to this current work.Proof.To begin the proof, for s (k) , ŝ(k) ∈ R 2k , define Now, we recall the backward channel relationship (20): where Ŝ(k) n and W n are mutually independent zero-mean, Gaussian random vectors with independent entries, corresponding to the optimal compression process and its respective distortion.From this relationship we obtain n and Ŝ(k) n can be expressed via the conditional PDF as: where (a) follows from (D.8).Since Ŝ(k) n and W n are Gaussian and mutually independent and since the product of two multivariate Gaussian PDFs is also a multivariate Gaussian PDF [36,Sec. 3], it follows from (D.9) that S is a diagonal matrix with each diagonal element taking the value of the corresponding temporal variance at the respective index i ∈ {1, 2, . . .k}. i.e., Relationship (D.10) implies that the joint limit distribution satisfies p S (k) . Consequently, we can define an asymptotic backward channel that satisfies (D.10) via the expression: Next, by convergence of the joint PDF p and the convergence is uniform in k ∈ N and in s (k) T , ŝ(k) T T ∈ R 2k .Then, by the continuous mapping theorem (CMT) [37,Thm. 7.7], we have . Now, using the extended CMT [37,Thm. 7.24] 9 , we will show that ǫ for each k ∈ N , following the same approach of the proof for [17, Lemma B.2] 10 .Then, since , where it also follows from the proof of [17, Lemma B.2] that the 8 Please refer to the footnote 6 on page 30.n }n∈N and a zero-mean Gaussian random vector with independent entries X (k) , such that X ǫ , we will show that the following two properties hold: is separable 11 .
P2 For any convergent sequence s To prove property P1, we show that For the asynchronously sampled source process, we note that σ we have that Pr U (k) 2 > β < η, and thus U (k) is separable.
By the assumption in this lemma it follows that ∀η > 0 there exists n 0 (η) > 0 such that for all n > n 0 (η) we have that ∀w for all sufficiently large k ∈ N .Consequently, for all s (k) T , ŝ(k) T T ∈ R 2k , n > n 0 (η) and a sufficiently large k ∈ N , it follows from (D.8) that Following the continuity of p S (k) This satisfies condition P2 for the extended CMT; Therefore, by the extended CMT, we have that α), uniformly over k ∈ N .Thus, for all sufficiently small η > 0, ∃k 0 ∈ N , n 0 (η, k 0 ) ∈ N such that ∀n > n 0 (η, k 0 ), and ∀k > k 0 , the mutual information density rate converges as k increases 14 [38,Thm. 18.1]: Let (Fn) be a sequence of density functions and let φn denote the characteristic function of Fn.Suppose that g(θ) := lim φn(θ) exists for all θ ∈ R, and that g(•) is continuous at 0. Then g = φF for some distribution function F , and Fn (dist.)−→ n→∞ F . 15 Information stable channels can be described as having the property that the input that maximizes mutual information and its corresponding output behave ergodically [15].Also, the information stability was further defined in [16,Sec.IV] by applying the fact that ergodic theory is consequential to the law of large numbers.[14, Eq. (3.9.2)]:A general source V = {V n } ∞ n=1 is said to be information-stable if almost surely to the finite and deterministic mutual information rate [14, Thm.5.9.1] 16 .Since almost sure convergence implies convergence in distribution [37,Lemma 7.21], this proves the lemma.
C. Showing that R ǫ (D) = lim sup n→∞ R n (D) This section completes the proof to Theorem 4. We note from ( 14) that the RDF for the source process We now state the following lemma characterizing the asymptotic statistics of the optimal reconstruction Ŝ(k) n process and the respective noise process W  [i] has a convergent subsequence, and we let n 1 < n 2 < . . .denote the indexes of this convergent subsequence and let 16 [14, Thm.5.9.1] holds for a subadditive distortion measure [14, Eqn.(5.9.2)];The MSE distortion measure, which was used in this research, is additive (and thus also subadditive). 17Every bounded infinite subset of R k has a limit point in R k .ǫ ; Ŝ(l) ǫ ) ≤ 1 l log M l ≤ R ǫ + η.Since for every arbitrarily small η → 0, this inequality holds for all l > l 0 (η), i.e., for all sufficiently large l, it follows that R ǫ ≥ lim sup

√ − 1 .
use | • | to denote the absolute value, ⌊d⌋, d ∈ R, to denote the floor function and d + , d ∈ R, to denote the max{0, d}.δ[•] denotes the Kronecker delta function: δ[n] = 1 for n = 0 and δ[n] = 0 otherwise, and E{•} denotes the stochastic expectation.The sets of positive integers, integers, rational numbers, real numbers, positive numbers, and complex numbers are denoted by N , Z, Q, R, R ++ , and C, respectively.The cumulative distribution function (CDF) is denoted by F X (x) Pr (X ≤ x) and the probability density function (PDF) of a CT random variable (RV) is denoted by p X (x).We represent a real Gaussian distribution with mean µ and variance σ 2 by the notation N (µ, σ 2 ).All logarithms are taken to base-2, and j = Lastly, for any sequence y[i], i ∈ N , and positive integer k ∈ N , y (k) denotes the column vector y[1], . . ., y[k]T .

Fig. 1 .
Fig. 1.Source coding block diagram variance D n [m] = D for all n ∈ N and m = 1, 2, . . ., p n then by applying the inverse DCD to W (pn) n [i], the resulting scalar DT process W n [i] is wide sense stationary; and in fact IID with 17, Lemma B.1] and [17, Lemma B.5], respectively, and parts of Lemma D.2 coincide with [17, Lemma B.2]. V. NUMERICAL EXAMPLES In this section we demonstrate the insights arising from our RDF characterization via numerical examples.Recalling that Theorem 4 states the RDF for asynchronously sampled CT WSCS Gaussian process, R ǫ (D), as the limit supremum of a sequence of RDFs corresponding to DT memoryless WSCS Gaussian source processes {R n (D)} n∈N , we first consider the convergence of {R n (D)} n∈N in Subsection V-A.Next, in Subsection V-B we study the variation of the RDF of the sampled CT process due to changes in the sampling rate and in the sampling time offset.

Def. 5
for all j ∈ {1, . . ., r − 1} as follows: Apply the code R (b•r) , b • r to the first b • r samples of S[i] and then concatenate each codeword by j zeros to obtain a source code having codewords of length b • r + j.The average distortion (i.e., see (2)) of the resulting R (b•r+j) , b • r + j code is given by: we construct a code R (b+1)•r , (b + 1) • r with codewords whose length is (b + 1) • r, i.e., an integer multiple of r, by adding r − j zeros at the end of each codeword of the code R (b•r+j) , b • r + j .

n 1 β
t→x lim n→∞ fn(t) = lim n→∞ lim t→x fn(t) 2 [31, Thm.2.42]:Every bounded infinite subset of R k has a limit point in R k 3 [31, Thm.3.17]: Let {sn} be a sequence of real numbers; Let E be the set of numbers x (in the extended real number system) containing all limits of all subsequences of {sn}.Then...... lim inf n→∞ sn ∈ E. Lemma C.2.Given assumptions AS1-AS2, the sequence of RVs Zk,n k,n∈N satisfies lim n→∞ p− lim sup k→∞ Zk,n = inf β ∈ R lim n→∞ lim inf k→∞ Fk,n (β) = 1 .(C.6) Proof.Since by assumption AS1, for every n ∈ N , every convergent subsequence of Zk,n k∈N converges in distribution as k → ∞ to a deterministic scalar, it follows that every convergent subsequence of Fk,n (β) converges as k → ∞ to a step function, which is the CDF of the corresponding sublimit of Zk,n .In particular, the limit lim inf k→∞ Fk,n (β) is a step function representing the CDF of the deterministic scalar ζ n , i.e., ≥ ζ n .(C.7) Since, by Lemma C.1, AS2 implies that the limit lim n→∞ lim inf k→∞ Fk,n (β) exists 4 , then lim n→∞ ζ n exists.

n
, p S (k) n s (k) , converges to the PDF of S (k) ǫ , p S (k) ǫ s (k) , uniformly in s (k) ∈ R k and in k ∈ N .Next, we show in Corollary D.1 that S ǫ uniformly in k ∈ N .To that aim, let us define the set K {1, 2, . . ., k} and consider the kdimensional zero-mean, memoryless random vectors S

6 [ 35 ,
Thm.1]: If, for a sequence {pn(x)}n∈N of densities, lim n→∞ pn(x) = p(x) for almost all x in R, then a sufficient condition that lim n→∞ S pn(x)dx = S p(x)dx, uniformly for all Borel sets S in R, is that p(x) be a density.

F
entries and let PDFs p Ŝ(k) n ŝ(k) and p W (k) n w (k) , respectively, denote their PDFs.Consider two other zero-mean Gaussian random vectors Ŝ(k) and p W (k) ǫ w (k) , respectively, such that lim n→∞ p Ŝ(k) n ŝ(k) = p Ŝ(k) ǫ ŝ(k) uniformly in ŝ(k) ∈ R k and uniformly with respect to k ∈ N , and lim n→∞ p W (k) n w (k) = p W (k) ǫ w (k) uniformly in w (k) ∈ R k and uniformly with respect to k ∈ N .Then, the RVs Z′ k,n F opt Sn, Ŝn and Z ′ k,ǫ F Sǫ, Ŝǫ , defined via (D.1) satisfy Z′ k,n F opt Sǫ, Ŝǫ uniformly over k ∈ N .

n
, see (D.7), and (b) follows since W (k) n and Ŝ(k) n are mutually independent.The joint PDF of S (k)

0≤t≤Tps σ 2
Sc (t)].By the independence of W (k) ǫ and Ŝ(k) ǫ , and by the fact that their mean is zero, we have, from (D.14) that

S
n[i] (for fixed length coding and MSE distortion measure) is given by: R n (D) = inf F Ŝn,Sn : dS(F Ŝn,Sn )≤D p

n= σ 2 Sn [i] − σ 2 Ŝn
used in the backward channel relationship (21): Lemma D.4.Consider the RDF-achieving distribution with distortion D for compression of a vector Gaussian source process S (k) n characterized by the backward channel (21).Then, there exists a subsequence in the index n ∈ N denoted n 1 < n 2 < . .., such that for the RDF-achieving distribution, the sequences of reproduction vectors { Ŝ(k) nl } l∈N and backward channel noise vectors {W ) = p Ŝ(k) ǫ ŝ(k) uniformly in ŝ(k) ∈ R k and uniformly with respect to k ∈ N , as well as lim l→∞ p W (k) n l w (k) = p W (k) ǫ w (k) uniformly in w (k) ∈ R k and uniformly with respect to k ∈ N , where p Ŝ(k) ǫ ŝ(k) and p W (k) ǫ w (k) are Gaussian PDFs.Proof.Recall from the analysis of the RDF for WSCS processes that for each n ∈ N , the marginal distributions of the RDF-achieving reproduction process Ŝn [i] and the backward channel noise W n [i] is Gaussian, memoryless, zero-mean, and with variances σ 2 Ŝn [i] E Ŝn [i] 2 and E W n [i] 2 , the sequences of reproduction vectors { Ŝ(k) n } n∈N and backward channel noise vectors {W (k) n } n∈N are zero-mean Gaussian with independent entries for each k ∈ N .Since σ 2 Sn [i] ≤ max

kF
→ ∞.Therefore, by Theorem 3 it holds that lim l→∞ p− lim sup k→∞ Z′ k,nl F opt Sn l , Ŝn l Sǫ, Ŝǫ = R ǫ (D).(D.19)From(14) we have that R n (D) = p− lim sup k→∞ Z′ k,n F opt Sn, Ŝn , then from (D.19), it follows that R ǫ (D) ≤ lim l→∞ R nl (D) follows since, by [31, Def.3.16], the limit of every subsequence is not greater than the limit superior.Noting that F ′ Sǫ, Ŝǫ is Gaussian by Lemma D.4 concludes the proof.Lemma D.6.The RDF of {S ǫ[i]} satisfies R ǫ (D) ≥ lim sup n→∞ R n (D).Proof.To prove this lemma, we first show that for a joint distribution F Sǫ, Ŝǫ which achieves a rate-distortion pair (R ǫ , D) it holds that R ǫ ≥ E{Z ′ k,ǫ (F ′ Sǫ, Ŝǫ )}: Recall that (R ǫ , D) is an achievable ratedistortion pair for the source {S ǫ [i]}, namely, there exists a sequence of codes {C l } whose rate-distortion approach (R ǫ , D) when applied to {S ǫ [i]}, This implies that for any η > 0 there exists l 0 (η) such that ∀l > l 0 (η) it holds that C l has a code rate R l = 1 l log 2 M l satisfying R l ≤ R ǫ + η by (3).Recalling Def. 4, the source code maps S (l) ǫ into a discrete index J l ∈ {1, 2, . . ., M l }, which is in turn mapped into Ŝ(l) ǫ , i.e., S(l)ǫ → J l → Ŝ(l) ǫ form a Markov chain.Since J l is a discrete random variable taking values in {1, 2, . . ., M l }, it holds thatlog 2 M l ≥ H(J l ) (a) ≥ I(S (l) ǫ ; J l ) ǫ ; J l ) = H(J l ) − H(J l |S (l)ǫ ) which is not larger than H(J l ) as J l takes discrete values; while (b) follows from the data processing inequality [5, Ch. 2.8].Now, (D.21) implies that for each l > l 0 (η), the reproduction obtained using the code C l satisfies 1 l I(S (l)

.F
ǫ ).Hence, replacing the blocklength symbol from l to k, as1  k Let Z ′ kl,ǫ F ′ Sǫ, Ŝǫ be a subsequence of E Z ′ k,ǫ (F ′ Sǫ, Ŝǫ) with the indexes k 1 < k 2 < . . .such that its limit equals the limit superior.i.e., lim l→∞ Since by Lemma D.2, the sequence of non-negative RVs Z′ kl,n F optSn, Ŝn n∈N convergences in distribution to Z ′ kl,ǫ F ′ Sǫ, Ŝǫas n → ∞ uniformly in k ∈ N , it follows from18 [40,Thm.3.5] that E Z ′ kl,ǫ F ′ Sǫ, Ŝǫ = lim n→∞ E Z′ kl,n F opt Sn, Ŝn.Also, we define a family of distributionsF(D) such that F(D) = {F S, Ŝ : D F S, Ŝ ≤ D}.Consequently, Eq. (D.22) can now be written as:R ǫ (D) ≥ lim sup k→∞ E Z ′ k,ǫ F ′ F ′ Sǫ,Ŝ is uniform with respect to k l , thus the limits are interchangeable [31, Thm.7.11] 19 ; (b) follows since the limit of the subsequence E Z′ kl,n F opt Sn, Ŝn exists in the index n, and is therefore equivalent to the limit superior, lim sup n→∞ E Z′ kl,n F opt Sn, Ŝn [31, Page 57]; and (c) holds since mutual information is the expected value of the mutual information density rate [5, Eqn.(2.30)].Finally, we recall that in the proof of Lemma D.3 it was established that the backward channel for the RDF at the distortion constraint D, defined in (21), is information stable, hence for such backward channels, we have from [41, Thm.1] that the minimum rate is defined as R n (D) = lim k→∞ inf k.Substituting this into equation (D.23) yields the result: 18 [40, Thm.3.5] states that if Xn are uniformly integrable and Xn (dist.)Thm.7.11: Suppose fn → f uniformly in a set E in a metric space.Let x be a limit point of E.... lim t→x lim n→∞ fn(t) = lim n→∞ lim t→x fn(t) and let σ 2 Sn [i] r Sn[i, 0]denote its periodic variance.We next restate Corollary 1 in terms of ǫ n as follows: Sn [m], m = 1, 2, ..., p n , denote the variance of the m th component of S pn[i].The rate-distortion function is given by: Sn [i], obtained from S c (t) by sampling with a sampling interval of T s (n) = Tps p+ǫn .Let S pn [i] denote the memoryless stationary multivariate random process obtained by applying the DCD to S n [i] and let σ 2 Sn [m] we let D n [m] min σ 2 Sn [m], θ n , and θ n is selected s.t.
n [i] is characterized in Corollary 1 via the RDF of the multivariate stationary process S (pn) n [i] obtained via a p n -dimensional DCD applied to S n [i].Next, we recall that the relationship between the source process S (pn) n [i] and the optimal reconstruction process, denoted by Ŝ(pn) n [i], is characterized in [5, Ch. 10.3.3] via a linear, multivariate, time-invariant backward channel with a p n × 1 additive vector noise process W (pn) θ n ; θ n denotes the reverse waterfilling threshold defined in Prop. 1 for the index n, and is selected such that D = 1 the low MSE regime, i.e., at a given distortion D which is not larger than the source variance.The RDF is stated in the following theorem, which applies to both synchronous sampling as well as asynchronous sampling:Theorem 4. Consider a DT source {S ǫ [i]} ∞i=1 obtained by sampling a CT WSCS source, whose period of statistics is T ps , at intervals T s .Then, for any distortion constraint D such that D < min numerically evaluate the RDF versus the distortion constraint D ∈ [0.05, 0.19] for [34,,[34, Sec.IV].Moreover, by Theorem 1, for every α > 0 it holds that when any rate R achievable when compressing the original source S c (t) with distortion not larger that D is achievable when compressing the scaled source α • S c (t) with distortion not larger than α 2 • D. Note that if for the source S c (t) the distortion satisfies D < min Lemma B.1]5, which was applied for a Gaussian noise process with independent entries and variance above 1 2π . .In addition, since the convergence of the PDFs is uniform in k ∈ N , the convergence of the CDFs is also uniform in k ∈ N .Sǫ, Ŝǫ Satisfy the Conditions of Thm. 3 Sǫ, Ŝǫ uniformly in k ∈ N for the optimal zero-mean Gaussian reproduction vectors with independent entries.Lemma D.3 proves that for any fixed n, Z′ k,n F opt ǫ

T
[29,separable [37, Pg. 101], i.e., we show that ∀η > 0, there exists β > 0 such that Pr U (k) 2 > β < η.To that aim, recall first that by Markov's inequality[29, Pg. 114], it follows that Pr ( Sǫ, Ŝǫ and that this convergence is uniform for sufficiently large k.Finally, since the CDFs of k• Z′ k,n F opt Sǫ, Ŝǫ obtained at α ∈ R are equivalent to the CDFs of Z′ k,n F opt Lemma D.3.Let n ∈ N be given.Every subsequence of Z′ multivariate memoryless additive stationary Gaussian noise channel, which is an information stable channel [39, Sec.1.5] 15 .For such channels in which the source and its reproduction obey the RDF-achieving joint distribution F opt [21]F opt Ŝn,Sn k∈N , indexed by k l , converges in distribution, in the limit as l → ∞, to a finite deterministic scalar.Proof.Recall that the RVs Z′ k,n F opt Ŝn,Sn represent the mutual information density rate between k samples of the source process S n [i] and the corresponding samples of its reproduction process Ŝn [i],where these processes are jointly distributed via the Gaussian distribution measure F opt Ŝn,Sn.Further, recall that the relationship between the source signal and the reproduction process which achieves the RDF can be described via the backward channel in(21)for a Gaussian source.The channel (21) is a memoryless additive WSCS Gaussian noise channel with period p n , thus, by[21], it can be equivalently represented as a p n × 1