Upper Bounds for the Rate Distortion Function of Finite-Length Data Blocks of Gaussian WSS Sources

In this paper, we present upper bounds for the rate distortion function (RDF) of finite-length data blocks of Gaussian wide sense stationary (WSS) sources and we propose coding strategies to achieve such bounds. In order to obtain those bounds, we previously derive new results on the discrete Fourier transform (DFT) of WSS processes.


Introduction
In [1], Pearl gave an upper bound for the rate distortion function (RDF) of finite-length data blocks of Gaussian wide sense stationary (WSS) sources and proved that such bound tends to the RDF of the source when the size of the data block grows.However, he did not give a coding strategy to achieve his bound for a given block length.
In this paper, we present two new upper bounds for the RDF of finite-length data blocks of Gaussian WSS sources and we propose coding strategies to achieve these two bounds for a given block length.Since our bounds are tighter than the one given by Pearl, they also tend to the RDF of the source when the size of the data block grows.In order to obtain our bounds, we previously derive new results on the discrete Fourier transform (DFT) of WSS processes.
It should be mentioned that our coding strategies allow us to deal with Gaussian WSS sources as if they were memoryless.This fact can be used, for instance, to consider Gaussian WSS sources in [2].
The paper is organized as follows.In Section 2 we set up notation and review the mathematical definitions and results used in the rest of the paper.In Section 3 we obtain several results on the DFT of WSS processes which will be applied in Section 4. Finally, in Section 4 we present two new upper bounds for the RDF of finite-length data blocks of Gaussian WSS sources and we propose coding strategies to achieve such bounds.In this section, we also present a numerical example to illustrate the difference between Pearl's bound and our bounds.

Notation
In this paper, N, Z, R, and C denote the set of natural numbers (i.e., the set of positive integers), the set of integer numbers, the set of (finite) real numbers, and the set of (finite) complex numbers, respectively.R n×1 is the set of all real n-dimensional (column) vectors.I n denotes the n × n identity matrix, * stands for conjugate transpose, denotes transpose, and λ k (A), k ∈ {1, . . ., n}, are the eigenvalues of an n × n Hermitian matrix A arranged in decreasing order.E stands for expectation, i is the imaginary unit, and Re and Im denote real and imaginary parts, respectively.If z ∈ C, then and, if z k ∈ C for all k ∈ {1, . . ., n}, then we denote by z n:1 the n-dimensional (column) vector given by If x k is a random variable for all k ∈ N, we denote by {x k :k ∈ N} the corresponding random process.
We finish this subsection by reviewing the concept of square Toeplitz matrix.

Definition 1.
An n × n Toeplitz matrix is an n × n matrix of the form Consider a function f : R → C that is continuous and 2π-periodic.For every n ∈ N, we denote by T n ( f ) the n × n Toeplitz matrix given by where {t k } k∈Z is the sequence of Fourier coefficients of f : It should be mentioned that T n ( f ) is Hermitian for all n ∈ N if and only if f is a real function (see [3] (Theorem 4.4.1)).Furthermore, in this case, from [3] (Theorem 4.4.2),we have (1)

DFT of Real Vectors
In this subsection, we recall a well-known property of the DFT of real vectors.

Lemma 1.
Let n ∈ N. Consider x k , y k ∈ C for all k ∈ {1, . . ., n}.Suppose that y n:1 is the DFT of x n:1 , i.e., where V n is the n × n Fourier unitary matrix Then, the two following assertions are equivalent: (1) x n:1 ∈ R n×1 .

RDF of Real Gaussian WSS Processes
Kolmogorov gave in [4] the following formula for the rate distortion function (RDF) of a real zero-mean Gaussian n-dimensional vector x: where θ is a real number satisfying R x (D) can be thought of as the minimum rate (measured in nats) at which one must encode (compress) x in order to be able to recover it with a mean square error (MSE) per dimension not larger than D, that is: where x denotes the estimation of x and • 2 is the spectral norm.We now review the definition of WSS process with continuous power spectral density (PSD).
Definition 2. Let f : R → R be continuous and 2π-periodic.A random process {x k : k ∈ N} is said to be WSS with PSD f if it has constant mean (i.e., E( If {x k : k ∈ N} is a real zero-mean Gaussian WSS process with continuous PSD f satisfying min( f ) > 0 and D ∈ (0, min( f )], then from Equations (1) and (2), we obtain We recall that the RDF of the source (process) is given by R(D) = lim n→∞ R x n:1 (D).

DFT of WSS Processes
In this section, we present several new results on the DFT of WSS processes in one theorem.
Theorem 1.Consider a WSS process {x k : k ∈ N} with continuous PSD f .Let n ∈ N and y n:1 = V * n x n:1 .

Upper Bounds for the RDF of Finite-Length Data Blocks of Gaussian WSS Sources
Let {x k : k ∈ N} be a real zero-mean Gaussian WSS process with continuous PSD f and min( f ) > 0. For a given block length n ∈ N and a distortion D ∈ (0, min( f )], Pearl presented in [1] an upper bound of R x n:1 (D), namely: where C n ( f ) is the matrix defined in Equation (8).In the following theorem, we give two new upper bounds of R x n:1 (D), denoted by R x n:1 (D) and Ȓx n:1 (D), that are tighter than the one given by Pearl.
Theorem 2. Consider a real zero-mean Gaussian WSS process {x k : k ∈ N} with continuous PSD f and min( f ) > 0. Let D ∈ (0, min( f )].If n ∈ N and y n:1 is the DFT of x n:1 , then where R x n:1 (D) is given by and Proof.We divide the proof into four steps: Step 1: We show that R x n:1 (D) ≤ R x n:1 (D).We encode y n 2 , . . ., y n separately with for all j ∈ n 2 , . . ., n , where n 2 denotes the smallest integer higher than or equal to n 2 .Observe that if j ∈ { n 2 , . . ., n − 1} with j = n 2 Equation ( 19) is equivalent to From Lemma 1, y j = y n−j for all j ∈ {1, . . ., n 2 − 1}, and y j ∈ R with j ∈ { n 2 , n} ∩ N. Let x n:1 := V n y n:1 , where with y j := y n−j for all j ∈ {1, . . ., n 2 − 1}.Applying Lemma 1 yields As V * n is unitary and the spectral norm is unitarily invariant, we have Consequently, Step 2: We show that R x n:1 (D) ≤ Ȓx n:1 (D).To do that, we only need to prove that for all j ∈ { n 2 , . . ., n − 1} with j = n 2 .Fix j ∈ { n 2 , . . ., n − 1} with j = n 2 .We encode Re y j and Im y j separately with Let y j := Re(y j ) + i Im(y j ).We have Consequently, Step 3: We show that Ȓx n: . From Equations ( 2) and ( 5), we obtain and applying Equations ( 2), ( 6) and ( 7), the arithmetic mean-geometric mean (AM-GM) inequality, and Lemma 1 yields Hence, from Equation (9), if n is even, we have and, if n is odd, is yielded.
Step 4: We show Equation (18).Applying Equation (3) yields is positive definite, and applying the AM-GM inequality yields where tr stands for trace and • F is the Frobenius norm.From [5] (Lemma 4), we obtain Finally, observe that Theorem 2 also provides coding strategies to achieve the two new bounds of R x n:1 (D) presented: R x n:1 (D) and Ȓx n:1 (D).Specifically, Theorem 2 shows that R x n:1 (D) can be achieved by encoding y k separately, with k ∈ { n 2 , . . ., n}, instead of encoding x n:1 jointly, and that Ȓx n:1 (D) can be achieved by encoding separately the real part and the imaginary part of y k instead of encoding y k when k / ∈ { n 2 , n}.Therefore, although R x n:1 (D) is a tighter bound, the coding strategy associated with Ȓx n:1 (D) is simpler.It should be mentioned that, in order to achieve R x n:1 (D) and Ȓx n:1 (D), an optimal coding method of Gaussian random variables is required.

2 =Figure 1 .
Figure 1.Numerical example of the upper bounds presented in Theorem 2.