Upper Bounds for the Rate Distortion Function of Finite-Length Data Blocks of Gaussian WSS Sources

Abstract

In this paper, we present upper bounds for the rate distortion function (RDF) of finite-length data blocks of Gaussian wide sense stationary (WSS) sources and we propose coding strategies to achieve such bounds. In order to obtain those bounds, we previously derive new results on the discrete Fourier transform (DFT) of WSS processes.

1. Introduction

In [1], Pearl gave an upper bound for the rate distortion function (RDF) of finite-length data blocks of Gaussian wide sense stationary (WSS) sources and proved that such bound tends to the RDF of the source when the size of the data block grows. However, he did not give a coding strategy to achieve his bound for a given block length.

In this paper, we present two new upper bounds for the RDF of finite-length data blocks of Gaussian WSS sources and we propose coding strategies to achieve these two bounds for a given block length. Since our bounds are tighter than the one given by Pearl, they also tend to the RDF of the source when the size of the data block grows. In order to obtain our bounds, we previously derive new results on the discrete Fourier transform (DFT) of WSS processes.

It should be mentioned that our coding strategies allow us to deal with Gaussian WSS sources as if they were memoryless. This fact can be used, for instance, to consider Gaussian WSS sources in [2].

The paper is organized as follows. In Section 2 we set up notation and review the mathematical definitions and results used in the rest of the paper. In Section 3 we obtain several results on the DFT of WSS processes which will be applied in Section 4. Finally, in Section 4 we present two new upper bounds for the RDF of finite-length data blocks of Gaussian WSS sources and we propose coding strategies to achieve such bounds. In this section, we also present a numerical example to illustrate the difference between Pearl’s bound and our bounds.

2. Preliminaries

2.1. Notation

In this paper, $\mathbb{N}$, $\mathbb{Z}$, $\mathbb{R}$, and $\mathbb{C}$ denote the set of natural numbers (i.e., the set of positive integers), the set of integer numbers, the set of (finite) real numbers, and the set of (finite) complex numbers, respectively. ${\mathbb{R}}^{n\times 1}$ is the set of all real n-dimensional (column) vectors. $I_n$ denotes the $n\times n$ identity matrix, $\ast$ stands for conjugate transpose, ⊤ denotes transpose, and ${\lambda }_k\left(A\right)$, $k\in \{1,\dots ,n\}$, are the eigenvalues of an $n\times n$ Hermitian matrix A arranged in decreasing order. E stands for expectation, $\mathrm{i}$ is the imaginary unit, and $\mathrm{Re}$ and $\mathrm{Im}$ denote real and imaginary parts, respectively. If $z\in \mathbb{C}$, then

$$\widehat{z}:=\left(\begin{array}{c}\mathrm{Re}\left(z\right)\\ \mathrm{Im}\left(z\right)\end{array}\right)\in {\mathbb{R}}^{2\times 1}$$

and, if $z_k\in \mathbb{C}$ for all $k\in \{1,\dots ,n\},$ then we denote by $z_{n:1}$ the n-dimensional (column) vector given by

$$z_{n:1}:=\left(\begin{array}{c}{z}_n\\ z_{n-1}\\ z_{n-2}\\ ⋮\\ z_1\end{array}\right).$$

If $x_k$ is a random variable for all $k\in \mathbb{N}$, we denote by $\{x_k:k\in \mathbb{N}\}$ the corresponding random process.

We finish this subsection by reviewing the concept of square Toeplitz matrix.

Definition 1.

An $n\times n$ Toeplitz matrix is an $n\times n$ matrix of the form

$$\left(\begin{array}{ccccc}{t}_0& t_{-1}& t_{-2}& \cdots & t_{1-n}\\ t_1& t_0& t_{-1}& \cdots & t_{2-n}\\ t_2& t_1& t_0& \cdots & t_{3-n}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ t_{n-1}& t_{n-2}& t_{n-3}& \cdots & t_0\end{array}\right),$$

where $t_k\in \mathbb{C}$ with $k\in \{1-n,\dots ,n-1\}$.

Consider a function $f:\mathbb{R}\to \mathbb{C}$ that is continuous and $2\pi$-periodic. For every $n\in \mathbb{N}$, we denote by $T_n\left(f\right)$ the $n\times n$ Toeplitz matrix given by

$$T_n\left(f\right):={\left(t_{j-k}\right)}_{j,k=1}^n,$$

where ${\left\{t_k\right\}}_{k\in \mathbb{Z}}$ is the sequence of Fourier coefficients of f:

$$t_k=\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right){\mathrm{e}}^{-k\omega \mathrm{i}}d\omega \forall k\in \mathbb{Z}.$$

It should be mentioned that $T_n\left(f\right)$ is Hermitian for all $n\in \mathbb{N}$ if and only if f is a real function (see [3] (Theorem 4.4.1)). Furthermore, in this case, from [3] (Theorem 4.4.2), we have

$$min\left(f\right)\le {\lambda }_n\left(T_n\left(f\right)\right)\le {\lambda }_1\left(T_n\left(f\right)\right)\le max\left(f\right)\forall n\in \mathbb{N}.$$

(1)

2.2. DFT of Real Vectors

In this subsection, we recall a well-known property of the DFT of real vectors.

Lemma 1.

Let $n\in \mathbb{N}$. Consider $x_k,y_k\in \mathbb{C}$ for all $k\in \{1,\dots ,n\}$. Suppose that $y_{n:1}$ is the DFT of $x_{n:1}$, i.e.,

$$y_{n:1}=V_n^{*}{x}_{n:1},$$

where $V_n$ is the $n\times n$ Fourier unitary matrix

$${\left[V_n\right]}_{j,k}:=\frac{1}{\sqrt{n}}{\mathrm{e}}^{-\frac{2\pi (j-1)(k-1)}{n}\mathrm{i}},j,k\in \{1,\dots ,n\}.$$

Then, the two following assertions are equivalent:

(1) 

$x_{n:1}\in {\mathbb{R}}^{n\times 1}$.

(2) 

$y_j=\overline{y_{n-j}}$ for all $j\in \{1,\dots ,n-1\}$ and $y_n\in \mathbb{R}$.

2.3. RDF of Real Gaussian WSS Processes

Kolmogorov gave in [4] the following formula for the rate distortion function (RDF) of a real zero-mean Gaussian n-dimensional vector $\mathrm{x}$:

$$R_{\mathrm{x}}\left(D\right)=\frac{1}{n}\sum _{k=1}^{n}max\left\{0,\frac{1}{2}ln\frac{{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{\theta }\right\},$$

(2)

where $\theta$ is a real number satisfying

$$D=\frac{1}{n}\sum _{k=1}^{n}min\left\{\theta ,{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)\right\}.$$

$R_{\mathrm{x}}\left(D\right)$ can be thought of as the minimum rate (measured in nats) at which one must encode (compress) $\mathrm{x}$ in order to be able to recover it with a mean square error (MSE) per dimension not larger than D, that is:

$$\frac{E\left({∥\mathrm{x}-\tilde{\mathrm{x}}∥}_2^2\right)}{n}\le D,$$

where $\tilde{\mathrm{x}}$ denotes the estimation of $\mathrm{x}$ and ${\parallel ·\parallel }_2$ is the spectral norm.

We now review the definition of WSS process with continuous power spectral density (PSD).

Definition 2.

Let $f:\mathbb{R}\to \mathbb{R}$ be continuous and $2\pi$-periodic. A random process $\{x_k:k\in \mathbb{N}\}$ is said to be WSS with PSD f if it has constant mean (i.e., $E\left(x_{k_1}\right)=E\left(x_{k_2}\right)$ for all $k_1,k_2\in \mathbb{N})$ and $\left\{E\left(x_{n:1}{x}_{n:1}^{*}\right)\right\}=\left\{T_n\left(f\right)\right\}$.

If $\{x_k:k\in \mathbb{N}\}$ is a real zero-mean Gaussian WSS process with continuous PSD f satisfying $min\left(f\right)>0$ and $D\in (0,min(f\left)\right]$, then from Equations (1) and (2), we obtain

$$R_{x_{n:1}}\left(D\right)=\frac{1}{2n}\sum _{k=1}^{n}ln\frac{{\lambda }_k\left(T_n\left(f\right)\right)}{D}=\frac{1}{2n}ln\frac{det\left(T_n\left(f\right)\right)}{D^n}\forall n\in \mathbb{N}.$$

(3)

We recall that the RDF of the source (process) is given by $R\left(D\right)={lim}_{n\to \infty }{R}_{x_{n:1}}\left(D\right)$.

3. DFT of WSS Processes

In this section, we present several new results on the DFT of WSS processes in one theorem.

Theorem 1.

Consider a WSS process $\{x_k:k\in \mathbb{N}\}$ with continuous PSD f. Let $n\in \mathbb{N}$ and $y_{n:1}=V_n^{*}{x}_{n:1}$.

(1) 

If $j\in \{1,\dots ,n\},$ then

$$min\left(f\right)\le E\left({\left|x_j\right|}^2\right)\le max\left(f\right)$$

(4)

and

$$min\left(f\right)\le E\left({\left|y_j\right|}^2\right)\le max\left(f\right)$$

(5)

(2) 

If the process $\{x_k:k\in \mathbb{N}\}$ is real and $j\in \{1,\dots ,n-1\}$ with $j\ne \frac{n}{2}$ then

$$\frac{min\left(f\right)}{2}\le E\left({\left(\mathrm{Re}\left(y_j\right)\right)}^2\right)\le \frac{max\left(f\right)}{2}$$

(6)

and

$$\frac{min\left(f\right)}{2}\le E\left({\left(\mathrm{Im}\left(y_j\right)\right)}^2\right)\le \frac{max\left(f\right)}{2}.$$

(7)

Proof. 

(1) Since

$$E\left({\left|x_j\right|}^2\right)={\left[E\left(x_{n:1}{x}_{n:1}^{*}\right)\right]}_{n-j+1,n-j+1}={\left[T_n\left(f\right)\right]}_{n-j+1,n-j+1}=t_0=T_1\left(f\right),\forall j\in \{1,\dots ,n\}$$

from Equation (1), we obtain Equation (4).

Let

$${\widehat{C}}_n\left(f\right):=V_n{\mathrm{diag}}_{1\le j\le n}\left({\left[V_n^{*}{T}_n\left(f\right)V_n\right]}_{j,j}\right)V_n^{*},$$

(8)

where ${\mathrm{diag}}_{1\le j\le n}\left(a_j\right)={\left(a_j{\delta }_{j,k}\right)}_{j,k=1}^n$ with $\delta$ being the Kronecker delta and $a_j\in \mathbb{C}$ for all $j\in \{1,\dots ,n\}$. As

$$E\left(y_{n:1}{y}_{n:1}^{*}\right)=E\left(V_n^{*}{x}_{n:1}{x}_{n:1}^{*}{\left(V_n^{*}\right)}^{*}\right)=V_n^{*}E\left(x_{n:1}{x}_{n:1}^{*}\right){\left(V_n^{*}\right)}^{*}=V_n^{*}{T}_n\left(f\right)V_n,$$

we have

$${\widehat{C}}_n\left(f\right)=V_n{\mathrm{diag}}_{1\le j\le n}\left({\left[E\left(y_{n:1}{y}_{n:1}^{*}\right)\right]}_{j,j}\right)V_n^{*}.$$

Hence,

$$\left\{{\lambda }_j\left({\widehat{C}}_n\left(f\right)\right):j\in \{1,\dots ,n\}\right\}=\left\{{\left[E\left(y_{n:1}{y}_{n:1}^{*}\right)\right]}_{j,j}:j\in \{1,\dots ,n\}\right\}=\left\{E\left({\left|y_j\right|}^2\right),j\in \{1,\dots ,n\}\right\}.$$

(9)

Equation (5) now follows by taking $N=1$ in [5] (Lemma 6).

(2) Fix $j\in \{1,\dots ,n-1\}$ with $j\ne \frac{n}{2}$. Since

$$\begin{array}{cc}\hfill y_j& ={\left[y_{n:1}\right]}_{n-j+1,1}={\left[V_n^{*}{x}_{n:1}\right]}_{n-j+1,1}=\sum _{k=1}^n{\left[V_n^{*}\right]}_{n-j+1,k}{\left[x_{n:1}\right]}_{k,1}=\sum _{k=1}^n\overline{{\left[V_n\right]}_{k,n-j+1}}{\left[x_{n:1}\right]}_{k,1}\hfill \\ & =\sum _{k=1}^n\frac{1}{\sqrt{n}}{\mathrm{e}}^{\frac{2\pi (k-1)(n-j)}{n}\mathrm{i}}{\left[x_{n:1}\right]}_{k,1}=\sum _{k=1}^n\frac{1}{\sqrt{n}}{\mathrm{e}}^{2\pi (k-1)\mathrm{i}}{\mathrm{e}}^{-\frac{2\pi (k-1)j}{n}\mathrm{i}}{\left[x_{n:1}\right]}_{k,1}=\sum _{k=1}^n\frac{1}{\sqrt{n}}{\mathrm{e}}^{-\frac{2\pi (k-1)j}{n}\mathrm{i}}{\left[x_{n:1}\right]}_{k,1}\hfill \\ & =\frac{1}{\sqrt{n}}\sum _{k=1}^n\left(cos\frac{2\pi (1-k)j}{n}+\mathrm{i}sin\frac{2\pi (1-k)j}{n}\right)x_{n-k+1},\hfill \end{array}$$

we obtain

$$\begin{array}{cc}\hfill & E\left(\widehat{y_j}{\left(\widehat{y_j}\right)}^{\top }\right)\hfill \\ & =E\left(\left(\begin{array}{c}\mathrm{Re}\left(y_j\right)\\ \mathrm{Im}\left(y_j\right)\end{array}\right)\left(\begin{array}{cc}\mathrm{Re}\left(y_j\right)& \mathrm{Im}\left(y_j\right)\end{array}\right)\right)=E\left(\begin{array}{cc}{\left(\mathrm{Re}\left(y_j\right)\right)}^2& \mathrm{Re}\left(y_j\right)\mathrm{Im}\left(y_j\right)\\ \mathrm{Im}\left(y_j\right)\mathrm{Re}\left(y_j\right)& {\left(\mathrm{Im}\left(y_j\right)\right)}^2\end{array}\right)\hfill \\ & =\frac{1}{n}\sum _{k_1,k_2=1}^n\left(\begin{array}{cc}cos{\displaystyle \frac{2\pi (1-k_1)j}{n}}cos\frac{2\pi (1-k_2)j}{n}E\left(x_{n-k_1+1}{x}_{n-k_2+1}\right)& cos{\displaystyle \frac{2\pi (1-k_1)j}{n}}sin{\displaystyle \frac{2\pi (1-k_2)j}{n}}E\left(x_{n-k_1+1}{x}_{n-k_2+1}\right)\\ sin{\displaystyle \frac{2\pi (1-k_1)j}{n}}cos{\displaystyle \frac{2\pi (1-k_2)j}{n}}E\left(x_{n-k_1+1}{x}_{n-k_2+1}\right)& sin{\displaystyle \frac{2\pi (1-k_1)j}{n}}sin{\displaystyle \frac{2\pi (1-k_2)j}{n}}E\left(x_{n-k_1+1}{x}_{n-k_2+1}\right)\end{array}\right)\hfill \\ & ={\displaystyle \frac{1}{n}}\sum _{k_1,k_2=1}^n\left(\begin{array}{cc}cos{\displaystyle \frac{2\pi (1-k_1)j}{n}}cos{\displaystyle \frac{2\pi (1-k_2)j}{n}}{t}_{k_1-k_2}& cos{\displaystyle \frac{2\pi (1-k_1)j}{n}}sin{\displaystyle \frac{2\pi (1-k_2)j}{n}}{t}_{k_1-k_2}\\ sin{\displaystyle \frac{2\pi (1-k_1)j}{n}}cos{\displaystyle \frac{2\pi (1-k_2)j}{n}}{t}_{k_1-k_2}& sin{\displaystyle \frac{2\pi (1-k_1)j}{n}}sin{\displaystyle \frac{2\pi (1-k_2)j}{n}}{t}_{k_1-k_2}\end{array}\right).\hfill \end{array}$$

(10)

We begin by proving Equation (6). Applying Equation (10) yields

$$\begin{array}{c}E\left({\left(\mathrm{Re}\left(y_j\right)\right)}^2\right)=\frac{1}{n}\sum _{k_1,k_2=1}^{n}cos\frac{2\pi (1-k_1)j}{n}cos\frac{2\pi (1-k_2)j}{n}{t}_{k_1-k_2}\hfill \\ =\frac{1}{n}\sum _{k_1,k_2=1}^{n}cos\frac{2\pi (1-k_1)j}{n}cos\frac{2\pi (1-k_2)j}{n}\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right){\mathrm{e}}^{-(k_1-k_2)\omega \mathrm{i}}d\omega \hfill \\ =\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right)\left(\frac{1}{\sqrt{n}}\sum _{k_1=1}^{n}cos\frac{2\pi (1-k_1)j}{n}{\mathrm{e}}^{-k_1\omega \mathrm{i}}\right)\left(\frac{1}{\sqrt{n}}\sum _{k_2=1}^{n}cos\frac{2\pi (1-k_2)j}{n}{\mathrm{e}}^{k_2\omega \mathrm{i}}\right)d\omega \hfill \\ =\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right)\overline{\left(\frac{1}{\sqrt{n}}{\sum }_{k_1=1}^{n}cos\frac{2\pi (1-k_1)j}{n}{\mathrm{e}}^{k_1\omega \mathrm{i}}\right)}\left(\frac{1}{\sqrt{n}}\sum _{k_2=1}^{n}cos\frac{2\pi (1-k_2)j}{n}{\mathrm{e}}^{k_2\omega \mathrm{i}}\right)d\omega \hfill \\ =\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right){\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}cos\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega ,\hfill \end{array}$$

and consequently,

$$\begin{array}{cc}\hfill min\left(f\right)\frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}cos\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega & \le E\left({\left(\mathrm{Re}\left(y_j\right)\right)}^2\right)\hfill \\ & \le max\left(f\right)\frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}cos\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega .\hfill \end{array}$$

Observe that to finish the proof of Equation (6), we only need to show that

$$\frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}cos\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega =\frac{1}{2}.$$

(11)

Since

$$\frac{1}{2\pi }{\int }_0^{2\pi }{\mathrm{e}}^{m\omega \mathrm{i}}d\omega =\left\{\begin{array}{cc}1,\hfill & \mathrm{if}m=0,\hfill \\ 0,\hfill & \mathrm{if}m\in \mathbb{Z}\backslash \left\{0\right\},\hfill \end{array}\right.$$

(12)

we obtain

$$\begin{array}{cc}\hfill & \frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}cos\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega \hfill \\ & =\frac{1}{n}\sum _{k_1,k_2=1}^{n}cos\frac{2\pi (1-k_1)j}{n}cos\frac{2\pi (1-k_2)j}{n}\frac{1}{2\pi }{\int }_0^{2\pi }{\mathrm{e}}^{-(k_1-k_2)\omega \mathrm{i}}d\omega \hfill \\ & =\frac{1}{n}\sum _{k=1}^n{\left(cos\frac{2\pi (1-k)j}{n}\right)}^2=\frac{1}{n}\sum _{k=1}^n\left(1-{\left(sin\frac{2\pi (1-k)j}{n}\right)}^2\right)\hfill \\ & =1-\frac{1}{n}\sum _{k=1}^n{\left(sin\frac{2\pi (1-k)j}{n}\right)}^2.\hfill \end{array}$$

(13)

As ${\mathrm{e}}^{\frac{4\pi j}{n}\mathrm{i}}\ne 1$ from the formula for the partial sums of the geometric series (see, e.g., [6] (p. 388)), we have

$$\sum _{k=1}^n{\mathrm{e}}^{\frac{4\pi (k-1)j}{n}\mathrm{i}}=\sum _{h=0}^{n-1}{\mathrm{e}}^{\frac{4\pi hj}{n}\mathrm{i}}=\sum _{h=0}^{n-1}{\left({\mathrm{e}}^{\frac{4\pi j}{n}\mathrm{i}}\right)}^h=\frac{1-{\left({\mathrm{e}}^{\frac{4\pi j}{n}\mathrm{i}}\right)}^n}{1-{\mathrm{e}}^{\frac{4\pi j}{n}\mathrm{i}}}=\frac{1-{\mathrm{e}}^{4\pi j\mathrm{i}}}{1-{\mathrm{e}}^{\frac{4\pi j}{n}\mathrm{i}}}=0.$$

(14)

Applying (14) and the basic trigonometric formula $cos\left(2x\right)=1-2{sin}^{2}x$ (see, e.g., [6] (p. 97)) yields

$$\begin{array}{cc}\hfill \frac{1}{n}\sum _{k=1}^n{\left(sin\frac{2\pi (1-k)j}{n}\right)}^2& =\frac{1}{n}\sum _{k=1}^n\frac{1-cos\frac{4\pi (1-k)j}{n}}{2}=\frac{1}{n}\left[\frac{n}{2}-\frac{1}{2}\sum _{k=1}^{n}cos\frac{4\pi (k-1)j}{n}\right]\hfill \\ & =\frac{1}{2}-\frac{1}{2n}\sum _{k=1}^n\mathrm{Re}\left({\mathrm{e}}^{\frac{4\pi (k-1)j}{n}\mathrm{i}}\right)=\frac{1}{2}-\frac{1}{2n}\mathrm{Re}\left(\sum _{k=1}^n{\mathrm{e}}^{\frac{4\pi (k-1)j}{n}\mathrm{i}}\right)=\frac{1}{2},\hfill \end{array}$$

(15)

and, thus, from Equation (13), we obtain Equation (11), and, therefore, Equation (6) holds.

Finally, we prove Equation (7). From Equation (10), we obtain

$$\begin{array}{cc}\hfill E\left({\left(\mathrm{Im}\left(y_j\right)\right)}^2\right)& =\frac{1}{n}\sum _{k_1,k_2=1}^{n}sin\frac{2\pi (1-k_1)j}{n}sin\frac{2\pi (1-k_2)j}{n}{t}_{k_1-k_2}\hfill \\ & =\frac{1}{n}\sum _{k_1,k_2=1}^{n}sin\frac{2\pi (1-k_1)j}{n}sin\frac{2\pi (1-k_2)j}{n}\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right){\mathrm{e}}^{-(k_1-k_2)\omega \mathrm{i}}d\omega \hfill \\ & =\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right)\left(\frac{1}{\sqrt{n}}\sum _{k_1=1}^{n}sin\frac{2\pi (1-k_1)j}{n}{\mathrm{e}}^{-k_1\omega \mathrm{i}}\right)\left(\frac{1}{\sqrt{n}}\sum _{k_2=1}^{n}sin\frac{2\pi (1-k_2)j}{n}{\mathrm{e}}^{k_2\omega \mathrm{i}}\right)d\omega \hfill \\ & =\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right)\overline{\left(\frac{1}{\sqrt{n}}{\sum }_{k_1=1}^{n}sin\frac{2\pi (1-k_1)j}{n}{\mathrm{e}}^{k_1\omega \mathrm{i}}\right)}\left(\frac{1}{\sqrt{n}}\sum _{k_2=1}^{n}sin\frac{2\pi (1-k_2)j}{n}{\mathrm{e}}^{k_2\omega \mathrm{i}}\right)d\omega \hfill \\ & =\frac{1}{2\pi }{\int }_0^{2\pi }f\left(\omega \right){\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}sin\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega ,\hfill \end{array}$$

and, consequently,

$$\begin{array}{cc}\hfill min\left(f\right)\frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}sin\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega & \le E\left({\left(\mathrm{Im}\left(y_j\right)\right)}^2\right)\hfill \\ & \le max\left(f\right)\frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}sin\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega .\hfill \end{array}$$

Applying Equations (12) and (15) yields

$$\begin{array}{c}\frac{1}{2\pi }{\int }_0^{2\pi }{\left|\frac{1}{\sqrt{n}}\sum _{k=1}^{n}sin\frac{2\pi (1-k)j}{n}{\mathrm{e}}^{k\omega \mathrm{i}}\right|}^{2}d\omega \hfill \\ =\frac{1}{n}\sum _{k_1,k_2=1}^{n}sin\frac{2\pi (1-k_1)j}{n}sin\frac{2\pi (1-k_2)j}{n}\frac{1}{2\pi }{\int }_0^{2\pi }{\mathrm{e}}^{-(k_1-k_2)\omega \mathrm{i}}d\omega \hfill \\ =\frac{1}{n}\sum _{k=1}^n{\left(sin\frac{2\pi (1-k)j}{n}\right)}^2=\frac{1}{2},\hfill \end{array}$$

and, therefore, Equation (7) holds. ☐

4. Upper Bounds for the RDF of Finite-Length Data Blocks of Gaussian WSS Sources

Let $\{x_k:k\in \mathbb{N}\}$ be a real zero-mean Gaussian WSS process with continuous PSD f and $min\left(f\right)>0$. For a given block length $n\in \mathbb{N}$ and a distortion $D\in (0,min(f\left)\right]$, Pearl presented in [1] an upper bound of $R_{x_{n:1}}\left(D\right)$, namely:

$$\frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n},$$

where ${\widehat{C}}_n\left(f\right)$ is the matrix defined in Equation (8). In the following theorem, we give two new upper bounds of $R_{x_{n:1}}\left(D\right)$, denoted by ${\tilde{R}}_{x_{n:1}}\left(D\right)$ and ${\breve{R}}_{x_{n:1}}\left(D\right)$, that are tighter than the one given by Pearl.

Theorem 2.

Consider a real zero-mean Gaussian WSS process $\{x_k:k\in \mathbb{N}\}$ with continuous PSD f and $min\left(f\right)>0$. Let $D\in (0,min(f\left)\right]$. If $n\in \mathbb{N}$ and $y_{n:1}$ is the DFT of $x_{n:1},$ then

$$R_{x_{n:1}}\left(D\right)\le {\tilde{R}}_{x_{n:1}}\left(D\right)\le {\breve{R}}_{x_{n:1}}\left(D\right)\le \frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n},$$

(16)

where ${\tilde{R}}_{x_{n:1}}\left(D\right)$ is given by

$${\tilde{R}}_{x_{n:1}}\left(D\right)=\left\{\begin{array}{cc}\frac{R_{y_{\frac{n}{2}}}\left(D\right)+2{\sum }_{k=\frac{n}{2}+1}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+R_{y_n}\left(D\right)}{n},\hfill & \mathit{if}n\mathit{is}\mathit{even},\hfill \\ \frac{2{\sum }_{k=\frac{n+1}{2}}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+R_{y_n}\left(D\right)}{n},\hfill & \mathit{if}n\mathit{is}\mathit{odd}.\hfill \end{array}\right.$$

(17)

and

$${\breve{R}}_{x_{n:1}}\left(D\right)=\left\{\begin{array}{cc}\frac{R_{y_{\frac{n}{2}}}\left(D\right)+{\sum }_{k=\frac{n}{2}+1}^{n-1}\left(R_{\mathrm{Re}\left(y_k\right)}\left(\frac{D}{2}\right)+R_{\mathrm{Im}\left(y_k\right)}\left(\frac{D}{2}\right)\right)+R_{y_n}\left(D\right)}{n},\hfill & \mathit{if}n\mathit{is}\mathit{even},\hfill \\ \frac{{\sum }_{k=\frac{n+1}{2}}^{n-1}\left(R_{\mathrm{Re}\left(y_k\right)}\left(\frac{D}{2}\right)+R_{\mathrm{Im}\left(y_k\right)}\left(\frac{D}{2}\right)\right)+R_{y_n}\left(D\right)}{n},\hfill & \mathit{if}n\mathit{is}\mathit{odd}.\hfill \end{array}\right.$$

Furthermore,

$$R\left(D\right)=\underset{n\to \infty }{lim}{R}_{x_{n:1}}\left(D\right)=\underset{n\to \infty }{lim}{\tilde{R}}_{x_{n:1}}\left(D\right)=\underset{n\to \infty }{lim}{\breve{R}}_{x_{n:1}}\left(D\right)=\underset{n\to \infty }{lim}\frac{1}{2n}ln\frac{det({\widehat{C}}_n\left(f\right))}{D^n}=\frac{1}{4\pi }{\int }_0^{2\pi }ln\frac{f\left(\omega \right)}{D}d\omega .$$

(18)

Proof. 

We divide the proof into four steps:

Step 1: We show that $R_{x_{n:1}}\left(D\right)\le {\tilde{R}}_{x_{n:1}}\left(D\right)$. We encode $y_{\lceil \frac{n}{2}\rceil },\dots ,y_n$ separately with

$$E\left({\left|y_j-\widetilde{y_j}\right|}^2\right)\le D$$

(19)

for all $j\in \left\{⌈\frac{n}{2}⌉,\dots ,n\right\}$, where $\lceil \frac{n}{2}\rceil$ denotes the smallest integer higher than or equal to $\frac{n}{2}$. Observe that if $j\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}$ with $j\ne \frac{n}{2}$ Equation (19) is equivalent to

$$\frac{E\left({∥\widehat{y_j}-\widehat{\widetilde{y_j}}∥}_2^2\right)}{2}\le \frac{D}{2}.$$

From Lemma 1, $y_j=\overline{y_{n-j}}$ for all $j\in \{1,\dots ,\lceil \frac{n}{2}\rceil -1\}$, and $y_j\in \mathbb{R}$ with $j\in \{\frac{n}{2},n\}\cap \mathbb{N}$. Let $\widetilde{x_{n:1}}:=V_n\widetilde{y_{n:1}}$, where

$$\widetilde{y_{n:1}}=\left(\begin{array}{c}\widetilde{y_n}\\ ⋮\\ \widetilde{y_1}\end{array}\right),$$

with $\widetilde{y_j}:=\overline{\widetilde{y_{n-j}}}$ for all $j\in \{1,\dots ,\lceil \frac{n}{2}\rceil -1\}$. Applying Lemma 1 yields $\widetilde{x_{n:1}}\in {\mathbb{R}}^{n\times 1}$.

As $V_n^{*}$ is unitary and the spectral norm is unitarily invariant, we have

$$\begin{array}{cc}\hfill & \frac{E\left({∥x_{n:1}-\widetilde{x_{n:1}}∥}_2^2\right)}{n}\hfill \\ & =\frac{E\left({∥V_n^{*}{x}_{n:1}-V_n^{*}\widetilde{x_{n:1}}∥}_2^2\right)}{n}=\frac{E\left({∥y_{n:1}-V_n^{*}{V}_n\widetilde{y_{n:1}}∥}_2^2\right)}{n}=\frac{E\left({∥y_{n:1}-\widetilde{y_{n:1}}∥}_2^2\right)}{n}\hfill \\ & =\frac{1}{n}E\left(\sum _{j=1}^n{\left|y_j-\widetilde{y_j}\right|}^2\right)=\frac{1}{n}\sum _{j=1}^{n}E\left({\left|y_j-\widetilde{y_j}\right|}^2\right)=\frac{1}{n}\left(\sum _{j=1}^{\lceil \frac{n}{2}\rceil -1}E\left({\left|y_j-\widetilde{y_j}\right|}^2\right)+\sum _{k=\lceil \frac{n}{2}\rceil }^{n}E\left({\left|y_k-\widetilde{y_k}\right|}^2\right)\right)\hfill \\ & =\frac{1}{n}\left(\sum _{j=1}^{\lceil \frac{n}{2}\rceil -1}E\left({\left|\overline{y_{n-j}}-\overline{\widetilde{y_{n-j}}}\right|}^2\right)+\sum _{k=\lceil \frac{n}{2}\rceil }^{n}E\left({\left|y_k-\widetilde{y_k}\right|}^2\right)\right)=\frac{1}{n}\left(\sum _{k=n-\lceil \frac{n}{2}\rceil +1}^{n-1}E\left({\left|\overline{y_k}-\overline{\widetilde{y_k}}\right|}^2\right)+\sum _{k=\lceil \frac{n}{2}\rceil }^{n}E\left({\left|y_k-\widetilde{y_k}\right|}^2\right)\right)\hfill \\ & =\frac{1}{n}\left(\sum _{k=n-\lceil \frac{n}{2}\rceil +1}^{n-1}E\left({\left|\overline{y_k-\widetilde{y_k}}\right|}^2\right)+\sum _{k=\lceil \frac{n}{2}\rceil }^{n}E\left({\left|y_k-\widetilde{y_k}\right|}^2\right)\right)=\frac{1}{n}\left(\sum _{k=n-\lceil \frac{n}{2}\rceil +1}^{n-1}E\left({\left|y_k-\widetilde{y_k}\right|}^2\right)+\sum _{k=\lceil \frac{n}{2}\rceil }^{n}E\left({\left|y_k-\widetilde{y_k}\right|}^2\right)\right)\hfill \\ & \le \frac{1}{n}\left(\left(⌈\frac{n}{2}⌉-1\right)D+\left(n-⌈\frac{n}{2}⌉+1\right)D\right)=D.\hfill \end{array}$$

Consequently,

$$R_{x_{n:1}}\left(D\right)\le \left\{\begin{array}{cc}\frac{R_{y_{\frac{n}{2}}}\left(D\right)+2{\sum }_{k=\frac{n}{2}+1}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+R_{y_n}\left(D\right)}{n},\hfill & \mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ \frac{2{\sum }_{k=\frac{n+1}{2}}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+R_{y_n}\left(D\right)}{n},\hfill & \mathrm{if}n\mathrm{is}\mathrm{odd}.\hfill \end{array}\right.$$

Step 2: We show that ${\tilde{R}}_{x_{n:1}}\left(D\right)\le {\breve{R}}_{x_{n:1}}\left(D\right)$. To do that, we only need to prove that

$$2R_{\widehat{y_j}}\left(\frac{D}{2}\right)\le R_{\mathrm{Re}\left(y_j\right)}\left(\frac{D}{2}\right)+R_{\mathrm{Im}\left(y_j\right)}\left(\frac{D}{2}\right)$$

for all $j\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}$ with $j\ne \frac{n}{2}$. Fix $j\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}$ with $j\ne \frac{n}{2}$. We encode $\mathrm{Re}\left(y_j\right)$ and $\mathrm{Im}\left(y_j\right)$ separately with

$$E\left({\left(\mathrm{Re}\left(y_j\right)-\widetilde{\mathrm{Re}\left(y_j\right)}\right)}^2\right)\le \frac{D}{2}$$

and

$$E\left({\left(\mathrm{Im}\left(y_j\right)-\widetilde{\mathrm{Im}\left(y_j\right)}\right)}^2\right)\le \frac{D}{2}.$$

Let $\widetilde{y_j}:=\widetilde{\mathrm{Re}\left(y_j\right)}+\mathrm{i}\widetilde{\mathrm{Im}\left(y_j\right)}$. We have

$$\begin{array}{cc}\hfill \frac{E\left({∥\widehat{y_j}-\widehat{\widetilde{y_j}}∥}_2^2\right)}{2}& =\frac{E\left({\left(\mathrm{Re}\left(y_j\right)-\widetilde{\mathrm{Re}\left(y_j\right)}\right)}^2+{\left(\mathrm{Im}\left(y_j\right)-\widetilde{\mathrm{Im}\left(y_j\right)}\right)}^2\right)}{2}\hfill \\ & =\frac{1}{2}\left(E\left({\left(\mathrm{Re}\left(y_j\right)-\widetilde{\mathrm{Re}\left(y_j\right)}\right)}^2\right)+E\left({\left(\mathrm{Im}\left(y_j\right)-\widetilde{\mathrm{Im}\left(y_j\right)}\right)}^2\right)\right)\le \frac{1}{2}\left(\frac{D}{2}+\frac{D}{2}\right)=\frac{D}{2}.\hfill \end{array}$$

Consequently,

$$R_{\widehat{y_j}}\left(\frac{D}{2}\right)\le \frac{R_{\mathrm{Re}\left(y_j\right)}\left(\frac{D}{2}\right)+R_{\mathrm{Im}\left(y_j\right)}\left(\frac{D}{2}\right)}{2}.$$

Step 3: We show that ${\breve{R}}_{x_{n:1}}\left(D\right)\le \frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n}$. From Equations (2) and (5), we obtain

$$\begin{array}{c}\hfill R_{y_k}\left(D\right)=\frac{1}{2}ln\frac{E\left({\left|y_k\right|}^2\right)}{D},k\in \left\{\frac{n}{2},n\right\}\cap \mathbb{N},\end{array}$$

and applying Equations (2), (6) and (7), the arithmetic mean-geometric mean (AM-GM) inequality, and Lemma 1 yields

$$\begin{array}{c}{R}_{\mathrm{Re}\left(y_k\right)}\left(\frac{D}{2}\right)+R_{\mathrm{Im}\left(y_k\right)}\left(\frac{D}{2}\right)\hfill \\ =\frac{1}{2}ln\frac{E\left({\left(\mathrm{Re}\left(y_k\right)\right)}^2\right)}{\frac{D}{2}}+\frac{1}{2}ln\frac{E\left({\left(\mathrm{Im}\left(y_k\right)\right)}^2\right)}{\frac{D}{2}}=\frac{1}{2}ln\frac{E\left({\left(\mathrm{Re}\left(y_k\right)\right)}^2\right)E\left({\left(\mathrm{Im}\left(y_k\right)\right)}^2\right)}{{\left(\frac{D}{2}\right)}^2}\hfill \\ =\frac{1}{2}ln\frac{{\left(\sqrt{E\left({\left(\mathrm{Re}\left(y_k\right)\right)}^2\right)E\left({\left(\mathrm{Im}\left(y_k\right)\right)}^2\right)}\right)}^2}{{\left(\frac{D}{2}\right)}^2}\le \frac{1}{2}ln\frac{{\left(\frac{E\left({\left(\mathrm{Re}\left(y_k\right)\right)}^2\right)+E\left({\left(\mathrm{Im}\left(y_k\right)\right)}^2\right)}{2}\right)}^2}{{\left(\frac{D}{2}\right)}^2}\hfill \\ =\frac{1}{2}ln\frac{{\left(\frac{E\left({\left(\mathrm{Re}\left(y_k\right)\right)}^2+{\left(\mathrm{Im}\left(y_k\right)\right)}^2\right)}{2}\right)}^2}{{\left(\frac{D}{2}\right)}^2}=\frac{1}{2}ln\frac{{\left(E\left({\left|y_k\right|}^2\right)\right)}^2}{D^2}=\frac{1}{2}ln\frac{E\left({\left|y_k\right|}^2\right)E\left({\left|\overline{y_k}\right|}^2\right)}{D^2}\hfill \\ =\frac{1}{2}ln\frac{E\left({\left|y_k\right|}^2\right)E\left({\left|y_{n-k}\right|}^2\right)}{D^2}=\frac{1}{2}\left(ln\frac{E\left({\left|y_k\right|}^2\right)}{D}+ln\frac{E\left({\left|y_{n-k}\right|}^2\right)}{D}\right)\hfill \end{array}$$

for all $k\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}$ with $k\ne \frac{n}{2}$. Hence, from Equation (9), if n is even, we have

$$\begin{array}{cc}\hfill {\breve{R}}_{x_{n:1}}\left(D\right)& \le \frac{1}{2n}\left(ln\frac{E\left({\left|y_{\frac{n}{2}}\right|}^2\right)}{D}+\sum _{k=\frac{n}{2}+1}^{n-1}\left(ln\frac{E\left({\left|y_k\right|}^2\right)}{D}+ln\frac{E\left({\left|y_{n-k}\right|}^2\right)}{D}\right)+ln\frac{E\left({\left|y_n\right|}^2\right)}{D}\right)\hfill \\ & =\frac{1}{2n}\left(ln\frac{E\left({\left|y_{\frac{n}{2}}\right|}^2\right)}{D}+\sum _{k=\frac{n}{2}+1}^{n-1}ln\frac{E\left({\left|y_k\right|}^2\right)}{D}+\sum _{j=1}^{\frac{n}{2}-1}ln\frac{E\left({\left|y_j\right|}^2\right)}{D}+ln\frac{E\left({\left|y_n\right|}^2\right)}{D}\right)\hfill \\ & =\frac{1}{2n}\sum _{k=1}^{n}ln\frac{E\left({\left|y_k\right|}^2\right)}{D}=\frac{1}{2n}ln\frac{{\prod }_{k=1}^{n}E\left({\left|y_k\right|}^2\right)}{D^n}=\frac{1}{2n}ln\frac{det({\widehat{C}}_n\left(f\right))}{D^n}\hfill \end{array}$$

and, if n is odd,

$$\begin{array}{cc}\hfill {\breve{R}}_{x_{n:1}}\left(D\right)& \le \frac{1}{2n}\left(\sum _{k=\frac{n+1}{2}}^{n-1}\left(ln\frac{E\left({\left|y_k\right|}^2\right)}{D}+ln\frac{E\left({\left|y_{n-k}\right|}^2\right)}{D}\right)+ln\frac{E\left({\left|y_n\right|}^2\right)}{D}\right)\hfill \\ & =\frac{1}{2n}\left(\sum _{k=\frac{n+1}{2}}^{n-1}ln\frac{E\left({\left|y_k\right|}^2\right)}{D}+\sum _{j=1}^{\frac{n-1}{2}}ln\frac{E\left({\left|y_j\right|}^2\right)}{D}+ln\frac{E\left({\left|y_n\right|}^2\right)}{D}\right)\hfill \\ & =\frac{1}{2n}\sum _{k=1}^{n}ln\frac{E\left({\left|y_k\right|}^2\right)}{D}=\frac{1}{2n}ln\frac{{\prod }_{k=1}^{n}E\left({\left|y_k\right|}^2\right)}{D^n}=\frac{1}{2n}ln\frac{det({\widehat{C}}_n\left(f\right))}{D^n}\hfill \end{array}$$

is yielded.

Step 4: We show Equation (18). Applying Equation (3) yields

$$\begin{array}{cc}\hfill 0\le \frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n}-R_{x_{n:1}}\left(D\right)& =\frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n}-\frac{1}{2n}ln\frac{det\left(T_n\left(f\right)\right)}{D^n}\hfill \\ & =\frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{det\left(T_n\left(f\right)\right)}=\frac{1}{2n}ln\left(det\left({\widehat{C}}_n\left(f\right)\right)det\left({\left(T_n\left(f\right)\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2n}ln\left(det\left({\widehat{C}}_n\left(f\right)\right)det\left({\left(\sqrt{T_n\left(f\right)}\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2n}ln\left(det\left({\widehat{C}}_n\left(f\right)\right)det\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2n}ln\left(det\left({\widehat{C}}_n\left(f\right)\right)det\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)det\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2n}ln\left(det\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)det\left({\widehat{C}}_n\left(f\right)\right)det\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2n}lndet\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\widehat{C}}_n\left(f\right){\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\hfill \\ & =\frac{1}{2n}ln\prod _{k=1}^n{\lambda }_k\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\widehat{C}}_n\left(f\right){\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right),\hfill \end{array}$$

where $\sqrt{T_n\left(f\right)}:=U_n{\mathrm{diag}}_{1\le k\le n}\left(\sqrt{{\lambda }_k\left(T_n\left(f\right)\right)}\right)U_n^{-1}$ with $T_n\left(f\right)=U_n{\mathrm{diag}}_{1\le k\le n}\left({\lambda }_k\left(T_n\left(f\right)\right)\right)U_n^{-1}$ being a unitary diagonalization of $T_n\left(f\right)$. Since $\sqrt{T_n\left(f\right)}$ is Hermitian and ${\widehat{C}}_n\left(f\right)$ is positive definite (see [5] (Lemma 5)), ${\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\widehat{C}}_n\left(f\right){\left(\sqrt{T_n\left(f\right)}\right)}^{-1}$ is positive definite, and applying the AM-GM inequality yields

$$\begin{array}{cc}\hfill 0\le \frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n}-R_{x_{n:1}}\left(D\right)& \le \frac{1}{2n}ln\left({\left(\frac{1}{n}\sum _{k=1}^n{\lambda }_k\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\widehat{C}}_n\left(f\right){\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)}^n\right)\hfill \\ & =\frac{1}{2}ln\left(\frac{1}{n}\mathrm{tr}\left({\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\widehat{C}}_n\left(f\right){\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2}ln\left(\frac{1}{n}\mathrm{tr}\left({\widehat{C}}_n\left(f\right){\left(\sqrt{T_n\left(f\right)}\right)}^{-1}{\left(\sqrt{T_n\left(f\right)}\right)}^{-1}\right)\right)\hfill \\ & =\frac{1}{2}ln\left(\frac{1}{n}\mathrm{tr}\left({\widehat{C}}_n\left(f\right){\left(T_n\left(f\right)\right)}^{-1}\right)\right)\hfill \\ & \le \frac{1}{2}ln\left(\frac{\sqrt{n}}{n}{\parallel {\widehat{C}}_n\left(f\right){\left(T_n\left(f\right)\right)}^{-1}\parallel }_F\right)\hfill \\ & =\frac{1}{2}ln\left(\frac{1}{\sqrt{n}}{\parallel ({\widehat{C}}_n\left(f\right)-T_n\left(f\right)){\left(T_n\left(f\right)\right)}^{-1}+I_n\parallel }_F\right)\hfill \\ & \le \frac{1}{2}ln\left(\frac{1}{\sqrt{n}}\left(\parallel ({\widehat{C}}_n\left(f\right)-T_n\left(f\right)){\left(T_n\left(f\right)\right)}^{-1}{\parallel }_F+\sqrt{n}\right)\right)\hfill \\ & \le \frac{1}{2}ln\left(\frac{1}{\sqrt{n}}\left(\parallel {\widehat{C}}_n\left(f\right)-T_n{\left(f\right)\parallel }_F{\parallel {\left(T_n\left(f\right)\right)}^{-1}\parallel }_2+\sqrt{n}\right)\right)\hfill \\ & =\frac{1}{2}ln\left(\frac{\parallel T_n\left(f\right)-{\widehat{C}}_n{\left(f\right)\parallel }_F}{\sqrt{n}}\frac{1}{{\lambda }_n\left(T_n\left(f\right)\right)}+1\right)\hfill \\ & \le \frac{1}{2}ln\left(1+\frac{1}{min\left(f\right)}\frac{\parallel T_n\left(f\right)-{\widehat{C}}_n{\left(f\right)\parallel }_F}{\sqrt{n}}\right),\hfill \end{array}$$

(20)

where $\mathrm{tr}$ stands for trace and ${\parallel ·\parallel }_F$ is the Frobenius norm. From [5] (Lemma 4), we obtain

$$\underset{n\to \infty }{lim}\frac{1}{2}ln\left(1+\frac{1}{min\left(f\right)}\frac{\parallel T_n\left(f\right)-{\widehat{C}}_n{\left(f\right)\parallel }_F}{\sqrt{n}}\right)=0$$

and, therefore,

$$\underset{n\to \infty }{lim}\left(\frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n}-R_{x_{n:1}}\left(D\right)\right)=0.$$

Consequently, applying [7] (Theorem 5), we conclude that

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim}\frac{1}{2n}ln\frac{det\left({\widehat{C}}_n\left(f\right)\right)}{D^n}& =\underset{n\to \infty }{lim}{R}_{x_{n:1}}\left(D\right)=\underset{n\to \infty }{lim}\frac{1}{2n}ln\frac{det\left(T_n\left(f\right)\right)}{D^n}=\frac{1}{2}\underset{n\to \infty }{lim}\frac{1}{n}ln\prod _{k=1}^n\frac{{\lambda }_k\left(T_n\left(f\right)\right)}{D}\hfill \\ & =\frac{1}{2}\underset{n\to \infty }{lim}\frac{1}{n}\sum _{k=1}^{n}ln\frac{{\lambda }_k\left(T_n\left(f\right)\right)}{D}=\frac{1}{4\pi }{\int }_0^{2\pi }ln\frac{f\left(\omega \right)}{D}d\omega ☐.\hfill \end{array}$$

As an example, Figure 1 shows Equation (16) for the case in which $f\left(\omega \right)=0.1+{(\omega -\pi )}^6$ with $\omega \in [0,2\pi ]$, $D=\frac{min\left(f\right)}{2}=0.05$, and $n\le 100$.

Figure 1. Numerical example of the upper bounds presented in Theorem 2.

Finally, observe that Theorem 2 also provides coding strategies to achieve the two new bounds of $R_{x_{n:1}}\left(D\right)$ presented: ${\tilde{R}}_{x_{n:1}}\left(D\right)$ and ${\breve{R}}_{x_{n:1}}\left(D\right)$. Specifically, Theorem 2 shows that ${\tilde{R}}_{x_{n:1}}\left(D\right)$ can be achieved by encoding $y_k$ separately, with $k\in \{\lceil \frac{n}{2}\rceil ,\dots ,n\}$, instead of encoding $x_{n:1}$ jointly, and that ${\breve{R}}_{x_{n:1}}\left(D\right)$ can be achieved by encoding separately the real part and the imaginary part of $y_k$ instead of encoding $y_k$ when $k\notin \{\frac{n}{2},n\}$. Therefore, although ${\tilde{R}}_{x_{n:1}}\left(D\right)$ is a tighter bound, the coding strategy associated with ${\breve{R}}_{x_{n:1}}\left(D\right)$ is simpler. It should be mentioned that, in order to achieve ${\tilde{R}}_{x_{n:1}}\left(D\right)$ and ${\breve{R}}_{x_{n:1}}\left(D\right),$ an optimal coding method of Gaussian random variables is required.