A Low-Complexity and Asymptotically Optimal Coding Strategy for Gaussian Vector Sources

Abstract

In this paper, we present a low-complexity coding strategy to encode (compress) finite-length data blocks of Gaussian vector sources. We show that for large enough data blocks of a Gaussian asymptotically wide sense stationary (AWSS) vector source, the rate of the coding strategy tends to the lowest possible rate. Besides being a low-complexity strategy it does not require the knowledge of the correlation matrix of such data blocks. We also show that this coding strategy is appropriate to encode the most relevant Gaussian vector sources, namely, wide sense stationary (WSS), moving average (MA), autoregressive (AR), and ARMA vector sources.

1. Introduction

The rate distortion function (RDF) of a source provides the minimum rate at which data can be encoded in order to be able to recover them with a mean squared error (MSE) per dimension not larger than a given distortion.

In this paper, we present a low-complexity coding strategy to encode (compress) finite-length data blocks of Gaussian N-dimensional vector sources. Moreover, we show that for large enough data blocks of a Gaussian asymptotically wide sense stationary (AWSS) vector source, the rate of our coding strategy tends to the RDF of the source. The definition of AWSS vector process can be found in ([1] (Definition 7.1)). This definition was first introduced for the scalar case N = 1 (see ([2] (Section 6)) or [3]), and it is based on the Gray concept of asymptotically equivalent sequences of matrices [4].

A low-complexity coding strategy can be found in [5] for finite-length data blocks of Gaussian wide sense stationary (WSS) sources and in [6] for finite-length data blocks of Gaussian AWSS autoregressive (AR) sources. Both precedents deal with scalar processes. The low-complexity coding strategy presented in this paper generalizes the aforementioned strategies to Gaussian AWSS vector sources.

Our coding strategy is based on the block discrete Fourier transform (DFT), and therefore, it turns out to be a low-complexity coding strategy when the fast Fourier transform (FFT) algorithm is used. Specifically, the computational complexity of our coding strategy is $O(nNlogn)$, where n is the length of the data blocks. Besides being a low-complexity strategy, it does not require the knowledge of the correlation matrix of such data blocks.

We show that this coding strategy is appropriate to encode the most relevant Gaussian vector sources, namely, WSS, moving average (MA), autoregressive (AR), and ARMA vector sources. Observe that our coding strategy is then appropriate to encode Gaussian vector sources found in the literature, such as the corrupted WSS vector sources considered in [7,8] for the quadratic Gaussian CEO problem.

The paper is organized as follows. In Section 2, we obtain several new mathematical results on the block DFT, and we present an upper bound for the RDF of a complex Gaussian vector. In Section 3, using the results given in Section 2, we present a new coding strategy based on the block DFT to encode finite-length data blocks of Gaussian vector sources. In Section 4, we show that for large enough data blocks of a Gaussian AWSS vector source, the rate of our coding strategy tends to the RDF of the source. In Section 5, we show that our coding strategy is appropriate to encode WSS, MA, AR, and ARMA vector sources. In Section 6, conclusions and numerical examples are presented.

2. Preliminaries

2.1. Notation

In this paper $\mathbb{N}$, $\mathbb{Z}$, $\mathbb{R}$, and $\mathbb{C}$ are the set of positive integers, the set of integers, the set of real numbers, and the set of complex numbers, respectively. The symbol ⊤ denotes transpose and the symbol ∗ denotes conjugate transpose. ${\parallel ·\parallel }_2$ and ${\parallel ·\parallel }_F$ are the spectral and the Frobenius norm, respectively. $\lceil x\rceil$ denotes the smallest integer higher than or equal to x. E stands for expectation, ⊗ is the Kronecker product, and ${\lambda }_j(A)$, $j\in \{1,\dots ,n\}$, denote the eigenvalues of an $n\times n$ Hermitian matrix A arranged in decreasing order. ${\mathbb{R}}^{n\times 1}$ is the set of real n-dimensional (column) vectors, ${\mathbb{C}}^{m\times n}$ denotes the set of $m\times n$ complex matrices, $0_{m\times n}$ is the $m\times n$ zero matrix, $I_n$ denotes the $n\times n$ identity matrix, and $V_n$ is the $n\times n$ Fourier unitary matrix, i.e.,

$${\left[V_n\right]}_{j,k}=\frac{1}{\sqrt{n}}{\mathrm{e}}^{-\frac{2\pi (j-1)(k-1)}{n}\mathrm{i}},j,k\in \{1,\dots ,n\},$$

where $\mathrm{i}$ is the imaginary unit.

If $A_j\in {\mathbb{C}}^{N\times N}$ for all $j\in \{1,\dots ,n\}$, then ${\mathrm{diag}}_{1\le j\le n}(A_j)$ denotes the $n\times n$ block diagonal matrix with $N\times N$ blocks given by ${\mathrm{diag}}_{1\le j\le n}(A_j)={(A_j{\delta }_{j,k})}_{j,k=1}^n$, where $\delta$ is the Kronecker delta.

$\mathrm{Re}$ and $\mathrm{Im}$ denote the real part and the imaginary part of a complex number, respectively. If $A\in {\mathbb{C}}^{m\times n}$, then $\mathrm{Re}(A)$ and $\mathrm{Im}(A)$ are the $m\times n$ real matrices given by ${\left[\mathrm{Re}(A)\right]}_{j,k}=\mathrm{Re}({\left[A\right]}_{j,k})$ and ${\left[\mathrm{Im}(A)\right]}_{j,k}=\mathrm{Im}({\left[A\right]}_{j,k})$ with $j\in \{1,\dots ,m\}$ and $k\in \{1,\dots ,n\}$, respectively.

If $z\in {\mathbb{C}}^{N\times 1}$, then $\widehat{z}$ denotes the real $2N$-dimensional vector given by

$$\widehat{z}=\left(\begin{array}{c}\mathrm{Re}(z)\\ \mathrm{Im}(z)\end{array}\right).$$

If $z_k\in {\mathbb{C}}^{N\times 1}$ for all $k\in \{1,\dots ,n\}$, then $z_{n:1}$ is the $nN$-dimensional vector given by

$$z_{n:1}=\left(\begin{array}{c}{z}_n\\ z_{n-1}\\ ⋮\\ z_1\end{array}\right).$$

Finally, if $z_k$ is a (complex) random N-dimensional vector for all $k\in \mathbb{N}$, $\{z_k\}$ denotes the corresponding (complex) random N-dimensional vector process.

2.2. New Mathematical Results on the Block DFT

We first give a simple result on the block DFT of real vectors.

Lemma 1.

Let $n,N\in \mathbb{N}$. Consider $x_k\in {\mathbb{C}}^{N\times 1}$ for all $k\in \{1,\dots ,n\}$. Suppose that $y_{n:1}$ is the block DFT of $x_{n:1}$, i.e.,

$$y_{n:1}=\left(V_n^{*}\otimes I_N\right)x_{n:1}={\left(V_n\otimes I_N\right)}^{*}{x}_{n:1}.$$

(1)

Then the two following assertions are equivalent:

1. 

$x_{n:1}\in {\mathbb{R}}^{nN\times 1}$.

2. 

$y_k=\overline{y_{n-k}}$ for all $k\in \{1,\dots ,n-1\}$ and $y_n\in {\mathbb{R}}^{N\times 1}$.

Proof. 

See Appendix A. □

We now give three new mathematical results on the block DFT of random vectors that are used in Section 3.

Theorem 1.

Consider $n,N\in \mathbb{N}$. Let $x_k$ be a random N-dimensional vector for all $k\in \{1,\dots ,n\}$. Suppose that $y_{n:1}$ is given by Equation (1). If $k\in \{1,\dots ,n\}$, then

$${\lambda }_{nN}(E\left(x_{n:1}{x}_{n:1}^{*}\right))\le {\lambda }_N\left(E(x_k{x}_k^{*})\right)\le {\lambda }_1\left(E(x_k{x}_k^{*})\right)\le {\lambda }_1\left(E\left(x_{n:1}{x}_{n:1}^{*}\right)\right)$$

(2)

and

$${\lambda }_{nN}(E\left(x_{n:1}{x}_{n:1}^{*}\right))\le {\lambda }_N\left(E(y_k{y}_k^{*})\right)\le {\lambda }_1\left(E(y_k{y}_k^{*})\right)\le {\lambda }_1\left(E\left(x_{n:1}{x}_{n:1}^{*}\right)\right).$$

(3)

Proof. 

See Appendix B. □

Theorem 2.

Let $x_{n:1}$ and $y_{n:1}$ be as in Theorem 1. Suppose that $x_{n:1}$ is real. If $k\in \{1,\dots ,n-1\}\setminus \{\frac{n}{2}\}$, then

$$\frac{{\lambda }_{nN}(E\left(x_{n:1}{x}_{n:1}^{\top }\right))}{2}\le {\lambda }_{2N}\left(E\left(\widehat{y_k}{\widehat{y_k}}^{\top }\right)\right)\le {\lambda }_1\left(E\left(\widehat{y_k}{\widehat{y_k}}^{\top }\right)\right)\le \frac{{\lambda }_1(E\left(x_{n:1}{x}_{n:1}^{\top }\right))}{2}.$$

Proof. 

See Appendix C. □

Lemma 2.

Let $x_{n:1}$ and $y_{n:1}$ be as in Theorem 1. If $k\in \{1,\dots ,n\}$, then

1. 

$E\left(y_k{y}_k^{*}\right)={\left[{\left(V_n\otimes I_N\right)}^{*}E\left(x_{n:1}{x}_{n:1}^{*}\right)\left(V_n\otimes I_N\right)\right]}_{n-k+1,n-k+1}$.

2. 

$E\left(y_k{y}_k^{\top }\right)={\left[{\left(V_n\otimes I_N\right)}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(\overline{V_n\otimes I_N}\right)\right]}_{n-k+1,n-k+1}$.

3. 

$E\left(\widehat{y_k}{\widehat{y_k}}^{\top }\right)=\frac{1}{2}\left(\begin{array}{cc}\mathrm{Re}\left(E\left(y_k{y}_k^{*}\right)\right)+\mathrm{Re}\left(E\left(y_k{y}_k^{\top }\right)\right)& \mathrm{Im}\left(E\left(y_k{y}_k^{\top }\right)\right)-\mathrm{Im}\left(E\left(y_k{y}_k^{*}\right)\right)\\ \mathrm{Im}\left(E\left(y_k{y}_k^{*}\right)\right)+\mathrm{Im}\left(E\left(y_k{y}_k^{\top }\right)\right)& \mathrm{Re}\left(E\left(y_k{y}_k^{*}\right)\right)-\mathrm{Re}\left(E\left(y_k{y}_k^{\top }\right)\right)\end{array}\right)$.

Proof. 

See Appendix D. □

2.3. Upper Bound for the RDF of a Complex Gaussian Vector

In [9], Kolmogorov gave a formula for the RDF of a real zero-mean Gaussian N-dimensional vector $\mathrm{x}$ with positive definite correlation matrix $E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)$, namely,

$$R_{\mathrm{x}}(D)=\frac{1}{N}\sum _{k=1}^{N}max\left\{0,\frac{1}{2}ln\frac{{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{\theta }\right\}\forall D\in \left(0,\frac{\mathrm{tr}\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{N}\right],$$

(4)

where $\mathrm{tr}$ denotes trace and $\theta$ is a real number satisfying

$$D=\frac{1}{N}\sum _{k=1}^{N}min\left\{\theta ,{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)\right\}.$$

If $D\in \left(0,{\lambda }_N\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)\right]$, an optimal coding strategy to achieve $R_{\mathrm{x}}(D)$ is to encode ${\left[\mathrm{z}\right]}_{1,1},\dots ,{\left[\mathrm{z}\right]}_{N,1}$ separately, where $\mathrm{z}=U^{\top }\mathrm{x}$ with U being a real orthogonal eigenvector matrix of $E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)$ (see ([6] (Corollary 1))). Observe that in order to obtain U, we need to know the correlation matrix $E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)$. This coding strategy also requires an optimal coding method for real Gaussian random variables. Moreover, as $0<D\le {\lambda }_N\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)\le \frac{1}{N}{\sum }_{k=1}^N{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)=\frac{\mathrm{tr}\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{N}$, if $D\in \left(0,{\lambda }_N\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)\right]$, then from Equation (4) we obtain

$$R_{\mathrm{x}}(D)=\frac{1}{N}\sum _{k=1}^N\frac{1}{2}ln\frac{{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{D}=\frac{1}{2N}ln\frac{{\prod }_{k=1}^N{\lambda }_k\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{D^N}=\frac{1}{2N}ln\frac{det\left(E\left(\mathrm{x}{\mathrm{x}}^{\top }\right)\right)}{D^N}.$$

(5)

We recall that $R_{\mathrm{x}}(D)$ can be thought of as the minimum rate (measured in nats) at which $\mathrm{x}$ can be encoded (compressed) in order to be able to recover it with an MSE per dimension not larger than D, that is:

$$\frac{E\left({∥\mathrm{x}-\tilde{\mathrm{x}}∥}_2^2\right)}{N}\le D,$$

where $\tilde{\mathrm{x}}$ denotes the estimation of $\mathrm{x}$.

The following result gives an upper bound for the RDF of a complex zero-mean Gaussian N-dimensional vector (i.e., a real zero-mean Gaussian $2N$-dimensional vector).

Lemma 3.

Consider $N\in \mathbb{N}$. Let z be a complex zero-mean Gaussian N-dimensional vector. If $E\left(\widehat{z}\widehat{z}{}^{\top }\right)$ is a positive definite matrix, then

$$R_{\widehat{z}}(D)\le \frac{1}{2N}ln\frac{det\left(E\left(z{z}^{*}\right)\right)}{{(2D)}^N}\forall D\in \left(0,{\lambda }_{2N}\left(E\left(\widehat{z}\widehat{z}{}^{\top }\right)\right)\right].$$

(6)

Proof. 

We divide the proof into three steps:

Step 1: We prove that $E\left(z{z}^{*}\right)$ is a positive definite matrix. We have

$$E\left(\widehat{z}\widehat{z}{}^{\top }\right)=\left(\begin{array}{cc}E\left(\mathrm{Re}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)& E\left(\mathrm{Re}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)\\ E\left(\mathrm{Im}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)& E\left(\mathrm{Im}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)\end{array}\right)$$

and

$$\begin{array}{ccc}\hfill E\left(z{z}^{*}\right)& =& E\left(\left(\mathrm{Re}(z)+\mathrm{i}\mathrm{Im}(z)\right)\left({\left(\mathrm{Re}(z)\right)}^{\top }-\mathrm{i}{\left(\mathrm{Im}(z)\right)}^{\top }\right)\right)\hfill \\ & =& E\left(\mathrm{Re}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)+E\left(\mathrm{Im}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)+\mathrm{i}E\left(\mathrm{Im}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)-\mathrm{i}E\left(\mathrm{Re}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right).\hfill \end{array}$$

Consider $u\in {\mathbb{C}}^{N\times 1}$, and suppose that $u^{*}E\left(z{z}^{*}\right)u=0$. We only need to show that $u=0_{N\times 1}$. As $E\left(\widehat{z}\widehat{z}{}^{\top }\right)$ is a positive definite matrix and

$$\begin{array}{ccc}\hfill {\left(\begin{array}{c}u\\ -\mathrm{i}u\end{array}\right)}^{*}E\left(\widehat{z}\widehat{z}{}^{\top }\right)\left(\begin{array}{c}u\\ -\mathrm{i}u\end{array}\right)& =& {\left(\begin{array}{c}u\\ -\mathrm{i}u\end{array}\right)}^{*}\left(\begin{array}{c}E\left(\mathrm{Re}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)u-\mathrm{i}E\left(\mathrm{Re}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)u\\ E\left(\mathrm{Im}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)u-\mathrm{i}E\left(\mathrm{Im}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)u\end{array}\right)\hfill \\ & =& u^{*}E\left(\mathrm{Re}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)u-\mathrm{i}{u}^{*}E\left(\mathrm{Re}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)u\hfill \\ & & +\mathrm{i}{u}^{*}E\left(\mathrm{Im}(z){\left(\mathrm{Re}(z)\right)}^{\top }\right)u+u^{*}E\left(\mathrm{Im}(z){\left(\mathrm{Im}(z)\right)}^{\top }\right)u\hfill \\ & =& u^{*}E\left(z{z}^{*}\right)u=0,\hfill \end{array}$$

we obtain $\left(\begin{array}{c}u\\ -\mathrm{i}u\end{array}\right)=0_{2N\times 1}$, or equivalently $u=0_{N\times 1}$.

Step 2: We show that $det\left(E\left(\widehat{z}\widehat{z}{}^{\top }\right)\right)\le \frac{{\left(det\left(E\left(z{z}^{*}\right)\right)\right)}^2}{2^{2N}}$. We have $E\left(z{z}^{*}\right)={\mathsf{\Lambda }}_c+\mathrm{i}{\mathsf{\Lambda }}_s$, where ${\mathsf{\Lambda }}_c=E\left(\mathrm{Re}(z)\left(\mathrm{Re}(z)\right){}^{\top }\right)+E\left(\mathrm{Im}(z)\left(\mathrm{Im}(z)\right){}^{\top }\right)$ and ${\mathsf{\Lambda }}_s=E\left(\mathrm{Im}(z)\left(\mathrm{Re}(z)\right){}^{\top }\right)-\left(E\left(\mathrm{Im}(z)\left(\mathrm{Re}(z)\right){}^{\top }\right)\right){}^{\top }$. Applying ([10] (Corollary 1)), we obtain

$$\begin{array}{cc}\hfill det\left(E\left(\widehat{z}\widehat{z}{}^{\top }\right)\right)& \le \frac{det\left({\mathsf{\Lambda }}_c+{\mathsf{\Lambda }}_s{\mathsf{\Lambda }}_c^{-1}{\mathsf{\Lambda }}_s\right)det\left({\mathsf{\Lambda }}_c\right)}{2^{2N}}=\frac{det\left(I_N+{\mathsf{\Lambda }}_s{\mathsf{\Lambda }}_c^{-1}{\mathsf{\Lambda }}_s{\mathsf{\Lambda }}_c^{-1}\right){\left(det\left({\mathsf{\Lambda }}_c\right)\right)}^2}{2^{2N}}\hfill \\ & =\frac{det\left(\left(I_N+\mathrm{i}{\mathsf{\Lambda }}_s{\mathsf{\Lambda }}_c^{-1}\right)\left(I_N-\mathrm{i}{\mathsf{\Lambda }}_s{\mathsf{\Lambda }}_c^{-1}\right)\right){\left(det\left({\mathsf{\Lambda }}_c\right)\right)}^2}{2^{2N}}=\frac{det\left({\mathsf{\Lambda }}_c+\mathrm{i}{\mathsf{\Lambda }}_s\right)det\left({\mathsf{\Lambda }}_c-\mathrm{i}{\mathsf{\Lambda }}_s\right)}{2^{2N}}\hfill \\ & =\frac{det\left(E\left(z{z}^{*}\right)\right)det\left(\overline{E\left(z{z}^{*}\right)}\right)}{2^{2N}}=\frac{det\left(E\left(z{z}^{*}\right)\right)\overline{det\left(E\left(z{z}^{*}\right)\right)}}{2^{2N}}=\frac{{\left(det\left(E\left(z{z}^{*}\right)\right)\right)}^2}{2^{2N}}.\hfill \end{array}$$

Step 3: We now prove Equation (6). From Equation (5), we conclude that

$$\begin{array}{c}\hfill R_{\widehat{z}}(D)=\frac{1}{4N}ln\frac{det\left(E\left(\widehat{z}\widehat{z}{}^{\top }\right)\right)}{D^{2N}}\le \frac{1}{4N}ln\frac{{\left(det\left(E\left(z{z}^{*}\right)\right)\right)}^2}{{(2D)}^{2N}}=\frac{1}{2N}ln\frac{det\left(E\left(z{z}^{*}\right)\right)}{{(2D)}^N}.\end{array}$$

 □

3. Low-Complexity Coding Strategy for Gaussian Vector Sources

In this section (see Theorem 3), we present our coding strategy for Gaussian vector sources. To encode a finite-length data block $x_{n:1}$ of a Gaussian N-dimensional vector source $\{x_k\}$, we compute the block DFT of $x_{n:1}$ ($y_{n:1}$) and we encode $y_{\lceil \frac{n}{2}\rceil },\dots ,y_n$ separately with $\frac{E\left({∥y_k-\widetilde{y_k}∥}_2^2\right)}{N}\le D$ for all $k\in \left\{⌈\frac{n}{2}⌉,\dots ,n\right\}$ (see Figure 1).

Figure 1. Proposed coding strategy for Gaussian vector sources. In this figure, $Encode{r}_k$ ($Decode{r}_k$) denotes the optimal encoder (decoder) for the Gaussian N-dimensional vector $y_k$ with $k\in \left\{⌈\frac{n}{2}⌉,\dots ,n\right\}$.

We denote by ${\tilde{R}}_{x_{n:1}}(D)$ the rate of our strategy. Theorem 3 also provides an upper bound of ${\tilde{R}}_{x_{n:1}}(D)$. This upper bound is used in Section 4 to prove that our coding strategy is asymptotically optimal whenever the Gaussian vector source is AWSS.

In Theorem 3 $C_{A_n}$ denotes the matrix $(V_n\otimes I_N){\mathrm{diag}}_{1\le k\le n}\left({\left[{(V_n\otimes I_N)}^{*}{A}_n(V_n\otimes I_N)\right]}_{k,k}\right){(V_n\otimes I_N)}^{*}$, where $A_n\in {\mathbb{C}}^{nN\times nN}$.

Theorem 3.

Consider $n,N\in \mathbb{N}$. Let $x_k$ be a random N-dimensional vector for all $k\in \{1,\dots ,n\}$. Suppose that $x_{n:1}$ is a real zero-mean Gaussian vector with a positive definite correlation matrix (or equivalently, ${\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)>0$). Let $y_{n:1}$ be the random vector given by Equation (1). If $D\in \left(0,{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right]$, then

$$\begin{array}{cc}\hfill R_{x_{n:1}}& (D)\le {\tilde{R}}_{x_{n:1}}(D)\le \frac{1}{2nN}ln\frac{\det \left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{D^{nN}},\hfill \end{array}$$

(7)

where

$${\tilde{R}}_{x_{n:1}}(D)=\left\{\begin{array}{cc}\frac{R_{y_{\frac{n}{2}}}\left(D\right)+2{\sum }_{k=\frac{n}{2}+1}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+R_{y_n}(D)}{n}\hfill & ifniseven,\hfill \\ \frac{2{\sum }_{k=\frac{n+1}{2}}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+R_{y_n}(D)}{n}\hfill & ifnisodd.\hfill \end{array}\right.$$

Moreover,

$$\begin{array}{c}\hfill 0\le \frac{1}{2nN}ln\frac{\det \left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{D^{nN}}-R_{x_{n:1}}(D)\le \frac{1}{2}ln\left(1+\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{nN}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}\right).\end{array}$$

(8)

Proof. 

We divide the proof into three steps:

Step 1: We show that $R_{x_{n:1}}(D)\le {\tilde{R}}_{x_{n:1}}(D)$. From Lemma 1, $y_k=\overline{y_{n-k}}$ for all $k\in \{1,\dots ,\lceil \frac{n}{2}\rceil -1\}$, and $y_k\in {\mathbb{R}}^{N\times 1}$ with $k\in \{\frac{n}{2},n\}\cap \mathbb{N}$. We encode $y_{\lceil \frac{n}{2}\rceil },\dots ,y_n$ separately (i.e., if n is even, we encode $y_{\frac{n}{2}},\widehat{y_{\frac{n}{2}+1}},\dots ,\widehat{y_{n-1}},y_n$ separately, and if n is odd, we encode $\widehat{y_{\frac{n+1}{2}}},\dots ,\widehat{y_{n-1}},y_n$ separately) with

$$\frac{E\left({∥\widehat{y_k}-\widetilde{\widehat{y_k}}∥}_2^2\right)}{2N}\le \frac{D}{2},k\in \left\{⌈\frac{n}{2}⌉\dots ,n-1\right\}\setminus \left\{\frac{n}{2}\right\}$$

and

$$\frac{E\left({∥y_k-\widetilde{y_k}∥}_2^2\right)}{N}\le D,k\in \left\{\frac{n}{2},n\right\}\cap \mathbb{N}.$$

Let $\widetilde{x_{n:1}}=\left(V_n\otimes I_N\right)\widetilde{y_{n:1}}$ with

$$\widetilde{y_{n:1}}=\left(\begin{array}{c}\widetilde{y_n}\\ ⋮\\ \widetilde{y_1}\end{array}\right),$$

where $\widehat{\widetilde{y_k}}=\widetilde{\widehat{y_k}}$ for all $k\in \{\lceil \frac{n}{2}\rceil \dots ,n-1\}\setminus \{\frac{n}{2}\}$, and $\widetilde{y_k}=\overline{\widetilde{y_{n-k}}}$ for all $k\in \{1,\dots ,\lceil \frac{n}{2}\rceil -1\}$. Applying Lemma 1 yields $\widetilde{x_{n:1}}\in {\mathbb{R}}^{nN\times 1}$. As ${\left(V_n\otimes I_N\right)}^{*}$ is unitary and ${\parallel ·\parallel }_2$ is unitarily invariant, we have

$$\begin{array}{cc}\hfill \frac{E\left({∥x_{n:1}-\widetilde{x_{n:1}}∥}_2^2\right)}{nN}& =\frac{E\left({∥{\left(V_n\otimes I_N\right)}^{*}{x}_{n:1}-{\left(V_n\otimes I_N\right)}^{*}\widetilde{x_{n:1}}∥}_2^2\right)}{nN}\hfill \\ & =\frac{E\left({∥y_{n:1}-\widetilde{y_{n:1}}∥}_2^2\right)}{nN}=\frac{1}{nN}\sum _{k=1}^{n}E\left({∥y_k-\widetilde{y_k}∥}_2^2\right)\hfill \\ & =\frac{1}{nN}\left(2\sum _{k_1\in \{\lceil \frac{n}{2}\rceil \dots ,n-1\}\setminus \{\frac{n}{2}\}}E\left({∥y_{k_1}-\widetilde{y_{k_1}}∥}_2^2\right)+\sum _{k_2\in \{\frac{n}{2},n\}\cap \mathbb{N}}E\left({∥y_{k_2}-\widetilde{y_{k_2}}∥}_2^2\right)\right)\hfill \\ & =\frac{1}{nN}\left(2\sum _{k_1\in \{\lceil \frac{n}{2}\rceil \dots ,n-1\}\setminus \{\frac{n}{2}\}}E\left({∥\widehat{y_{k_1}}-\tilde{\widehat{y_{k_1}}}∥}_2^2\right)+\sum _{k_2\in \{\frac{n}{2},n\}\cap \mathbb{N}}E\left({∥y_{k_2}-\widetilde{y_{k_2}}∥}_2^2\right)\right)\hfill \\ & \le \left\{\begin{array}{cc}\frac{1}{nN}\left(2\left(\frac{n}{2}-1\right)ND+2ND\right)\hfill & \mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ \frac{1}{nN}\left(2\left(n-\frac{n+1}{2}\right)ND+ND\right)\hfill & \mathrm{if}n\mathrm{is}\mathrm{odd},\hfill \end{array}\right\}=D.\hfill \end{array}$$

Consequently,

$$R_{x_{n:1}}(D)\le \left\{\begin{array}{cc}\frac{N{R}_{y_{\frac{n}{2}}}\left(D\right)+2N{\sum }_{k=\frac{n}{2}+1}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+N{R}_{y_n}(D)}{nN}\hfill & \mathrm{if}n\mathrm{is}\mathrm{even},\hfill \\ \frac{2N{\sum }_{k=\frac{n+1}{2}}^{n-1}{R}_{\widehat{y_k}}\left(\frac{D}{2}\right)+N{R}_{y_n}(D)}{nN}\hfill & \mathrm{if}n\mathrm{is}\mathrm{odd},\hfill \end{array}\right\}={\tilde{R}}_{x_{n:1}}(D).$$

Step 2: We prove that ${\tilde{R}}_{x_{n:1}}(D)\le \frac{1}{2nN}ln\frac{\det \left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{D^{nN}}$. From Equations (3) and (5), we obtain

$$R_{y_k}(D)=\frac{1}{2N}ln\frac{det\left(E\left(y_k{y}_k^{\top }\right)\right)}{D^N},k\in \left\{\frac{n}{2},n\right\}\cap \mathbb{N},$$

(9)

and applying Theorem 2 and Equation (5) yields

$$R_{\widehat{y_k}}\left(\frac{D}{2}\right)=\frac{1}{4N}ln\frac{det\left(E\left(\widehat{y_k}{\widehat{y_k}}^{\top }\right)\right)}{{\left(\frac{D}{2}\right)}^{2N}},k\in \left\{1,\dots ,n-1\right\}\setminus \left\{\frac{n}{2}\right\}.$$

(10)

From Lemma 3, we have

$$\begin{array}{cc}\hfill & {\tilde{R}}_{x_{n:1}}(D)\hfill \\ \hfill & \le \frac{1}{n}\left[2\sum _{k_1\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}\setminus \{\frac{n}{2}\}}\frac{1}{2N}ln\frac{det\left(E\left(y_{k_1}{y}_{k_1}^{*}\right)\right)}{D^N}+\sum _{k_2\in \{\frac{n}{2},n\}\cap \mathbb{N}}\frac{1}{2N}ln\frac{det\left(E\left(y_{k_2}{y}_{k_2}^{*}\right)\right)}{D^N}\right]\hfill \\ \hfill & =\frac{1}{2nN}\left[\sum _{k_1\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}\setminus \{\frac{n}{2}\}}\left(ln\frac{det\left(E\left(y_{k_1}{y}_{k_1}^{*}\right)\right)}{D^N}+ln\frac{\overline{det\left(E\left(y_{k_1}{y}_{k_1}^{*}\right)\right)}}{D^N}\right)+\sum _{k_2\in \{\frac{n}{2},n\}\cap \mathbb{N}}ln\frac{det\left(E\left(y_{k_2}{y}_{k_2}^{*}\right)\right)}{D^N}\right]\hfill \\ \hfill & =\frac{1}{2nN}\left[\sum _{k_1\in \{\lceil \frac{n}{2}\rceil ,\dots ,n-1\}\setminus \{\frac{n}{2}\}}\left(ln\frac{det\left(E\left(y_{k_1}{y}_{k_1}^{*}\right)\right)}{D^N}+ln\frac{det\left(E\left(y_{n-k_1}{y}_{n-k_1}^{*}\right)\right)}{D^N}\right)\right.\hfill \\ \hfill & \left.+\sum _{k_2\in \{\frac{n}{2},n\}\cap \mathbb{N}}ln\frac{det\left(E\left(y_{k_2}{y}_{k_2}^{*}\right)\right)}{D^N}\right]\hfill \\ \hfill & =\frac{1}{2nN}\sum _{k=1}^{n}ln\frac{det\left(E\left(y_k{y}_k^{*}\right)\right)}{D^N}=\frac{1}{2nN}ln\frac{{\prod }_{k=1}^{n}det\left(E\left(y_k{y}_k^{*}\right)\right)}{D^{nN}}.\hfill \end{array}$$

As

$$\begin{array}{cc}\hfill & \left\{{\lambda }_j(E\left(y_k{y}_k^{*}\right)):j\in \{1,\dots ,N\},k\in \{1,\dots ,n\}\right\}=\left\{{\lambda }_j({\left[E\left(y_{n:1}{y}_{n:1}^{*}\right)\right]}_{k,k}):j\in \{1,\dots ,N\},k\in \{1,\dots ,n\}\right\}\hfill \\ \hfill & =\left\{{\lambda }_j\left({\left[\left(V_n\otimes I_N\right){}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)\right]}_{k,k}\right):j\in \{1,\dots ,N\},k\in \{1,\dots ,n\}\right\}\hfill \\ \hfill & =\left\{{\lambda }_j\left({\mathrm{diag}}_{1\le k\le n}\left({\left[\left(V_n\otimes I_N\right){}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)\right]}_{k,k}\right)\right):j\in \{1,\dots ,nN\}\right\}\hfill \\ \hfill & =\left\{{\lambda }_j\left(\left(V_n\otimes I_N\right){\mathrm{diag}}_{1\le k\le n}\left({\left[\left(V_n\otimes I_N\right){}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)\right]}_{k,k}\right){\left(V_n\otimes I_N\right)}^{-1}\right):j\in \{1,\dots ,nN\}\right\}\hfill \\ \hfill & =\left\{{\lambda }_j\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right):j\in \{1,\dots ,nN\}\right\},\hfill \end{array}$$

(11)

we obtain

$$\begin{array}{c}\hfill \prod _{k=1}^{n}det\left(E\left(y_k{y}_k^{*}\right)\right)=\prod _{k=1}^n\prod _{j=1}^N{\lambda }_j\left(E\left(y_k{y}_k^{*}\right)\right)=\prod _{j=1}^{nN}{\lambda }_j\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)=det\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right).\end{array}$$

Step 3: We show Equation (8).

As $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$ is a positive definite matrix (or equivalently, $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$ is Hermitian and ${\lambda }_j(E\left(x_{n:1}{x}_{n:1}^{\top }\right))>0$ for all $j\in \{1,\dots ,nN\}$), ${\left(V_n\otimes I_N\right)}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)$ is Hermitian. Hence, ${\left[{\left(V_n\otimes I_N\right)}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)\right]}_{k,k}$ is Hermitian for all $k\in \{1,\dots ,n\}$, and therefore, ${\mathrm{diag}}_{1\le k\le n}\left({\left[{\left(V_n\otimes I_N\right)}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)\right]}_{k,k}\right)$ is also Hermitian. Consequently, $\left(V_n\otimes I_N\right){\mathrm{diag}}_{1\le k\le n}\left({\left[{\left(V_n\otimes I_N\right)}^{*}E\left(x_{n:1}{x}_{n:1}^{\top }\right)\left(V_n\otimes I_N\right)\right]}_{k,k}\right){\left(V_n\otimes I_N\right)}^{*}$ is Hermitian, and applying Equations (3) and (11), we have that $C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}$ is a positive definite matrix.

Let $E\left(x_{n:1}{x}_{n:1}^{\top }\right)=U{\mathrm{diag}}_{1\le j\le nN}\left({\lambda }_j\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right)U^{-1}$ be an eigenvalue decomposition (EVD) of $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$, where U is unitary. Thus, $\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}=U{\mathrm{diag}}_{1\le j\le nN}\left(\sqrt{{\lambda }_j\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}\right)U^{*}$ and ${\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}=U{\mathrm{diag}}_{1\le j\le nN}\left(\frac{1}{\sqrt{{\lambda }_j\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}}\right)U^{*}$.

Since ${\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}$ is Hermitian and $C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}$ is a positive definite matrix, ${\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}{C}_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}$ is also a positive definite matrix.

From Equation (5), we have

$$R_{x_{n:1}}(D)=\frac{1}{2nN}ln\frac{det\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}{D^{nN}},$$

(12)

and applying the arithmetic mean-geometric mean inequality yields

$$\begin{array}{cc}\hfill 0& \le \frac{1}{2nN}ln\frac{det\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{D^{nN}}-R_{x_{n:1}}(D)\hfill \\ \hfill & =\frac{1}{2nN}ln\frac{det\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{det\left(E(x_{n:1}{x}_{n:1}^{\top })\right)}=\frac{1}{2nN}ln\frac{det\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{det\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)det\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}\hfill \\ \hfill & =\frac{1}{2nN}ln\left(det\left({\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)det\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)det\left({\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)\right)\hfill \\ \hfill & =\frac{1}{2nN}lndet\left({\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}{C}_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)\hfill \\ \hfill & =\frac{1}{2nN}ln\prod _{j=1}^{nN}{\lambda }_j\left({\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}{C}_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)\hfill \\ \hfill & \le \frac{1}{2nN}ln\left({\left(\frac{1}{nN}\sum _{j=1}^{nN}{\lambda }_j\left({\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}{C}_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)\right)}^{nN}\right)\hfill \\ \hfill & =\frac{1}{2}ln\left(\frac{1}{nN}\mathrm{tr}\left({\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}{C}_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)\right)\hfill \\ \hfill & =\frac{1}{2}ln\left(\frac{1}{nN}\mathrm{tr}\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}{\left(\sqrt{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}^{-1}\right)\right)\hfill \\ \hfill & =\frac{1}{2}ln\left(\frac{1}{nN}\mathrm{tr}\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}\right)\right)\hfill \\ \hfill & \le \frac{1}{2}ln\left(\frac{\sqrt{nN}}{nN}{∥C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}{\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}∥}_F\right)\hfill \\ \hfill & =\frac{1}{2}ln\left(\frac{1}{\sqrt{nN}}{∥\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}-E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right){\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}+I_{nN}∥}_F\right)\hfill \\ \hfill & \le \frac{1}{2}ln\left(\frac{1}{\sqrt{nN}}\left({∥\left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}-E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right){\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}∥}_F+\sqrt{nN}\right)\right)\hfill \\ \hfill & \le \frac{1}{2}ln\left(\frac{1}{\sqrt{nN}}\left({∥C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}-E\left(x_{n:1}{x}_{n:1}^{\top }\right)∥}_F{∥{\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}∥}_2+\sqrt{nN}\right)\right)\hfill \\ \hfill & =\frac{1}{2}ln\left(1+\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{nN}{\lambda }_{nN}(E\left(x_{n:1}{x}_{n:1}^{\top }\right))}\right).\hfill \end{array}$$

 □

In Equation (12), $R_{x_{n:1}}(D)$ is written in terms of $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$. ${\tilde{R}}_{x_{n:1}}(D)$ can be written in terms of $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$ and $V_n$ by using Lemma 2 and Equations (9) and (10).

As our coding strategy requires the computation of the block DFT, its computational complexity is $O(nNlogn)$ whenever the FFT algorithm is used. We recall that the computational complexity of the optimal coding strategy for $x_{n:1}$ is $O(n^2{N}^2)$ since it requires the computation of $U_n^{\top }{x}_{n:1}$, where $U_n$ is a real orthogonal eigenvector matrix of $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$. Observe that such eigenvector matrix $U_n$ also needs to be computed, which further increases the complexity. Hence, the main advantage of our coding strategy is that it notably reduces the computational complexity of coding $x_{n:1}$. Moreover, our coding strategy does not require the knowledge of $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$. It only requires the knowledge of $E\left(\widehat{y_k}{\widehat{y_k}}^{\top }\right)$, with $k\in \{\lceil \frac{n}{2}\rceil \dots ,n\}$.

It should be mentioned that Equation (7) provides two upper bounds for the RDF of finite-length data blocks of a real zero-mean Gaussian N-dimensional vector source $\{x_k\}$. The greatest upper bound in Equation (7) was given in [11] for the case in which the random vector source $\{x_k\}$ is WSS, and therefore, the correlation matrix of the Gaussian vector, $E\left(x_{n:1}{x}_{n:1}^{\top }\right)$, is a block Toeplitz matrix. Such upper bound was first presented by Pearl in [12] for the case in which the source is WSS and $N=1$. However, neither [11] nor [12] propose a coding strategy for $\{x_k\}$.

4. Optimality of the Proposed Coding Strategy for Gaussian AWSS Vector Sources

In this section (see Theorem 4), we show that our coding strategy is asymptotically optimal, i.e., we show that for large enough data blocks of a Gaussian AWSS vector source $\{x_k\}$, the rate of our coding strategy, presented in Section 3, tends to the RDF of the source.

We begin by introducing some notation. If $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$ is a continuous and $2\pi$-periodic matrix-valued function of a real variable, we denote by $T_n(X)$ the $n\times n$ block Toeplitz matrix with $N\times N$ blocks given by

$$T_n(X)={({\mathsf{X}}_{j-k})}_{j,k=1}^n,$$

where ${\{{\mathsf{X}}_k\}}_{k\in \mathbb{Z}}$ is the sequence of Fourier coefficients of X:

$${\mathsf{X}}_k=\frac{1}{2\pi }{\int }_0^{2\pi }{\mathrm{e}}^{-k\omega \mathrm{i}}X(\omega )d\omega \forall k\in \mathbb{Z}.$$

If $A_n$ and $B_n$ are $nN\times nN$ matrices for all $n\in \mathbb{N}$, we write $\{A_n\}\sim \{B_n\}$ when the sequences $\{A_n\}$ and $\{B_n\}$ are asymptotically equivalent (see ([13] (p. 5673))), that is, $\{\parallel A_n{\parallel }_2\}$ and $\{\parallel B_n{\parallel }_2\}$ are bounded and

$$\underset{n\to \infty }{lim}\frac{\parallel A_n-B_n{\parallel }_F}{\sqrt{n}}=0.$$

The original definition of asymptotically equivalent sequences of matrices was given by Gray (see ([2] (Section 2.3)) or [4]) for $N=1$.

We now review the definition of the AWSS vector process given in ([1] (Definition 7.1)). This definition was first introduced for the scalar case N = 1 (see ([2] (Section 6)) or [3]).

Definition 1.

Let $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, and suppose that it is continuous and $2\pi$-periodic. A random N-dimensional vector process $\{x_k\}$ is said to be AWSS with asymptotic power spectral density (APSD) X if it has constant mean (i.e., $E(x_{k_1})=E(x_{k_2})$ for all $k_1,k_2\in \mathbb{N})$ and $\{E\left(x_{n:1}{x}_{n:1}^{*}\right)\}\sim \{T_n(X)\}$.

We recall that the RDF of $\{x_k\}$ is defined as ${lim}_{n\to \infty }{R}_{x_{n:1}}(D)$.

Theorem 4.

Let $\{x_k\}$ be a real zero-mean Gaussian AWSS N-dimensional vector process with APSD X. Suppose that ${inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)>0$. If $D\in \left(0,{inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right]$, then

$$\underset{n\to \infty }{lim}{R}_{x_{n:1}}(D)=\underset{n\to \infty }{lim}{\tilde{R}}_{x_{n:1}}(D)=\frac{1}{4\pi N}{\int }_0^{2\pi }ln\frac{det(X(\omega ))}{D^N}d\omega .$$

(13)

Proof. 

We divide the proof into two steps:

Step 1: We show that ${lim}_{n\to \infty }{R}_{x_{n:1}}(D)=\frac{1}{4\pi N}{\int }_0^{2\pi }ln\frac{det(X(\omega ))}{D^N}d\omega$. From Equation (12), ([1] (Theorem 6.6)), and ([14] (Proposition 2)) yields

$$\begin{array}{cc}\hfill \underset{n\to \infty }{lim}{R}_{x_{n:1}}(D)& =\underset{n\to \infty }{lim}\frac{1}{2nN}ln\frac{{\prod }_{k=1}^{nN}{\lambda }_k\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}{D^{nN}}=\underset{n\to \infty }{lim}\frac{1}{2nN}\sum _{k=1}^{nN}ln\frac{{\lambda }_k\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}{D}\hfill \\ \hfill & =\frac{1}{4\pi }{\int }_0^{2\pi }\frac{1}{N}\sum _{k=1}^{N}ln\frac{{\lambda }_k(X(\omega ))}{D}d\omega =\frac{1}{4\pi N}{\int }_0^{2\pi }ln\frac{det(X(\omega ))}{D^N}d\omega .\hfill \end{array}$$

Step 2: We prove that ${lim}_{n\to \infty }{R}_{x_{n:1}}(D)={lim}_{n\to \infty }{\tilde{R}}_{x_{n:1}}(D)$. Applying Equations (7) and (8), we obtain

$$\begin{array}{cc}\hfill 0& \le {\tilde{R}}_{x_{n:1}}(D)-R_{x_{n:1}}(D)\le \frac{1}{2nN}ln\frac{\det \left(C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}\right)}{D^{nN}}-R_{x_{n:1}}(D)\hfill \\ \hfill & \le \frac{1}{2}ln\left(1+\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{nN}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}\right)\hfill \\ \hfill & \le \frac{1}{2}ln\left(1+\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{nN}{inf}_{m\in \mathbb{N}}{\lambda }_{mN}\left(E\left(x_{m:1}{x}_{m:1}^{\top }\right)\right)}\right)\forall n\in \mathbb{N}.\hfill \end{array}$$

(14)

To finish the proof, we only need to show that

$$\underset{n\to \infty }{lim}\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{n}}=0.$$

(15)

Let $C_n(X)$ be the $n\times n$ block circulant matrix with $N\times N$ blocks defined in ([13] (p. 5674)), i.e.,

$$C_n(X)=(V_n\otimes I_N){\mathrm{diag}}_{1\le k\le n}\left(X\left(\frac{2\pi (k-1)}{n}\right)\right){(V_n\otimes I_N)}^{*}\forall n\in \mathbb{N}.$$

Observe that

$$\begin{array}{cc}\hfill C_{C_n(X)}=& (V_n\otimes I_N){\mathrm{diag}}_{1\le k\le n}\left({\left[{(V_n\otimes I_N)}^{*}{C}_n(X)(V_n\otimes I_N)\right]}_{k,k}\right){(V_n\otimes I_N)}^{*}\hfill \\ \hfill =& (V_n\otimes I_N){\mathrm{diag}}_{1\le k\le n}\left({\left[{\mathrm{diag}}_{1\le j\le n}\left(X\left(\frac{2\pi (j-1)}{n}\right)\right)\right]}_{k,k}\right){(V_n\otimes I_N)}^{*}\hfill \\ \hfill =& (V_n\otimes I_N){\mathrm{diag}}_{1\le k\le n}\left(X\left(\frac{2\pi (k-1)}{n}\right)\right){(V_n\otimes I_N)}^{*}=C_n(X)\forall n\in \mathbb{N}.\hfill \end{array}$$

Consequently, as the Frobenius norm is unitarily invariant, we have

$$\begin{array}{cc}\hfill & {∥C_n(X)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F={∥C_{C_n(X)}-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F\hfill \\ \hfill & ={∥(V_n\otimes I_N){\mathrm{diag}}_{1\le k\le n}\left({\left[{(V_n\otimes I_N)}^{*}\left(C_n(X)-E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)(V_n\otimes I_N)\right]}_{k,k}\right){(V_n\otimes I_N)}^{*}∥}_F\hfill \\ \hfill & ={∥{\mathrm{diag}}_{1\le k\le n}\left({\left[{(V_n\otimes I_N)}^{*}\left(C_n(X)-E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)(V_n\otimes I_N)\right]}_{k,k}\right)∥}_F\hfill \\ \hfill & \le {∥{(V_n\otimes I_N)}^{*}\left(C_n(X)-E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)(V_n\otimes I_N)∥}_F={∥C_n(X)-E\left(x_{n:1}{x}_{n:1}^{\top }\right)∥}_F\forall n\in \mathbb{N}.\hfill \end{array}$$

Therefore,

$$\begin{array}{cc}\hfill 0\le & \frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{n}}\le \frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_n(X)∥}_F}{\sqrt{n}}+\frac{{∥C_n(X)-C_{E\left(x_{n:1}{x}_{n:1}^{\top }\right)}∥}_F}{\sqrt{n}}\hfill \\ \hfill \le & 2\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-C_n(X)∥}_F}{\sqrt{n}}\le 2\left(\frac{{∥E\left(x_{n:1}{x}_{n:1}^{\top }\right)-T_n(X)∥}_F}{\sqrt{n}}+\frac{{∥T_n(X)-C_n(X)∥}_F}{\sqrt{n}}\right)\forall n\in \mathbb{N}.\hfill \end{array}$$

(16)

Since $\{E\left(x_{n:1}{x}_{n:1}^{\top }\right)\}\sim \{T_n(X)\}$, Equation (16) and ([1] (Lemma 6.1)) yields Equation (15). □

Observe that the integral formula in Equation (13) provides the value of the RDF of the Gaussian AWSS vector source whenever $D\in \left(0,{inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right]$. An integral formula of such an RDF for any $D>0$ can be found in ([15] (Theorem 1)). It should be mentioned that ([15] (Theorem 1)) generalized the integral formulas previously given in the literature for the RDF of certain Gaussian AWSS sources, namely, WSS scalar sources [9], AR AWSS scalar sources [16], and AR AWSS vector sources of finite order [17].

5. Relevant AWSS Vector Sources

WSS, MA, AR, and ARMA vector processes are frequently used to model multivariate time series (see, e.g., [18]) that arise in any domain that involves temporal measurements. In this section, we show that our coding strategy is appropriate to encode the aforementioned vector sources whenever they are Gaussian and AWSS.

It should be mentioned that Gaussian AWSS MA vector (VMA) processes, Gaussian AWSS AR vector (VAR) processes, and Gaussian AWSS ARMA vector (VARMA) processes are frequently called Gaussian stationary VMA processes, Gaussian stationary VAR processes, and Gaussian stationary VARMA processes, respectively (see, e.g., [18]). However, they are asymptotically stationary but not stationary, because their corresponding correlation matrices are not block Toeplitz.

5.1. WSS Vector Sources

In this subsection (see Theorem 5), we give conditions under which our coding strategy is asymptotically optimal for WSS vector sources.

We first recall the well-known concept of WSS vector process.

Definition 2.

Let $X:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, and suppose that it is continuous and $2\pi$-periodic. A random N-dimensional vector process $\{x_k\}$ is said to be WSS (or weakly stationary) with PSD X if it has constant mean and $\{E\left(x_{n:1}{x}_{n:1}^{*}\right)\}=\{T_n(X)\}$.

Theorem 5.

Let $\{x_k\}$ be a real zero-mean Gaussian WSS N-dimensional vector process with PSD X. Suppose that ${min}_{\omega \in [0,2\pi ]}{\lambda }_N\left(X(\omega )\right)>0$ (or equivalently, $det(X(\omega ))\ne 0$ for all $\omega \in \mathbb{R}$). If $D\in \left(0,{min}_{\omega \in [0,2\pi ]}{\lambda }_N\left(X(\omega )\right)\right]$, then

$$\underset{n\to \infty }{lim}{R}_{x_{n:1}}(D)=\underset{n\to \infty }{lim}{\tilde{R}}_{x_{n:1}}(D)=\frac{1}{4\pi N}{\int }_0^{2\pi }ln\frac{det\left(X(\omega )\right)}{D^N}d\omega .$$

Proof. 

Applying ([1] (Lemma 3.3)) and ([1] (Theorem 4.3)) yields $\{E\left(x_{n:1}{x}_{n:1}^{\top }\right)\}=\{T_n(X)\}\sim \{T_n(X)\}$. Theorem 5 now follows from ([14] (Proposition 3)) and Theorem 4. □

Theorem 5 was presented in [5] for the case $N=1$ (i.e., just for WSS sources but not for vector WSS sources).

5.2. VMA Sources

In this subsection (see Theorem 6), we give conditions under which our coding strategy is asymptotically optimal for VMA sources.

We start by reviewing the concept of VMA process.

Definition 3.

A real zero-mean random N-dimensional vector process $\{x_k\}$ is said to be MA if

$$x_k=w_k+\sum _{j=1}^{k-1}{\mathsf{G}}_{-j}{w}_{k-j}\forall k\in \mathbb{N},$$

where ${\mathsf{G}}_{-j}$, $j\in \mathbb{N}$, are real $N\times N$ matrices, $\{w_k\}$ is a real zero-mean random N-dimensional vector process, and $E\left(w_{k_1}{w}_{k_2}^{\top }\right)={\delta }_{k_1,k_2}\mathsf{\Lambda }$ for all $k_1,k_2\in \mathbb{N}$ with Λ being a real $N\times N$ positive definite matrix. If there exists $q\in \mathbb{N}$ such that $G_{-j}=0_{N\times N}$ for all $j>q$, then $\{x_k\}$ is called a VMA(q) process.

Theorem 6.

Let $\{x_k\}$ be as in Definition 3. Assume that ${\{{\mathsf{G}}_k\}}_{k=-\infty }^{\infty }$, with ${\mathsf{G}}_0=I_N$ and ${\mathsf{G}}_k=0_{N\times N}$ for all $k\in \mathbb{N}$, is the sequence of Fourier coefficients of a function $G:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, which is continuous and $2\pi$-periodic. Suppose that $\{T_n(G)\}$ is stable (that is, $\{\parallel {(T_n(G))}^{-1}{\parallel }_2\}$ is bounded). If $\{x_k\}$ is Gaussian and $D\in \left(0,{inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right]$, then

$$\underset{n\to \infty }{lim}{R}_{x_{n:1}}(D)=\underset{n\to \infty }{lim}{\tilde{R}}_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}.$$

(17)

Moreover, $R_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}$ for all $n\in \mathbb{N}$.

Proof. 

We divide the proof into three steps:

Step 1: We show that $det\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)={\left(det(\mathsf{\Lambda })\right)}^n$ for all $n\in \mathbb{N}$. From ([15] (Equation (A3))) we have that $\left\{E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right\}=\left\{T_n(G)T_n(\mathsf{\Lambda }){\left(T_n(G)\right)}^{*}\right\}$. Consequently,

$$\begin{array}{cc}\hfill det\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)& =det\left(T_n(G)\right)det\left(T_n(\mathsf{\Lambda })\right)\overline{det\left(T_n(G)\right)}={|det\left(T_n(G)\right)|}^2{\left(det(\mathsf{\Lambda })\right)}^n={\left(det(\mathsf{\Lambda })\right)}^n\forall n\in \mathbb{N}.\hfill \end{array}$$

Step 2: We prove the first equality in Equation (17). Applying ([15] (Theorem 2)), we obtain that $\{x_k\}$ is AWSS. From Theorem 4, we only need to show that ${inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)>0$. We have

$$\begin{array}{cc}\hfill & {\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)=\frac{1}{{\lambda }_1\left({\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}\right)}=\frac{1}{{∥{\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}∥}_2}=\frac{1}{{∥{\left(T_n(G)T_n(\mathsf{\Lambda }){\left(T_n(G)\right)}^{*}\right)}^{-1}∥}_2}\hfill \\ \hfill & =\frac{1}{{∥{\left({\left(T_n(G)\right)}^{-1}\right)}^{*}{T}_n({\mathsf{\Lambda }}^{-1}){\left(T_n(G)\right)}^{-1}∥}_2}\ge \frac{1}{{∥{\left({\left(T_n(G)\right)}^{-1}\right)}^{*}∥}_2{∥T_n({\mathsf{\Lambda }}^{-1})∥}_2{∥{\left(T_n(G)\right)}^{-1}∥}_2}\hfill \\ \hfill & =\frac{1}{{∥{\left(T_n(G)\right)}^{-1}∥}_2^2{\lambda }_1({\mathsf{\Lambda }}^{-1})}=\frac{{\lambda }_N(\mathsf{\Lambda })}{{∥{\left(T_n(G)\right)}^{-1}∥}_2^2}\ge \frac{{\lambda }_N(\mathsf{\Lambda })}{{\left({sup}_{m\in \mathbb{N}}{∥{\left(T_m(G)\right)}^{-1}∥}_2\right)}^2}>0\forall n\in \mathbb{N}.\hfill \end{array}$$

Step 3: We show that $R_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}$ for all $n\in \mathbb{N}$. Applying Equation (12) yields

$$\begin{array}{c}\hfill R_{x_{n:1}}(D)=\frac{1}{2nN}ln\frac{{\left(det(\mathsf{\Lambda })\right)}^n}{D^{nN}}=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}\forall n\in \mathbb{N}.\end{array}$$

 □

5.3. VAR AWSS Sources

In this subsection (see Theorem 7), we give conditions under which our coding strategy is asymptotically optimal for VAR sources.

We first recall the concept of VAR process.

Definition 4.

A real zero-mean random N-dimensional vector process $\{x_k\}$ is said to be AR if

$$x_k=w_k-\sum _{j=1}^{k-1}{\mathsf{F}}_{-j}{x}_{k-j}\forall k\in \mathbb{N},$$

where ${\mathsf{F}}_{-j}$, $j\in \mathbb{N}$, are real $N\times N$ matrices, $\{w_k\}$ is a real zero-mean random N-dimensional vector process, and $E\left(w_{k_1}{w}_{k_2}^{\top }\right)={\delta }_{k_1,k_2}\mathsf{\Lambda }$ for all $k_1,k_2\in \mathbb{N}$ with Λ being a real $N\times N$ positive definite matrix. If there exists $p\in \mathbb{N}$ such that $F_{-j}=0_{N\times N}$ for all $j>p$, then $\{x_k\}$ is called a VAR(p) process.

Theorem 7.

Let $\{x_k\}$ be as in Definition 4. Assume that ${\{{\mathsf{F}}_k\}}_{k=-\infty }^{\infty }$, with ${\mathsf{F}}_0=I_N$ and ${\mathsf{F}}_k=0_{N\times N}$ for all $k\in \mathbb{N}$, is the sequence of Fourier coefficients of a function $F:\mathbb{R}\to {\mathbb{C}}^{N\times N}$, which is continuous and $2\pi$-periodic. Suppose that $\{T_n(F)\}$ is stable and $det\left(F(\omega )\right)\ne 0$ for all $\omega \in \mathbb{R}$. If $\{x_k\}$ is Gaussian and $D\in \left(0,{inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right]$, then

$$\underset{n\to \infty }{lim}{R}_{x_{n:1}}(D)=\underset{n\to \infty }{lim}{\tilde{R}}_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}.$$

(18)

Moreover, $R_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}$ for all $n\in \mathbb{N}$.

Proof. 

We divide the proof into three steps:

Step 1: We show that $det\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)={\left(det(\mathsf{\Lambda })\right)}^n$ for all $n\in \mathbb{N}$. From ([19] (Equation (19))), we have that $\left\{E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right\}=\left\{{\left(T_n(F)\right)}^{-1}{T}_n(\mathsf{\Lambda }){\left({\left(T_n(F)\right)}^{*}\right)}^{-1}\right\}$. Consequently,

$$\begin{array}{cc}\hfill det\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)& =\frac{det\left(T_n(\mathsf{\Lambda })\right)}{det\left(T_n(F)\right)det\left({\left(T_n(F)\right)}^{*}\right)}=\frac{{\left(det(\mathsf{\Lambda })\right)}^n}{|det\left(T_n(F)\right){|}^2}={\left(det(\mathsf{\Lambda })\right)}^n\forall n\in \mathbb{N}.\hfill \end{array}$$

Step 2: We prove the first equality in Equation (18). Applying ([15] (Theorem 3)), we obtain that $\{x_k\}$ is AWSS. From Theorem 4, we only need to show that ${inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)>0$. Applying ([1] (Theorem 4.3)) yields

$$\begin{array}{cc}\hfill {\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)& =\frac{1}{{∥{\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}∥}_2}=\frac{1}{{∥{\left({\left(T_n(F)\right)}^{-1}{T}_n(\mathsf{\Lambda }){\left({\left(T_n(F)\right)}^{*}\right)}^{-1}\right)}^{-1}∥}_2}\hfill \\ \hfill & \ge \frac{{\lambda }_N(\mathsf{\Lambda })}{{∥T_n(F)∥}_2^2}\ge \frac{{\lambda }_N(\mathsf{\Lambda })}{{\left({sup}_{m\in \mathbb{N}}{∥T_m(F)∥}_2\right)}^2}>0\forall n\in \mathbb{N}.\hfill \end{array}$$

Step 3: We show that $R_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}$ for all $n\in \mathbb{N}$. This can be directly obtained from Equation (12). □

Theorem 7 was presented in [6] for the case of $N=1$ (i.e., just for AR sources but not for VAR sources).

5.4. VARMA AWSS Sources

In this subsection (see Theorem 8), we give conditions under which our coding strategy is asymptotically optimal for VARMA sources.

We start by reviewing the concept of VARMA process.

Definition 5.

A real zero-mean random N-dimensional vector process $\{x_k\}$ is said to be ARMA if

$$x_k=w_k+\sum _{j=1}^{k-1}{\mathsf{G}}_{-j}{w}_{k-j}-\sum _{j=1}^{k-1}{\mathsf{F}}_{-j}{x}_{k-j}\forall k\in \mathbb{N},$$

where ${\mathsf{G}}_{-j}$ and ${\mathsf{F}}_{-j}$, $j\in \mathbb{N}$, are real $N\times N$ matrices, $\{w_k\}$ is a real zero-mean random N-dimensional vector process, and $E\left(w_{k_1}{w}_{k_2}^{\top }\right)={\delta }_{k_1,k_2}\mathsf{\Lambda }$ for all $k_1,k_2\in \mathbb{N}$ with Λ being a real $N\times N$ positive definite matrix. If there exists $p,q\in \mathbb{N}$ such that $F_{-j}=0_{N\times N}$ for all $j>p$ and $G_{-j}=0_{N\times N}$ for all $j>q$, then $\{x_k\}$ is called a VARMA(p,q) process (or a VARMA process of (finite) order (p,q)).

Theorem 8.

Let $\{x_k\}$ be as in Definition 5. Assume that ${\{{\mathsf{G}}_k\}}_{k=-\infty }^{\infty }$, with ${\mathsf{G}}_0=I_N$ and ${\mathsf{G}}_k=0_{N\times N}$ for all $k\in \mathbb{N}$, is the sequence of Fourier coefficients of a function $G:\mathbb{R}\to {\mathbb{C}}^{N\times N}$ which is continuous and $2\pi$-periodic. Suppose that ${\{{\mathsf{F}}_k\}}_{k=-\infty }^{\infty }$, with ${\mathsf{F}}_0=I_N$ and ${\mathsf{F}}_k=0_{N\times N}$ for all $k\in \mathbb{N}$, is the sequence of Fourier coefficients of a function $F:\mathbb{R}\to {\mathbb{C}}^{N\times N}$ which is continuous and $2\pi$-periodic. Assume that $\{T_n(G)\}$ and $\{T_n(F)\}$ are stable, and $det\left(F(\omega )\right)\ne 0$ for all $\omega \in \mathbb{R}$. If $\{x_k\}$ is Gaussian and $D\in \left(0,{inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)\right]$, then

$$\underset{n\to \infty }{lim}{R}_{x_{n:1}}(D)=\underset{n\to \infty }{lim}{\tilde{R}}_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}.$$

(19)

Moreover, $R_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}$ for all $n\in \mathbb{N}$.

Proof. 

We divide the proof into three steps:

Step 1: We show that $det\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)={\left(det(\mathsf{\Lambda })\right)}^n$ for all $n\in \mathbb{N}$. From ([15] (Appendix D)) and ([1] (Lemma 4.2)), we have that $\left\{E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right\}=\left\{{(T_n(F))}^{-1}{T}_n(G)T_n(\mathsf{\Lambda }){\left(T_n(G)\right)}^{*}{({(T_n(F))}^{*})}^{-1}\right\}$. Consequently,

$$\begin{array}{cc}\hfill det\left(E\left({\mathbf{x}}_{n:1}{\mathbf{x}}_{n:1}^{\top }\right)\right)& =\frac{|det\left(T_n(G)\right){|}^2{\left(det(\mathsf{\Lambda })\right)}^n}{|det\left(T_n(F)\right){|}^2}={\left(det(\mathsf{\Lambda })\right)}^n\forall n\in \mathbb{N}.\hfill \end{array}$$

Step 2: We prove the first equality in Equation (19). Applying ([15] (Theorem 3)), we obtain that $\{x_k\}$ is AWSS. From Theorem 4, we only need to show that ${inf}_{n\in \mathbb{N}}{\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)>0$. Applying ([1] (Theorem 4.3)) yields

$$\begin{array}{cc}\hfill & {\lambda }_{nN}\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)=\frac{1}{{∥{\left(E\left(x_{n:1}{x}_{n:1}^{\top }\right)\right)}^{-1}∥}_2}=\frac{1}{{∥{\left({(T_n(F))}^{-1}{T}_n(G)T_n(\mathsf{\Lambda }){\left(T_n(G)\right)}^{*}{({(T_n(F))}^{*})}^{-1}\right)}^{-1}∥}_2}\hfill \\ \hfill & \ge \frac{{\lambda }_N(\mathsf{\Lambda })}{{∥T_n(F)∥}_2^2{∥{\left(T_n(G)\right)}^{-1}∥}_2^2}\ge \frac{{\lambda }_N(\mathsf{\Lambda })}{{\left({sup}_{m\in \mathbb{N}}{∥T_m(F)∥}_2\right)}^2{\left({sup}_{m\in \mathbb{N}}{∥{\left(T_m(G)\right)}^{-1}∥}_2\right)}^2}>0\forall n\in \mathbb{N}.\hfill \end{array}$$

Step 3: We show that $R_{x_{n:1}}(D)=\frac{1}{2N}ln\frac{det(\mathsf{\Lambda })}{D^N}$ for all $n\in \mathbb{N}$. This can be directly obtained from Equation (12). □

6. Numerical Examples

We first consider four AWSS vector processes, namely, we consider the zero-mean WSS vector process in ([20] (Section 4)), the VMA(1) process in ([18] (Example 2.1)), the VAR(1) process in ([18] (Example 2.3)), and the VARMA(1,1) process in ([18] (Example 3.2)). In ([20] (Section 4)), $N=2$ and the Fourier coefficients of its PSD X are

$$X_0=\left(\begin{array}{cc}2.0002& 0.7058\\ 0.7058& 2.0000\end{array}\right),X_{-1}=X_1^{*}=\left(\begin{array}{cc}-0.3542& 0.1016\\ 0.1839& -0.2524\end{array}\right),X_{-2}=X_2^{*}=\left(\begin{array}{cc}-0.0923& 0.0153\\ 0.1490& 0.0696\end{array}\right),$$

$$X_{-3}=X_3^{*}=\left(\begin{array}{cc}-0.1443& -0.0904\\ 0.0602& 0.0704\end{array}\right),X_{-4}=X_4^{*}=\left(\begin{array}{cc}-0.0516& -0.0603\\ 0& 0\end{array}\right),$$

and $X_j=0_{2\times 2}$ with $\left|j\right|>4$. In ([18] (Example 2.1)), $N=2$, $G_{-1}$ is given by

$$\left(\begin{array}{cc}-0.8& -0.7\\ 0.4& -0.6\end{array}\right),$$

(20)

$G_{-j}=0_{2\times 2}$ for all $j\in \mathbb{N}$, and

$$\mathsf{\Lambda }=\left(\begin{array}{cc}4& 1\\ 1& 2\end{array}\right).$$

(21)

In ([18] (Example 2.3)), $N=2$, $F_{-j}=0_{2\times 2}$ for all $j\in \mathbb{N}$, and $F_{-1}$ and $\mathsf{\Lambda }$ are given by Equations (20) and (21), respectively. In ([18] (Example 3.2)), $N=2$,

$$G_{-1}=\left(\begin{array}{cc}0.6& -0.3\\ -0.3& -0.6\end{array}\right),F_{-1}=\left(\begin{array}{cc}-1.2& 0.5\\ -0.6& -0.3\end{array}\right),\mathsf{\Lambda }=\left(\begin{array}{cc}1& 0.5\\ 0.5& 1.25\end{array}\right),$$

$G_{-j}=0_{2\times 2}$ for all $j\in \mathbb{N}$, and $F_{-j}=0_{2\times 2}$ for all $j\in \mathbb{N}$.

Figure 2, Figure 3, Figure 4 and Figure 5 show $R_{x_{n:1}}(D)$ and ${\tilde{R}}_{x_{n:1}}(D)$ with $n\le 100$ and $D=0.001$ for the four vector processes considered, by assuming that they are Gaussian. The figures bear evidence of the fact that the rate of our coding strategy tends to the RDF of the source.

Figure 2. Considered rates for the wide sense stationary (WSS) vector process in ([20] (Section 4)).

Figure 3. Considered rates for the VMA(1) process in ([18] (Example 2.1)).

Figure 4. Considered rates for the VAR(1) process in ([18] (Example 2.3)).

Figure 5. Considered rates for the VARMA(1,1) process in ([18] (Example 3.2)).

We finish with a numerical example to explore how our method performs in the presence of a perturbation. Specifically, we consider a perturbed version of the WSS vector process in ([20] (Section 4)) (Figure 6). The correlation matrices of the perturbed process are

$$T_n(X)+\left(\begin{array}{cc}{0}_{2n-2\times 2n-2}& 0_{2n-2\times 2}\\ 0_{2\times 2n-2}& I_2\end{array}\right),n\in \mathbb{N}.$$

Figure 6. Considered rates for the perturbed WSS vector process with D = 0.001.

7. Conclusions

The computational complexity of coding finite-length data blocks of Gaussian N-dimensional vector sources can be reduced by using the low-complexity coding strategy presented here instead of the optimal coding strategy. Specifically, the computational complexity is reduced from $O(n^2{N}^2)$ to $O(nNlogn)$, where n is the length of the data blocks. Moreover, our coding strategy is asymptotically optimal (i.e., the rate of our coding strategy tends to the RDF of the source) whenever the Gaussian vector source is AWSS and the considered data blocks are large enough. Besides being a low-complexity strategy, it does not require the knowledge of the correlation matrix of such data blocks. Furthermore, our coding strategy is appropriate to encode the most relevant Gaussian vector sources, namely, WSS, MA, AR, and ARMA vector sources.