Designing Bivariate Auto-Regressive Timeseries with Controlled Granger Causality

Abstract

In this manuscript, we analyze a bivariate vector auto-regressive (VAR) model in order to draw the design principle of a timeseries with a controlled statistical inter-relationship. We show how to generate bivariate timeseries with given covariance and Granger causality (or, equivalently, transfer entropy), and show the trade-off relationship between these two types of statistical interaction. In principle, covariance and Granger causality are independently controllable, but the feasible ranges of their values which allow the VAR to be proper and have a stationary distribution are constrained by each other. Thus, our analysis identifies the essential tri-lemma structure among the stability and properness of VAR, the controllability of covariance, and that of Granger causality.

1. Introduction

1.1. Background and Motivation

In the field of cognitive psychology, the human perception of the life-likeness (called animacy perception) of one or multiple moving geometric patterns has been studied for decades [1,2,3,4,5]. There are multiple findings on the effect of “synchrony” or “temporal contingency” between multiple moving points on animacy perception. Findings from one line of research [2] have suggested that a higher degree of “temporal contingency” of the moving objects is related to a higher likelihood of animacy perception. Findings from the other line of research [6] have suggested that the highest “temporal contingency”, presented in the form of perfect synchronization, would decrease the likelihood of animacy perception.

These two lines of research have together suggested the existence of multiple types of “temporal contingency”. Nevertheless, this past research does not appear to clarify these types. Further, confusion surrounding these two distinct types of effects have led to two lines of apparently conflicting effects of “temporal contingency”.

With this potential conflict in the literature on animacy perception in mind, we explore a theoretical framework which can generate timeseries of multiple random variables with multiple distinct types of statistical dependency. One such system, which is sufficiently simple and readily manipulable, is vector auto-regression (VAR). Vector auto-regression is a random process for generating multivariate timeseries for a given set of parameters. In this manuscript, we specifically consider only bivariate VAR, which is a minimal system with interaction between two moving points.

1.2. Vector Auto-Regressive Model, Granger Causality, and Transfer Entropy

Importantly, bivariate VAR, a series of paired random variables $(x_t,y_t)$ for $t=0,1,\dots$, has two types of statistical dependency—that is, the correlation and Granger causality of a timeseries [7], which has been identified as transfer entropy [8] of the timeseries generated by a Gaussian process by [9]. The correlation between univariate series x and y is a statistical dependency between $x_t$ and $y_t$ in the limit $t\to \infty$ (if it exists), while a Granger causality from y to x is that between $x_t$ and $y_{t-1}$ given $x_{t-1}$ in the limit $t\to \infty$ (if it exists). Conceptually, correlation captures a type of similarity between two timeseries, whereas Granger causality captures the “reactiveness” of one timeseries to another.

Thus, given these differences in theory, our goal is to propose a theoretical method to generate a bivariate random timeseries with a desired correlation and two types. The Granger causality of VAR has also been considered in other fields. In econometrics, the field in which it was originally proposed [7], Granger causality has been used as a measure of interaction between a pair of economic timeseries [10,11,12]. It has also been used in general behavioral sciences [13,14], particularly in computational neuroscience [15,16,17,18]. Such an in-principle data generation technique would be vital to the testing of any hypothesis on the empirical nature of timeseries (e.g., animacy perception) in the empirical sciences using VAR and Granger causality, as mentioned above. To our knowledge, however, there has been no mathematical analysis of the theoretical limitation of such a data generation technique for a given statistics.

Thus, we first need to explore the mathematical relationship among the parameters in a VAR with the correlation and Granger causality in a timeseries generated from it. In this paper, we therefore explore a theoretical structure of bivariate VAR from the designer’s perspective, and analyze a mathematical limit to the extent which we can simultaneously control correlation and Granger causality of a bivariate timeseries.

This paper is written with the following structure. In Section 2, the VAR model is defined, from which a set of basic statistical properties of the VAR model are derived, such as Granger causality (Section 3) with a set of parameters. In Section 4, the existence of the stationary distribution of the VAR is analyzed. This is a foundation which sets the limit of a controllable set of parameters. In Section 5, we give a method to derive the parameters of VAR for a given set of statistics in bivariate timeseries. In Section 6, the mathematical analysis provided in this paper is summarized and a remark on the design principle of bivariate timeseries generated by VAR is added.

2. Vector Auto-Regression (VAR)

In theory, Granger causality (GC) is the transfer entropy of random variables in a bivariate vector auto-regression (VAR) model up to a constant factor of 2, if the VAR model has a stationary distribution [9]. Thus, it is straightforward to start with the bivariate VAR and derive its transfer entropy. In this way, we can derive a rich mathematical relationship between GC and the properties of VAR, rather than just a statistics of the bivariate timeseries.

Definition 1.

For some real vector $\mu \in {\mathbb{R}}^2$ and some positive-definite matrix $\mathsf{\Sigma }\in {\mathbb{R}}^{2\times 2}$, suppose the random variable ${ϵ}_t$ for every integer $t=0,1,\dots$ is drawn from the bivariate normal distribution $N\left({ϵ}_t\right|\mu ,\mathsf{\Sigma })$ with mean μ and variance Σ. Define the initial vector by $v_0=\left(\begin{array}{c}{x}_0\\ y_0\end{array}\right)$, and for $t\ge 0$ and a given coefficient matrix with real entries $A:=\left(\begin{array}{cc}{a}_{0,0}& a_{0,1}\\ a_{1,0}& a_{1,1}\end{array}\right)\in {\mathbb{R}}^{2\times 2}$, define the random variable $v_t$ by

$$v_{t+1}:=A{v}_t+{ϵ}_t.$$

(1)

Then, bivariate vector auto-regression is defined by the semi-infinite series of random variables $V=(v_0,v_1,v_2,\dots )$.

In general, one can generate a timeseries $v_0,v_1,\dots$ by fixing a set of the VAR parameters, the coefficient matrix A, and the base covariance matrix $\mathsf{\Sigma }$, where the base mean vector $\mu$ is omitted as its effect is lost in the limit $t\to \infty$ when the VAR is stationary. The stationary correlation (covariance) $\widehat{\mathsf{\Sigma }}$ and Granger causality $G_0$ and $G_1$ defined later are the statistics of the timeseries generated by a VAR model (Figure 1). In what follows, we first explore the forward relationship of how the statistics $\widehat{\mathsf{\Sigma }}$ and $G_0,G_1$ are given by the VAR parameters $(A,\mathsf{\Sigma })$. We then consider the backward relationship in which the VAR parameters $(A,\mathsf{\Sigma })$ suffice to generate a timeseries with given desired timeseries statistics $(G_0,G_1,\widehat{\mathsf{\Sigma }})$.

2.1. Marginal Distribution of the VAR at Each Step

Lemma 1

(Marginal distribution of the VAR random variables at each step). The VAR model with the initial vector $v_0\in {\mathbb{R}}^2$ and the coefficient matrix $A\in {\mathbb{R}}^{2\times 2}$ has the bivariate normal distribution

$$N\left(v_t\right|{\mu }_t,{\mathsf{\Sigma }}_t)$$

as its marginal distribution of the random variable $v_t$ at each step $t=0,1,\dots$, where

$${\mu }_t:=A^t{v}_0,{\mathsf{\Sigma }}_t=\sum _{s=0}^t{A}^s\mathsf{\Sigma }{\left(A^s\right)}^{\top }.$$

Proof. 

By Definition 1, Lemma 1 holds for $t=0$. For $t+1>0$, we prove Lemma 1 by assuming that it holds up to $t\ge 0$. By this assumption held for t, we have the distribution of $v_t\in {\mathbb{R}}^2$ as the bivariate normal distribution

$$N\left(v_t\right|{\mu }_t,{\mathsf{\Sigma }}_t)={\left(2\pi \right)}^{-1}{\left|{\mathsf{\Sigma }}_t\right|}^{-\frac{1}{2}}{e}^{-\frac{1}{2}{(v_t-{\mu }_t)}^{\top }{\mathsf{\Sigma }}_t^{-1}(v_t-{\mu }_t)}$$

with its mean ${\mu }_t$ and its covariance matrix ${\mathsf{\Sigma }}_t$. Then, the random variable $A{v}_t$ is distributed by the normal distribution

$$\begin{array}{ccc}\hfill N\left(A{v}_t\right|A{\mu }_t,A{\mathsf{\Sigma }}_t{A}^{\top })& =& {\left(2\pi \right)}^{-1}{\left|A{\mathsf{\Sigma }}_t{A}^{\top }\right|}^{-\frac{1}{2}}\hfill \\ \hfill & \times & e^{-\frac{1}{2}{\left(A(v_t-{\mu }_t)\right)}^{\top }{\left(A{\mathsf{\Sigma }}_t{A}^{\top }\right)}^{-1}\left(A(v_t-{\mu }_t)\right)}\hfill \end{array}$$

(2)

with its mean $A{\mu }_t$ and the covariance matrix $A{\mathsf{\Sigma }}_t{A}^{\top }$. The random variable ${ϵ}_t$ is distributed by the following normal distribution:

$$N\left({ϵ}_t\right|0,\mathsf{\Sigma })={\left(2\pi \right)}^{-1}{\left|\mathsf{\Sigma }\right|}^{-\frac{1}{2}}{e}^{-\frac{1}{2}{ϵ}_t^{\top }{\mathsf{\Sigma }}^{-1}{ϵ}_t}.$$

Thus, by VAR, Equation (1), we have the random variable

$$v_{t+1}:=A{v}_t+{ϵ}_t$$

which has a distribution calculated by the following integral:

$$P\left(v_{t+1}\right)={\int }_{{ϵ}_t\in {\mathbb{R}}^2}N(v_{t+1}-{ϵ}_t|A{\mu }_t,A{\mathsf{\Sigma }}_t{A}^{\top })N\left({ϵ}_t\right|0,\mathsf{\Sigma })\mathrm{d}{ϵ}_t.$$

Calculating this, we have

$$P\left(v_{t+1}\right)=N\left(v_{t+1}|A{\mu }_t,A{\mathsf{\Sigma }}_t{A}^{\top }+\mathsf{\Sigma }\right).$$

(3)

Thus defining by ${\mu }_{t+1}:=A{\mu }_t$ and ${\mathsf{\Sigma }}_{t+1}=A{\mathsf{\Sigma }}_t{A}^{\top }+\mathsf{\Sigma }$, Lemma 1 holds for $t+1$. By expanding this, we have the Lemma 1 for any integer $t\ge 0$. □

2.2. Stability of VAR: Lyapunov Equation

By Lemma 1, the mean and covariance matrix of the random variable at the $t\mathrm{th}$ step are

$${\mu }_t=A^t{v}_0\mathrm{and}{\mathsf{\Sigma }}_t=\sum _{s=0}^t{A}^s\mathsf{\Sigma }{\left(A^s\right)}^{\top }.$$

From this, we have the stationary distribution

$$\underset{t\to \infty }{lim}N\left(v_t\right|{\mu }_t,{\mathsf{\Sigma }}_t),$$

if and only if the absolute values of all the eigenvalues ${\lambda }_0,{\lambda }_1\in \mathbb{C}$ of the coefficient matrix A are less than 1. If there is such a stationary distribution, we call the VAR stable, and its stationary distribution is the bivariate normal distribution

$$N\left(\widehat{v}\right|{\mathbf{0}}_2,\widehat{\mathsf{\Sigma }}),$$

where the stationary mean vector $\widehat{v}\in {\mathbb{R}}^2$ and stationary covariance matrix $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}^{2\times 2}$ are defined as follows. If the VAR is stable, we have the following Lyapunov equation of the stationary covariance matrix $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}^{2\times 2}$:

$$\widehat{\mathsf{\Sigma }}=\mathsf{\Sigma }+A\widehat{\mathsf{\Sigma }}{A}^{\top }.$$

(4)

The Lyapunov equation is solved analytically by

$$\mathrm{vec}\left(\widehat{\mathsf{\Sigma }}\right)={\left(I_4-A\otimes A\right)}^{-1}\mathrm{vec}\left(\mathsf{\Sigma }\right),$$

(5)

where $I_d\in {\mathbb{R}}^{d\times d}$ is the $d$ order identity matrix, ⊗ denotes the Kronecker product, and $\mathrm{vec}\left(X\right)$ for any matrix $X={\left(x_{i,j}\right)}_{i=1,\dots ,n,j=1,\dots ,m}$ is the vectorization operator $\mathrm{vec}\left(\right):{\mathbb{R}}^{n\times m}\to {\mathbb{R}}^{nm\times 1}$ defined by

$$\mathrm{vec}\left(X\right):={(x_{1,1},x_{2,1},\dots ,x_{n,1},\dots ,x_{1,m},x_{2,m},\dots ,x_{n,m})}^{\top }.$$

The Lyapunov Equation (4) has the solution for $\widehat{\mathsf{\Sigma }}$ if the VAR is stable, but not vice versa. This is shown by Lemma 4.

3. Transfer Entropy and Granger Causality

In [9], the transfer entropy of an appropriate triplet of variables in the VAR model is shown to be equivalent to Granger causality up to the constant factor 2. Following this guide, we define this quantity as the Granger causality of the VAR model, below.

Although this relationship has been known in a more general form [9], we re-derive it for bivariate VAR in order to later analyze the structure of VAR and GCs in depth—for example, its upper and lower bounds (Lemma 3), stability (Section 4), and design principle (Section 5).

Definition 2.

If VAR with its random variables $v_t={\left(x_t,y_t\right)}^{\top }\in {\mathbb{R}}^2$ for $t=0,1,\dots$ is stable, transfer entropy from y to x is defined by

$$T_{y\to x}:=\underset{t\to \infty }{lim}(H\left(x_{t+1}\right|x_t)-H\left(x_{t+1}\right|x_t,y_t)),$$

and the transfer entropy from x to y is defined by

$$T_{x\to y}:=\underset{t\to \infty }{lim}(H\left(y_{t+1}\right|y_t)-H\left(y_{t+1}\right|y_t,x_t)),$$

where the differential entropy of random variable x with its probability density function P is

$$H\left(x\right):=-{\int }_{x\in \Omega }P\left(x\right)logP\left(x\right)\mathrm{d}x,$$

and the conditional entropy is

$$H\left(x\right|y):=H(x,y)-H(y).$$

In particular, we call two times of transfer entropy Granger causality denoted by

$$G_0=2T_{y\to x}\mathit{and}{G}_1=2T_{x\to y}.$$

(6)

Specifically, GCs are specifically written by the terms of the VAR parameters in the following lemma.

Lemma 2

(Granger causality). If a stable VAR has its covariance matrix, coefficient matrix, and stationary matrix

$$\mathsf{\Sigma }=\left(\begin{array}{cc}{\sigma }_{0,0}& {\sigma }_{0,1}\\ {\sigma }_{1,0}& {\sigma }_{1,1}\end{array}\right),A=\left(\begin{array}{cc}{a}_{0,0}& a_{0,1}\\ a_{1,0}& a_{1,1}\end{array}\right),\widehat{\mathsf{\Sigma }}=\left(\begin{array}{cc}{\widehat{\sigma }}_{0,0}& {\widehat{\sigma }}_{0,1}\\ {\widehat{\sigma }}_{1,0}& {\widehat{\sigma }}_{1,1}\end{array}\right),$$

each Granger causality of this VAR for $i=0,1$ is

$$\begin{array}{ccc}\hfill G_i& =& log\left(1+\frac{a_{i,1-i}^2\det \left(\widehat{\mathsf{\Sigma }}\right)}{{\widehat{\sigma }}_{i,i}{\sigma }_{i,i}}\right).\hfill \end{array}$$

(7)

Proof. 

In general, the differential entropy of multivariate normal distribution $N\left(v\right|\mu ,\mathsf{\Sigma })$ is

$$H\left(v\right)=\frac{1}{2}log\left|2\pi e\mathsf{\Sigma }\right|,$$

where $e\approx 2.71$ is Napier’s constant. For the joint probability distribution of $v_t={(x_t,y_t)}^{\top }$

$$P\left(v_{t+1}\right|v_t)=N\left(v_{t+1}\right|A{v}_t,\mathsf{\Sigma }),$$

the two marginal probability distributions of $x_t,y_t$ are

$$P\left(x_{t+1}\right|v_t)=N\left(x_{t+1}\right|(1,0)A{v}_t,{\sigma }_{0,0})\mathrm{and}P\left(y_{t+1}\right|v_t)=N\left(y_{t+1}\right|(0,1)A{v}_t,{\sigma }_{1,1}).$$

Thus, the conditional entropy of $x_{t+1}$ and $y_{t+1}$ given $v_t={(x_t,y_t)}^{\top }$ are

$$H\left(x_{t+1}\right|x_t,y_t)=\frac{1}{2}log|2\pi e{\sigma }_{0,0}|\mathrm{and}H\left(y_{t+1}\right|x_t,y_t)=\frac{1}{2}log\left|2\pi e{\sigma }_{1,1}\right|.$$

With the conditional probability distribution and marginal probability distribution

$$P\left(v_{t+1}\right|v_t)=N\left(v_{t+1}\right|A{v}_t,\mathsf{\Sigma })\mathrm{and}P\left(v_t\right)=N\left(v_t\right|A^t{v}_0,{\mathsf{\Sigma }}_t),$$

the joint probability distribution of $v_t$ and $v_{t+1}$ is their product

$$P(v_{t+1},v_t)=N\left(v_{t+1}\right|A{v}_t,\mathsf{\Sigma })N\left(v_t\right|A^t{v}_0,{\mathsf{\Sigma }}_t).$$

Specifically, this quad-variate normal distribution is

$$\begin{array}{ccc}\hfill P(v_{t+1},v_t)& =& e^{-\frac{1}{2}{(v_{t+1}-A{v}_t)}^{\top }{\mathsf{\Sigma }}^{-1}(v_{t+1}-A{v}_t)-\frac{1}{2}{(v_t-A^t{v}_0)}^{\top }{\mathsf{\Sigma }}_t^{-1}(v_t-A^t{v}_0)}{\left(2\pi \right)}^{-2}{\left|\mathsf{\Sigma }\right|}^{-\frac{1}{2}}{\left|{\mathsf{\Sigma }}_t\right|}^{-\frac{1}{2}}.\hfill \end{array}$$

Applying the identities

$$v_{t+1}-A{v}_t=v_{t+1}-A^{t+1}{v}_0-A(v_t-A^t{v}_0),$$

$${\mathsf{\Sigma }}_t^{\prime }:=\left(\begin{array}{cc}\mathsf{\Sigma }+A{\mathsf{\Sigma }}_t{A}^{\top }& A{\mathsf{\Sigma }}_t\\ {\mathsf{\Sigma }}_t{A}^{\top }& {\mathsf{\Sigma }}_t\end{array}\right)={\left(\begin{array}{cc}{\mathsf{\Sigma }}^{-1}& -{\mathsf{\Sigma }}^{-1}A\\ -A^{\top }{\mathsf{\Sigma }}^{-1}& {\mathsf{\Sigma }}_t^{-1}+A{\mathsf{\Sigma }}^{-1}{A}^{\top }\end{array}\right)}^{-1},$$

and $\left|\mathsf{\Sigma }\right||{\mathsf{\Sigma }}_t|=|{\mathsf{\Sigma }}_t^{\prime }|$ to $P(v_{t+1},v_t)$, we have

$$P(v_{t+1},v_t)=N\left(v_t^{\prime }\right|{\mu }_t^{\prime },{\mathsf{\Sigma }}_t^{\prime }),$$

where

$$v_t^{\prime }:=\left(\begin{array}{c}{v}_{t+1}\\ v_t\end{array}\right),{\mu }_t^{\prime }:=\left(\begin{array}{c}{A}^{t+1}{v}_0\\ A^t{v}_0\end{array}\right),{\mathsf{\Sigma }}_t^{\prime }:=\left(\begin{array}{cc}\mathsf{\Sigma }+A{\mathsf{\Sigma }}_t{A}^{\top }& A{\mathsf{\Sigma }}_t\\ {\mathsf{\Sigma }}_t{A}^{\top }& {\mathsf{\Sigma }}_t\end{array}\right).$$

From this joint probability distribution $P(v_{t+1},v_t)$, we drive the marginal distributions

$$P(x_{t+1},x_t)=N(x_{t+1},x_t|{\mu }_{t,0},{\mathsf{\Sigma }}_{t,0}),P(y_{t+1},y_t)=N(y_{t+1},y_t|{\mu }_{t,1},{\mathsf{\Sigma }}_{t,1}),$$

where for $i=0,1$ the mean vectors and covariance matrices are defined as follows:

$$\begin{array}{ccc}\hfill {\mu }_{t,i}& :=& \left(\begin{array}{c}{e}_i^{\top }{A}^{t+1}{v}_0\\ e_i^{\top }{A}^t{v}_0\end{array}\right)=(I_2\otimes e_i){\mu }_t^{\prime },\hfill \\ \hfill {\mathsf{\Sigma }}_{t,i}& :=& \left(\begin{array}{cc}{e}_i^{\top }(\mathsf{\Sigma }+A{\mathsf{\Sigma }}_t{A}^{\top })e_i& e_i^{\top }A{\mathsf{\Sigma }}_t{e}_i\\ e_i^{\top }{\mathsf{\Sigma }}_t{A}^{\top }{e}_i& e_i^{\top }{\mathsf{\Sigma }}_t{e}_i\end{array}\right)={(I_2\otimes e_i)}^{\top }{\mathsf{\Sigma }}_t^{\prime }(I_2\otimes e_i),\hfill \end{array}$$

with the unit vectors $e_0:={(1,0)}^{\top }$, $e_1:={(0,1)}^{\top }$.

Thus, we have the joint entropy of $x_t$ and $x_{t+1}$

$$\begin{array}{ccc}\hfill H(x_{t+1},x_t)& =& \frac{1}{2}log\left|2\pi e{\mathsf{\Sigma }}_{t,0}\right|\hfill \end{array}$$

(8)

$$\begin{array}{ccc}& =& \frac{1}{2}log{\left(2\pi e\right)}^2|e_0^{\top }(\mathsf{\Sigma }+A{\mathsf{\Sigma }}_t{A}^{\top })e_0{e}_0^{\top }{\mathsf{\Sigma }}_t{e}_0-{\left(e_0^{\top }A{\mathsf{\Sigma }}_t{e}_0\right)}^2|,\hfill \end{array}$$

(9)

and the marginal distribution of $x_t$

$$H\left(x_t\right)=\frac{1}{2}log2\pi e\left|e_0^{\top }{\mathsf{\Sigma }}_t{e}_0\right|.$$

(10)

Using these, we have the conditional entropy

$$H\left(x_{t+1}\right|x_t)=\frac{1}{2}log2\pi \frac{e_0^{\top }(\mathsf{\Sigma }+A{\mathsf{\Sigma }}_t{A}^{\top })e_0{e}_0^{\top }{\mathsf{\Sigma }}_t{e}_0-{\left(e_0^{\top }A{\mathsf{\Sigma }}_t{e}_0\right)}^2}{|e_0^{\top }{\mathsf{\Sigma }}_t{e}_0|}.$$

(11)

By the stability of VAR, the Lyapunov Equation (4) holds, and this conditional entropy in the limit $t\to \infty$ is

$$\begin{array}{ccc}\hfill \underset{t\to \infty }{lim}H\left(x_{t+1}\right|x_t)& =& \frac{1}{2}log2\pi \frac{{\left(e_0^{\top }\widehat{\mathsf{\Sigma }}{e}_0\right)}^2-{\left(e_0^{\top }A\widehat{\mathsf{\Sigma }}{e}_0\right)}^2}{|e_0^{\top }\widehat{\mathsf{\Sigma }}{e}_0|}.\hfill \end{array}$$

(12)

Applying Definition 2 and denoting by entries in the stationary covariance matrix $\widehat{\mathsf{\Sigma }}=\left(\begin{array}{cc}{\widehat{\sigma }}_{0,0}& {\widehat{\sigma }}_{0,1}\\ {\widehat{\sigma }}_{1,0}& {\widehat{\sigma }}_{1,1}\end{array}\right)$, we have

$$G_0=2T_{y\to x}=log\frac{{\left({\widehat{\sigma }}_{0,0}\right)}^2-{(a_{0,0}{\widehat{\sigma }}_{0,0}+a_{0,1}{\widehat{\sigma }}_{1,0})}^2}{{\widehat{\sigma }}_{0,0}{\sigma }_{0,0}}.$$

(13)

Similarly, we have

$$G_1=2T_{x\to y}=log\frac{{\left({\widehat{\sigma }}_{1,1}\right)}^2-{(a_{1,0}{\widehat{\sigma }}_{0,1}+a_{1,1}{\widehat{\sigma }}_{1,1})}^2}{{\widehat{\sigma }}_{1,1}{\sigma }_{1,1}}.$$

(14)

Let us define for $i=0,1$

$${\delta }_i:={\left({\widehat{\sigma }}_{i,i}\right)}^2-{(a_{i,i}{\widehat{\sigma }}_{i,i}+a_{i,1-i}{\widehat{\sigma }}_{1-i,i})}^2\mathrm{and}{\delta }_i^{\prime }:={\widehat{\sigma }}_{i,i}{\sigma }_{i,i}.$$

(15)

By the Lyapunov equation, ${\sigma }_{i,i}={\widehat{\sigma }}_{i,i}-e_i^{\top }A\widehat{\mathsf{\Sigma }}{A}^{\top }{e}_i$. Applying this to ${\delta }_i^{\prime }$, we have

$${\delta }_i={\widehat{\sigma }}_{i,i}^2-e_i^{\top }A\left(\begin{array}{cc}{\widehat{\sigma }}_{i,i}^2& {\widehat{\sigma }}_{i,i}{\widehat{\sigma }}_{1-i,i}\\ {\widehat{\sigma }}_{i,i}{\widehat{\sigma }}_{1-i,i}& {\widehat{\sigma }}_{1-i,i}^2\end{array}\right)A^{\top }{e}_i,$$

(16)

$${\delta }_i^{\prime }={\widehat{\sigma }}_{i,i}^2-e_i^{\top }A\left(\begin{array}{cc}{\widehat{\sigma }}_{i,i}{\widehat{\sigma }}_{0,0}& {\widehat{\sigma }}_{i,i}{\widehat{\sigma }}_{0,1}\\ {\widehat{\sigma }}_{i,i}{\widehat{\sigma }}_{1,0}& {\widehat{\sigma }}_{i,i}{\widehat{\sigma }}_{1,1}\end{array}\right)A^{\top }{e}_i.$$

(17)

As ${\delta }_i-{\delta }_i^{\prime }=a_{i,1-i}^2\det \left(\widehat{\mathsf{\Sigma }}\right)$ and $G_0=log\left(1+\frac{{\delta }_i-{\delta }_i^{\prime }}{{\delta }_i^{\prime }}\right)$, we have

$$\begin{array}{c}\hfill G_i=log\left(1+\frac{a_{i,1-i}^2\det \left(\widehat{\mathsf{\Sigma }}\right)}{{\widehat{\sigma }}_{i,i}{\sigma }_{i,i}}\right).\end{array}$$

 □

The Granger causality has its lower and upper bounds in theory. Although these bounds may be further narrowed by considering the stability of the VAR, what follows below are the theoretical bounds regardless of the stability of the VAR.

Lemma 3

(The upper and lower bound for Granger causality). For each $i=0,1$, Granger causality $G_i$ has the following bounds:

$$0\le G_i\le log{\gamma }_i,$$

(18)

where ${\gamma }_i:=\frac{{\widehat{\sigma }}_{i,i}}{{\sigma }_{i,i}}\ge 1$ due to the Lyapunov Equation (4). The lower bound $G_i=0$ is given only if

$$a_{i,1-i}^2\det \left(\widehat{\mathsf{\Sigma }}\right)=0.$$

(19)

The upper bound $G_i=log{\gamma }_i$ is given only if

$$a_{i,i}{\widehat{\sigma }}_{i,i}+a_{i,1-i}{\widehat{\sigma }}_{1-i,i}=0.$$

(20)

Proof. 

As the stationary covariance matrix is (semi-)positive definite, $\det \left(\widehat{\mathsf{\Sigma }}\right)\ge 0$. Thus, the lower bound of Granger causality is $G_i\ge log\left(1\right)=0$ and this bound is only reacheable when $a_{i,1-i}^2\det \left(\widehat{\mathsf{\Sigma }}\right)=0$.

Modifying (13) and (14), for $i=0,1$ we have

$${(a_{i,i}{\widehat{\sigma }}_{i,i}+a_{i,1-i}{\widehat{\sigma }}_{1-i,i})}^2={\widehat{\sigma }}_{i,i}\left({\widehat{\sigma }}_{i,i}-{\sigma }_{i,i}{e}^{G_i}\right).$$

(21)

As ${(a_{i,i}{\widehat{\sigma }}_{i,i}+a_{i,1-i}{\widehat{\sigma }}_{1-i,i})}^2\ge 0$ and ${\widehat{\sigma }}_{i,i}>0$,

$$G_i\le log{\gamma }_i.$$

This upper bound holds only if $a_{i,i}{\widehat{\sigma }}_{i,i}+a_{i,1-i}{\widehat{\sigma }}_{1-i,i}=0$. □

The upper bound Lemma 3 can also be obtained by the following information-theoretic identity:

$$\underset{t\to \infty }{lim}I(x_{t-1};x_t)+I(x_t;x_{t-1}|x_{t-1})=\underset{t\to \infty }{lim}I(x_t;x_{t-1},x_{t-1}),$$

where ${lim}_{t\to \infty }I(x_t;x_{t-1}|x_{t-1})=\frac{1}{2}{G}_0$ is the transfer entropy, ${lim}_{t\to \infty }I(x_t;x_{t-1},x_{t-1})=\frac{1}{2}log\frac{{\widehat{\sigma }}_{0,0}}{{\sigma }_{0,0}}$, and

$$\underset{t\to \infty }{lim}I(x_{t-1};x_t)=\frac{1}{2}log\frac{{\widehat{\sigma }}_{0,0}^2}{\left|\begin{array}{cc}\widehat{\mathsf{\Sigma }}& \widehat{\mathsf{\Sigma }}{A}^{\top }\\ A\widehat{\mathsf{\Sigma }}& \widehat{\mathsf{\Sigma }}\end{array}\right|}=\frac{1}{2}\left(log{\gamma }_0-G_0\right).$$

4. Stability and Constraints of VAR

In this study, we primarily consider the class of stable VAR models with a proper set of parameters. In this class, the statistical nature of any VAR is characterized by the base covariance matrix $\mathsf{\Sigma }\in {\mathbb{R}}^{2\times 2}$, coefficient matrix $A\in {\mathbb{R}}^{2\times 2}$, and stationary covariance matrix $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}^{2\times 2}$. Let us denote the set of (strictly) positive definite matrices by

$${\mathbb{R}}_{+}^{2\times 2}:=\left\{M\in {\mathbb{R}}^{2\times 2}|\det \left(M\right)>0\mathrm{and}\mathrm{tr}\left(M\right)>0\right\},$$

and the set of coefficient matrices of stable VAR models

$${\mathbb{R}}_{*}^{2\times 2}:=\left\{M\in {\mathbb{R}}^{2\times 2}|-1+|\mathrm{tr}\left(M\right)|<\det \left(M\right)<1\right\}.$$

We will briefly show that the stable set ${\mathbb{R}}_{*}^{2\times 2}$ includes all and only coefficient matrices of stable bivariate VAR models.

With this notation of the set of matrices, the two conditions that any proper VAR model needs to satisfy are as follows.

Stability 

Any stable VAR model has both of the eigenvalues ${\lambda }_0,{\lambda }_1$ of its coefficient matrix A meeting $|{\lambda }_0|,|{\lambda }_1|<1$.

Properness 

To have a proper (non-degenerated) bivariate normal distribution in a VAR model, its base covariance matrix $\mathsf{\Sigma }$ and stationary covariance matrix $\widehat{\mathsf{\Sigma }}$ need to satisfy $\mathsf{\Sigma },\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$. The set of positive-definite matrices is equivalently written with the entries of the following matrix:

$${\mathbb{R}}_{+}^{2\times 2}=\left\{C\in {\mathbb{R}}^{2\times 2}\mid C_{0,0}>0,C_{1,1}>0,\mathrm{and}{C}_{0,0}{C}_{1,1}-C_{0,1}{C}_{1,0}>0\right\}.$$

(22)

4.1. Stability of VAR

As stated previously in Section 2.2, the stability of VAR is primarily characterized by the eigenvalues of the coefficient matrix A. However, this condition is equivalent to $A\in {\mathbb{R}}_{*}^{2\times 2}$, as shown by the following lemma.

Lemma 4.

A given bivariate VAR model with its coefficient matrix $A\in {\mathbb{R}}^{2\times 2}$ is stable if and only if

$$|\mathrm{tr}\left(A\right)|-1<\det \left(A\right)<1.$$

(23)

Proof. 

Let $\lambda$ be an eigenvalue of the coefficient matrix A. Such an eigenvalue then satisfies

$$f\left(\lambda \right)=\left|A-\lambda I_2\right|={\lambda }^2-\mathrm{tr}\left(A\right)\lambda +\det \left(A\right)=0.$$

(24)

If a VAR is stable, this eigenvalue needs to satisfy $\left|\lambda \right|<1$. As (24) is rewritten by

$$f\left(\lambda \right)={\left(\lambda -\frac{1}{2}\mathrm{tr}\left(A\right)\right)}^2-\frac{1}{4}\left(\mathrm{tr}{\left(A\right)}^2-4\det \left(A\right)\right),$$

(25)

we analyze this condition on (24) for the following two cases with $\lambda$ being real or non-real:

• If $\lambda$ is real, this stability condition is equivalent to

  $$\mathrm{tr}{\left(A\right)}^2\le 4\det \left(A\right),f\left(1\right)>0,f(-1)>0,\left|\mathrm{tr}\left(A\right)\right|<2.$$

  (26)
• If $\lambda$ is not real, this stability condition is equivalent with

  $$\mathrm{tr}{\left(A\right)}^2<4\det \left(A\right),{\left|\lambda \right|}^2<1.$$

  (27)

If $\lambda$ of (24) is non-real (Case 2), $\lambda$ (and its conjugate) is

$$\lambda =\frac{1}{2}\mathrm{tr}\left(A\right)\pm \frac{j}{2}\sqrt{|\mathrm{tr}{\left(A\right)}^2-4\det \left(A\right)|},$$

(28)

with the imaginary unit denoted by j.

With the inequality (27), the stability condition in this case is

$${\left(\frac{\mathrm{tr}\left(A\right)}{2}\right)}^2<{\left|\lambda \right|}^2=\det \left(A\right)<1.$$

(29)

If $\lambda$ of (24) is real, $\mathrm{tr}{\left(A\right)}^2-4\det \left(A\right)\ge 0$ and

$$\begin{array}{c}\hfill f\left(1\right)=1-\mathrm{tr}\left(A\right)+\det \left(A\right)>0\end{array}$$

(30)

$$\begin{array}{c}\hfill f(-1)=1+\mathrm{tr}\left(A\right)+\det \left(A\right)>0\end{array}$$

(31)

$$\begin{array}{c}\hfill |\mathrm{tr}\left(A\right)|<2.\end{array}$$

(32)

Combining (30) and (31), we have $|\mathrm{tr}\left(A\right)|-1<\det \left(A\right)$. This inequality with (26),

$$C_0<\det \left(A\right)\le {\left(\frac{\mathrm{tr}\left(A\right)}{2}\right)}^2<C_1,$$

(33)

where $C_0:=\left|\mathrm{tr}\left(A\right)\right|-1$ and $C_1:=min\left(1,{\left(\frac{1+\det \left(A\right)}{2}\right)}^2\right)$. Find for an arbitrary $A\in {\mathbb{R}}^{2\times 2}$ we have the following two inequalities:

$$|\mathrm{tr}\left(A\right)|-1\le {\left(\frac{1}{2}\mathrm{tr}\left(A\right)\right)}^2,$$

(34)

and

$$\det \left(A\right)\le {\left(\frac{1+\det \left(A\right)}{2}\right)}^2.$$

(35)

The inequality (34) holds equality for and only for $\mathrm{tr}\left(A\right)=2$, and the inequality (34) holds equality for and only for $\det \left(A\right)=1$. As both of these equality conditions do not hold under (29), (29) is equivalent to

$$C_0<{\left(\frac{\mathrm{tr}\left(A\right)}{2}\right)}^2<\det \left(A\right)<C_1.$$

(36)

Integrating the two inequalities (33) for real $\lambda$ and (36) for non-real $\lambda$, the VAR with the coefficient matrix A is stable if

$$C_0<\det \left(A\right)<C_1$$

(37)

and

$$C_0<{\left(\frac{\mathrm{tr}\left(A\right)}{2}\right)}^2<C_1.$$

(38)

As the inequality (37) implies $0<\frac{1+\det \left(A\right)}{2}<1$ and $\mathrm{tr}\left(A\right)<2$, (38) is equivalent to

$${\left(\frac{\mathrm{tr}\left(A\right)}{2}\right)}^2<{\left(\frac{1+\det \left(A\right)}{2}\right)}^2.$$

(39)

As the upper bound for $\det \left(A\right)$ in (37) can be implied by $\det \left(A\right)<1$, it is equivalent to

$$|\mathrm{tr}\left(A\right)|-1<\det \left(A\right)<1.$$

(40)

Thus, the pair of inequalities (37) and (38) for A is equivalent to the single inequality (40) for A. □

4.2. Stability and Existence of the Solution for the Lyapunov Equation

Intuitively, it would be reasonable if there was a stationary covariance matrix $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$ satisfying the Lyapunov Equation (4), if the coefficient matrix is $A\in {\mathbb{R}}_{*}^{2\times 2}$. However, this is not trivial, as the opposite may not be always true: the existence of $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$ does not imply $A\in {\mathbb{R}}_{*}^{2\times 2}$. This relationship between A and $\widehat{\mathsf{\Sigma }}$ is stated by the following Theorem 1.

Theorem 1.

There is a stationary covariance matrix $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$ satisfying the Lyapunov Equation (4), if the coefficient matrix is $A\in {\mathbb{R}}_{*}^{2\times 2}$. However, the existence of $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$ does not imply $A\in {\mathbb{R}}_{*}^{2\times 2}$.

Proof. 

Find the identity

$$\begin{array}{cc}\det \left(I_4-A\otimes A\right)\hfill & =\det \left(\left(I_2-a_{0,0}A\right)\left(I_2-a_{1,1}A\right)-a_{0,1}{a}_{1,0}{A}^2\right)\hfill \\ \hfill & =\det \left(I_2-A\mathrm{tr}\left(A\right)+A^2\det \left(A\right)\right)\hfill \\ \hfill & ={\left(1-\det \left(A\right)\right)}^2\left(\left(1-a_{0,1}-a_{0,0}^2\right)\left(1-a_{0,1}-a_{1,1}^2\right)-a_{0,1}{a}_{1,0}\mathrm{tr}(A)\right)\hfill \\ \hfill & ={\left(1-\det \left(A\right)\right)}^2\left({\left(1+\det \left(A\right)\right)}^2-\mathrm{tr}{\left(A\right)}^2\right).\hfill \end{array}$$

(41)

By Lemma 4 and (41), we have $\det \left(I_4-A\otimes A\right)>0$. Thus, Lyapunov Equation (5) has the solution for $\widehat{\mathsf{\Sigma }}$, as the matrix $\left(I_4-A\otimes A\right)$ is invertible. The converse of this theorem does not hold, as we construct a counter-example of the coefficient matrix A such that $\det \left(I_4-A\otimes A\right)<0$, with which there is a $\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$, but such a VAR is not stable. □

5. Design of Bivariate Timeseries Given GCs

The goal of this study was to derive a design principle of bivariate timeseries generated by a VAR model with the desired correlation and two types of Granger causality. In this section, we explore the inter-dependent relationships among the variables in the VAR. This analysis revealed a trade-off limitation in designing these variables of timeseries. Specifically, a timeseries with a certain range of desired Granger causality cannot be realized by a stable VAR, in which no stationary covariance is defined in theory.

The set of parameters in any stable VAR model includes

• The coefficient matrix A;
• The base covariance matrix $\mathsf{\Sigma }$;
• The stationary covariance matrix $\widehat{\mathsf{\Sigma }}$; and
• The two types of Granger causality $G_0,G_1$.

There are equality constraints on these variables:

• The variables $A,\mathsf{\Sigma },\widehat{\mathsf{\Sigma }}$ need to satisfy the Lyapunov Equation (4).
• Granger causality $G_i$ ($i=0,1$) is the function of $a_{i,1-i}$, ${\sigma }_{i,i}$, and $\widehat{\mathsf{\Sigma }}$ (Lemma 2).

Besides, it is important to know the feasibility of a set of parameters in VAR, which constrains the range of these variables:

• Stability: $A\in {\mathbb{R}}_{*}^{2\times 2}$ (Section 4);
• Properness: $\mathsf{\Sigma },\widehat{\mathsf{\Sigma }}\in {\mathbb{R}}_{+}^{2\times 2}$ (Section 4) and ${\sigma }_{i,i}\le {\widehat{\sigma }}_{i,i}$ due to the existence of a solution for the Lyapunov equation; and
• The bound for each Granger causality: $G_i\in [0,log{\gamma }_i]$ (Lemma 3).

The Lyapunov Equation (4) on the matrices can be decomposed into the three equations on the scalar variables as follows. For a coefficient matrix $A=\left(\begin{array}{cc}{a}_{0,0}& a_{0,1}\\ a_{1,0}& a_{1,1}\end{array}\right)$, let us define two vectors by

$$a_0:=\left(\begin{array}{c}{a}_{0,0}\\ a_{0,1}\end{array}\right),a_1:=\left(\begin{array}{c}{a}_{1,0}\\ a_{1,1}\end{array}\right).$$

The Lyapunov equation is then equivalently written with these vectors $a_0,a_1$ by the set of the three equations

$$\begin{array}{ccc}\hfill {\widehat{\sigma }}_{0,0}-{\sigma }_{0,0}& =& a_0^{\top }\widehat{\mathsf{\Sigma }}{a}_0\hfill \end{array}$$

(42)

$$\begin{array}{ccc}\hfill {\widehat{\sigma }}_{1,1}-{\sigma }_{1,1}& =& a_1^{\top }\widehat{\mathsf{\Sigma }}{a}_1\hfill \end{array}$$

(43)

$$\begin{array}{ccc}\hfill {\widehat{\sigma }}_{0,1}-{\sigma }_{0,1}& =& a_0^{\top }\widehat{\mathsf{\Sigma }}{a}_1.\hfill \end{array}$$

(44)

Equations (42) and (43) above imply that each of the vectors $a_0$ and $a_1$ are on an ellipsis on each of their planes. This gives the lower bound for ${\widehat{\sigma }}_{i,i}\ge {\sigma }_{i,i}$ (i.e., one condition of the properness above), as $x^{\top }\widehat{\mathsf{\Sigma }}x\ge 0$ for any $x\in {\mathbb{R}}^2$ with a positive-definite matrix $\widehat{\mathsf{\Sigma }}$.

Fixing $G_0$ and $G_1$ imposes each of the two vectors $a_0$ and $a_1$ on the two parallel lines by

$${\left(a_i^{\top }{\widehat{\sigma }}_i\right)}^2={\tau }_i^2,$$

(45)

where

$${\widehat{\sigma }}_i:={({\widehat{\sigma }}_{i,0},{\widehat{\sigma }}_{i,1})}^{\top },{\tau }_i^2:={\widehat{\sigma }}_{i,i}^2\left(1-{\gamma }_i^{-1}{e}^{G_i}\right).$$

Thus, the solution of $a_i$ which satisfies the Lyapunov equation and the fixed Granger causality is the four intersections of the ellipsis and the two parallel lines (Figure 2). This ellipsis is obtained by scaling and shearing transformation to the standard circle $a_{0,0}^2+a_{0,1}^2=1$. This observation gives the angular parametrization of the solution vector ${(a_{0,0},a_{0,1})}^{\top }$, which is explicitly stated by Lemma 5 in the next section.

5.1. Solution A of the Lyapunov Equality Given $\widehat{\mathsf{\Sigma }}$, $G_0$, and $G_1$

In what follows, we start with the derivation of the coefficient matrix A as a root of the equality constraint by the Lyapunov Equation (4) and the Granger causality, for a fixed proper $\widehat{\mathsf{\Sigma }}$, ${\sigma }_{i,i}$ and $G_i$ for each $i=0,1$. The following Lemma 5 gives a necessary condition for the coefficient matrix $A\in {\mathbb{R}}^{2\times 2}$ to satisfy the equality conditions above. Note, however, that such a solution A in this equation does not guarantee the stability of the corresponding VAR (i.e., $A\in {\mathbb{R}}_{*}^{2\times 2}$). This sufficiency is explored in Section 5.2.

Lemma 5.

For a given set of parameters, a positive-definite matrix $\widehat{\mathsf{\Sigma }}=\left(\begin{array}{cc}{\widehat{\sigma }}_{0,0}& {\widehat{\sigma }}_{0,1}\\ {\widehat{\sigma }}_{1,0}& {\widehat{\sigma }}_{1,1}\end{array}\right)\in {\mathbb{R}}_{+}^{2\times 2}$, ${\sigma }_{i,i}\in (0,{\widehat{\sigma }}_{i,i}),G_i\in [0,log{\gamma }_i]$ for each $i=0,1$, suppose that a coefficient matrix $A=\left(\begin{array}{cc}{a}_{0,0}& a_{0,1}\\ a_{1,0}& a_{1,1}\end{array}\right)\in {\mathbb{R}}^{2\times 2}$, satisfies the set of the equations

$$\left\{\begin{array}{c}{a}_0^{\top }\widehat{\mathsf{\Sigma }}{a}_0={\widehat{\sigma }}_{0,0}-{\sigma }_{0,0}\hfill \\ a_1^{\top }\widehat{\mathsf{\Sigma }}{a}_1={\widehat{\sigma }}_{1,1}-{\sigma }_{1,1}\hfill \\ {\left({\widehat{\sigma }}_0^{\top }{a}_0\right)}^2={\tau }_0^2\hfill \\ {\left({\widehat{\sigma }}_1^{\top }{a}_1\right)}^2={\tau }_1^2\hfill \end{array}\right.,$$

(46)

where for $i=0,1$

$${\widehat{\sigma }}_i:={({\widehat{\sigma }}_{i,0},{\widehat{\sigma }}_{i,1})}^{\top },{\tau }_i^2:={\widehat{\sigma }}_{i,i}^2\left(1-{\gamma }_i^{-1}{e}^{G_i}\right).$$

Any coefficient matrix A of a root of this Equation (46) is in the form

$$A={\left(\begin{array}{cc}{S}_0\left(\begin{array}{c}cos{\theta }_0\\ sin{\theta }_0\end{array}\right)& P_2{S}_1\left(\begin{array}{c}cos{\theta }_1\\ sin{\theta }_1\end{array}\right)\end{array}\right)}^{\top },$$

(47)

where each pair of the angles ${\theta }_0\in [0,2\pi )$ and ${\theta }_1\in [0,2\pi )$ takes one of the two or four pairs satisfying for each $i=0,1$

$${sin}^2{\theta }_i=\frac{e^{G_i}-1}{{\gamma }_i-1}$$

(48)

and

$$P_2:=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right),S_i:=\sqrt{1-{\gamma }_i^{-1}}\left(\begin{array}{cc}1& -\frac{{\widehat{\sigma }}_{i,1-i}}{{\widehat{\sigma }}_{i,i}}\\ 0& 1\end{array}\right)\left(\begin{array}{cc}1& 0\\ 0& \frac{{\widehat{\sigma }}_{i,i}}{\sqrt{\det \left(\widehat{\mathsf{\Sigma }}\right)}}\end{array}\right).$$

Proof. 

Find that the following pair of equations in (46) is symmetric under exchange of $i=0,1$:

$$\left\{\begin{array}{c}{a}_i^{\top }\widehat{\mathsf{\Sigma }}{a}_i={\widehat{\sigma }}_{i,i}-{\sigma }_{i,i}\hfill \\ {\left({\widehat{\sigma }}_i^{\top }{a}_i\right)}^2={\tau }_i^2\hfill \end{array}\right..$$

(49)

Thus, we solve this for $i=0$ below, and it holds for $i=1$.

Solving the second equation of (49) for $a_{0,0}$, we have

$$a_{0,0}=\frac{\pm {\tau }_0-a_{0,1}{\widehat{\sigma }}_{0,1}}{{\widehat{\sigma }}_{0,0}}.$$

(50)

Inserting this into the first equation of (49), we have

$$a_{0,1}^2=\frac{{\widehat{\sigma }}_{0,0}^2(1-{\gamma }_0^{-1})-{\tau }_0^2}{\det \left(\widehat{\mathsf{\Sigma }}\right)}.$$

(51)

Inserting ${\tau }_0^2={\widehat{\sigma }}_{0,0}^2\left(1-{\gamma }_0^{-1}{e}^{G_0}\right)$, we have

$$a_{0,1}=\pm \sqrt{\frac{{\widehat{\sigma }}_{0,0}{\sigma }_{0,0}\left(e^{G_0}-1\right)}{\det \left(\widehat{\mathsf{\Sigma }}\right)}}.$$

(52)

Inserting this into (50), we have at most four vectors $a_0={(a_{0,0},a_{0,1})}^{\top }$ as the solution of (49) for $i=1$:

$$a_0=\left(\begin{array}{c}{c}_0-\frac{{\widehat{\sigma }}_{0,1}}{{\widehat{\sigma }}_{0,0}}{d}_0\\ d_0\end{array}\right),\left(\begin{array}{c}{c}_0+\frac{{\widehat{\sigma }}_{0,1}}{{\widehat{\sigma }}_{0,0}}{d}_0\\ -d_0\end{array}\right),\left(\begin{array}{c}-c_0-\frac{{\widehat{\sigma }}_{0,1}}{{\widehat{\sigma }}_{0,0}}{d}_0\\ d_0\end{array}\right),\left(\begin{array}{c}-c_0+\frac{{\widehat{\sigma }}_{0,1}}{{\widehat{\sigma }}_{0,0}}{d}_0\\ -d_0\end{array}\right),$$

(53)

where for $i=0,1$

$$c_i:=\sqrt{1-{\gamma }_i^{-1}{e}^{G_i}},d_i:=\sqrt{\frac{{\widehat{\sigma }}_{i,i}{\sigma }_{i,i}\left(e^{G_i}-1\right)}{\det \left(\widehat{\mathsf{\Sigma }}\right)}}.$$

By symmetry to $i=0,1$, there are at most four vectors as the solution of (49) for $i=1$:

$$P_2{a}_1=\left(\begin{array}{c}{a}_{1,1}\\ a_{1,0}\end{array}\right)=\left(\begin{array}{c}{c}_1-\frac{{\widehat{\sigma }}_{1,0}}{{\widehat{\sigma }}_{1,1}}{d}_1\\ d_1\end{array}\right),\left(\begin{array}{c}{c}_1+\frac{{\widehat{\sigma }}_{1,0}}{{\widehat{\sigma }}_{1,1}}{d}_1\\ -d_1\end{array}\right),\left(\begin{array}{c}-c_1-\frac{{\widehat{\sigma }}_{1,0}}{{\widehat{\sigma }}_{1,1}}{d}_1\\ d_1\end{array}\right),\left(\begin{array}{c}-c_1+\frac{{\widehat{\sigma }}_{1,0}}{{\widehat{\sigma }}_{1,1}}{d}_1\\ -d_1.\end{array}\right)$$

(54)

Find that these four solution vectors parameterized by

$$a_0=S_0\left(\begin{array}{c}cos{\theta }_0\\ sin{\theta }_0\end{array}\right)\mathrm{and}{a}_1=P_2{S}_1\left(\begin{array}{c}cos{\theta }_1\\ sin{\theta }_1\end{array}\right),$$

satisfy (49), if $({\theta }_0,{\theta }_1)$ holds (48), with the trigonometric identity ${cos}^2{\theta }_i+{sin}^2{\theta }_i=1$ and

$$S_0^{\top }\widehat{\mathsf{\Sigma }}{S}_0=\left({\widehat{\sigma }}_{0,0}-{\sigma }_{0,0}\right)I_2\mathrm{and}{S}_1^{\top }{P}_2^{\top }\widehat{\mathsf{\Sigma }}{P}_2{S}_1=\left({\widehat{\sigma }}_{1,1}-{\sigma }_{1,1}\right)I_2.$$

(55)

□

5.2. Sufficiency of the Solution

For a given set of parameters, Lemma 5 in the previous section gives a set of solutions of the coefficient matrix A for the Lyapunov equation. Note that not all of these solutions A are feasible, in the sense that they satisfy all constraints such as the stability of $A\in {\mathbb{R}}_{*}^{2\times 2}$ and the properness of $\mathsf{\Sigma }\in {\mathbb{R}}_{+}^{2\times 2}$, in which $\mathsf{\Sigma }$ can be derived from Lyapunov Equation (4) given A and $\widehat{\mathsf{\Sigma }}$. The following lemmas provide the sufficient condition for a solution A by checking the properness of $\mathsf{\Sigma }$ and the stability of A.

Lemma 6.

Suppose A is a solution of Equation (46) in Lemma 5, represented by a pair of $({\theta }_0,{\theta }_1)$. In this case, $\mathsf{\Sigma }\in {\mathbb{R}}_{+}^{2\times 2}$, if and only if

$$cos\widehat{\eta }-{\left({\gamma }_0{\gamma }_1\right)}^{-\frac{1}{2}}\le {\widehat{\gamma }}_0{\widehat{\gamma }}_{1}cos\left(\widehat{\eta }-{\theta }_0-{\theta }_1\right)\le cos\widehat{\eta }+{\left({\gamma }_0{\gamma }_1\right)}^{-\frac{1}{2}},$$

(56)

where $\widehat{\eta }\in [0,2\pi ]$ is the angler parametrization of the correlation coefficient defined by $cos\widehat{\eta }:=\frac{{\widehat{\sigma }}_{0,1}}{\sqrt{{\widehat{\sigma }}_{0,0}{\widehat{\sigma }}_{1,1}}}$ and ${\widehat{\gamma }}_i=\sqrt{1-{\gamma }_i^{-1}}$.

Proof. 

Applying the polar representation of the $a_0,a_1$ in (47) in Lemma 5 to the third Equation (44) of the Lyapunov equation, we have

$${\widehat{\sigma }}_{0,1}-{\sigma }_{0,1}=\sqrt{\left({\widehat{\sigma }}_{0,0}-{\sigma }_{0,0}\right)\left({\widehat{\sigma }}_{1,1}-{\sigma }_{1,1}\right)}cos\left(\widehat{\eta }-{\theta }_0-{\theta }_1\right).$$

(57)

By the positive definiteness of $\mathsf{\Sigma }$, ${\sigma }_{0,1}^2\le {\sigma }_{0,0}{\sigma }_{1,1}$. This inequality applied to (57) gives the lemma. □

If we have

$${\widehat{\gamma }}_0{\widehat{\gamma }}_1\le cos\widehat{\eta }+{\left({\gamma }_0{\gamma }_1\right)}^{-\frac{1}{2}}\mathrm{and}cos\widehat{\eta }-{\left({\gamma }_0{\gamma }_1\right)}^{-\frac{1}{2}}\le -{\widehat{\gamma }}_0{\widehat{\gamma }}_1,$$

Equation (56) holds for any pair of angles $({\theta }_0,{\theta }_1)$. This condition is

$$|cos\widehat{\eta }|\le {\left({\gamma }_0{\gamma }_1\right)}^{-\frac{1}{2}}-{\widehat{\gamma }}_0{\widehat{\gamma }}_1,$$

or

$$|{\widehat{\sigma }}_{0,1}|\le \sqrt{{\sigma }_{0,0}{\sigma }_{1,1}}-\sqrt{\left({\widehat{\sigma }}_{0,0}-{\sigma }_{0,0}\right)\left({\widehat{\sigma }}_{1,1}-{\sigma }_{1,1}\right)}.$$

(58)

On the other hand, (56) holds for any $0<\widehat{\eta }<\pi$ (equivalently $-1<cos\widehat{\eta }<1$) if

$$\frac{\sqrt{{\gamma }_0{\gamma }_1}-1}{\sqrt{({\gamma }_0-1)({\gamma }_1-1)}}\le cos({\theta }_0+{\theta }_1)\le \frac{\sqrt{{\gamma }_0{\gamma }_1}+1}{\sqrt{({\gamma }_0-1)({\gamma }_1-1)}}.$$

(59)

These bounds (58) and (59) mean that the range of feasible GCs (${\theta }_0,{\theta }_1$) and the range of feasible correlation $cos\widehat{\eta }$ are in a trade-off relationship in general.

Lemma 7.

The VAR with the correlation $|cos(\widehat{\eta }\left)\right|<1$ and Granger causality ${\theta }_0,{\theta }_1$ in the angular form is stable if and only if

$$-sin\widehat{\eta }+|{\widehat{\gamma }}_{0}sin\left(\widehat{\eta }-{\theta }_0\right)+{\widehat{\gamma }}_{1}sin\left(\widehat{\eta }-{\theta }_1\right)|<{\widehat{\gamma }}_0{\widehat{\gamma }}_{1}sin\left(\widehat{\eta }-{\theta }_0-{\theta }_1\right)<sin\widehat{\eta }.$$

(60)

Proof. 

Using the angular notation of the solution A with ${\theta }_0,{\theta }_1$, we have

$$\det \left(A\right)={\widehat{\gamma }}_0{\widehat{\gamma }}_1\frac{sin\left(\widehat{\eta }-{\theta }_0-{\theta }_1\right)}{sin\widehat{\eta }}\mathrm{and}\mathrm{tr}\left(A\right)=\frac{{\widehat{\gamma }}_{0}sin\left(\widehat{\eta }-{\theta }_0\right)+{\widehat{\gamma }}_{1}sin\left(\widehat{\eta }-{\theta }_1\right)}{sin\widehat{\eta }}.$$

Inserting these into stability condition (23), we have the inequality (60). □

As well as Lemma 6, Lemma 7 leads the trade-off relationship between $\widehat{\mathsf{\Sigma }}$ in the angle from $\widehat{\eta }$ and A in the angle from ${\theta }_0,{\theta }_1$. In general, this bound further narrows the upper and lower bounds given by Lemma 3. In general, correlation is limited to close to zero if one wishes for higher Granger causality. On the other hand, the two types of Granger causality are limited to close to zero if one wishes for a higher correlation in the absolute value.

6. Concluding Remarks

6.1. Summary and Potential Usage of the Algorithm

In this paper, we explored the relationship between the VAR parameters and timeseries statistics (Figure 1), and identified the trade-off limitation between the stationary covariance $\widehat{\mathsf{\Sigma }}$ and Granger causality $G_0,G_1$ (Lemma 6). This suggests that the following Algorithm 1 will generate a timeseries with desired statistics.

Algorithm 1: Compute a VAR parameter set for the desired statistics

Data: Desired timeseries statistics $(G_0,G_1,\widehat{\mathsf{\Sigma }},{\sigma }_{0,0},{\sigma }_{1,1})$ in the feasible range satisfy both inequalities (56) in Lemma 6 and (60) in Lemma 7. 1 Derive four sets of VAR parameters $(A,\mathsf{\Sigma })$ for the given timeseries statistics $(G_0,G_1,\widehat{\mathsf{\Sigma }},{\sigma }_{0,0},{\sigma }_{1,1})$ by Lemma 5 2 Choose one of the four sets of VAR parameters $(A,\mathsf{\Sigma })$.  Result: The VAR parameters $(A,\mathsf{\Sigma })$.

This data-generation algorithm can be used to generate surrogate data [19], which can be used to test whether an empirical timeseries is a sample from a VAR with a given correlation and Granger causality. This algorithm is also useful in analyzing to what extent a class of VAR timeseries varies under the same statistics.

6.2. Validity of Granger Causality Estimated on Empirical Timeseries

Our analysis also warns that not all Granger causality (or transfer entropy) is “valid”, in the sense that its underlying VAR model is not stable and thus the Granger causality is undefined in theory. In theory, we can identify some value of Granger causality for a finite empirical time series, which is generated by an underlying unstable VAR model without any stationary statistics. Such timeseries statistics will diverge in the long run, but it may be difficult to identify this with a finite empirical timeseries. This asymmetry—namely, that the Granger causality can be calculated numerically but does not guarantee the stability of the underlying VAR—is explicitly demonstrated by Theorem 1. For a given empirical timeseries $v=(v_0,v_1,\dots ,v_T)\in {\mathbb{R}}^{2\times (T+1)}$, one should calculate not just the Granger causality but also its validity by checking (1) $A\in {\mathbb{R}}_{*}^{2\times 2}$, (2) $\mathsf{\Sigma }\in {\mathbb{R}}_{+}^{2\times 2}$, and (3) ${\mathsf{\Sigma }}_{0,1}\le {\widehat{\mathsf{\Sigma }}}_{0,1}$, by calculating the maximum likelihood estimator of $\left(A\right(v),\mathsf{\Sigma }(v\left)\right)$, such as

$$A\left(v\right):=V_{1,0}{V}_{0,0}^{-1}\mathrm{and}\mathsf{\Sigma }\left(v\right)=V_{1,1}-V_{1,0}{V}_{0,0}^{-1}{V}_{0,1},$$

where

$$V_{i,j}:=T^{-1}\sum _{t=1}^T{v}_{t-1+i}{v}_{t-1+j}^{\top }.$$

In fact, $\mathsf{\Sigma }\left(v\right)$ is always (semi-)positive definite, as it takes the form of the Schur complement of the (semi-)positive-definite matrix $\left(\begin{array}{cc}{V}_{0,0}& V_{0,1}\\ V_{1,0}& V_{1,1}\end{array}\right)$. Thus, this maximum likelihood estimator readily satisfies condition (2).

6.3. Future Work

In this paper, we limit the VAR model to be bivariate for simplicity of analysis. We expect it is possible to generalize the current result to any higher dimensional VAR model. In such a generalization, feasible boundaries for the stable VAR models may require further effort to understand.