This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

Transfer entropy is a frequently employed measure of conditional co-dependence in non-parametric analysis of Granger causality. In this paper, we derive analytical expressions for transfer entropy for the multivariate exponential, logistic, Pareto (type ℐ – ℐ

Granger causality is a well-known concept based on dynamic co-dependence [

Information theory has increasingly become a useful complement to the existing repertoire of methodologies in mathematical statistics [

The aim of this paper is to present closed-form expressions for transfer entropy for a number of non-Gaussian, unimodal, skewed distributions used in the modeling of occurrence rates, rare events and ‘fat-tailed’ phenomena in biological, physical and actuarial sciences [

The specific choice of these distributions is contingent upon the existence of unique expressions for the corresponding probability density functions and Shannon entropy expressions. A counter-example is given by the multivariate gamma distribution, which although derived in a number of tractable formats under certain preconditions [

Another remark shall be dedicated to stable distributions. Such distributions are limits of appropriately scaled sums of independent and identically distributed variables. The general tractability of distributions with this property lies in their “attractor” behavior and their ability to accommodate skewness and heavy tails. Other than the Gaussian distribution (stable by the Central Limit Theorem), the Cauchy-Lorentz distribution and the Lévy distribution are considered to be the only stable distributions that can be expressed analytically. However, the latter lacks analytical expressions for Shannon entropy in the multivariate case. Expressions for Shannon entropy and transfer entropy for the multivariate Gaussian distribution have been derived in [

As a brief methodological introduction, we will go through a conceptual sketch of Granger causality, the formulation of the linear models underlying the above-mentioned test statistic, and the definition of transfer entropy before deriving the expressions for our target distributions.

Employment of Granger causality is common practice within cause-effect analysis of dynamic phenomena where the cause temporally precedes the effect and where the information embedded in the cause about the effect is unique. Formulated using probability theory, under _{0}, given

where ⫫ denotes probabilistic independence. Henceforth, for the sake of convenience, we implement the following substitutions: _{t}

The statement above can be tested by comparing the two conditional probability densities: _{X|Z}_{X|Y Z}

In parametric analysis of Granger causality, techniques of linear regression have been the dominant choice. Under fulfilled assumptions of ordinary least squares’ regression and stationarity, the hypothesis in

where the ^{2}). Traditionally, the F-distributed Granger-Sargent test [_{ε}/_{η}^{2}-distributed under the null hypothesis, and non-central ^{2}-distributed under the alternate hypothesis. There are two types of multivariate generalizations of

where the last equality follows the scheme presented in [

Transfer entropy, a non-parametric measure of co-dependence is identical to (conditional) mutual information measured in _{S}

where

Interestingly, for Gaussian variables one can show that

In the following, we shall look at closed-form expressions for transfer entropy for the multivariate exponential, logistic, Pareto (type ℐ – ℐ

In this section we will derive the expression for transfer entropy for the multivariate exponential distribution. The remaining derivations follow an identical scheme and are presented in the

The multivariate exponential density function for a

where ^{d}_{i}_{i}_{i}

Thus, transfer entropy for a set of multivariate exponential variables can be formulated as:

which, after simplifications, reduces to

where _{X}_{X}_{Y}_{Z}

In any regard,

The distributions discussed in this paper are frequently subject to the modeling of natural phenomena, and utilized frequently within biological, physical and actuarial engineering. Events distributed according to any of the discussed distributions are not suitable for analysis using linear models and require non-parametric models of analysis or transformations where feasible.

The focus of this paper has been on non-parametric modeling of Granger causality using transfer entropy. Our results show that the expressions for transfer entropy for the multivariate exponential, logistic, Pareto (type ℐ – ℐ

As underlined by our result, the value of transfer entropy depends in a declining manner on the multivariate distribution parameter

The authors wish to thank John Hertz for insightful discussions and feedback. MJM has been supported by the Magnussons Fund at the Royal Swedish Academy of Sciences and the European Unions Seventh Framework Programme (FP7/2007-2013) under grant agreement #258068, EU-FP7-Systems Microscopy NoE. MJM and JT have been supported by the Swedish Research Council grant #340-2012-6011.

The authors declare no conflicts of interest.

Mehrdad Jafari-Mamaghani and Joanna Tyrcha designed, performed research and analyzed the data; Mehrdad Jafari-Mamaghani wrote the paper. All authors read and approved the final manuscript.

The multivariate logistic density function for a

with ^{d}_{i}

where

which, after simplifications, using the identity

reduces to

The multivariate Pareto density function of type ℐ – ℐ

with ^{d}_{i}_{i}_{i}_{i}

Pareto ℐℐℐ by setting

Pareto ℐℐ by setting _{i}

Pareto ℐ by setting _{i}_{i}_{i}

For the multivariate Pareto distribution in

Thus, the transfer entropy for the multivariate Pareto density function of type ℐ – ℐ

which, after simplifications, reduces to

The multivariate Burr density function for a

with ^{d}_{i}_{i}_{i}

Thus, the transfer entropy for the multivariate Burr distribution can be formulated as:

which, after simplifications, reduces to

The multivariate Cauchy-Lorentz density function for a

for ^{d}

Thus, the transfer entropy

which, after simplifications, using the identity in

where

is obtained after a simplification of the digamma function.