# Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Data and Methods

#### 2.1. Causality Assessment Methods

#### 2.1.1. Granger Causality Analysis

#### 2.2. Estimation of GC

#### 2.3. Transfer Entropy

#### 2.4. Potential Causes of Observed Difference

#### 2.5. TE Estimation

#### 2.6. Data

#### 2.6.1. Dataset

#### 2.6.2. Preprocessing

#### 2.6.3. Computing the Components

**Figure 1.**Location of areas dominated by specific components of the surface air temperature data using VARIMAX-rotated PCA decomposition. For each location, the color corresponding to the component with maximal intensity was used. White dots represent approximate centers of mass of the components, used in subsequent figures for visualization of the nodes of the networks.

#### 2.6.4. Estimating the Dimensionality of the Data

#### 2.7. Network Construction

#### 2.8. Reliability Assessment

#### 2.8.1. Model

#### 2.8.2. Implementation Details

## 3. Results

#### 3.1. Weighted Causality Networks

**Figure 2.**Reliability of causality network detection using different causality estimators, and the similarity to linear causality network estimates using the Fourier surrogate model. For each estimator, six causality networks are estimated, one for each decade-long section of model stationary data (a Fourier surrogate realization of the original data). Black: the height of the bar corresponds to the average Spearman’s correlation across all 15 pairs of decades. White: the height of the bar corresponds to the average Spearman’s correlation of nonlinear causality network and linear causality network across 6 decades.

**Figure 3.**The variability of causality network detection using different causality estimators and the similarity to linear causality network estimates for the original data. For each estimator, six causality networks are estimated, one for each decade of the data. Black: the height of the bar corresponds to the average Spearman’s correlation across all 15 pairs of decades. White: the height of the bar corresponds to the average Spearman’s correlation of nonlinear causality network and linear causality network across 6 decades.

**Figure 4.**The reliability of causality network detection using different causality estimators and the similarity to linear causality network estimates for the stationary model constructed as multivariate AR(1) surrogate of the original data. For each estimator, six causality networks are estimated, one for each decade of modeled stationary data. Black: the height of the bar corresponds to the average Spearman’s correlation across all 15 pairs of decades. White: the height of the bar corresponds to the average Spearman’s correlation of nonlinear causality network and linear causality network across 6 decades.

**Figure 5.**The reliability of causality network detection using different causality estimators and the similarity to linear causality network estimates for the stationary model constructed as multivariate AR(1) surrogate of the original data. For each estimator, six causality networks are estimated, each for a separate realization of the multivariate AR(1) process fitted to the original data. Black: the height of the bar corresponds to the average Spearman’s correlation across all 15 pairs of decades. White: the height of the bar corresponds to the average Spearman’s correlation of nonlinear causality network and linear causality network across 6 decades.

#### 3.2. Unweighted Causality Networks

**Figure 6.**The reliability of causality network detection using different causality estimators and the similarity to linear causality network estimates. For each estimator, six causality networks are estimated, one for each decade of modeled stationary data. Black: the height of the bar corresponds to the average Jaccard similarity coefficient across all 15 pairs of decades. White: the height of the bar corresponds to the average Jaccard similarity coefficient of nonlinear causality network and linear causality network across 6 decades.

#### 3.3. Components and Resulting Networks

**Figure 7.**Causality network obtained by averaging the results for the six decades (total time span 1948–2007) for decomposed data (67 components represented by center of mass). Only the 100 strongest links are shown. For each decade, the network was estimated by linear Granger causality.

**Figure 8.**Causality network obtained by averaging the results for the six decades (total time span 1948–2007) for decomposed data (67 components represented by center of mass). Only the 100 strongest links are shown. For each decade, the network was estimated by (nonlinear) transfer entropy using the equiqantal binning method with $Q=2$.

## 4. Discussion

**Figure 9.**Causality network obtained by averaging the results for the six decades (total time span 1948–2007) for decomposed data (67 components represented by center of mass). Only the 100 strongest links are shown. For each decade, the network was detected by the fully multivariate linear Granger causality.

**Figure 10.**The reliability of causality network detection using different causality estimators and the similarity to linear causality network estimates for the Fourier surrogates model. For each estimator, six causality networks are estimated, one for each decade-long section of the model stationary data (a Fourier surrogate realization of the original data). Black: the height of the bar corresponds to the average Spearman’s correlation across all 15 pairs of decades. White: the height of the bar corresponds to the average Spearman’s correlation of nonlinear causality network and linear causality network across 6 decades.

**Figure 11.**Causality network obtained by averaging the results for the six decades (total time span 1948–2007) for gridded data (162 spatial locations). Only the 200 strongest links are shown. For each decade, the network was estimated by linear Granger causality.

**Figure 12.**Causality network obtained by averaging the results for the six decades (total time span 1948–2007) for gridded data (162 spatial locations). Only the 200 strongest links are shown. For each decade, the network was estimated by (nonlinear) conditional mutual information, using the equiqantal binning method with $Q=2$.

## 5. Conclusions

## Acknowledgements

## References

- Newman, M.E.J. The structure and function of complex networks. SIAM Rev.
**2003**, 45, 167–256. [Google Scholar] [CrossRef] - Boccaletti, S.; Latora, V.; Moreno, Y.; Chavez, M.; Hwang, D.U. Complex networks: Structure and dynamics. Phys. Rep.
**2006**, 424, 175–308. [Google Scholar] [CrossRef] - Tsonis, A.; Roebber, P. The architecture of the climate network. Physica A
**2004**, 333, 497–504. [Google Scholar] [CrossRef] - Tsonis, A.A.; Swanson, K.L.; Roebber, P.J. What do networks have to do with climate? Bull. Am. Meteorol. Soc.
**2006**. [Google Scholar] [CrossRef] - Yamasaki, K.; Gozolchiani, A.; Havlin, S. Climate networks around the globe are significantly affected by El Nino. Phys. Rev. Lett.
**2008**. [Google Scholar] [CrossRef] [PubMed] - Malik, N.; Bookhagen, B.; Marwan, N.; Kurths, J. Analysis of spatial and temporal extreme monsoonal rainfall over South Asia using complex networks. Clim. Dyn.
**2012**, 39, 971–987. [Google Scholar] [CrossRef] - Steinhaeuser, K.; Ganguly, A.R.; Chawla, N.V. Multivariate and multiscale dependence in the global climate system revealed through complex networks. Clim. Dyn.
**2012**, 39, 889–895. [Google Scholar] [CrossRef] - Steinhaeuser, K.; Chawla, N.V.; Ganguly, A.R. Complex Networks in Climate Science: Progress, Opportunities and Challenges. In Proceedings of the 2010 Conference on Intelligent Data Understanding, CIDU 2010, Mountain View, CA, USA, 5–6 October 2013; pp. 16–26.
- Donges, J.F.; Zou, Y.; Marwan, N.; Kurths, J. The backbone of the climate network. EPL
**2009**. [Google Scholar] [CrossRef] - Donges, J.F.; Zou, Y.; Marwan, N.; Kurths, J. Complex networks in climate dynamics. Eur. Phys. J.
**2009**, 174, 157–179. [Google Scholar] [CrossRef] - Hlinka, J.; Hartman, D.; Vejmelka, M.; Novotna, D.; Paluš, M. Non-linear dependence and teleconnections in climate data: Sources, relevance, nonstationarity. Clim. Dyn.
**2012**. [Google Scholar] [CrossRef] - Ebert-Uphoff, I.; Deng, Y. Causal discovery for climate research using graphical models. J. Clim.
**2012**, 25, 5648–5665. [Google Scholar] [CrossRef] - Ebert-Uphoff, I.; Deng, Y. A new type of climate network based on probabilistic graphical models: Results of boreal winter versus summer. Geophys. Res. Lett.
**2012**, 39, 5648–5665. [Google Scholar] [CrossRef] - Granger, C.W. Investigating causal relations by econometric model and cross-spectral methods. Econometrica
**1969**, 37, 424–438. [Google Scholar] [CrossRef] - Vejmelka, M.; Palus, M. Inferring the directionality of coupling with conditional mutual information. Phys. Rev. E
**2008**. [Google Scholar] [CrossRef] [PubMed] - Schreiber, T. Measuring information transfer. Phys. Rev. Lett.
**2000**, 85, 461–464. [Google Scholar] [CrossRef] [PubMed] - Kistler, R.; Kalnay, E.; Collins, W.; Saha, S.; White, G.; Woollen, J.; Chelliah, M.; Ebisuzaki, W.; Kanamitsu, M.; Kousky, V.; et al. The NCEP-NCAR 50-year reanalysis: Monthly means CD-ROM and documentation. Bull. Am. Meteorol. Soc.
**2001**, 82, 247–267. [Google Scholar] [CrossRef] - Kalnay, E.; Kanamitsu, M.; Kistler, R.; Collins, W.; Deaven, D.; Gandin, L.; Iredell, M.; Saha, S.; White, G.; Woollen, J.; et al. The NCEP/NCAR 40-year reanalysis project. Bull. Am. Meteorol.Soc.
**1996**, 77, 437–471. [Google Scholar] [CrossRef] - Wiener, N. Modern Mathermatics for Engineers; McGraw-Hill: New York, NY, USA, 1956; chapter The theory of prediction; pp. 165–190. [Google Scholar]
- Ding, M.; Chen, Y.; Bressler, S.L. Granger Causality: Basic Theory and Application to Neuroscience. In Handbook of Time Series Analysis; Wiley-VCH Verlag GmbH & Co. KGaA: Weinheim, Germany, 2006; pp. 437–460. [Google Scholar]
- Geweke, J.F. Measurement of linear dependence and feedback between multiple time series. J. Am. Stat. Assoc.
**1982**, 79, 907–915. [Google Scholar] [CrossRef] - Geweke, J.F. Measures of conditional linear dependence and feedback between time series. J. Am. Stat. Assoc.
**1984**, 79, 907–915. [Google Scholar] [CrossRef] - Barnett, L.; Barrett, A.B.; Seth, A.K. Granger causality and transfer entropy are equivalent for gaussian variables. Phys. Rev. Lett.
**2009**, 103. [Google Scholar] [CrossRef] [PubMed] - Palus, M.; Albrecht, V.; Dvorak, I. Information theoretic test for nonlinearity in time series. Phys. Lett.
**1993**, 175, 203–209. [Google Scholar] [CrossRef] - Frenzel, S.; Pompe, B. Partial mutual information for coupling analysis of multivariate time series. Phys. Rev. Lett.
**2007**, 99. [Google Scholar] [CrossRef] [PubMed] - Kaiser, H. The varimax criterion for analytic rotation in factor analysis. Psychometrika
**1958**, 23, 187–200. [Google Scholar] [CrossRef] - Schwarz, G. Estimating the dimension of a model. Ann. Stat.
**1978**, 5, 461–464. [Google Scholar] [CrossRef] - Benjamini, Y.; Hochberg, Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. Series B-Methodol.
**1995**, 57, 289–300. [Google Scholar] - Prichard, D.; Theiler, J. Generating surrogate data for time series with several simultaneously measured variables. Phys. Rev. Lett.
**1994**, 73, 951–954. [Google Scholar] [CrossRef] [PubMed] - Palus, M. Detecting phase synchronization in noisy systems. Phys. Lett.
**1997**, 235, 341–351. [Google Scholar] [CrossRef] - Zou, Y.; Romano, M.C.; Thiel, M.; Marwan, N.; Kurths, J. Inferring indirect coupling by means of recurrences. Int. J. Bifurc. Chaos
**2011**, 21, 1099–1111. [Google Scholar] [CrossRef] - Runge, J.; Heitzig, J.; Petoukhov, V.; Kurths, J. Escaping the curse of dimensionality in estimating multivariate transfer entropy. Phys. Rev. Lett.
**2012**. [Google Scholar] [CrossRef] [PubMed] - Runge, J.; Heitzig, J.; Marwan, N.; Kurths, J. Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy. Phys. Rev.
**2012**. [Google Scholar] [CrossRef] [PubMed] - Baccala, L.A.; Sameshima, K. Partial directed coherence: A new concept in neural structure determination. Biol. Cybern.
**2001**, 84, 463–474. [Google Scholar] [CrossRef] [PubMed] - Kaminski, M.; Ding, M.; Truccolo, W.A.; Bressler, S.L. Evaluating causal relations in neural systems: Granger causality, directed transfer function and statistical assessment of significance. Biol. Cybern.
**2001**, 85, 145–157. [Google Scholar] [PubMed] - Hlinka, J.; Palus, M.; Vejmelka, M.; Mantini, D.; Corbetta, M. Functional connectivity in resting-state fMRI: Is linear correlation sufficient? NeuroImage
**2011**, 54, 2218–2225. [Google Scholar] [CrossRef] [PubMed] - Palus, M.; Novotna, D. Testing for nonlinearity in weather records. Phys. Lett.
**1994**, 193, 67–74. [Google Scholar] [CrossRef] - Palus, M.; Novotna, D. Enhanced Monte Carlo Singular System Analysis and detection of period 7.8 years oscillatory modes in the monthly NAO index and temperature records. Nonlinear Process. Geophys.
**2004**, 11, 721–729. [Google Scholar] [CrossRef] - Palus, M.; Novotna, D. Quasi-biennial oscillations extracted from the monthly NAO index and temperature records are phase-synchronized. Nonlinear Process. Geophys.
**2006**, 13, 287–296. [Google Scholar] [CrossRef] - Palus, M.; Novotna, D. Phase-coherent oscillatory modes in solar and geomagnetic activity and climate variability. J. Atmos. Solar-terr. Phys.
**2009**, 71, 923–930. [Google Scholar] [CrossRef] - Palus, M.; Hartman, D.; Hlinka, J.; Vejmelka, M. Discerning connectivity from dynamics in climate networks. Nonlinear Process. Geophys.
**2011**, 18, 751–763. [Google Scholar] [CrossRef] - Feliks, Y.; Ghil, M.; Robertson, A.W. Oscillatory climate modes in the eastern mediterranean and their synchronization with the north atlantic oscillation. J. Clim.
**2010**, 23, 4060–4079. [Google Scholar] [CrossRef] - Boucharel, J.; Dewitte, B.; du Penhoat, Y.; Garel, B.; Yeh, S.W.; Kug, J.S. ENSO nonlinearity in a warming climate. Clim. Dyn.
**2011**, 37, 2045–2065. [Google Scholar] [CrossRef] - Osprey, S.M.; Ambaum, M.H.P. Evidence for the chaotic origin of northern annular mode variability. Geophys. Res. Lett.
**2011**. [Google Scholar] [CrossRef] - Mokhov, I.I.; Smirnov, D.A.; Nakonechny, P.I.; Kozlenko, S.S.; Seleznev, E.P.; Kurths, J. Alternating mutual influence of El-Nino/Southern Oscillation and Indian monsoon. Geophys. Res. Lett.
**2011**. [Google Scholar] [CrossRef]

© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Hlinka, J.; Hartman, D.; Vejmelka, M.; Runge, J.; Marwan, N.; Kurths, J.; Paluš, M. Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information. *Entropy* **2013**, *15*, 2023-2045.
https://doi.org/10.3390/e15062023

**AMA Style**

Hlinka J, Hartman D, Vejmelka M, Runge J, Marwan N, Kurths J, Paluš M. Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information. *Entropy*. 2013; 15(6):2023-2045.
https://doi.org/10.3390/e15062023

**Chicago/Turabian Style**

Hlinka, Jaroslav, David Hartman, Martin Vejmelka, Jakob Runge, Norbert Marwan, Jürgen Kurths, and Milan Paluš. 2013. "Reliability of Inference of Directed Climate Networks Using Conditional Mutual Information" *Entropy* 15, no. 6: 2023-2045.
https://doi.org/10.3390/e15062023