Complex dynamical systems consist of interacting subsystems. It is important to detect these interactions and quantify their strength to improve process understanding. Especially in the climate system, detecting the interactions could provide insights into system dynamics. These interactions can be contemplated as information exchanged or transferred among subsystems. Shannon [1
] introduced a mathematical theory for quantifying the information contained in the context of data compression and transmission. Recently, there has been a surge in applications of information theory in a wide range of fields, for example, uncertainty propagation [2
], neurosciences [5
], climate sciences [7
], earth system sciences [10
], turbulence research [14
], and networks and synchronization in dynamical systems [15
Often in climate studies, the relationship among subsystems is assessed with correlation analysis, empirical orthogonal functions, and linear regressions given time series. Among them, correlation analysis is a parametric method used in identifying linear interactions. For nonlinear interactions, mutual information, a nonparametric method [1
], is often used. Mutual information reveals any shared information between two subsystems. However, both correlation and mutual information are symmetric, i.e., they cannot distinguish between a drive and a response system. For a better understanding of the dynamics, detection of the directionality of interactions is essential. The time-lagged cross-correlation and time-lagged mutual information methods are frequently used for this purpose [17
]. The authors of [19
] observed that these two methods are sensitive to autocorrelations which often obscure detection and quantification of the interaction mechanisms. For example, in his study, a spurious interaction between tropical east Pacific and the northern tropical Atlantic is detected at a time lag 3–6 months while cross-correlations are applied, while the information theoretic-based asymmetric measures detected an interaction between tropical east Pacific and the northern tropical Atlantic at a time lag of 1 month. This interaction time lag is physically consistent with the advection speed of Pacific–Atlantic Walker circulation.
For deeper insights into detecting drive and response linear interactions, the author of [20
] proposed a test based on the Wiener principle. According to the Granger test, X causes Y if the past of the system X assists in predicting the future of the system, Y. In a statistical sense, if the error variance of the optimal linear prediction of Y future state based on the past of X and Y has a smaller error variance than considering the past of Y alone, then X causes, Y. A nonparametric method equivalent to Granger causality known as transfer entropy (TE) was proposed by [21
] (for linear Gaussian systems, they are equivalent up to a factor of 2). The TE measures if any additional information is provided by the past of the source system, which assists in predicting the future state of the destination system. In other words, it measures the divergence or deviation between the entropy rates of the destination’s own past and the past included from a source system. The TE, unlike mutual information, is an asymmetric quantity and hence, it can detect drive and response interactions. It is worth noting that, in nature, the interactions do not merely consist of driving and responding systems, but also systems which drive each other simultaneously with different interaction strength. Hence, any reliable asymmetric estimate, e.g., TE, should reveal this underlying behavior. Many information-theory-based methods were spawned based on a similar principle to that of TE, for example, momentary information transfer, Information Transfer to Y, and Information transfer to X [22
]. These methods have their own applications and limitations.
The simplest estimation of TE uses a multivariate Gaussian model assuming linear interactions between the subsystems. This parametric estimation is hereafter referred to as TE-linear. While the parametric estimator TE-linear is straightforward to calculate, the nonparametric TE estimation is notoriously challenging. Some of the common nonparametric estimation techniques of TE in the literature include the binning, kernel density, and k-nearest neighbor. These estimators are sensitive to the parameter selection in their implementation, such as the bin width selection in the binning estimator, the kernel width in the kernel density estimator, and the number of nearest neighbors while applying the k-nearest neighbor estimator. Unfortunately, no clear consensus is reached among the scientific community on selecting these free parameters. As a result of this dependency, spurious detection of information exchange between the system components could arise. Regardless, TE has been widely used—for example, the authors of [8
] applied TE in the identification of primary drivers of recent climate variability and quantified their influence on climate variability. Their results suggested that greenhouse gases are primary contributors to the recent climate variability. The authors of [23
] studied the information exchange between the South Atlantic anomaly and global sea level for the last 300 years using TE and concluded that larger information is exchanged from the south Atlantic anomaly to global sea-level rise than vice versa. However, these studies relied only on a single TE estimation technique (binning). It is still unclear if TE nonparametric estimations with the free parameters reliably produce numerically consistent estimations. Furthermore, these estimators are also sensitive to the length of the time series. For example, robust kernel estimation asks for sufficient data. Hence, before applying TE to climate phenomena, we tested various estimators of TE along with their sensitivity on time series length with idealized systems where the system dynamics is expected or known.
Liang and Kleeman [24
], realizing that information exchange could be derived rigorously rather than axiomatically, developed another method called it information flow (IF), which is derived from the first principles of information theory. In their framework, the information source and destination are abstracted as system components and thus derive the information flow between these dynamical components. The information flow between the source system Y and destination system X is equal to the difference between the time evolution of marginal entropy of X and entropy of X excluding the influence of, Y. To apply the information flow method, the time evolution of the marginal probabilities must be computed. This time evolution of the marginal probabilities, in turn, depends on the system dynamics. If the system dynamics is unknown, IF becomes difficult to apply. The information flow method was successfully applied to Heńon maps, the Rössler system, and truncated Burgers–Hopf with their respective system dynamics known [25
]. Unlike that of IF, the calculation of TE do not require system dynamics. Hence, given two climate time series, TE is straightforward to apply. Liang [26
] proposed a simple and concise maximum likelihood estimator of IF for linear systems which is easy and straightforward to apply without system dynamics. This estimator is a very important result for the climate community as it bridges the gap between theory and real-world applications. From hereafter, this maximum likelihood estimator is referred to as information flow-linear (IF-linear). IF-linear has been successfully applied in detecting the causal structure between CO
and global temperatures [7
], changing the relationship between the convection over the Western Tibetan Plateau and the sea surface temperature in the Northern Bay of Bengal [27
] and forecasting the tropical cyclone genesis over the Northwest Pacific through identifying the causal factors in cyclone–climate interactions [28
In this work, our aim was to apply information theory methods to detect interactions between climate phenomena. Moreover, we were provided with a limited amount of temporal series of climate data and with their dynamics unknown. Hence, we focused on IF-linear and TE methods. We wanted to find out if these methods are consistently able to detect the directionality of the interactions for climate phenomena. Having in mind that the nature of both methodologies differs, we first wanted to understand if this could lead to differences in detecting the information exchange from climate time series. On the other hand, TE methods are highly sensitive to the choice of free parameters and with time series length, which often might lead to brittle information exchange detections. Thus, we initially checked if the different estimators (IF-linear, TE-linear, TE-binning, TE-kernel, and TE-kraskov) detect the directionality of the interactions using various temporal series lengths as realized by idealized systems, whose dynamics and directionality of the interactions is expected or known. These systems consist of uni- and bidirectional coupled linear and nonlinear systems. Thereafter, we applied these methods to the Lorenz-96 system [29
], which is known to mimic the mid-latitude atmosphere behavior. Finally, we experimentally applied these methods to (1) Indo-Pacific sea surface temperatures interbasin coupling and (2) the relation between North Atlantic Oscillation (NAO) and winter near-surface air temperatures over Europe. One of the limitations of this study is that we rigorously tested and applied various estimators to two-dimensional systems only. For a detailed and excellent review on the applications of TE on high dimensional interactions, refer to [30
This paper is organized as follows. Section 2
comprises the background material for IF-linear, TE, and its estimation techniques such as TE-linear, TE-binning, TE-kernel, and TE k-nearest neighbor. In Section 3
, the abovementioned methods are applied to uni- and bidirectional coupled linear and nonlinear systems, the Lorenz-96 system, and then to climate phenomena. Results are also discussed in this section. Finally, conclusions are drawn in Section 4
This work targeted detecting and quantifying interactions in climate phenomena through asymmetric methods from information theory, IF, and TE. However, due to the difficulty in their estimations, we initially tested various estimators of these methods to idealized systems and then to two important climate phenomena. We limited our discussions only to two-dimensional systems.
The parametric estimators assuming linearity, such as the rigorously derived IF-linear and axiomatically proposed TE-linear, detected and reliably quantified the unidirectional and bidirectional information exchange in the idealized linear systems. IF-linear was able to detect the unidirectional information exchange for the tested unidirectional nonlinear system, whereas the TE-linear failed to do so. For the bidirectional nonlinear Heńon maps, both linear estimators failed to detect and quantify the information exchange. Hence, care has to be taken if linear information exchange measures is applied in climate system diagnosis, especially if the system variables have non-Gaussian distributions. However, these two estimators, IF-linear and TE-linear, were robust and reliable for the discussed linear systems and, in addition, IF-linear also for a weakly nonlinear system. For all the idealized systems discussed here, the nonlinear implementation of IF might reveal the interactions, but since we focused on climate applications given time series with an unknown dynamical model, we used IF-linear, which does not require system dynamics.
Among the nonparametric estimators, the TE-binning failed to be useful as a robust estimator. Even though the TE-kernel and TE-kraskov passed the idealized tests, their implementations had to be tuned to get consistent numerical results. Slow convergence of information exchange with the TE-kraskov estimator with increase in time series length was also noted. Therefore, we concluded that both reliable nonparametric estimators should be jointly applied and their implementation should be optimized for consistent results before any quantitative interpretation of the investigated nonlinear system is drawn. This conclusion is conditioned on the availability of long enough data time series. The composite use of TE-kernel and TE-kraskov showed that the dynamics of the Lorenz-96 model is dominated by the slow subsystem.
For real climate applications, i.e., information exchange between the Indian and Pacific Oceans, the parametric and reliable nonparametric estimators showed a significant bidirectional information exchange. Moreover, the time lag of significant information exchange from Pacific to the Indian Ocean was about 2 to 3 months. An instantaneous information exchange from the Indian to Pacific Ocean was detected and also with a time lag of about 10 to 12 months. The respective spatial patterns over the Indian and Pacific Oceans revealed a significant bidirectional information exchange. Hence, given the consistent estimations, we concludef that a bidirectional information exchange exists between the Pacific and Indian Oceans, as expected from literature [64
]. However, given the limitations of TE and IF-linear, a possibility of a hidden influence by another system cannot be ruled out. This requires further analysis.
For the relation of NAO and European winter air temperatures, the estimators showed significant bidirectional information exchange. The process mechanism from NAO to European temperature is often discussed in the literature [71
]. However, the measured information exchange from European temperatures to the NAO cannot be explained by a straightforward process chain. This indicates an influence from a third hidden variable as a common driver.
Thus, even though TE and IF-linear are useful measures which allow for quantification of interactions and their directionality, their limitations and the system at hand need to be taken into account carefully before drawing any conclusions from their estimations. Hence, we propose a composite use of the information theory methods with parameter testing for various applications, for example, as a robust model evaluation framework. While this study was limited in investigating the relationship between two systems, in the future study, the authors plan to investigate interaction measures based on information theory in higher-dimensional climate system networks.