This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

The use of transfer entropy has proven to be helpful in detecting which is the verse of dynamical driving in the interaction of two processes,

This paper is about the use of quantities, referred to as information dynamical quantities (IDQ), derived from the Shannon information [

The purpose of applying a certain data analysis technique to a completely known model is to investigate the potentiality of the analysis tool in retrieving the expected information, preparing it for future applications to real systems. The choice of the GOY model as a test-bed for the IDQ-based CPA is due both to its high complexity, rendering the test rather solid with respect to the possible intricacies expected in natural systems, and to its popularity in the scientific community, due to how faithfully it simulates real features of turbulence.

In order to focus on how IDQ-based CPA tools are applied in the study of coupled dynamical processes, let us consider two processes,

where the terms,

Through the study of the IDQs obtained from

The IDQs discussed here have been introduced and developed over some decades. After Shannon’s work [

The tools of Kantz and Schreiber were augmented in [

TE was soon adopted as a time series analysis tool in many complex system fields, such as space physics [

A very important issue is, namely, the physical meaning of the IDQs described before. Indeed, while the concept of Shannon entropy is rather clear and has been related to the thermodynamical entropy in classical works, such as [

The paper is organized as follows.

A short review of the IDQs is done in Section 2. Then, the use of MI and TE to discriminate the “driver” and the “driven” process is criticized, and new normalized quantities are introduced, more suitable for analyzing the cross-predictability in dynamical interactions of different “intrinsic” randomness (normalizing the information theoretical quantities, modifying them with respect to their initial definitions, is not a new thing: in [

The innovative feature of the IDQs described here is the introduction of a variable delay,

In Section 3, the transfer entropy analysis (TEA) is applied to synthetic time series obtained from the GOY model of turbulence, both in the form already described, e.g., in [

In all our reasoning, we will use four time series: those representing two processes,

(in _{t}

is understood.

Shannon entropy is defined for a stochastic process,

quantifying the uncertainty on _{t}

For two interacting processes,

The instantaneous MI shared by

positive _{t}

About _{t}_{t}_{t}

There may be reasons to choose to use the MI instead of, say, cross-correlation between

In the context of information theory (IT), we state that a process,

turns out to be very useful for this purpose. DMI is clearly a quantity with which cross-predictability is investigated.

In [

In practice, the TE provides the amount of knowledge added to

The easiest way to compare the two verses of cross-predictability is of course that of taking the difference between the two:

as done in [

Consider taking the difference between _{Y}_{→}_{X}_{X}_{→}_{Y}_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{τ}_{Y}_{→}_{X}_{t}_{τ}

and a normalized transfer entropy (NTE):

or equally:

These new quantities, _{Y}_{→}_{X}_{Y}_{→}_{X}

The positivity of Δ_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{t}_{τ}_{t}_{τ}_{Y}_{→}_{X}_{Y}_{→}_{X}_{0}, and the conditioned ones equal both to _{0}; clearly, one has:

and the quantity, Δ_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}

Before applying the calculation of the quantities, Δ_{Y}_{→}_{X}_{Y}_{→}_{X}

This section considers an example in which we know what must be expected, and apply our analysis tools to it to check and refine them. In this case, the application of the normalized quantities instead of the traditional ones revised in Section 2 is investigated in some detail. In the chosen example, the IDQs are used to recognize the existence of cascades in a synthetic model of fluid turbulence [

Some theoretical considerations are worth being done in advance. The quantities described above are defined using instantaneous pmfs,

The system at hand is described in [

The physical variable that evolves is the velocity of the fluid, which is assigned as a value, _{h}_{h}_{h}_{h}_{h}_{h}_{n}_{n}_{n}_{n}_{n}

_{0} being the fundamental, lowest wavenumber and _{n}_{+1} = _{n}_{n}_{n}_{+1}.

Each Fourier mode, _{n}_{h}_{n}_{n}_{4}_{,n}_{n}

where ^{*} is the complex conjugate of _{n}_{n}_{+1}, _{n}_{+2}, _{n}_{−1} and _{n}_{−2} in a non-linear way, and in addition, it possesses a linear coupling to itself via the dissipative term,
_{n}^{−3}(1+^{−7} were used. The integration procedure is the one due to Adam and Bashfort, described in [^{−4}.

Even if the lattice is 1D, the equations in (

The stirring force pumps energy and momentum into the fluid, injecting them at the fourth scale, and the energy and momentum are transferred from _{4} to all the other modes via the non-linear couplings. There is a scale for each _{n}_{n}_{n}_{0}, or of other _{n}

The quantity represented along the ordinate axis of

where [_{1}_{2}] is a time-interval taken after a sufficiently long time, such that the system (_{n}_{1} is “many times” the largest eddy turnover time.

The energetic, and informatic, behavior of the GOY system is critically influenced by the form of the dissipative term,
_{n}_{n}_{n}_{n}_{m}_{n}_{≫}_{m}_{n}_{≪}_{m}

To get the above target, and to investigate the application of TEA to the GOY model and illustrate the advantages of using the new normalized quantities discussed in Section 2, we selected three non-consecutive shells. In particular, the choice:

is made. The real parts of _{9}, _{13} and _{17} are reported in _{4}. For each of the selected shells, we considered very long time series of the corresponding energy _{n}_{n}^{2}. The typical length of the considered time series is of many (≃ 1

The quantities, Δ_{1→2} and Δ_{1→3}, and Δ_{1→2} and Δ_{1→3}, can be calculated as functions of the delay, _{i}_{→}_{j}_{i}_{→}_{j}_{i}_{j}_{1→2}, _{2→1}, _{1→3}, _{3→1} and the corresponding quantities normalized, _{1→2}, _{2→1}, _{1→3} and _{3→1}, give the results portrayed in _{#1}, that pertains to the 1 mode (with

The use of non-adjacent shells to calculate the transfer of information is a choice: the interaction between nearby shells is obvious from

All the plots show clearly that there is a direct cascade for short delays. The first noticeable difference between the transfer entropies and the normalized transfer entropies is that in the #1 ↔ #3 coupling, a non-understandable inverse regime appears after about 4_{#1}, when the “traditional” transfer entropy is used. Instead, the use of the normalized quantities suggests decoupling after long times (after about 6_{#1}). A comparison between the #1 ↔ #2 and #1 ↔ #3 interactions is also interesting: the maximum of the “direct cascade” coupling is reached at less than 0.5_{#1} for both the interactions if the TEs are used. However in the plot of _{1→2}, _{2→1}, _{1→3} and _{3→1}, some time differences appear; this is clarified when difference quantities are plotted, as in

Both the analyses diagnose a prevalence of the smaller-onto-larger wavenumber drive for sufficiently small delays: the Δ_{1→2} indicates a driving of _{9} onto _{13} (Mode #1 onto Mode #2) for _{#1}, while Δ_{1→3} indicates a driving of _{9} onto _{17} (Mode #1 onto Mode #3) for _{#1}. This is expected, due to how the system (_{1→2} (_{1→3} (_{1→2}_{1→3}_{#1}, the modes appear to become decoupled, since Δ_{1→2}_{1→3}

The misleading response of the Δ_{t}_{τ}_{n}_{9} (_{13} (_{9} (_{17} (_{m}_{n}_{1→2} (_{1→2}_{1→3}_{1→3} (

Another observation that deserves to be made is about the maxima of Δ_{1→2} (_{1→3} (_{1→2}_{1→3}_{9}_{13}), and for the interaction, (_{9}_{17}). In the plots of Δ_{1→2} (_{1→3} (_{1→2} at _{#1} and a maximum for Δ_{1→3} just slightly before this. It appears that the characteristic time of interaction of _{9} with _{13} is slightly larger than of _{9} with _{17}: this is a little bit contradictory, because of the _{9} to _{13} and then to _{17}.

What happens in the plots of Δ_{1→2}_{1→3}_{1→3}_{#1}, which comes a little bit before the maximum of Δ_{1→2}_{1→3}_{#1}, so that maybe different processes in the interaction, (_{9}_{17}), are emerging. Actually, distant wavenumbers may interact through several channels, and more indirect channels enter the play as the wavenumbers become more distant. This might explain the existence _{9}_{17}), as indicated by the plot of Δ_{1→3} (_{#1}; and a later one within the interval (1.5_{#1}_{#1}).

The differences between the TEAs performed with Δ

As far as the surrogate data test used to produce the level of confidence in ^{4} surrogate data copies have been realized, by randomizing the Fourier phases. In each of these surrogate datasets, the delayed transfer entropy was calculated. Than, a statistical analysis, with a confidence threshold of five percent, was performed. A similar level of confidence was obtained also for the results in

Mutual information and transfer entropy are increasingly used to discern whether relationships exist between variables describing interacting processes, and if so, what is the dominant direction of dynamical influence in those relationships? In this paper, these IDQs are normalized in order to account for potential differences in the intrinsic stochasticity of the coupled processes.

A process,

The normalized transfer entropy is particularly promising, as has been illustrated for a synthetic model of fluid turbulence, namely the GOY model. The results obtained here about the transfer entropy and its normalized version for the interactions between Fourier modes of this model point towards the following conclusions.

The fundamental characteristics of the GOY model non-linear interactions, expected by construction, are essentially re-discovered via the TEA of its Fourier components, both using the unnormalized IDQs and the normalized ones: the prevalence of the large-to-small scale cascade; the locality of the interactions in the _{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}_{Y}_{→}_{X}

An indication is then obtained that for the irregular non-linear dynamics at hand, the use of the TEA via Δ_{Y}_{→}_{X}

The systematic application of the TEA via Δ_{Y}_{→}_{X}

The authors declare no conflict of interest.

The time-average instantaneous power spectral density (PSD) of the velocity field of the Gledzer–Ohkitana–Yamada (GOY) model after a certain transitory regime. The modes chosen for the transfer entropy analysis (TEA) are indicated explicitly, together with the _{n}

Time series plots showing the real part of the processes, _{9} (_{13} (_{17} (_{4}, of the scale forced.

The quantities, _{1→2}, _{2→1}, _{1→3}, _{3→1}, _{1→2}, _{2→1}, _{1→3} and _{3→1} (see Section 2) calculated for the three modes of the GOY model chosen. In the case of the quantities _{a}_{→}_{b}_{#1} = _{9}. Note that transfer entropy is always positive, indicating one always learns something from observing another mode; transfer entropy decreases as

Comparison between TEA via the “traditional” transfer entropies (TEs) (^{4} surrogate data realizations, as described in the text.