Evaluating Temporal Correlations in Time Series Using Permutation Entropy, Ordinal Probabilities and Machine Learning

Abstract

Time series analysis comprises a wide repertoire of methods for extracting information from data sets. Despite great advances in time series analysis, identifying and quantifying the strength of nonlinear temporal correlations remain a challenge. We have recently proposed a new method based on training a machine learning algorithm to predict the temporal correlation parameter, $\alpha$, of flicker noise (FN) time series. The algorithm is trained using as input features the probabilities of ordinal patterns computed from FN time series, $x_{\alpha }^{FN}\left(t\right)$, generated with different values of $\alpha$. Then, the ordinal probabilities computed from the time series of interest, $x\left(t\right)$, are used as input features to the trained algorithm and that returns a value, ${\alpha }_e$, that contains meaningful information about the temporal correlations present in $x\left(t\right)$. We have also shown that the difference, $\mathrm{\Omega }$, of the permutation entropy (PE) of the time series of interest, $x\left(t\right)$, and the PE of a FN time series generated with $\alpha ={\alpha }_e$, $x_{{\alpha }_e}^{FN}\left(t\right)$, allows the identification of the underlying determinism in $x\left(t\right)$. Here, we apply our methodology to different datasets and analyze how ${\alpha }_e$ and $\mathrm{\Omega }$ correlate with well-known quantifiers of chaos and complexity. We also discuss the limitations for identifying determinism in highly chaotic time series and in periodic time series contaminated by noise. The open source algorithm is available on Github.

1. Introduction

Thanks to huge advances in data science and computing power, a wide repertoire of time series analysis methods [1,2,3,4,5,6,7] are available for the quantitative characterization of time series and are routinely used in all fields of science and technology, social sciences, economy and finance, etc. Since different methods have different requirements and involve different approximations, no single method can be expected to perform well over all types of data. Therefore, despite huge advances, extracting reliable information from stochastic or high dimensional signals remains a challenge. As any algorithm will return, at least, a number (i.e., a “feature” that encapsulates some property of the time series), in order to interpret the information in the obtained features and to assess the performance of different algorithms, appropriate surrogates [8] or a “reference model” (where the systems that generates the data are known) need to be used. The comparison of the features obtained from the time series of interest with those obtained from surrogate time series or from reference time series allows testing (and even quantifying) some particular property of the time series of interest.

We have recently proposed a new method for estimating the strength of the temporal correlations in a given time series, which uses flicker noise (FN), a fully stochastic process, as the reference model [9]. A FN time series, $x_{\alpha }^{FN}\left(t\right)$, is characterized by a power spectrum $P\left(f\right)\propto 1/f^{\alpha }$, with $\alpha$ being a parameter that quantifies the temporal correlations present in the signal [10]. The method proposed in [9] combines the use of symbolic ordinal analysis [11,12] and machine learning (ML): We utilize the ordinal probabilities computed from FN time series generated with different $\alpha$ values as input features to a ML algorithm. The algorithm is trained to return the value of ${\alpha }_{\mathrm{e}}$, which estimates the real $\alpha$ value of a FN time series from the $D!$ probabilities of the ordinal patterns of length D, calculated from the same time series. Then, after the training stage, the ordinal probabilities now computed from a time series of interest, $x\left(t\right)$, are provided as input features to the ML algorithm that returns a value, ${\alpha }_e$, which encapsulates reliable information about the strength of the temporal correlations present in $x\left(t\right)$. By calculating the difference, $\mathrm{\Omega }$, between the permutation entropy (PE) of $x\left(t\right)$ and that of a FN time series generated with $\alpha ={\alpha }_e$, $x_{{\alpha }_e}^{FN}\left(t\right)$, we were also able to identify determinism in $x\left(t\right)$.

Our approach is, thus, based on reducing a large number of features (with $D=6$, we have $D!=720$ ordinal probabilities) to only two: the ${\alpha }_e$ value returned by the ML algorithm; and the permutation entropy, $\overline{S}$, computed with the ordinal probabilities. Dimensionality reduction is a well known technique [13,14,15] that has been used to tackle a variety of problems. With ordinal probabilities, for instance, it is possible to distinguish between noise and chaos by reducing the set of probabilities to only two features—the permutation entropy and the complexity; or the permutation entropy and the Fisher information—as demonstrated in [16,17,18,19]. In our methodology, we not only apply dimensionality reduction but also use a fully stochastic “reference” FN time series: we compare the value of $\overline{S}$ of the time series of interest with that of a FN time series generated with $\alpha ={\alpha }_e$. We have shown that the entropy difference, $\mathrm{\Omega }$, may provide good contrast for distinguishing fully stochastic time series from a time series with a degree of determinism [9].

The method we proposed has in fact a large degree of flexibility because, instead of ordinal analysis and the permutation entropy, different symbolization rules [20,21] and different entropies [22,23,24] could be tested. In addition, while we use a simple artificial neural network, other algorithms could be evaluated. Different combinations may provide different results and particular combinations may result in optimized performance for the analysis of particular types of time series.

We have shown that the algorithm returns meaningful information even from time series that are very short: for the synthetic examples considered in [9], we could distinguish whether the dynamics is mainly chaotic or stochastic with only 100 data points. However, an open question is as follows: What are the limitations in terms of the level of chaos, the level of noise and the length of the time series? Here, we address this issue by using as examples the time series generated with the Logistic map, the $\beta x$ map and the Schuster map. We also address the following questions: Can we distinguish a highly chaotic time series from a stochastic one? Can we identify a periodic signal hidden by noise? In addition, to gain insight into the information encapsulated by ${\alpha }_e$ and $\mathrm{\Omega }$, we contrast them with well known quantifiers of chaos and complexity: the maximum Lyapunov exponent and the ordinal-based statistical complexity measure [25].

The organization of the paper is as follows: Section 2 describes the methodology, Section 3 presents the datasets and systems analyzed, Section 4 presents the results and Section 5 presents the discussion and our conclusions.

2. Methodology

The methodology proposed in [9] can be described in a few steps:

1.

Calculate the ordinal probabilities (OPs) of a large set of FN time series generated with different values of $\alpha$ and use them as features to train a ML algorithm to return the (known) value of ${\alpha }_{\mathrm{e}}$;

2.

Calculate the OPs of the time series of interest, $x\left(t\right)$, and use them as features to the trained ML algorithm, which returns a value ${\alpha }_e$ (see Section 2.1);

3.

Generate a FN time series with $\alpha ={\alpha }_e$ and calculate its permutation entropy (PE), ${\overline{S}}_{\mathrm{FN}}$ (see Section 2.2);

4.

Calculate the relative difference, $\mathrm{\Omega }$, between the PE of $x\left(t\right)$, $\overline{S}$ and ${\overline{S}}_{\mathrm{FN}}$:

$$\mathrm{\Omega }=\frac{|{\overline{S}}_{\mathrm{FN}}-\overline{S}|}{{\overline{S}}_{\mathrm{FN}}};$$

(1)

5.

Use the value of ${\alpha }_e$ to quantify the strength of the temporal correlations in the time series of interest and use the value of $\mathrm{\Omega }$ to identify underlying determinism: if $\mathrm{\Omega }\approx 0$, $x\left(t\right)$ is mainly stochastic, otherwise there is some determinism.

In the implementation proposed in [9], the probabilities of the 720 ordinal patterns of length $D=6$ (described in the next section) were used as input features to the ML algorithm. Then, these features were reduced to only two: the scalar value returned by the ML algorithm, ${\alpha }_e$, which quantifies the temporal correlations presented in the time series of interest $x\left(t\right)$; and the permutation entropy, $\overline{S}$ (Equation (4)). Then, the value of $\overline{S}$ was compared with the PE of a FN time series generated with $\alpha ={\alpha }_e$ and the relative difference, $\mathrm{\Omega }$ (Equation (1)), was found to provide contrast for identifying determinism in $x\left(t\right)$. The value of $\mathrm{\Omega }$ can be used to organize a set of time series according to their level of stochasticity (the lower the value of $\mathrm{\Omega }$, the larger the stochasticity level) and, by appropriately selecting a threshold value, $\mathrm{\Omega }$ can be used to classify the time series into two categories: mainly stochastic and mainly deterministic.

2.1. Machine Learning Algorithm

A wide range of ML algorithms are available nowadays. Since we want to regress the information of the features ($D!$ probabilities) into one real value, ${\alpha }_e$ (a classical scalar regression problem), an appropriate simple option is a feed forward artificial neural network (ANN). Mathematically, the ANN can de described as follows. Considering a set of inputs $\mathbf{X}$ of $D!$ features with output ${\alpha }_{\mathrm{e}}$, the ANN can be sketched by the following:

$$\begin{array}{ccc}\hfill {\mathbf{X}}^{\prime }& =& f(\mathbf{W}\ast \mathbf{X}+\mathbf{b}),\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill {\alpha }_{\mathrm{e}}& =& f^{\prime }({\mathbf{W}}^{\prime }\ast {\mathbf{X}}^{\prime }+{\mathbf{b}}^{\prime }),\hfill \end{array}$$

(3)

where $\mathbf{W}$ and ${\mathbf{W}}^{\prime }$ are matrices of weights, $\mathbf{b}$ and ${\mathbf{b}}^{\prime }$ are biases column vectors, f and $f^{\prime }$ correspond to activation functions and the “*” symbols corresponds to a tensor product. In this sense, ${\mathbf{X}}^{\prime }$ is the result of the transformation $f(\mathbf{W}\ast \mathbf{X}+\mathbf{b})$, which can be understood as a new representation of the inputs $\mathbf{X}$. The elements of the tensors $\mathbf{W},{\mathbf{W}}^{\prime },\mathbf{b},{\mathbf{b}}^{\prime }$ are the parameters of the ANN, which are calibrated in the training state. ANNs are well known, and we refer the reader to our previous work [9] for details about the network structure and the training procedure. We remark that our ANN is a fast and automatic tool and it performed well in all the cases we tested, with a computational time and in a standard notebook, of a few seconds for the analysis of time series with ${10}^6$ data points. However, we do not claim that an ANN is an optimal choice, since different ML algorithms may be even more efficient. It is important to notice also that the optimal choice will likely depend on the characteristics of the time series (length, frequency content, level of noise, etc.).

2.2. Ordinal Analysis and Permutation Entropy

Ordinal analysis and the permutation entropy were proposed by Bandt and Pompe [11] almost 20 years ago and they are now well-known. For their interdisciplinary applications, we refer the reader to a recent Focus Issue [12].

Here, we compute the ordinal patterns of length $D=6$ with the algorithm proposed in [26]. For $D=6$, there are $6!=720$ possible patterns. The patterns are calculated with an overlap of $D-1$ data points, i.e., for a time series with N data points, $N-D$ ordinal patterns are obtained, which are then used to evaluate the ordinal probabilities $\mathcal{P}=\left\{\mathcal{P}\right(i);i=1,\cdots ,D!\}$. Then, the normalized permutation entropy is calculated as follows.

$$\overline{S}=-\frac{{\displaystyle \sum _{i=1}^{D!}}\mathcal{P}\left(i\right)ln\left[\mathcal{P}\right(i)]}{ln[D!]}.$$

(4)

2.3. Quantifiers of Chaos and Complexity

The Lyapunov exponents measure the local divergence of infinitesimally close trajectories, and positive exponents indicate chaos [27]. While there are important challenges when calculating them from high-dimensional and/or noisy data [3,28], in the case of a one-dimensional dynamical system for which its governing equation is known, the Lyapunov exponent, $x_{n+1}=f\left(x_n\right)$, can be straightforwardly calculated as follows.

$$\lambda =\underset{n\to \infty }{lim}\frac{1}{n}\sum _{n}ln\left[\right|f^{\prime }\left(x_n\right)\left|\right].$$

(5)

A popular measure for characterizing complex systems is the statistical complexity measure [25] that takes into account the distance to the most regular state and the most random states of the system. It is defined as follows:

$$C=\overline{S}\left(\mathcal{P}\right)Q(\mathcal{P},{\mathcal{P}}_e),$$

(6)

where $\overline{S}$ is the entropy and Q is the distance between the distribution that describes the state of the system, $\mathcal{P}$, and the equilibrium distribution, ${\mathcal{P}}_e$, that maximizes the entropy.

As we use the probabilities of the ordinal patterns [16,29], $\overline{S}$ is the normalized permutation entropy and ${\mathcal{P}}_e=\{1/D!,1/D!,\cdots ,1/D!\}$. The distance between $\mathcal{P}$ and ${\mathcal{P}}_e$ is calculated with the Jensen–Shannon divergence:

$$Q(\mathcal{P},{\mathcal{P}}_e)=Q_0\left\{S\left(\frac{\mathcal{P}+{\mathcal{P}}_e}{2}\right)-\frac{S\left(\mathcal{P}\right)}{2}-\frac{S\left({\mathcal{P}}_e\right)}{2}\right\},$$

(7)

where $Q_0$ a normalization factor $Q_0=-2{\left\{\left(\frac{D!+1}{D!}\right)ln[D!+1]-2ln[2D!]+ln[D!]\right\}}^{-1}$.

3. Datasets

We analyze time series generated by the following stochastic or deterministic dynamical systems. Typical time series examples are presented in Figure 1.

3.1. Flicker Noise

Flicker noise (FN), also called as colored noise, is used for training the ML algorithm. FN time series are stochastic and characterized by a temporal correlation coefficient $\alpha$. The power spectrum of this signal is given by $1/f^{\alpha }$, and different $\alpha$ values result in different “colors”. We used the open Python library colorednoise.py [30,31] to generate FN time series with different $\alpha$ values. An example is shown in panel (e) of Figure 1.

3.2. Uniform Noise

Uniform noise is a stochastic process with no memory and uniform distribution in $[0,1]$. Panels (a) and (b) of Figure 1 depict an example of a uniformly distributed white noise and its PDF, respectively.

3.3. Random Walk

A one-dimensional random walk time series is defined by $x_{n+1}=x_n+{ϵ}_n$, where $ϵ$ is a memory-less stochastic Gaussian process with mean 0 and standard deviation 1.

3.4. Periodic or Chaotic Signals Contaminated by Noise

In order to generate time series that represent periodic or chaotic signals contaminated by noise, we used the map equation $Z_n=(1-\eta )X_n+\eta Y_n$. For a periodic signal, $X_n$ is the sin map, $X_{n+1}=sin(2\pi n/\tau )$ with period $\tau$; for a chaotic signal, $X_n$ is the $\beta x$ map (see below). In both cases, $X_t$ is normalized $[0,1]$, $Y_t$ is a uniform white noise and $\eta \in [0,1]$ controls the stochastic component of $Z_t$; for $\eta =0$, the signal is fully deterministic (periodic or chaotic depending on the map used), while for $\eta =1$ the signal is fully stochastic and memory-less.

3.5. Logistic Map

The Logistic map is a popular nonlinear dynamical system defined by $x_{n+1}=r{x}_n(1-x_n)$, where r is the control parameter that allows us to obtain periodic or chaotic signals.

3.6. $\beta x$ Map

The generalized Bernoulli chaotic map, also known as $\beta x$ map, is defined by $x_{n+1}=\beta x_n\left(\mathrm{mod}1\right)$, where $\beta$ controls the dynamical characteristic of the map. Panel (c) and (d) of Figure 1 depict the evolution of a $\beta x$ map with $\beta =2$ and its PDF, respectively. Here, we observe that it cannot be visually distinguished from uniform noise; however, the level of chaos can be estimated with the Lyapunov exponent, which has an exact solution in this case: $\lambda =ln\left(\beta \right)$ [27]. Therefore, the higher the $\beta$ parameter, the more chaotic the signal is.

3.7. Schuster Map

The Schuster map [32] generates intermittent signals with a power spectrum $P\left(f\right)\sim 1/f^z$ and is defined as $x_{n+1}=x_n+x_n^z,\left(\mathrm{mod}1\right)$, where we used z as a parameter control.

3.8. Lorenz System

The Lorenz system is also a well-known dynamical system [33], defined by three rate equations: $dx/dt=\sigma (y-x)$, $dy/dt=x(R-z)-y$ and $dz/dt=xy-bz.$ Here we use typical parameters $\sigma =16$, $R=45.92$ and $b=4$ that generate chaotic trajectories.

3.9. Rossler System

The Rossler system [34] is also a well-known dynamical system, defined by the equations $dx/dt=-y-z$, $dy/dt=x+ay$ and $dz/dt=b+z(x-c)$. Here, we used typical parameters for generating chaotic trajectories: $a=b=0.2$ and $c=5.7$.

3.10. Three Waves System

This system is composed of three first-order autonomous differential equations written in terms of the complex slowly varying wave amplitude [35] $d{C}_1/dt={\gamma }_1{C}_1+C_2{C}_3$, $d{C}_2/dt={\gamma }_2{C}_2-C_1{C}_3^{\ast }+i\delta C_2$ and $d{C}_3/dt={\gamma }_3{C}_3-C_1{C}_2^{\ast }$, where $C_j\in \mathbb{C}$ and we used ${\gamma }_1=1$, ${\gamma }_2={\gamma }_3=-17$ and $\delta =2$. The chaotic dynamics can be observed at the amplitude variations of $|C_1|$, which is depicted in panel (f) of Figure 1.

3.11. Hindmarsh–Rose Model

The Hindmarsh–Rose model [36] is a set of three rate equations that models the neural activity. An example is shown in panel (g) of Figure 1. The equations are $dx/dt=y-a{x}^3+b{x}^2-z+I$, $dy/dt=c-d{x}^2-y$ and $dz/dt=r[s(x-x_r)-z]$ and the parameters used are $a=c=1$, $b=3$, $s=4$, $x_r=-8/5$ and $r=0.006$.

4. Results

A demonstration of the methodology is presented in Figure 2 that shows, in the plane ($\overline{S}$, ${\alpha }_{\mathrm{e}}$), the values obtained from time series generated with the dynamical systems described in the previous section. All the time series analyzed possesses $N=2^{14}$ points and the error bars represent the standard deviation over 1000 time series generated with different initial conditions or noise seeds.

Figure 2a presents results for discrete systems, while Figure 2b, for continuous systems. In both panels the black line represents FN signals generated with different values of $\alpha$, which are perfectly recovered by the ANN (that returns a value ${\alpha }_{\mathrm{e}}$ equal to $\alpha$). As expected, for ${\alpha }_{\mathrm{e}}=0$, $\overline{S}\approx 1$ since the FN signal is white noise [37]. For ${\alpha }_{\mathrm{e}}>0$, some ordinal patterns occur in the time series more frequently than others, and the value of $\overline{S}$ decreases.

In panel (a), the orange circle represents time series of uniform white noise, and the ANN returns the correct value ${\alpha }_{\mathrm{e}}=0$ (There is almost no dispersion in the returned value of ${\alpha }_{\mathrm{e}}=0$, therefore, the error bar is not shown). The red diamond represents random walk signals; for them, the ANN returns ${\alpha }_{\mathrm{e}}\approx 1.75$ (Again, there is almost no dispersion in the value of ${\alpha }_e$). We observe that the red diamond is very close to the black line, providing a clear indication of the highly stochastic nature of a time series generated by a one-dimensional random walk.

On the other hand, when we analyze chaotic signals (time series from $\beta x$ map with $\beta =2$, from logistic map with $r=4$ and from Schuster map with $z=0.5$), we observe that the distance between the symbols and the FN noise curve (black line) allows identifying the signals as not fully stochastic, i.e., the distance to the FN curve uncovers determinism in the signals.

For continuous dynamical systems, the results obtained with ordinal analysis strongly depend on the lag time between the data points, which can be performed in multiple ways, for instance using the maxima of each variable or even the first minimum of the mutual information [38]. While this dependence with the lag allows identifying different time scales in a complex signal [39,40], it renders it difficult to compare different signals. Therefore, for the continuous dynamical systems considered (Lorenz, Rossler, 3-waves and Hindmarsh–Rose), the time series analyzed here are the sequence of maxima of each variable of the system. Due to the fact that the variables obey different equations and have oscillations with different properties, we can expect obtaining different values of ${\alpha }_e$. Despite a large dispersion in the values obtained, it can be observed in Figure 2b that all the systems depict a substantial distance to the FN curve, clearly revealing that they are not fully stochastic. The dispersion in the permutation entropy values is of the order of 1%; therefore, vertical error bars are not shown.

4.1. Comparison with Standard Quantifiers

Next, we compare, for the three chaotic maps, the quantifiers obtained with our approach, ${\alpha }_{\mathrm{e}}$ and $\mathrm{\Omega }$, with well-known quantifiers of chaos and complexity including the Lyapunov exponent, $\lambda$ and the ordinal statistical complexity, C (described in Section 2.3).

The results are depicted in Figure 3. Panels (a), (b) and (c) show results for the $\beta x$ map. For $\beta <1$, the map is not chaotic since $\lambda =ln\left(\beta \right)<0$. In this range, $\mathrm{\Omega }$ decreases while ${\alpha }_{\mathrm{e}}$ remains constant. The $\beta x$ map is chaotic for $\beta >1$, since $\lambda >0$. At $\beta =1$, $\mathrm{\Omega }$ and ${\alpha }_{\mathrm{e}}$ vary abruptly and both decreases as $\beta$ increases. There is negative correlation between $\overline{S}$ and $\mathrm{\Omega }$ and also between $\lambda$ and $\mathrm{\Omega }$. If $\beta$ is too large, $\mathrm{\Omega }\approx 0$ and $\overline{S}\approx 1$ determinism can no longer be identified. We note that the small oscillations of ${\alpha }_{\mathrm{e}}$ capture the changes in dynamics when $\beta$ approaches an integer number since the PDF of $\beta x$ is homogeneous for integer $\beta$ but becomes inhomogeneous for non-integer values [41], which is not clearly observed in the other quantifiers.

In panels (d), (e) and (f) for the logistic map, periodic windows embedded in chaotic regions are detected in the Lyapunov exponent, and the quantifiers $\overline{S}$, C and ${\alpha }_{\mathrm{e}}$ also show abrupt variations. We note that $\mathrm{\Omega }$ identifies the deterministic nature of the signals since, for the entire interval of r, $\mathrm{\Omega }$ remains well above zero and $\overline{S}$ remains well below one.

Similar results are found for the Schuster map, panels (g), (h) and (i): $\mathrm{\Omega }$ is anti-correlated with $\lambda$ and confirms that the signal is always deterministic since relatively large $\mathrm{\Omega }$ values are observed for the entire interval of z. Even though a signal generated by the Schuster map has the same power spectrum as Flicker noise, its ${\alpha }_{\mathrm{e}}$ value varies non-monotonically with z [32], contrary to the line $\alpha ={\alpha }_e$ that is obtained from FN signals. This is due to the fact that signals generated by the Schuster map and FN signals have different sets of ordinal probabilities.

4.2. Influence of the Length of the Time Series

The results so far indicate that our methodology is very precise for characterizing chaotic and stochastic signals when long times series are analyzed (in Figure 2 and Figure 3 $N=2^{14}$). However, an important question is the following: What is the role of the time series size in the analysis? In order to address this issue, we investigate both stochastic and chaotic time series with different lengths N. Figure 4 displays the role of time series length N in $\overline{S}$ (a), in the output of the ANN ${\alpha }_{\mathrm{e}}$ (b) and in $\mathrm{\Omega }$ (c) for chaotic and for stochastic signals. Here, chaotic time series are represented by signals generated with the $\beta x$ map, while stochastic ones are given by uniform white noise and a random walk.

One can observe that, in general, for a small number of data points ($N<100$), our method is unable to distinguish the signals as all of them are overlapping. Interestingly, for $N\approx 100$, our method is able to distinguish the chaotic signal generated by the $\beta$ map with $\beta =2$ (green lines), since $\mathrm{\Omega }$ depicts values that are higher than those of the other signals. As N is increased, the signal of $\beta =3$ is also separated and characterized as chaotic. Surprisingly, for $N<D!=720$ (the number of features), we are already able to distinguish the chaotic signals of $\beta =2$ and $\beta =3$ from the stochastic ones (uniform and random walk) by using the quantifier $\mathrm{\Omega }$. The results improve as the number of points increases and, for $N\ge {10}^4$, the analysis stabilizes. Here, an important point is that even if the chaotic and stochastic signals have similar permutation entropy values ($\overline{S}$), $\mathrm{\Omega }$ is able to capture their difference. On the other hand, if the chaoticity of the signal is too high, as observed for $\beta =10$ (purple lines), $\overline{S}\approx 1$. In this case the ordinal probabilities of the signal are as uniform as those of white noise, which means that we are unable to identify determinism.

Figure 5 shows that the above observations remain robust when noise is added to the chaotic signal. As explained in Section 3.4, we control the amount of noise in the analyzed time series by varying the parameter $\eta$. If the time series is too long and has too much noise, $\mathrm{\Omega }$ does not identify determinism (blue region). On the other hand, if the time series is not long, $\mathrm{\Omega }$ identifies determinism even when there is no determinism for $\eta =1$ (orange region). Therefore, for a correct interpretation of the information contained in the value of $\mathrm{\Omega }$, we need to compare it with the value of $\overline{S}$ obtained from “reference” time series of the same length as the time series of interest, generated by a known stochastic process such as Flicker noise.

4.3. Analysis of Periodic Signals Contaminated by Noise

Another challenge to our methodology is the identification of periodic signals with added noise. The results obtained when varying the parameter $\eta$ (see Section 3.4) are depicted in Figure 6, where panel (a) shows the temporal correlation ${\alpha }_{\mathrm{e}}$ returned by the ANN and panel (b) shows the quantifier $\mathrm{\Omega }$, both as a function of $\eta$. Here, the periodic signals with $N=2^{14}$ points and different frequencies $\omega$ are analyzed.

First, one can observe that, for $\eta =0.0$, all signals are characterized as deterministic (high value of $\mathrm{\Omega }$) with a high temporal correlation (high value of ${\alpha }_{\mathrm{e}}$), which is expected from a periodic signal. Secondly, as we can also expect for $\eta =1.0$, all signals are identified as stochastic and memory-less (zero temporal correlation), since the added noise is white. However, for intermediate values of $\eta$, an interesting behavior is observed. The frequency of the signal is indeed important, because periodic signals with low frequencies are characterized as stochastic time series for lower values of $\eta$ (panel (b)). For instance, for $\omega <{10}^{-2}$, the signal is characterized as noisy even for very small values of $\eta$, while for signals with $\omega \approx {10}^{-1}$, the deterministic nature of the original time series is kept for $\eta <0.35$. As expected, the temporal correlation of the signals ${\alpha }_{\mathrm{e}}$ (panel (a)) also depends on the frequency $\omega =2\pi /\tau$. As $\eta$ is increased, the higher the frequency of the signal, the slower the way ${\alpha }_{\mathrm{e}}\to 0$ (i.e., high-frequency signals loss memory slower than low-frequency signals). Since the noise being added is white and thus, is memory-less, one can expect that ${\alpha }_{\mathrm{e}}\to 0$ and $\mathrm{\Omega }\to 0$ for $\eta \to 1$. However, we see that the temporal correlation ${\alpha }_{\mathrm{e}}$ can be high even when $\mathrm{\Omega }$ is very small. Therefore, these signals are first identified as noisy with non-zero time correlation, and, subsequently, they are identified as noisy with no time correlation.

These results and conclusions are robust for different realizations of the noise, as is shown in Figure 7, which depicts curves with fixed values of $\omega$ (upper row) as a function of $\eta$. Panel (a) shows the temporal correlation coefficient ${\alpha }_{\mathrm{e}}$, while panel (b) shows the quantifier $\mathrm{\Omega }$. Following the same idea, the lower row shows three examples with fixed $\eta$ values as a function of the periodic signal frequency $\omega$. In all cases, the error bars represent the dispersion over 1000 different noise realizations, and we can observe that the trends discussed previously persist over different noise realizations.

4.4. Analysis of Two Stochastic Processes

In experimental stochastic systems, two (or more) stochastic dynamics are often present. In order to test the performance of the algorithm in this situation, we consider the same approach as in the previous section, $Z_n=(1-\eta )X_n+\eta Y_n$, but now $X_n$ is not a periodic signal but a Flicker noise. The results are depicted in Figure 8, where panels (a) and (b) present ${\alpha }_{\mathrm{e}}$ and $\mathrm{\Omega }$, respectively, and are calculated from 1000 FN time series with $\alpha =0$ (purple), $\alpha =1$ (brown) and $\alpha =2$ (pink). For $\eta =0$ (FN noise) ${\alpha }_e=\alpha$, as $\eta$ increases due to the influence of the uniform white noise, ${\alpha }_e$ decreases to 0 and $\alpha =2$ (pink line) decays faster due to the slower dynamics, while $\alpha =0$ (purple) does not change since there is no time correlation in both $X_n$ and $Y_n$. $\mathrm{\Omega }$ is extremely low for all cases ($<0.01$) identifying the full stochasticity of the time series.

5. Discussions and Conclusions

We have analyzed stochastic and deterministic time series using an algorithm [9] that automatically reduces the dimensionality of the feature space from 720 probabilities of ordinal patterns to 2 features: the degree of stochasticity ($\mathrm{\Omega }$) and the strength of the temporal correlations (${\alpha }_{\mathrm{e}}$). We have analyzed the performance and limitations of the algorithm, presenting applications to different datasets, including highly chaotic and periodic signals with added white noise.

For the analysis of chaotic time series, we have shown that ${\alpha }_{\mathrm{e}}$ and $\mathrm{\Omega }$ are able to capture the rich dynamics that chaotic systems can depict, where the transitions between periodic windows and chaos are evident. In general, negative correlation between the Lyapunov exponent and $\mathrm{\Omega }$ and between the permutation entropy and $\mathrm{\Omega }$ were found. For highly chaotic signals, when the time evolution of the system is very similar to a stochastic process, $\overline{S}\approx 1$ and $\mathrm{\Omega }\approx 0$ and our methodology characterizes highly chaotic signals as stochastic ones.

In addition, we have studied periodic signals contaminated with noise. In this case, our method captures the transition from deterministic time series to stochastic ones. However, we have shown that the period of the signal is indeed important, with fast signals being identified as deterministic even with large noise. We have found that when the noise contamination increases, periodic signals lose their deterministic feature but preserve a nonzero temporal correlation.

For future work, it will be interesting to analyze whether the performance of our methodology can be improved by using different lengths of the ordinal pattern, D, or different lags between the data points that define the ordinal patterns. It is well known that the ordinal patterns distribution varies with the time scale of the analysis [19] and it will also be interesting, for future work, to address the relevant question of whether our method is able to estimate, from the same time series, different values of ${\alpha }_{\mathrm{e}}$ by using different lags.

We remark that the automatic and easy-to-use time series analysis tool that we propose is here is freely available at [42]. We believe that it will be a valuable contribution to the wide repertoire of time series analysis tools that are available nowadays. Many applications are foreseen. As an example, for ultra-fast optical random number generation [43,44,45], it is crucial to generate optical chaotic signals that are as uncorrelated and as ‘‘pseudorandom’’ as possible, for which its deterministic nature is hidden by noise-like properties. Many different setups have been proposed to generate such broad-band, high-entropy optical signals [46,47,48,49,50]. Our algorithm allows an automatic comparison of the strength of the correlations and the level of randomness of signals generated by different setups. Moreover, the algorithm may be used to identify, in a given experimental setup, the optimal operation conditions and parameters that produce optical signals with the lowest temporal correlations (lowest ${\alpha }_e$) or with highest level of randomness (lowest $\mathrm{\Omega }$).