Including the Magnitude Variability of a Signal in the Ordinal Pattern Analysis

Abstract

One of the most popular and innovative methods to analyse signals is by using Ordinal Patterns (OPs). The OP encoding is based on transforming a (univariate) signal into a symbolic sequence of OPs, where each OP represents the number of permutations needed to order a small subset of the signal’s magnitudes. This implies that OPs are conceptually clear, methodologically simple to implement, and robust to noise, and that they can be applied to short signals. Moreover, they simplify the statistical analyses that can be carried out on a signal, such as entropy and complexity quantifications. However, because of the relative ordering, information about the magnitude of the signal at each timestamp is lost—this being one of the major drawbacks of this method. Here, we propose a way to use the signal magnitudes discarded in the OP encoding as a complementary variable to its permutation entropy. To illustrate our approach, we analyse synthetic trajectories from logistic and Hénon maps—with and without added noise—and real-world signals, including intracranial electroencephalographic recordings from rats in different sleep-wake states and frequency fluctuations in power grids. Our results show that, when complementing the permutation entropy with the variability in the signal magnitudes, the characterisation of these signals is improved and the results remain explainable. This implies that our approach can be useful for feature engineering and improving AI classifiers, as typical machine learning algorithms need complementary signal features as inputs to improve classification accuracy.

1. Introduction

Since the beginning of the century, the boundaries of data mining have been pushed due to the growing ability to obtain larger and more precise data sets. With increasing data availability, we need to improve how we extract, manage, and analyse data [1] to uncover the underlying mechanisms that generate the data or to quantify its uncertainty.

An entropy measures the average contents of the information, where information is understood as the degree of uncertainty in an outcome (as defined by Shannon [2]). If an outcome is highly unlikely to happen, then it carries significant information because it would be surprising to record it, such as the presence of an outlier or an artefact in a signal. However, if an outcome is highly likely to happen, then it carries insignificant information because one would expect it to appear, such as a periodic signal. Hence, entropy is highest when any outcome is equally likely to happen, corresponding to a uniform probability distribution that conveys the maximum uncertainty regarding all possible outcomes [2].

One of the most successful entropy methods introduced to characterise signals is the permutation entropy [3]. Permutation entropy quantifies the average content of information in an Ordinal Pattern (OP) sequence, which is obtained from the signal by dividing it into a series of embedded vectors [3,4]. Each OP represents the number of permutations needed to order the signal magnitudes within each embedded vector. The resultant symbolic sequence is used to find the OP probabilities distribution. It is easier to perform statistical quantifications—known as OP analysis—with these probabilities than from the original signal, such as quantifying the uncertainty and complexity of the signal [5,6,7].

Due to its simplicity and robustness to noise, OP analysis (along with complexity calculations) has had remarkable success [8,9,10], being used to distinguish between chaotic and stochastic signals [11,12,13,14,15,16,17,18], as well as to characterise electrophysiological signals [19,20,21,22,23,24,25,26], laser dynamics [27,28,29,30,31,32,33,34], climate systems [35,36,37,38,39,40,41,42], and financial trends [43,44,45,46,47,48,49,50], to name a few. However, one of the main drawbacks in OP analysis is that the magnitude of the signal at each timestamp is discarded, solely keeping the ordinal relationship between the signal magnitudes.

To include this missing information, previous works have proposed modifications of the permutation entropy, such as modified permutation-entropy [51], weighted-permutation entropy [52], amplitude-aware permutation entropy [53], improved permutation entropy [54], and continuous ordinal patterns [55]. These methods and approaches introduce ad hoc assumptions that are supported by the effectiveness of the resultant modification to the permutation entropy measure in improving the characterisation of different datasets, but lack a theoretical framework that can validate their usage, limits, and scalability in general scenarios.

Here, we propose an alternative approach, which is to include the standard deviation of the signal magnitudes in the OP embedded vectors as a complementary variable to the permutation entropy—specifically, the OP-averaged logarithm of the standard deviation of the magnitudes in the ordinal pattern embedded vectors. The formalism behind our approach is justified in the works of Politi [56,57], who showed that this OP-averaged standard deviation of the signal magnitudes is needed—along with the information dimension of the system [58]—to make the permutation entropy of a signal tend towards its Kolmogorov–Sinai (KS) entropy [59,60]. KS entropy is a rigorously defined observable with invariant characteristics, contrary to permutation entropy, which can depend on the signal length and embedding choice. This provides fundamental ground to our approach, which, instead of modifying the permutation entropy, evaluates two easily accessible contributions to the Kolmogorov–Sinai entropy.

Because this tendency towards Kolmogorov–Sinai entropy [59,60] is only achieved when using the information dimension of the signal (which is tricky to find in finite real-world signals), we propose using this OP-averaged quantity of the standard deviation as a complementary variable to the permutation entropy value instead of a measure that combines both, discarding the need to find the information dimension and increasing the explainability of our results. In particular, we show that signal characterisation can be improved when using these standard deviations to complement the permutation entropy analysis, where we focus mostly on calculating the Rényi min-entropy [61]. Our conclusions are based on analysing numerically generated trajectories from coupled logistic [62,63,64] and Hénon [65] maps (with and without observational noise), real-world signals from intracranial electroencephalographic recordings of rats in different sleep–wake states, and frequency fluctuations from 4 locations in the European power grid.

2. Materials and Methods

2.1. Signals: General Notions

We only consider digital signals, i.e., where time is discrete and the magnitudes are quantised. These signals can be numerically generated (synthetic) or experimentally measured, where their digital nature is due to the precision of the computer or to analogue-to-digital converters, respectively. The signals we analyse come from a pair of coupled logistic maps, a Hénon map,—both being synthetic bi-variate trajectories—intracranial electroencephalographic (EEG) recordings from rats and frequency recordings at 4 locations in the European power grid

We write a signal as ${\left\{x_t\right\}}_{t=1}^T=\{x_1,x_2,\dots ,x_T\}$, where $x_t$ is the magnitude at the discrete time index $t\in \mathbb{N}$, $x_1$ is the initial state, and T is the length of the signal. A signal can be resampled using an embedding delay $\tau \in \mathbb{N}$, such that $\{x_1,x_2,\dots ,x_T\}\mapsto \{x_1,x_{1+\tau },\dots ,x_{1+n\tau }\}$, where $n=\lfloor (T-1)/\tau \rfloor$ is the smallest integer closest to $(T-1)/\tau$. This resampling can filter the high frequencies in a signal, but we set $\tau =1$ for all our analyses.

2.2. Synthetic Models: Map Iterates

We generate bi-variate signals from coupled, identical, logistic maps by iterating the following equations:

$$\left\{\begin{array}{cc}{x}_{t+1}=\hfill & \left(1-\epsilon \right)f\left(x_t\right)+\epsilon f\left(y_t\right),\hfill \\ y_{t+1}=\hfill & \left(1-\epsilon \right)f\left(y_t\right)+\epsilon f\left(x_t\right),\hfill \end{array}\right.$$

(1)

where $f\left(z\right)=rz(1-z)$ is the logistic mapping (with $z=x_t$ or $y_t$ for $t=1,\dots ,T$), $r\in (3,4]\subset \mathbb{R}$ is the control parameter, and $\epsilon \in [0,1]\subset \mathbb{R}$ is the coupling strength between the maps. When $\epsilon =0$ in Equation (1), the x and y maps are decoupled, i.e., they are isolated. As r is increased from $r=3$ to $r=4$, an isolated logistic map undergoes a series of period-doubling bifurcations, taking the solutions from periodic to chaotic trajectories [62]. When $0<\epsilon \le 1$, the maps are coupled and the resultant trajectories can become more complex (including intermittent and hysterical behaviours) [63,64].

The Hénon map is given by [65]

$$\left\{\begin{array}{cc}{x}_{t+1}=\hfill & 1-a{x}_t^2+y_t,\hfill \\ y_{t+1}=\hfill & b{x}_t,\hfill \end{array}\right.$$

(2)

where a and b are the control parameters, which, depending on their values, can generate periodic (e.g., when $a=1.0$ and $b=0.3$) or chaotic (e.g., when $a=1.4$ and $b=0.3$) trajectories.

For both maps (Equations (1) and (2)), we use the OP analysis of the x component for different control parameter values (i.e., r, $\epsilon$, and a), fixing its length to $T={10}^6$ after removing a transient of $\delta t={10}^3$ iterations from the initial condition $x_1=0.65$, $y_1=0.44$. In this way, we discard the bi-variate nature of these maps and focus on uni-variate signals. In Appendix A, we show the results when we analyse the y component instead. To analyse the effects of observational noise, we add white Gaussian noise to these signals by independently drawing identically distributed random numbers from a normal distribution and changing its strength (i.e., its standard deviation).

2.3. Animal Model: EEG Recordings

We use EEG recordings from 11 healthy and freely moving Wistar rats during their natural sleep–wake cycle, who can access food and water within the (sound-attenuated and Faraday-shielded) recording box. These rats have intracranially implanted electrodes, monitoring active wakefulness (AW), rapid eye movement (REM) sleep, and non-REM sleep. Details on the surgical procedure and experimental conditions can be found in Refs. [20,21,22]. The experiments are in agreement with Uruguay’s National Animal Care Law (No. 18611) and with the “Guide to the care and use of laboratory animals” [66]. These experiments were approved by the Institutional Animal Care Committee (Comisión de Etica en el Uso de Animales), Exp. No. 070153-000332-16.

The EEG signals we analyse are obtained by obtaining the differences between the electrode of interest and the Cerebellum (the reference). We focus on five electrodes, those bilaterally placed above the primary motor (M1) and somatosensory (S1) cortices, plus the right olfactory bulb (OB), discarding the two electrodes from the secondary visual (V2) cortex (because they are too close to the Cerebellum, increasing the relevance of observational noise). These EEGs have a sampling frequency of 1024 Hz and a resolution of 16 bits. To remove the degeneracies in the signal magnitudes due to the analogue-to-digital converter, we add white noise with an amplitude given by the range of the (electrode-dependent) EEG times $2^{-16}$.

AW is defined by low-voltage fast waves in M1, a strong theta rhythm in S1 (4–7 Hz), and relatively high electromyographic activity. REM sleep is defined by low-voltage fast-frontal waves, a regular theta rhythm in S1, and silent electromyography (excluding occasional twitches). NREM sleep is determined by the presence of high-voltage slow-cortical waves (1–4 Hz), sleep spindles in M1 and S1, and a reduction in electromyographic amplitudes. Additionally, a visual scoring is performed to discard artefacts and transitional states. The EEGs for these states are concatenations of artefact-free 10 s windows that meet each state criteria. To analyse them, we fix their lengths to $T=90\times 1024$, which is the shortest length of the concatenated EEGs.

2.4. Grid Model: Frequency Recording

We consider voltage frequency recordings at 4 different locations in the European synchronous electric power grid [67]. Under normal operations, the frequency of the grid fluctuates at around 50 Hz, due to the mismatch between power production and consumption. These fluctuations impact all the buses in the grid that follow the same overall variation around 50 Hz. On top of this common trend, additional signals coming from the grid dynamics or the spreading of disturbances might impact each bus differently. Two of the recordings were made in Germany, one in Karlsruhe, and the other in Oldenburg. The two other recordings were from Lisbon, Portugal, and Istanbul, Turkey. The overall datasets consist of 4 time-series of about 41 days of synchronous frequency recordings sampled at 1 Hz between July and August, 2019 [67]. For our analysis, we chose five different 24 h intervals, each of length T = 86,400.

2.5. Method: Encoding Signals into Ordinal Patterns

We followed Bandt–Pompe’s method [3] to encode uni-variate signals.

First, we divide the signal into quasi-non-overlapping vectors with D components, where $D>1$ is a natural number known as the embedding dimension. That is, we transform the signal ${\left\{x_t\right\}}_{t=1}^T$ to a series of vectors $\{x_1,\dots ,x_D\}$, $\{x_D,\dots ,x_{2D-1}\}$, $\{x_{2D-1},\dots ,x_{3D-2},\dots ,\{x_{(m-1)(D-1)+1},\dots ,x_{m(D-1)}\}$, where $m=\lceil T/(D-1)\rceil$ is the largest integer closest to $T/(D-1)$. Then, we transform each vector into an integer, ranging from 1 to $D!$, according to the number of permutations needed to order their elements in an increasing fashion (plus 1). These are known as Ordinal Patterns (OPs), and the overall process encodes the signal into a symbolic sequence that preserves the local relationships between consecutive time-points but discards their magnitudes.

For example, when $D=2$, if $x_1<x_2$, then $\{x_1,x_2\}\mapsto 1$, because no permutations are needed to order the two-element vector. If $x_1>x_2$, then $\{x_1,x_2\}\mapsto 2$, because we need one permutation. In total, there are $D!=2$ possible permutations between the components of the $D=2$ embedded vector. We encode all our signals using $D=3,4$, or 5, which implies symbolic sequences with a maximum of $D!=6,24$, or 120 different OPs, respectively. For the length $T=90\times 1024$ of the EEG signals, the average frequency of the appearance of any given OP when $D=5$ is approximately $T/D!=768$ (being higher for $D<5$). A similar number is obtained for the power grid frequency recordings. Consequently, we have high statistical power when finding the marginal probability distribution of the OPs and the other statistical measures.

2.6. Statistical Measures: Permutation Entropy, Rényi Min-Entropy, and Magnitude Variability

The information content of an OP is $log\left[1/P\left(\alpha \right)\right]$ (as defined by Shannon [2]), where $P\left(\alpha \right)$ is the probability of having OP $\alpha$ (such that ${\sum }_{\alpha =1}^{D!}P\left(\alpha \right)=1$). The Shannon entropy [2] of the OP sequence, H, is known as the permutation entropy [3], and is found from

$$H=\sum _{\alpha =1}^{D!}P\left(\alpha \right){log}_2\left[{\displaystyle \frac{1}{P\left(\alpha \right)}}\right]=〈{log}_2\left[{\displaystyle \frac{1}{P\left(\alpha \right)}}\right]〉,$$

(3)

with $\langle ·\rangle$ serving as the mean with respect to the probability distribution ${\left\{P\left(\alpha \right)\right\}}_{\alpha =1}^{D!}=\{P\left(1\right),\dots ,P(D!)\}$. The maximum value of H is $H_{max}={log}_2(D!)$, which is known as the Hartley or max-entropy and is achieved if and only if $P\left(\alpha \right)=1/D!$ for all $\alpha$ (a uniform distribution). We use ${log}_2$ in Equation (3) so that the unit is the bit.

To improve the differences in the permutation entropy values of ${\left\{P\left(\alpha \right)\right\}}_{\alpha =1}^{D!}$ when it is close to the uniform distribution, we use the Rényi min-entropy (in bits), $H_{\infty }$, which is defined by [61]

$$H_{\infty }=\underset{q\to \infty }{lim}{\displaystyle \frac{1}{1-q}}{log}_2\left[\sum _{\alpha =1}^{D!}{P}^q\left(\alpha \right)\right]=\underset{\alpha }{min}\left\{{log}_2\left[{\displaystyle \frac{1}{P\left(\alpha \right)}}\right]\right\}=-{log}_2\left[\underset{\alpha }{max}\left\{P\left(\alpha \right)\right\}\right].$$

(4)

This means that the information content of any OP sequence is bounded between $H_{max}$ and $H_{\infty }$.

To quantify the variability of the signal magnitudes within the OPs, we use

$$\langle {log}_2\left[{\sigma }_j\right]\rangle =\sum _{\alpha =1}^{D!}P\left(\alpha \right){log}_2\left[{\sigma }_j\left(\alpha \right)\right],$$

(5)

where $\langle ·\rangle$ is the mean with respect to the OP probability distribution (as in Equation (3)) and ${\sigma }_j\left(\alpha \right)$ is the standard deviation of the signal magnitudes at the j-th component (with $j=1,\dots ,D$) of all the embedded vectors that correspond to the OP symbol $\alpha$ (with $\alpha =1,\dots ,D!$) [56,57]. An example of the resultant values of Equation (5) for an EEG of the right primary motor cortex (rM1) of a representative rat during AW is shown in Figure 1.

We note that the variation in the values of $\langle {log}_2\left[{\sigma }_{\alpha }\left(j\right)\right]\rangle$ for different entries of j is minimal, as illustrated in Figure 1. Therefore, we work with the average value (but any choice of j would hold similar results and our conclusions would remain unchanged). Namely,

$${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}={\displaystyle \frac{1}{D}}\sum _{j=1}^D\langle {log}_2\left[{\sigma }_j\right]\rangle .$$

(6)

Consequently, when using Equation (3) [or Equation (4)] we can quantify the average [minimum] information content in an OP sequence, but lose the information from the magnitudes that compose the embedded vectors forming the OPs. In contrast, using Equation (6) we can quantify the average magnitude variability of these embedded vectors, complementing the information provided by H [or $H_{\infty }$].

3. Results

3.1. Ordinal Pattern Analysis of the Noiseless Map Iterates

From the analysis of the OP sequences for the coupled logistic maps (Equation (1)) and Hénon map (Equation (2)) for different parameters, we can see that when the dynamics change slightly, the min-entropy $H_{\infty }$ (Equation (4)) can have small variations, but the average magnitude variability ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ (Equation (6)) can change drastically.

Figure 2 shows that when the coupling strength is $\epsilon =0.01$, the red circle ($r=3.60$) and the cyan diamond ($r=3.80$) have similar $H_{\infty }$ values, close to 2 and $2.05$ bits (vertical axis), respectively. In contrast, their ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ values differ by an order of magnitude, being close to $-5.38$ and $-4.22$ (horizontal axis), respectively. Similarly, when $\epsilon =0.2$, the red triangle ($r=3.60$) and the black star ($r=3.75$) have an ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ differing by an order of magnitude (horizontal axis) but similar $H_{\infty }$ values, which are close to $1.92$ and $1.95$ bits (vertical axis), respectively.

However, the opposite effect is also observed in Figure 2. For example, when $\epsilon =0.01$, the black square ($r=3.75$) and the yellow triangle pointing upward ($r=3.9$) have significantly different $H_{\infty }$ values (close to $1.82$ and $2.62$ bits, respectively), but their ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ is similar (close to $-4.35$ and $-4.40$, respectively). The same happens for $\epsilon =0.2$, where the black star ($r=3.75$) and the yellow triangle pointing downward ($r=3.9$) have similar ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ values (close to $-4.55$ and $-4.50$, respectively) but significantly different $H_{\infty }$ (close to $1.94$ and $2.60$ bits, respectively).

The dynamics of the coupled logistic maps shown in Figure 2 follow the bifurcation-like diagram of Figure 3. We can see that the dynamical changes driving the similarities (or dissimilarities) in $H_{\infty }$ or ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ values in Figure 2 are nearly unnoticeable in Figure 3 by performing a direct visual inspection. This implies that the dynamical changes must be happening at smaller scales than the length of the signal, which is likely why the OP encoding is able to capture the different chaoticities (i.e., different $H_{\infty }$ values) emerging for the different control parameters and local changes in the magnitude of the $x_t$ signals (i.e., different ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ values). Nevertheless, these dynamical changes are revealed by the maximum Lyapunov exponent (MLE) of the bidimensional system [68], as can be seen from

Table 1. Maximum Lyapunov exponent $\lambda$ (MLE) of the coupled logistic maps for the parameters in Figure 2. These are found by using Wolf’s et al. method [68], using an embedding dimension of 2, a time delay of 1, a maximum number of iterations of 10 to track for divergence (which avoided saturation), and a $10\%$ minimum separation to avoid temporal neighbours.

$(r,\epsilon )$	$(3.6,0.01)$	$(3.75,0.01)$	$(3.8,0.01)$	$(3.9,0.01)$
$\lambda$	$0.16$	$0.39$	$0.42$	$0.45$
$(r,\epsilon )$	$(3.6,0.2)$	$(3.75,0.2)$	$(3.8,0.2)$	$(3.9,0.2)$
$\lambda$	$0.18$	$0.36$	$0.43$	$0.49$

.

We draw similar conclusions when analysing the Hénon map. For example, the red circle ($a=1.15$) and the black square ($a=1.20$) in Figure 4 have similar $H_{\infty }$ values (close to $1.83$ and $1.85$ bits, respectively), but have different ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$: close to $-2.85$ for the red circle and $-2.68$ for the black square, respectively.

In these cases ($a=1.15$, $a=1.20$, $a=1.34$, $a=1.35$, $a=1.40$, and $a=1.405$), the dynamics of the map have a chaotic regime, with minimal apparent differences (similarly to Figure 3)—this can be corroborated using the bifurcation diagram of the Hénon map (see [69] for a bifurcation diagram). However, as a is increased for the analysed values, the chaoticity of the signal is also increased. This can be quantified using the Lyapunov exponents of the system (as

Table 2. Maximum Lyapunov exponent $\lambda$ (MLE) of the Hénon map for the parameters in Figure 4. These are found as shown in Table 1.

a	$1.15$	$1.2$	$1.34$	$1.35$	$1.4$	$1.405$
$\lambda$	$0.27$	$0.30$	$0.36$	$0.37$	$0.42$	$0.41$

shows), and as Figure 4 shows, it can also be quantified by using $H_{\infty }$ and ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$.

Overall, these results (Figure 2 and Figure 4) show that using the OP Rényi min-entropy in conjunction with the average signal variability per OP improves the characterisation of the underlying map dynamics. Moreover, we can see that these results (Figure 2 and Figure 4) scale with the noise and the use of different embedding dimensions D because ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ follows a power-law behaviour as a function of $D\tau$, with an exponent that depends on the noise strength (see [56,57] for details), which can be useful to distinguish between chaotic and stochastic signals. This is corroborated in Figure 5 and Figure 6 for the coupled logistic maps and Hénon map, respectively. Moreover, these results remain invariant when using the other coordinate, as Figure A1 and Figure A2 show in the Appendix A, which is expected from Takens embedding theorem [70].

3.2. Ordinal Pattern Analysis of the EEG Recordings

The sleep–wake states of active wakefulness (AW), rapid-eye movement (REM) sleep, and non-REM (NREM) sleep have different electrophysiological characteristics (see Section 2.3). However, when focusing solely on frequency or permutation entropy analyses, some of these differences are lost [20,21,22]. Here, we show that using the average signal variability per OP (${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$) improves the differentiation between the states, even when accounting for natural inter-animal variability.

The results of the EEGs for the three sleep–wake states coming from the right OB, M1, and S1 of 11 rats are shown in Figure 7—we can find similar results for the left hemisphere’s M1 and S1 cortices. Panels B (OB), C (M1), and D (S1) show that AW has the highest value of $H_{\infty }$ for all animals—with two exceptions, in the S1 cortex (panel D, for red filled circles)—but mid-range values of ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$. This implies that the Rényi min-entropy can distinguish considerably well between wakefulness and sleep states. However, the values of $H_{\infty }$ for REM and NREM are similar for all electrodes.

In contrast, the values of ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ for REM and NREM sleep differ by approximately an order of magnitude in most animals. This is in accordance with the type of waves present in these sleep states: REM has low-voltage fast-frontal waves but NREM has high-voltage slow-cortical waves (see Section 2.3). Consequently, the AW, REM, and NREM states are fairly differentiated when using $H_{\infty }$ and ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ simultaneously. We note that this differentiation is improved as the cortical location considered is further away from the reference electrode, which in our case is the right OB and Cerebellum, respectively. We also note that these results and conclusions remain unchanged when using H (Equation (3)) instead of $H_{\infty }$ (Equation (4)).

3.3. Ordinal Pattern Analysis of Grid Frequency Recordings

The grid frequencies at all the buses in the European grid typically follow the same overall trend, which is close to 50 Hz on average [71]. This global variation is due to the mismatch between power generation and consumption, and is typically slow compared to the grid intrinsic timescales. The main differences between each recording are the fluctuations around that common trend. The Rényi min-entropy vs. average magnitude variability plane is shown in Figure 8 for 5 different 24 h recordings from four different locations in the synchronous European grid. As expected, one observes that, for each day (different symbols) the values of ${\mathrm{avg}}_j\left\{\langle {log}_2\left[{\sigma }_j\right]\rangle \right\}$ at the different locations (different colours) are very similar. Indeed, the amplitude of the time-series is mainly due to the variation in the common signal around 50 Hz. Interestingly, the value of $H_{\infty }$ for each location seems to stay the same for different time intervals, i.e., for KA, OL, PT, and TU, respectively, at close to $2.4$ bits, $2.8$ bits, $3.1$ bits, and $3.7$ bits. This ordering of $H_{\infty }$ can be understood from the locations in the grid where the recordings were performed. It was found that the frequency fluctuations at one bus from the common overall trend are determined by the inverse of its resistance centrality in the grid [72]. One therefore expects recordings at the periphery of the grid to be more impacted by noise than those closer to the centre of the grid. Both KA and OL are rather central in the grid, which suggests that these recordings are essentially in line with the common trend around 50 Hz. On the other hand, PT and TU are peripheral in the grid, which suggests that their recordings are more impacted by randomness around the common trend. Thus, the Rényi min-entropy should be smaller for KA and OL compared to PT and TU, which is what can be observed in Figure 8.

4. Discussion

For many real-world systems, the intrinsic dynamics are so complex that inferring a mathematical model for the underlying dynamical process generating signal measurements becomes a very challenging task—if feasible at all. An alternative approach is to analyse the evolution of the information contained in the signals one can measure. While this approach fails to provide a detailed model for the microscopic dynamics, if successful, then it allows for different dynamical regimes and changes in the parameters of the system to be characterised.

With this in mind, here we propose to include the magnitude variability in the signal in the ordinal pattern (OP) encoding. Our aim is to complement the permutation entropy analysis, to improve the characterisation of the dynamical behaviours observed from the time-series, and to maintain explainable results. For example, if instead of using our complementary variables, we combine them, then there would be cases when the resultant modified PE value could not tell us whether it is increased (or decreased) because the PE value increased or because the standard deviations of the amplitudes increased (or decreased).

We first tested our approach on synthetically generated signals from chaotic bidimensional mappings under different parameter values. Specifically, we used coupled logistic maps (Equation (1)) and the Hénon map (Equation (2)). We analysed the signals coming from one of the two coordinates available for these maps because the OP encoding can only be applied to univariate signals.

The main reason to choose bidimensional mappings is that there are proofs showing that the permutation entropy of one-dimensional mappings directly relates to the Kolmogorov–Sinai entropy [73,74], which is not the case for higher-dimensional mappings. Because of this difference, it is expected that the characterisation of a dynamical regime by the permutation entropy for a higher-dimensional system is incomplete. We show that this problem is mitigated by including the standard deviation of the signal (in the embedded vectors forming the OPs) to the permutation entropy quantification (Figure 2 and Figure 4). We also find that our approach improves the characterisation of the dynamical regimes even under significant levels of observational noise and different choices of embedding dimension (Figure 5 and Figure 6).

We then analysed real-world EEG signals registered intracranially from 11 rats under free conditions throughout the sleep–wake cycle. We considered these signals because they come from a system—the brain—where the underlying microscopic dynamics are unknown (i.e., we lack a differential equation that models the system), there are inherent noise sources affecting the quality of the signal measurements, and the system has been hypothesised to have some level of chaoticity [75]. Moreover, in practice, the polysomnographic classification of sleep-wake states requires highly trained professionals to recognise characteristic electrophysiological patterns that vary according to the sleep stage, plus depend on anatomy and individual variability. However, the electrophysiological variability introduced by experimental manipulations (in research settings) or disease (in clinical settings), requires that whichever automatic sleep scoring classifier used is interpretable and not a black box.

Our results show that, for most cortical locations, independently of the embedding dimension used and the natural inter-animal variability, the states of active wakefulness, rapid-eye movement (REM) sleep, and non-REM sleep can be distinguished in the plane formed by the Rényi min-entropy and signal variability (Figure 7).

Finally, we analysed frequency recordings in the European electric power grid. We found that the standard deviation of the signals at different locations is similar during the same time intervals. This is expected, as the frequency in the whole grid essentially fluctuates around a common value close to 50 Hz with similar fluctuation magnitudes. We also found that the permutation entropy is higher for recordings at the periphery of the grid compared to those closer to the centre, which can be explained by the centrality of the recorded buses. Overall, our method identified the important features of the grid frequency dynamics.

Our analyses are limited in terms of the choice of embedding delay and overlap between consecutive embedded vectors; namely, for all our results $\tau =1$ and the embedded vectors are quasi-non-overlapping, only sharing a single data point from the signal. The reason to restrict $\tau$ to 1 is that we can consider the entire signal, whereas $\tau >1$ implies consideration of sub-sampled signals. On the other hand, increasing the overlapping of the embedded vectors creates artificial correlations between consecutive OPs (which are irrelevant for most permutation entropy calculations, but can affect conditional or transfer entropy calculations). For example, with a larger overlap than the one we choose here, two consecutive embedded vectors with $D=3$ would share two points, such that $\{x_t,x_{t+1},x_{t+2}\}$ and $\{x_{t+1},x_{t+2},x_{t+3}\}$. This would imply that, if $x_{t+1}>x_{t+2}$, then the OP for the second embedded vector would be conditioned to this increasing relation that is present in the previous embedded vector, and all other OP possibilities would be forbidden (i.e., the OP would be artificially forced to take on particular OP symbols). Consequently, our choice of encoding parameters allows us to keep all the signal and have a maximum number of embedded vectors with null redundancy between consecutive ones.

Finally, we note that a natural extension to our approach could consider a three-dimensional space, composed of the permutation entropy, the signal variability within the embedded vectors, and a complexity measure, such as the Jensen–Shannon complexity, which could improve the characterisation of other dynamical regimes (such as non-chaotic ones). Moreover, in practical applications where the signals are short, instead of considering the standard deviation of the signal for the embedded vectors, one could consider the inter-quartile range, which is a statistically robust descriptor that is unaffected by outliers.