Ordinal Patterns in Heartbeat Time Series: An Approach Using Multiscale Analysis

In this paper, we simultaneously use two different scales in the analysis of ordinal patterns to measure the complexity of the dynamics of heartbeat time series. Rényi entropy and weighted Rényi entropy are the entropy-like measures proposed in the multiscale analysis in which, with the new scheme, four parameters are involved. First, the influence of the variation of the new parameters in the entropy values is analyzed when different groups of subjects (with cardiac diseases or healthy) are considered. Secondly, we exploit the introduction of multiscale analysis in order to detect differences between the groups.


Introduction
Entropy-like measures are a powerful tool in the study of time series. The encoded information in a time series has turned out to be a valuable source of information. Since the seminal paper of Bandt and Pompe in 2002 [1], permutation entropy (PE) has been used to measure the complexity of different types of processes, since the order relation established in the time series allows finding patterns linking with the complexity of the system; see [2] for a review about permutation entropy and its applications. In particular, biomedical time series have been the focus of great interest, including neuronal signals and heart rate series [3]. Nevertheless, new variants of entropy-like measures have arisen in the last years, considering the nature of the series that is object of study [4], since differences appear when we consider different versions of entropy and different values of the parameters. Therefore, the analysis of the behavior of each of them, taking into account the final purpose that has been fixed, is relevant.
A natural approach to the study of possible cardiac problems is the analysis of heartbeat series. The activity of human beings influences heartbeat series, causing non-stationary series in healthy subjects. Cardiac diseases should be revealed in heartbeat series though significant changes in complexity. Difficulties in measuring that complexity have been considered and preliminary results can be found in different papers where the idea of extracting information from the time series is presented. Thus, Pincus and collaborators use approximate entropy (AE) in several papers to quantify the complexity and the regularity of biological time series [5,6]. In [7], symbolic dynamics and renormalized entropy were used to detect abnormalities in heart rate variability (HRV) in patients that had been classified as low-risk using traditional methods. In 2002, [8] a sample entropy analysis of neonatal heart rate variability was considered, taking into account the abnormal heart rate characteristics appearing in neonatal sepsis. Atrial fibrillation (AF) is a common cardiac arrhythmia in which irregular patterns of electrical activation are usually found in the atria [9]. Cammarota and Rogora (2005) studied the independence of non-stationary heartbeat series

Terminology and Notation: Ordinal Patterns
Although most of the notions are well known, we recall them for the sake of completeness. The definitions that follow, with slightly differences, can be found in the literature; see for instance [1,3,4,18,19]. The starting point is a real time series (x n ) T n=1 where ordinal patterns of length m are considered. The length of the series must be big enough, since the procedure of reconstruction of the attractor evaluating the associated probability distribution requires enough data. Following [20], a required condition is T > 5m!.
Let m ∈ N be a fixed natural number, S m be the group of permutations of length m, and π = (r 1 , r 2 , . . . , r m ) ∈ S m . The vector (x 1 , x 2 , . . . , x m ) ∈ R m is said to be π-type if Observe that the number of possible ordinal patterns of length m is given by the cardinality of S m , that is |S m | = m!.
The length of the patterns is called the embedding dimension. Following [1], low orders of m are recommended for practical purposes. We will use m = 3, 4, and 5.
The following definition, where delay parameter τ is included, gives the relative frequency of the pattern associated with the permutation π ∈ S m . Definition 1 ([1,4]). Let (x n ) T n=1 be a time series, m ∈ N and π ∈ S m , then the relative frequency of π, denoted by p τ (π), is given by Permutation entropy (PE) was defined by Bandt and Pompe [1]. Let (x n ) T n=1 be a real time series, m, τ ∈ N. Then the permutation entropy (PE) is given by where π ∈ S m and τ ∈ N. The Rényi entropy variant [21] generalizes the permutation entropy [22]. The definition follows; see for instance [4] (Definition 4).

Definition 2.
Let (x n ) T n=1 be a time series, α = 1, m, τ ∈ N. Then, the Rényi permutation entropy of (x n ) T n=1 (RPE), is given by where p τ (π) is given as in Definition 1. When the parameters are fixed, we write RPE for short.
Permutation entropy (PE), also called Shannon entropy (SE), is obtained when α tends to 1 in Rényi permutation entropy. Liang [16] reported that the class of permutation entropy with the best results when EEG during different anesthesia states are considered is Rényi entropy. Moreover, the choice of the parameter is not a trivial question; see [23].
The weighted version of Rényi entropy uses weighted frequencies in which not only the order is considered but also the differences between the values of the pattern. The idea is illustrated in Figure 1. Observe that (2, 3, 1) and (2.5, 3, 2) are the same pattern, namely (3,1,2), but the distance between the values are different. The definition follows.

Definition 3.
Let (x n ) T n=1 be a time series, m ∈ N and π ∈ S m , then the weighted relative frequency of π, denoted by p τ ω (π), is given by is of π − type} for each π ∈ S m , and Normalized versions of entropy-like measures are obtained by dividing by log(m!), which is the maximal possible value for PE and RPE. We will consider the normalized versions throughout the paper.
The multiscale process that we will follow has different steps. The description follows. Let (x n ) T n=1 be a time series and s ∈ N the scale factor. The coarse-grained time series (x s n ) is given by denotes the integer part of T s . We will consider for our analysis the range 1 ≤ s ≤ 20. Now, fixing the embedding dimension, m, the delay, τ, and the Rényi parameter, α, the multiscale Rényi permutation entropy is given by whereas, the weighted multiscale Rényi permutation entropy (WMRPE) is given by Observe that if s = 1, then the MRPE (WMRPE) is simply the RPE (WRPE).

The Data
The data corresponds to a collection of 15 heart beat intervals (RR-interval) freely available on Physionet (http://www.physionet.org/challenge/chaos). Time series n1rr, n2rr, n3rr, n4rr, and n5rr correspond to healthy subjects, a1rr, a2rr, a3rr, a4rr, and a5rr are time series in atrial fibrillation, and finally c1rr, c2rr, c3rr, c4rr, and c5rr are time series in congestive heart failure. Time series are not filtered and were obtained from continuous ambulatory (Holter) electrocardiograms. Each time series is about 24 h long (roughly 100,000 intervals). More information about the recordings referring to healthy and congestive heart failure subjects can be obtained on the web. No additional details about the time series in atrial fibrillation are given. Figure 2 shows the data for a subject in each group. Although they are different, in some cases is not easy to classify a subject only by visual inspection. One anonymous reviewer has suggested the possibility of analyzing the differences between the groups when the images are similar using a fuzzy technique of quantification of the distances in digital images using fuzzy divergence [24], which is quite a different approach from this one.

Numerical Analysis
A preliminary numerical approach computing PE gives us the general idea that entropy values from a subject in the AF group are higher than the other groups (see Figure 3), where (PE) has been computed for each subject, taking m = 3 and s = 1. The delay parameter τ ∈ N takes values in the range 1 ≤ τ ≤ 100. The CHF group has been drawn in blue, green has been used for the H group, and the values obtained from the AF group have been represented in red. The deviation of the values of PE entropy with respect to its mean for each subject in the AF group is smaller than the rest of the groups. A simple look at the graphics reveals differences between the groups. We can also observe that entropy values that come from the CHF group and the healthy group are mixed and are not easy to distinguish.
In [13] permutation min-entropy is used to analyze this data collection. Here, we will analyze these series using normalized MRPE and WMRPE entropies. All procedures have been implemented with a code "ad hoc". First, we analyze the behavior of each group (the mean) for the embedding dimension m = 3, m = 4, and m = 5, taking into account that when the value of m increases, so does the computational time. We fix α = 1 2 and α = 2 as reference values. Regarding the scale factor s, it ranges over the interval 1 ≤ s ≤ 20 and τ ranges over 1 ≤ τ ≤ 100. The results are shown in  In Figure 4, we have fixed α = 1 2 and have computed the average of RPE and WRPE for the CHF group, considering the embedding dimensions m = 3 (dark blue), m = 4 (light blue), and m = 5 (green) in the 3D graph. We can observe in the contour plots that changes in the values of the entropy, that is, changes in the complexity of the dynamics, are presented for values close to the boundary, namely τ = 1 and s = 1. When the value of the embedding dimension increases, so does the rate at which the entropy decreases with respect to parameter values τ and s. We can see that the variability of the values of the entropy for α = 1 2 when τ and s increase is greater when m grows, although the values of the entropy are lower. Finally, for a fixed τ, changes in the values of the scale parameter s means changes in the values of the RPE and in the values of the WRPE. The blue color represents the congestive heart failure (CHF) group, green has been used for the healthy (H) group, and red for the atrial fibrillation (AF) group. The delay parameter τ has been represented in the OX axis. In the middle, the same situation has been represented using box plots, grouping the subjects belonging to the same group. Finally, in the right, we have taken the mean with respect to τ when 1 ≤ τ ≤ 100 for each subject and have represented the corresponding box plot. Regarding the relationship between the WRPE and the RPE for a fixed m and α = 1 2 , we can observe that it depends on the scale value s; see Figure 5, where the RPE is drawn in blue and the WRPE in red. This relationship can remain unnoticed if multiscale analysis is not involved. Thus, for s = 1 we can observe that for τ ≥ 3, the WRPE is greater than or equal to the RPE. Nevertheless, when s = 2 the order relation changes and there is no rule to determine if an entropy measure is greater than the other one. For values of s ≥ 3, the relationship turns around, and the RPE is greater than the WRPE. The limit case τ = 1 and close values of τ to this gives a different behavior. Observe that for s = 1, the WRPE is higher than the RPE for most values of τ. Nevertheless, for s ≥ 3 the situation changes.
Fixing α = 1 2 , we compute the average of the RPE and WRPE for the H group. We have drawn RPE in the top row for m = 3 (dark blue), m = 4 (light blue), and m = 5 (green) and the contour plots for each value of the embedding dimension m; see Figure 6. The bottom row shows the results obtained for the WRPE. Again, roughly speaking, as m increases the values of Rényi entropy decreases. We can see that the behavior with respect to the projections for the CHF and H groups are different. Moreover, in the case of the H group, the values of the weighted Rényi entropy are lower than those of the Rényi entropy for most values of τ, even for the case of s = 1, compared to what happened in the case of CHF group; see Figure 7. We recall that the CHF group and the H group were not easy to distinguish. and all the differences must be taken into account.  Finally, we compute the RPE and WRPE for the AF group following the previous schedule. The results are shown in Figure 8, and they are quite different from the results that were obtained for the previous groups. The relationship between the values of RPE and WRPE for m = 3 are shown in Figure 9. Again, for s = 1 we can see that there are values of τ for which the WRPE is greater than the RPE, and the situation changes when s increases. Thus, only the H group has a different behavior in respect to this fact.   For s = 1 and s = 2 there are values of τ such that the WRPE is greater than the RPE. Figure 11 shows the RPE (blue) and WRPE (red) results for m = 3 and s = 1, s = 2, s = 3, and s = 5. Figure 11. Average Rényi entropy (RPE, blue) and average weighted Rényi entropy (WRPE, red) for the CHF group and α = 2, m = 3, and s = 1, 2, 3, and 5. Observe that for s = 1, the WRPE is higher for most values of τ; nevertheless, for s = 5 the situation changes.    Following the previous analysis, we have been able to observe some differences between them. Now, we are interested in comparing the groups by analyzing what are the best parameter values to differentiate the three groups. For that, we compute the entropies of each subject and take the maximum and the minimum value in each group for each fixed parameter value. More concretely, fixing the parameter values s, m, α, and τ, we say that two groups are differentiated by the entropy measure if the intervals limited by their corresponding maximum and minimum values are disjoint, and the difference between two groups is the minimum distance between any two values belonging to different groups. Namely, fixing m, α, s, and τ, we compute for a group G Fixing s, α, m, and τ, we say that group G 1 and group G 2 are differentiated by the MRPE (resp. WMRPE) if . We will investigate the existence of parameter values in which the three groups are differentiated simultaneously. It means that a relationship of order G1 > G2 > G3 should be given. In addition, in those situations we will compute the difference between G1 and G2 and the difference between G2 and G3, and we will compute the minimum of both. This value will be called the difference between the three groups. We are interested in finding the parameter values by which the groups are differentiated and the difference is maximal. Observe that although the number of subjects is small, a negative answer to this question also gives valuable information. For this, we fix α and consider three different cases: m = 3, m = 4, and m = 5. Figure 16 shows the results for α = 1 2 . In 3D graphs (Figure 16), the OX axis represents the parameter s, where 1 ≤ s ≤ 20, in the OY axis is represented the parameter τ, 1 ≤ τ ≤ 100, and finally in the OZ axis is represented the difference between the three groups, when the groups are differentiated by MRPE. The 2D graphs show the values (s, τ) for which the three groups are disjoint in the sense described above. Whereas in the 3D plots we can see not only the parameter values in which the three groups are disjoint but also the difference between them. When m changes, the shape of the distribution of the points (s, τ) is similar although the number of points increases, namely for m = 3 there are 168 points, for m = 4 there are 283 points, and for m = 5 there are 348 points. The maximum difference increases with the parameter m, as we can see in Figure 16. The difference attains its maximum values when τ = 1. The relation of order between the three groups is, in all cases, AF > H > CHF. More details are given in Table 1, where the results for m = 3 and α = 1 2 are shown.
When we compute the WMRPE instead of the MRPE, the results are different. The values that appear in Table 2 show that in general terms, the WMRPE is worse than the MRPE for differentiating the three groups. 2 , values of s and τ for which the three groups are separated by the multiscale Rényi permutation entropy (MRPE). In the 3D graphs, the OX axis represents the parameter s, where 1 ≤ s ≤ 20, in the OY axis is represented the parameter τ, 1 ≤ τ ≤ 100, and finally in the OZ axis is represented the difference between the three groups, when the groups are differentiated by the MRPE.  Now we consider a fixed value of α greater than 1; we follow with the reference value α = 2. Figure 17 shows the results. The shape and the distribution of the points (s, τ) for a fixed embedding dimension is similar to the results obtained for α = 1 2 , but we can observe that the maximum difference is greater than the one obtained in the above case.
For α = 2, when the weighted version is considered, we obtain the points and the differences that have been collected in Table 3, taking 3 ≤ m ≤ 5, 1 ≤ s ≤ 20, and 1 ≤ τ ≤ 100. In the 3D graphs, the OX axis represents the parameter s, where 1 ≤ s ≤ 20, in the OY axis is represented the parameter τ, 1 ≤ τ ≤ 100, and finally in the OZ axis is represented the difference between the three groups, when the groups are differentiated by the MRPE.
Following the previous results, we see that the maximum value has been attained for s = 8 and τ = 1. To see the influence of the parameter α, we set s = 8 and τ = 1, and we compute the RPE values in the OY axis for each value of α (represented in the OX axis); see Figure 18. We can see that when α increases, the entropy values tend to a limit value.
For s = 1, we can observe that there is a large quantity of values of τ for which the three groups are differentiated, although the differences in these values are significantly lower than in the previous case. For s = 1, we simply get RPE with delay, τ, that now runs over a set of cardinality equal to 100. This means that for a fixed m, we have two parameters α and τ. Fixed the embedding dimension m = 3, we compute the Rényi entropy for τ ∈ N taking values in the interval 1 ≤ τ ≤ 100. Figures 19-21 show the results for α = 1 2 , α = 2, and α = 100, respectively. We can see that the AF group has a different behavior from the healthy and CHF groups. It is possible to find values of τ for which the three groups can be differentiated. More details are included in the Appendix A (Tables A1 and A2), where different values of α have been considered for m = 3 and s = 1. It is observed that when α decreases, the entropy values of the different groups are closer. Table 3. Fixing α = 2, values for the parameters m, s, τ ∈ N in the ranges 3 ≤ m ≤ 5, 1 ≤ s ≤ 20, and 1 ≤ τ ≤ 100 such that the CHF group, H group, and AF group are disjoint. The difference and the order relation between them are also included.     Rényi entropy for α = 100 and embedding dimension m = 3. We have drawn the Rényi entropy for the CHF group in blue, for the healthy group in green, and for the AF group in red on the left. A zoomed-in view of the right figure is shown in the middle, where the differences between the three groups when τ runs over the range can be appreciated. Finally, box plots for each group have been represented on the right.

Conclusions
In this paper, we propose a multiscale analysis with RPE and WRPE to analyze heartbeat time series, where four parameters are considered in the study of the behavior of a heartbeat time series. Our purposes are twofold. On the one hand, we have considered the behavior of the entropy measure when parameters change in a range. The introduction of new parameters makes the analysis more difficult and more expensive in computational terms, but on the other hand, when we add new parameters, this allows a better fit to the research objective. From this point of view, it is natural to ask if the introduction of two additional scales gives non-redundant information. In this direction, numerical analysis shows differences between the three groups considered, namely a congestive heart failure (CHF) group, healthy (H) group, and atrial fibrillation (AF) group. Moreover, an additional characteristic for the healthy group is obtained, as we can see that this group has a different behavior in the relationship between the RPE and WRPE when s changes. On the other hand, we compare the results obtained for the RPE and WRPE, and we find the values for which the three groups can be differentiated using segments limited by the minimum and maximum values that have been attained in each group. The groups are called differentiated if the intervals are disjoint. In addition, we compute the values of s and τ that simultaneously differentiate the three groups, taking the two reference values of α = 1 2 and α = 2 and embedding dimensions m = 3, 4 and 5. The numerical approach shows that even the RPE is better to differentiate than the WRPE.
The results look promising in the sense that they show that the introduction of multiscale analysis of Rényi entropy reveals more details about the behavior of each group and highlights non-redundant differences between them. The results that have been obtained are a first step in the idea that multiscale analysis should be taken into account to obtain an adequate interpretation of ordinal patterns in physiological terms, in order to consider all of the potential information that they can reveal.
Funding: The author has been supported by the Grant MTM2017-84079-P from Agencia Estatal de Investigación (AEI) y Fondo Europeo de Desarrollo Regional (FEDER).