Accelerating the Computation of Entropy Measures by Exploiting Vectors with Dissimilarity

Yun Lu 1 ID , Mingjiang Wang 1,*, Rongchao Peng 2 ID and Qiquan Zhang 1 1 Key Laboratory of Shenzhen Internet of Things Terminal Technology, Harbin Institute of Technology Shenzhen Graduate School, Shenzhen 518055, China; luyun@stu.hit.edu.cn (Y.L.); zhangqiquan_hit@163.com (Q.Z.) 2 School of Biomedical Sciences, The Chinese University of Hong Kong, Hong Kong, China; pengxiaotu@126.com * Correspondence: mjwang@hit.edu.cn; Tel.: +86-755-2603-3791


Introduction
In recent years, increasing attention has been paid to entropy measures for quantifying electroencephalogram (EEG) signals in neurological disease diagnosis and brain function assessment [1][2][3], including for monitoring of the depth of anesthesia [4,5], automatic detection of epileptic seizures [6,7] and estimation of cognitive workload [8,9], etc.Although numerous entropy measures have been proposed, such as approximate entropy (ApEn) [10], distribution entropy [11], bubble entropy [12], short-term Rényi entropy [13][14][15] and so on, many of them, such as ApEn, sample entropy (SpEn) [16], fuzzy entropy [17] and multiscale entropy (MSE) [18,19], require hundreds of data points for computation and involve high time costs.According to their definitions, their computational methods are based on phase space reconstruction by making use of many data points [20] and as such, arbitrary two vectors in the reconstructed phase space are required for distance calculation to determine whether the paired vectors are similar, leading to a great computational load.
In some specific applications of these entropy measures, if the computation is not performed fast enough, it is not practical for online neurological disease diagnosis or brain-state monitoring, Entropy 2017, 19, 598 3 of 21 Consider a time series X containing N data points: A set of vectors is reconstructed from the time series X, denoted as X (m) i , whose embedded dimensions are m.
The distance between two vectors is defined as The two vectors X (m) i and X (m) j are considered to be similar if the distance d X (m) i , X (m) j is within r, where r is the tolerance for accepting matches, For a given value of r, the counting variable C(i,j,r) is set to 1 when vectors X (m) i and X (m) j are similar to each other; otherwise, its value is set to zero.
For a given vector X i , the number of vectors X (m) j similar to vector Then, the probability of vector X (m) i similar to any other vectors can be expressed as Subsequently, as for ApEn, the first step is to take the natural logarithm of each P m i (r) and then average it over i The second step is to increase the embedded dimension to m + 1 and to repeat the above calculation steps from Equations ( 2)- (7) to obtain P m+1 i (r) and Φ m+1 (r).Finally, the ApEn is computed as As for SpEn, there is a slight difference from the ApEn.It firstly averages P m i (r) over i and then takes its natural logarithm.
After that, the embedded dimension to m + 1 is increased and the above calculation steps are repeated to obtain P m+1 i (r) and ψ m+1 (r).Finally, the SpEn is given by The SpEn differs from the ApEn mainly in the two ways [20]: (1) SpEn can remove the bias of self-matching; (2) SpEn performs additional operation prior to logarithmic operation so as to avoid the occurrence of the ln(0) in the calculations.Therefore, SpEn can require shorter data than ApEn and ApEn strongly depends on the length of data [16].What's more, SpEn has better relative consistency than ApEn [16].

Multiscale Entropy (MSE) and Time-Shift Multiscale Entropy (TSME)
The multiscale entropy (MSE) was introduced by Costa et al. and was developed to compute the corresponding SpEn in time series X over different scale factors [18,19].The computation of MSE consists of two steps.Firstly, for the time series X of length N, a new coarse-grained time series is constructed by averaging the non-overlapping data points from X at a scale factor τ.Then, the SpEn is computed to obtain an entropy value for the coarse-grained time series.In general, the coarse-grained time series is obtained from X according to the following equation: Many modified versions of MSE have been developed to describe the multiscale properties of time series [32], such as composite multiscale entropy [33] and time-shift multiscale entropy (TSME) [34].TSME was proposed by Pham [34], modifying the MSE over time series X of length N, with multiple time shifts at a given time interval kmax.The TSME is computed in the following three steps.Firstly, for a given time interval kmax, the k time-shift series is constructed from time series X by means of the known Higuchi's fractal dimension [34,35].Then, either the SpEn or ApEn is computed for all time-shift series, denoted as TSME β k , β = 1, 2, ..., k.Finally, the TSME for each k is defined as the mean value of all TSME β k where k = 1, 2, ..., kmax [34].With increasing k, the SpEn-based TSME β k can remain fairly constant, while the ApEn-based TSME β k tends to slightly decrease.The trend in the ApEn-based TSME β k should be due to the bias of self-matching in the computation of ApEn.Thus, the use of SpEn is more theoretically sound for computing TSME.

Vectors with Dissimilarity (VDS) Method
In order to accelerate the computation processes above, three strategies can be taken into consideration as shown below.First, the computational steps of Φ m (r), Φ m+1 (r), ψ m (r) and ψ m+1 (r) can be performed in parallel.Second, the distance d(X i ) and there is no need to calculate both distances.Third, the most time-consuming step is the distance calculation of ) and how to optimize this step is of the utmost importance for the overall computational load of the entropy measure.To the best of our knowledge, most of the published works adopted the first and the second strategies to reduce the computation time.Therefore, we focus on the third strategy to accelerate computation of entropy measures by exploiting VDS, which is referred to as the VDS method.
From a statistical point of view, according to the definitions and computation steps of entropy measures mentioned above, arbitrary two vectors with distances greater than r are generally superior in numbers.Thus, the number of the vectors with dissimilarity has a more obvious advantage than the number of the vectors with similarity and it is an extremely important to exploit a technique to determine whether the paired vectors are dissimilar before their distance calculations.Once the paired vectors are deemed to be dissimilar, there is no need to further calculate their distances, which can greatly reduce the computational time.For SpEn and ApEn, two vectors of dimensions m are considered to be similar if their distances are within r, i.e., Therefore, arbitrary two vectors of dimensions m are considered to be dissimilar if their scale components satisfy We put forward a decision method to decide if the two vectors are dissimilar.The decision quantity, Decis(i,j), is defined as follows: When the Decis(i,j) is greater than m × r, the corresponding paired vectors X (m) i and X (m) j are deemed to be dissimilar and this is referred to as the VDS decision.It can be proven as follows: Assume that the Decis(i,j) is greater than m × r, it can be given by the formula: Because of |a| + |b| ≥ |a + b|, we can derive the following equation From Equations ( 16) and ( 18), we can obtain In Equation (19), the sum of m absolute values, x i+k − x j+k where k is 0, 1, ..., m − 1, is larger than m × r.For the m absolute values, assuming all absolute values of x i+k − x j+k where k is 0, 1, ..., m − 1, are within r, then it can be deduced that By using reduction to absurdity, based on Equation (19), we can only deduce that The maximum value of all absolute values is greater than r.Therefore, the paired vectors X (m) i and X (m) j are dissimilar.By exploiting the VDS decision, some vectors can be predetermined to be dissimilar in the computation process and their distances are not necessary for further calculation for entropy measures.Furthermore, the vectors with dissimilarity are in fact superior in number, which also can be confirmed through experimental results in Section 3. Therefore, most of the time-consuming steps with regard to distance calculations between these paired vectors can be optimized during computation of entropy measures.

Implementation and Testing Method
In order to test the performance of the proposed VDS method in accelerating the computation of entropy measures taking SpEn as an example, we implemented the VDS method for accelerating the computation of SpEn in Figure 1.The details of implementation of the proposed VDS method for accelerating computation of SpEn are explained as follows.
(1) The input quantities are the time series X and computational parameters N, m, r.
(2) Two sets of vectors X (m) and X (m+1) are reconstructed from the original time series X.Their embedded dimensions are m and m + 1, respectively and the sizes are both L = N − m. (3) All scalar components of each vector X (m) (i) are added to obtain the Xsum(i).(4) For arbitrary two vectors, if the absolute value of Xsum (i) − Xsum (j) is greater than m × r, the Decis for the paired vectors X (m) (i) and X (m) (j) is set to 1 and then there are no further steps to calculate their distance.(5) Otherwise, the Decis is set to 0 and for the paired vectors there are some calculation steps needed to further calculate their distance in order to determine whether they are similar.An array C m is used for saving and updating the counting variable.If the distance of the paired vectors is within r, the C m (i) and C m (j) both increase by 1. (6) When the d(X (m) (i), X (m) (j)) ≤ r holds, the paired vectors of embedded dimension m are deemed to be similar.Then we continue to check whether the distance of the paired vectors of embedded dimension m + 1 is within r.Another array C m+1 is used for saving and updating the counting variable.(7) Finally, the SpEn is computed.
In order to better study the time efficiency of the VDS method, we also developed a conventional method to compute the SpEn as a contrast.For the conventional method, most of the calculation steps were the same as in the VDS method, only without the steps of the VDS decision.
Furthermore, for quantitative analysis of the VDS method to better understand the operation principle of accelerating computation, we developed a testing program to count the number of paired vectors for which Decis = 1 in Figure 1.The testing program added the some counting steps on the basis of the VDS method.We developed the testing program rather than directly using the VDS method to count, mainly because the counting steps would cause an increase in the running time of the VDS method, which was not conducive to evaluating the time efficiency correctly.

Experimental Data
We used two kinds of signals to evaluate time efficiency of computation of SpEn and TSME.One was random signal uniformly distributed in the interval (0, 1) and the other was the EEG signal.It is well known that most of physiological signals, for example EEG signals, have a characteristic of randomness in the time domain [36].The EEG signals were obtained from the EEG motor movement/imagery dataset (EEGMMID) [37] found at http://www.physionet.org.The EEG signals were recorded using the BCI2000 system from 64 electrodes as per the international 10-10 system.The subjects performed different motor/imagery tasks while 64-channel EEG signals were recorded and sampled at 160 samples per second.We optionally selected some subjects' EEG signals to use for computation of SpEn and TSME.

Accelerating Computations of Sample Entropy (SpEn) in Random Signals and White Noises
First of all, we used ten random signals to assess the VDS method in the accelerating computation of SpEn.Each random signal has 1000 data points uniformly distributed in the interval (0, 1), as shown in Figure 2.

Experimental Data
We used two kinds of signals to evaluate time efficiency of computation of SpEn and TSME.One was random signal uniformly distributed in the interval (0, 1) and the other was the EEG signal.It is well known that most of physiological signals, for example EEG signals, have a characteristic of randomness in the time domain [36].The EEG signals were obtained from the EEG motor movement/imagery dataset (EEGMMID) [37] found at http://www.physionet.org.The EEG signals were recorded using the BCI2000 system from 64 electrodes as per the international 10-10 system.The subjects performed different motor/imagery tasks while 64-channel EEG signals were recorded and sampled at 160 samples per second.We optionally selected some subjects' EEG signals to use for computation of SpEn and TSME.

Accelerating Computations of Sample Entropy (SpEn) in Random Signals and White Noises
First of all, we used ten random signals to assess the VDS method in the accelerating computation of SpEn.Each random signal has 1000 data points uniformly distributed in the interval (0, 1), as shown in Figure 2. To evaluate time efficiency of computation, we adopted both the VDS method and the conventional method to compute SpEn in the ten random signals and measured the computation time.The computational parameters of SpEn are N = 1000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each random signal.The measurement values of SpEn by the adoption of the VDS and conventional method are identical.Figure 3a presents the computation time of SpEn in the ten random signals.It is clearly observed that the computation of SpEn is greatly accelerated by the adoption of the VDS method.The mean computation time is calculated by averaging the ten random signals.For the conventional method the mean computation time of SpEn is 0.209 s but for the VDS method it is only 0.045 s.Consequently, the computation time of SpEn is significantly (p < 0.005) reduced, by 78.5%.The standard deviations of the computation time for the VDS and conventional method are 0.0042 and 0.0056, respectively.
For quantitative analysis, it is necessary to study how many vectors are determined to be dissimilar by the VDS decision in the computation process.Utilizing the testing program, the SpEn in the ten random signals was computed once again and the number of the paired vectors which were dissimilar as determined by the VDS decision was counted.Then, the percentages of these paired vectors were calculated.Figure 3b shows that the percentages of the paired vectors which are dissimilar as determined by the VDS decision are 84.7%,84.3%, 84.5%, 84.7%, 84.5%, 84.9%, 85.1%, 84.0%, 85.1% and 84.7% for the ten random signals during computation.The paired vectors which are dissimilar indicate that their distances are greater than r.This experimental result reveals the fact that arbitrary two vectors with dissimilarity are superior in numbers for the computation of entropy measures.
Furthermore, we analyzed several types of white noises of different power and mixed them with sinusoidal signal to evaluate the VDS method and compare against the conventional method.Six white noises' powers, −12 dBW, −9 dBW, −6 dBW, −3 dBW, 0 dBW and 3 dBW, respectively, were investigated and mixed with sinusoidal signal.Figure 4 presents the sinusoidal signal and the mixedsignals with white noises of power −12 dBW, −6 dBW and 0 dBW, respectively.To evaluate time efficiency of computation, we adopted both the VDS method and the conventional method to compute SpEn in the ten random signals and measured the computation time.The computational parameters of SpEn are N = 1000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each random signal.The measurement values of SpEn by the adoption of the VDS and conventional method are identical.Figure 3a presents the computation time of SpEn in the ten random signals.It is clearly observed that the computation of SpEn is greatly accelerated by the adoption of the VDS method.The mean computation time is calculated by averaging the ten random signals.For the conventional method the mean computation time of SpEn is 0.209 s but for the VDS method it is only 0.045 s.Consequently, the computation time of SpEn is significantly (p < 0.005) reduced, by 78.5%.The standard deviations of the computation time for the VDS and conventional method are 0.0042 and 0.0056, respectively.For quantitative analysis, it is necessary to study how many vectors are determined to be dissimilar by the VDS decision in the computation process.Utilizing the testing program, the SpEn in the ten random signals was computed once again and the number of the paired vectors which were dissimilar as determined by the VDS decision was counted.Then, the percentages of these paired vectors were calculated.Figure 3b shows that the percentages of the paired vectors which are dissimilar as determined by the VDS decision are 84.7%,84.3%, 84.5%, 84.7%, 84.5%, 84.9%, 85.1%, 84.0%, 85.1% and 84.7% for the ten random signals during computation.The paired vectors which are dissimilar indicate that their distances are greater than r.This experimental result reveals the fact that arbitrary two vectors with dissimilarity are superior in numbers for the computation of entropy measures.
Furthermore, we analyzed several types of white noises of different power and mixed them with sinusoidal signal to evaluate the VDS method and compare against the conventional method.Six white noises' powers, −12 dBW, −9 dBW, −6 dBW, −3 dBW, 0 dBW and 3 dBW, respectively, were investigated and mixed with sinusoidal signal.Figure 4 presents the sinusoidal signal and the mixed-signals with white noises of power −12 dBW, −6 dBW and 0 dBW, respectively.
(a) (b) The computation time of SpEn in ten random signals using the VDS method and conventional method, the computation time of SpEn is significantly (p < 0.005) reduced using the VDS method; (b) The percentages of vectors determined to be dissimilar by the VDS decision, for the 10 random signals, the percentages are 84.7%,84.3%, 84.5%, 84.7%, 84.5%, 84.9%, 85.1%, 84.0%, 85.1% and 84.7%, respectively, as determined by the VDS decision during computation.Figure 5a shows the computation times of SpEn in the six mixed-signals using the VDS and conventional method with the computational parameters N = 1000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each mixed-signal.It is clearly observed that the computation time of the VDS method is significantly (p < 0.005) lower than that of the conventional method.For the VDS method the changing trend of the computation time increases with the white noise's power increase.Figure 5b shows the measurement values of SpEn in the sinusoidal signal and six mixedsignals.The measurement values of SpEn by the adoption of the VDS and conventional method are identical.The larger power of the white noise the mixed-signal is mixed with, the greater value of SpEn in the mixed-signal is and the value of SpEn in the mixed-signal would reach a certain limit with the increase of the white noise's power.Figure 5a shows the computation times of SpEn in the six mixed-signals using the VDS and conventional method with the computational parameters N = 1000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each mixed-signal.It is clearly observed that the computation time of the VDS method is significantly (p < 0.005) lower than that of the conventional method.For the VDS method the changing trend of the computation time increases with the white noise's power increase.Figure 5b shows the measurement values of SpEn in the sinusoidal signal and six mixed-signals.
The measurement values of SpEn by the adoption of the VDS and conventional method are identical.The larger power of the white noise the mixed-signal is mixed with, the greater value of SpEn in the mixed-signal is and the value of SpEn in the mixed-signal would reach a certain limit with the increase of the white noise's power.

Accelerating Computation of Sample Entropy (SpEn) in EEG Signals
In order to demonstrate practical application of the VDS method in entropy measures, the computation of SpEn in EEG signals was investigated.The EEG signals were from the EEGMMID, which corresponded to the first set of experimental data of the subject labeled S011.Although the signals were of 64 channels in the records, the first ten channels of signals were used to compute SpEn.

Accelerating Computation of Sample Entropy (SpEn) in EEG Signals
In order to demonstrate practical application of the VDS method in entropy measures, the computation of SpEn in EEG signals was investigated.The EEG signals were from the EEGMMID, which corresponded to the first set of experimental data of the subject labeled S011.Although the signals were of 64 channels in the records, the first ten channels of signals were used to compute SpEn.

Accelerating Computation of Sample Entropy (SpEn) in EEG Signals
In order to demonstrate practical application of the VDS method in entropy measures, the computation of SpEn in EEG signals was investigated.The EEG signals were from the EEGMMID, which corresponded to the first set of experimental data of the subject labeled S011.Although the signals were of 64 channels in the records, the first ten channels of signals were used to compute SpEn.By adopting the VDS and conventional method, we computed the SpEn in the EEG signals and measured the computation time with the computational parameters N = 1000, m = 2, r = 0.2 × STD, where STD was the standard deviation of each EEG signal.Table 1 gives the measurement results of SpEn in the ten channels' EEG signals.The measurement values of SpEn by the adoption of the VDS and conventional method are identical.Figure 7a presents the computation time of SpEn in the ten channels' EEG signals.The mean computation times are 0.0434 s and 0.2063 s for the VDS and conventional method, respectively.The standard deviation of the computation time for the VDS method is greater than that for the conventional method.These experimental results indicate that the mean computation time of SpEn is greatly (p < 0.005) reduced using the VDS method, by 78.96%.
measured the computation time with the computational parameters N = 1000, m = 2, r = 0.2 × STD, where STD was the standard deviation of each EEG signal.Table 1 gives the measurement results of SpEn in the ten channels' EEG signals.The measurement values of SpEn by the adoption of the VDS and conventional method are identical.Figure 7a presents the computation time of SpEn in the ten channels' EEG signals.The mean computation times are 0.0434 s and 0.2063 s for the VDS and conventional method, respectively.The standard deviation of the computation time for the VDS method is greater than that for the conventional method.These experimental results indicate that the mean computation time of SpEn is greatly (p < 0.005) reduced using the VDS method, by 78.96%.
To count the number of vectors which were determined to be dissimilar by the VDS decision during computation, the SpEn in the ten channels' EEG signals was computed once again by the testing program.The percentages of vectors which are dissimilar as determined by the VDS decision in the computation process are calculated as shown in Figure 7b.The results are 84.7%,85.8%, 85.6%, 86.6%, 87.5%, 84.3%, 82.8%, 86.3%, 86.5% and 86.6% for the ten channels' EEG signals.The experimental results reveal that about 85.6% of arbitrary two vectors are deemed to be dissimilar for the ten channels' EEG signals, as determined by the VDS decision.Their distances are avoided in calculations to determine whether or not they are similar during the computation of the SpEn.We investigated the influence of parameter N and r on the time efficiency of computation.The computation of SpEn in the ten channels' EEG signals with five kinds of lengths was performed by using the VDS method and the conventional method, the computation time of SpEn is significantly (p < 0.005) reduced using the VDS method; (b) The percentages of vectors determined to be dissimilar by the VDS decision, for the ten channels' EEG signals the percentages are 84.7%,85.8%, 85.6%, 86.6%, 87.5%, 84.3%, 82.8%, 86.3%, 86.5% and 86.6%, respectively, as determined by the VDS decision in the computation process.
To count the number of vectors which were determined to be dissimilar by the VDS decision during computation, the SpEn in the ten channels' EEG signals was computed once again by the testing program.The percentages of vectors which are dissimilar as determined by the VDS decision in the computation process are calculated as shown in Figure 7b.The results are 84.7%,85.8%, 85.6%, 86.6%, 87.5%, 84.3%, 82.8%, 86.3%, 86.5% and 86.6% for the ten channels' EEG signals.The experimental results reveal that about 85.6% of arbitrary two vectors are deemed to be dissimilar for the ten channels' EEG signals, as determined by the VDS decision.Their distances are avoided in calculations to determine whether or not they are similar during the computation of the SpEn.
We investigated the influence of parameter N and r on the time efficiency of computation.The computation of SpEn in the ten channels' EEG signals with five kinds of lengths was performed by adopting the VDS method and the conventional method and involved measurement of the computation times.The five lengths selected were N = 500, N = 1000, N = 2000, N = 4000 and N = 8000 and the other computational parameters were m = 2 and r = 0.2 × STD, where STD was the standard deviation of each EEG signal.For each given length's EEG signals, the mean computation time of SpEn was calculated by averaging the ten channels' EEG signals, as shown in Table 2.For the lengths of N = 500, N = 1000, N = 2000, N = 4000, N = 8000, the time-reduced rates of computation of SpEn using the VDS method were 80.1%, 80.3%, 82.3%, 82.8% and 82.8%, respectively, as compared with the conventional method.We investigated the computation time of SpEn in the ten channels' EEG signals with different r parameters by the VDS method.The variable β was defined in the parameter r, r = β × STD.The value of β ranged from 0.05 to 0.3, with an increase by step length 0.05.The other computational parameters were N = 1000 and m = 2.We performed the computation of SpEn in ten channels' EEG signals by adopting the VDS method and measured their computation times.Figure 8a shows the changes in computation times of SpEn over different r parameters.For each channel's EEG signals, the computation time of SpEn increases to almost the same degree as the value of parameter r and the greater the value of parameter r is, the longer the computation time is.adopting the VDS method and the conventional method and involved measurement of the computation times.The five lengths selected were N = 500, N = 1000, N = 2000, N = 4000 and N = 8000 and the other computational parameters were m = 2 and r = 0.2 × STD, where STD was the standard deviation of each EEG signal.For each given length's EEG signals, the mean computation time of SpEn was calculated by averaging the ten channels' EEG signals, as shown in Table 2.For the lengths of N = 500, N = 1000, N = 2000, N = 4000, N = 8000, the time-reduced rates of computation of SpEn using the VDS method were 80.1%, 80.3%, 82.3%, 82.8% and 82.8%, respectively, as compared with the conventional method.We investigated the computation time of SpEn in the ten channels' EEG signals with different r parameters by the VDS method.The variable β was defined in the parameter r, r = β × STD.The value of β ranged from 0.05 to 0.3, with an increase by step length 0.05.The other computational parameters were N = 1000 and m = 2.We performed the computation of SpEn in ten channels' EEG signals by adopting the VDS method and measured their computation times.Figure 8a shows the changes in computation times of SpEn over different r parameters.For each channel's EEG signals, the computation time of SpEn increases to almost the same degree as the value of parameter r and the greater the value of parameter r is, the longer the computation time is.
We further checked the efficiency of the VDS method for different m parameters.The value of m ranged from 2 to 17 and the other computational parameters N = 1000 and r = 0.2 × STD. Figure 8b shows the computation time over different m parameters and the computation time is given as the means ± standard error.It is clearly observed that the mean computation time of SpEn increases with the increase of the value of parameter m.In order to evaluate the robustness of the VDS method, we carried out the computation of SpEn with the EEG signals of ten subjects.The ten subjects' EEG signals were from the EEGMMID, which was the first set of experimental data of the subjects labeled from S001 to S010, corresponding to the  We further checked the efficiency of the VDS method for different m parameters.The value of m ranged from 2 to 17 and the other computational parameters N = 1000 and r = 0.2 × STD. Figure 8b shows the computation time over different m parameters and the computation time is given as the means ± standard error.It is clearly observed that the mean computation time of SpEn increases with the increase of the value of parameter m.
In order to evaluate the robustness of the VDS method, we carried out the computation of SpEn with the EEG signals of ten subjects.The ten subjects' EEG signals were from the EEGMMID, which was the first set of experimental data of the subjects labeled from S001 to S010, corresponding to the electrode name Cz (numbering 11 in the records).The ten subjects' EEG signals are shown in Figure 9.  Figure 10 shows that the computation time of SpEn in the ten subjects' EEG signals by the VDS and conventional method.For the conventional method, the mean computation time of SpEn in the ten subjects' EEG signals is 0.199 s but for the VDS method the mean computation time of SpEn is 0.038 s, thus the computation time is significantly (p < 0.005) reduced by 80.9%.The standard deviation of the computation time for the VDS method is greater that for the conventional method.The measurement values of SpEns by adoption of the VDS and conventional method are identical and are 1.12, 1.52, 1.32, 1.40, 1.36, 1.80, 1.57, 0.97, 1.87 and 1.30, respectively.This experimental result indicates that the VDS method is sufficiently robust and effective for accelerating computation of SpEn in the EEG signals for different subjects.Figure 10 shows that the computation time of SpEn in the ten subjects' EEG signals by the VDS and conventional method.For the conventional method, the mean computation time of SpEn in the ten subjects' EEG signals is 0.199 s but for the VDS method the mean computation time of SpEn is 0.038 s, thus the computation time is significantly (p < 0.005) reduced by 80.9%.The standard deviation of the computation time for the VDS method is greater that for the conventional method.The measurement values of SpEns by adoption of the VDS and conventional method are identical and are 1.12, 1.52, 1.32, 1.40, 1.36, 1.80, 1.57, 0.97, 1.87 and 1.30, respectively.This experimental result indicates that the VDS method is sufficiently robust and effective for accelerating computation of SpEn in the EEG signals for different subjects.

Accelerating Computations of SpEn in Electromyography (EMG) and Electrocardiogram (ECG) Signals
In order to demonstrate the robustness and data-dependent of the VDS method, we further analyzed the electromyography (EMG) and electrocardiogram (ECG) signals.The EMG signals were obtained from the Physical Action Data Set [38], which included 10 normal and 10 aggressive physical actions that measured the human activity from 4 subjects.In our study, the EMG signals were selected from the channel measured on the right bicep of the first subject and contained eight normal physical actions: (1) bowing; (2) handshaking; (3) hugging; (4) seating; (5) walking; (6) jumping; (7) standing; and (8) running.The ECG signals were obtained from the MIT-BIH Arrhythmia Database [37], where contained 48 half-hour excerpts of two-channel ambulatory ECG recordings and were digitized at 360 samples per second.In our study, the ECG signals used were the recording data numbering 100, 101, 112 and 217 in the records.Figure 11 shows the first 3000 samples of the EMG signals of two normal actions, bowing and running and the first 2000 samples of the ECG signals numbering 100 in the records.
We computed the SpEn in the eight EMG signals using the VDS and conventional method with the computational parameters N = 3000, m = 2, r = 0.2 × STD, where STD was the standard deviation of each EMG signal.Figure 12a shows that the computation time of SpEn in the eight EMG signals.
For the conventional method the mean computation time of SpEn is 1.848 s but for the VDS method the mean computation time of SpEn is 0.396 s.Therefore the computation time is significantly (p < 0.005) reduced using the VDS method.The standard deviation of the computation time for the VDS method is greater than that for the conventional method.The measurement values of SpEn in the eight EMG signals by the adoption of the VDS and conventional method are identical and are 1.79, 0.49, 0.85, 2.18, 1.76, 1.13, 2.16 and 0.27, respectively.

Accelerating Computations of SpEn in Electromyography (EMG) and Electrocardiogram (ECG) Signals
In order to demonstrate the robustness and data-dependent of the VDS method, we further analyzed the electromyography (EMG) and electrocardiogram (ECG) signals.The EMG signals were obtained from the Physical Action Data Set [38], which included 10 normal and 10 aggressive physical actions that measured the human activity from 4 subjects.In our study, the EMG signals were selected from the channel measured on the right bicep of the first subject and contained eight normal physical actions: (1) bowing; (2) handshaking; (3) hugging; (4) seating; (5) walking; (6) jumping; (7) standing; and (8) running.The ECG signals were obtained from the MIT-BIH Arrhythmia Database [37], where contained 48 half-hour excerpts of two-channel ambulatory ECG recordings and were digitized at 360 samples per second.In our study, the ECG signals used were the recording data numbering 100, 101, 112 and 217 in the records.Figure 11 shows the first 3000 samples of the EMG signals of two normal actions, bowing and running and the first 2000 samples of the ECG signals numbering 100 in the records.
We computed the SpEn in the eight EMG signals using the VDS and conventional method with the computational parameters N = 3000, m = 2, r = 0.2 × STD, where STD was the standard deviation of each EMG signal.Figure 12a shows that the computation time of SpEn in the eight EMG signals.
For the conventional method the mean computation time of SpEn is 1.848 s but for the VDS method the mean computation time of SpEn is 0.396 s.Therefore the computation time is significantly (p < 0.005) reduced using the VDS method.The standard deviation of the computation time for the VDS method is greater than that for the conventional method.The measurement values of SpEn in the eight EMG signals by the adoption of the VDS and conventional method are identical and are 1.79, 0.49, 0.85, 2.18, 1.76, 1.13, 2.16 and 0.27, respectively.Figure 12b shows that the computation time of SpEn in the eight ECG signals with the computational parameters N = 2000, m = 2, r = 0.2 × STD, where STD is the standard deviation of each ECG signal.The mean computation times of SpEn are 0.892 s and 0.218 s corresponding to the VDS and the conventional method, respectively.For the ECG signals, the mean computation time is also significantly (p < 0.005) reduced using the VDS method and the standard deviation of the computation time for the VDS method is also greater that for the conventional method.Figure 12b shows that the computation time of SpEn in the eight ECG signals with the computational parameters N = 2000, m = 2, r = 0.2 × STD, where STD is the standard deviation of each ECG signal.The mean computation times of SpEn are 0.892 s and 0.218 s corresponding to the VDS and the conventional method, respectively.For the ECG signals, the mean computation time is also significantly (p < 0.005) reduced using the VDS method and the standard deviation of the computation time for the VDS method is also greater that for the conventional method.Figure 12b shows that the computation time of SpEn in the eight ECG signals with the computational parameters N = 2000, m = 2, r = 0.2 × STD, where STD is the standard deviation of each ECG signal.The mean computation times of SpEn are 0.892 s and 0.218 s corresponding to the VDS and the conventional method, respectively.For the ECG signals, the mean computation time is also significantly (p < 0.005) reduced using the VDS method and the standard deviation of the computation time for the VDS method is also greater that for the conventional method.

Accelerating Computations of Time-Shift Multiscale Entropy (TSME) and Approximate Entropy (ApEn)
In order to demonstrate application potential, we exploited the VDS method to achieve accelerating the computation of TSME in EEG signals.The TSME was proposed by Pham in 2017 [34] and the author has opened the original codes related to computation of TSME in his personal homepage [39].The EEG signals were from the EEGMMID, representing the first set of experimental data of the subjects labeled from S001 to S010 corresponding to electrode name Fz (numbering 34 in the records) as shown in Figure 13.

Accelerating Computations of Time-Shift Multiscale Entropy (TSME) and Approximate Entropy (ApEn)
In order to demonstrate application potential, we exploited the VDS method to achieve accelerating the computation of TSME in EEG signals.The TSME was proposed by Pham in 2017 [34] and the author has opened the original codes related to computation of TSME in his personal homepage [39].The EEG signals were from the EEGMMID, representing the first set of experimental data of the subjects labeled from S001 to S010 corresponding to electrode name Fz (numbering 34 in the records) as shown in Figure 13.9. We calculated the ApEns by adopting two algorithms, one exploited the VDS method to accelerate the computation of the ApEn and the other was a conventional algorithm without utilizing the VDS method.Figure 14 shows the computation time of ApEn in the ten subjects' EEG signals.The mean computation times are 0.0395 s and 0.206 s corresponding to the algorithm exploiting the VDS method and the conventional algorithm without exploiting VDS method, respectively.The computation time of ApEn is significantly (p < 0.005) reduced by utilizing the VDS method and the standard deviation of the computation time for the algorithm exploiting the VDS method is greater that for the conventional algorithm.These results indicate that the computation of ApEn can be greatly accelerated by exploiting the VDS method.The measurement values of ApEn by adoption of the two algorithms are identical.Table 5 gives the measurement values of ApEn and SpEn in the ten subjects' EEG signals.There are some issues that need to be pointed out.The methods are implemented with MATLAB2016 (The Mathworks Inc., Natick, MA, USA) and run on an ordinary computer (Intel Core i7 CPU @ 3.2 GHz; 16 GB RAM; Windows 7).It is well-known that different programming languages and computing speeds of hardware can lead to different executing times.However, the changing trends of relative computation time of between the VDS method and the conventional method are consistent, no matter what kinds of programming language and hardware are used in experiment.Thus, the experimental conclusions with respect to time efficiency of computation by the VDS method are reliable.
There are some limitations to the VDS method.On one hand, the level of optimization of the computation time by the VDS method depends on the statistic property of the signal itself.The decision quantity, Decis(i,j), is defined as the absolute value of the sum of all components in vectors ) (m i X minus the sum of all components in vectors ) (m j

X
and has some kind of statistical characteristic.Only when the Decis(i,j) of arbitrary two vectors is greater than m × r can the computation time of entropy measures be optimized.On the other hand, for the VDS method, the time efficiency of computation by the VDS method may reach a certain limit.This fact can be observed from the experimental results in Section 3.For five kinds of EEG signals of different lengths, the timereduced rate of computation of SpEn by the VDS method is about 82% compared with the conventional method.In practical applications, the longer the length of data, the greater the computation time.Thus, the computation time of SpEn may still be long for signals of long length, even though the VDS method is adopted.Despite these limitations, the proposed VDS method is still considered to be promising.One advantage is that the VDS method may be jointly applied with other fast algorithms such as the existing SKD algorithm [21], recursive sample entropy algorithm [23] and so on, to further accelerate computation of entropy measures.There is no doubt that the proposed method is applicable to other entropy measures, such as ApEn and MSE.This implies a potential for extensive application, which is of important value.

Conclusions
In this study, we reported a reliable method capable of accelerating the computation of entropy measures such as ApEn, SpEn, TSME and so on.The experimental results demonstrate convincingly that the computation of entropy measures can be sped up by exploiting the VDS method.Taking advantage of the VDS method, the time-reduced rates of the computation of SpEn in random signals and EEG signals are low at 78.5% and 78.9%, respectively, compared with the conventional method.By quantitative analyses, we find that the percentage of vectors determined as being dissimilar by There are some issues that need to be pointed out.The methods are implemented with MATLAB2016 (The Mathworks Inc., Natick, MA, USA) and run on an ordinary computer (Intel Core i7 CPU @ 3.2 GHz; 16 GB RAM; Windows 7).It is well-known that different programming languages and computing speeds of hardware can lead to different executing times.However, the changing trends of relative computation time of between the VDS method and the conventional method are consistent, no matter what kinds of programming language and hardware are used in experiment.Thus, the experimental conclusions with respect to time efficiency of computation by the VDS method are reliable.
There are some limitations to the VDS method.On one hand, the level of optimization of the computation time by the VDS method depends on the statistic property of the signal itself.The decision quantity, Decis(i,j), is defined as the absolute value of the sum of all components in vectors X (m) i minus the sum of all components in vectors X (m) j and has some kind of statistical characteristic.Only when the Decis(i,j) of arbitrary two vectors is greater than m × r can the computation time of entropy measures be optimized.On the other hand, for the VDS method, the time efficiency of computation by the VDS method may reach a certain limit.This fact can be observed from the experimental results in Section 3.For five kinds of EEG signals of different lengths, the time-reduced rate of computation of SpEn by the VDS method is about 82% compared with the conventional method.In practical applications, the longer the length of data, the greater the computation time.Thus, the computation time of SpEn may still be long for signals of long length, even though the VDS method is adopted.Despite these limitations, the proposed VDS method is still considered to be promising.One advantage is that the VDS method may be jointly applied with other fast algorithms such as the existing SKD algorithm [21], recursive sample entropy algorithm [23] and so on, to further accelerate computation of entropy measures.There is no doubt that the proposed method is applicable to other entropy measures, such as ApEn and MSE.This implies a potential for extensive application, which is of important value.

Conclusions
In this study, we reported a reliable method capable of accelerating the computation of entropy measures such as ApEn, SpEn, TSME and so on.The experimental results demonstrate convincingly that the computation of entropy measures can be sped up by exploiting the VDS method.Taking advantage of the VDS method, the time-reduced rates of the computation of SpEn in random signals and EEG signals are low at 78.5% and 78.9%, respectively, compared with the conventional method.By quantitative analyses, we find that the percentage of vectors determined as being dissimilar by the VDS decision in the computation process is about 84.5% for the random signals and there is an average of 85.6% for the EEG signals.For EEG signals of different lengths (N = 500, N = 1000, N = 2000, N = 4000, N = 8000), the time-reduced rate of the computation of SpEn by the VDS method is about 80.1~82.8%.We further demonstrate the use of the VDS method for success in accelerating the computations of not only SpEn in EMG and ECG signals but also TSME and ApEn in EEG signals.We confirm that the VDS method is effective and practical for accelerating the computation of entropy measures and deem that it has promising applications in the area of biomedical informatics.
The Matlab codes with respect to the VDS method for accelerating computation of SpEn, the conventional method for computation of SpEn, the testing program for quantitative analysis and the VDS method for accelerating the computation of TSME are all available at the author's personal homepage: https://sites.google.com/site/hitluyun/code/acemvds.

Figure 1 .
Figure 1.Implementation of the vectors with dissimilarity (VDS) method for accelerating the computation of sample entropy (SpEn).

Figure 1 .
Figure 1.Implementation of the vectors with dissimilarity (VDS) method for accelerating the computation of sample entropy (SpEn).

Figure 2 .
Figure 2. The random signal of 1000 data points uniformly distributed in the interval (0, 1).

Figure 2 .
Figure 2. The random signal of 1000 data points uniformly distributed in the interval (0, 1).

Figure 3 .
Figure 3. (a)The computation time of SpEn in ten random signals using the VDS method and conventional method, the computation time of SpEn is significantly (p < 0.005) reduced using the VDS method; (b) The percentages of vectors determined to be dissimilar by the VDS decision, for the 10 random signals, the percentages are 84.7%,84.3%, 84.5%, 84.7%, 84.5%, 84.9%, 85.1%, 84.0%, 85.1% and 84.7%, respectively, as determined by the VDS decision during computation.

Figure 3 .
Figure 3. (a)The computation time of SpEn in ten random signals using the VDS method and conventional method, the computation time of SpEn is significantly (p < 0.005) reduced using the VDS method; (b) The percentages of vectors determined to be dissimilar by the VDS decision, for the 10 random signals, the percentages are 84.7%,84.3%, 84.5%, 84.7%, 84.5%, 84.9%, 85.1%, 84.0%, 85.1% and 84.7%, respectively, as determined by the VDS decision during computation.

Figure 4 .
Figure 4.The sinusoidal signal and the mixed-signals with white noises of power −12 dBW, −6 dBW and 0 dBW, respectively.

Figure 4 .
Figure 4.The sinusoidal signal and the mixed-signals with white noises of power −12 dBW, −6 dBW and 0 dBW, respectively.

Figure 5 .
Figure 5. (a) The computation times of SpEn in the six mixed-signals using the VDS and conventional method; (b) The measurement values of SpEn in the sinusoidal signal and six mixed-signals.

Figure 6
shows the ten channels' EEG signals and the lengths of EEG signals are N = 1000.The vertical axis indicates the amplitude of EEG signals; the units are microvolts.

Figure 6 .
Figure 6.Electroencephalogram (EEG) signals of length N = 1000 from ten channels.The markers from GP1 to GP10 correspond to the ten different EEG signals.The vertical axis indicates the amplitude of the EEG signals.

Figure 5 .
Figure 5. (a) The computation times of SpEn in the six mixed-signals using the VDS and conventional method; (b) The measurement values of SpEn in the sinusoidal signal and six mixed-signals.

Figure 6 Figure 5 .
Figure 5. (a) The computation times of SpEn in the six mixed-signals using the VDS and conventional method; (b) The measurement values of SpEn in the sinusoidal signal and six mixed-signals.

Figure 6
shows the ten channels' EEG signals and the lengths of EEG signals are N = 1000.The vertical axis indicates the amplitude of EEG signals; the units are microvolts.

Figure 6 .
Figure 6.Electroencephalogram (EEG) signals of length N = 1000 from ten channels.The markers from GP1 to GP10 correspond to the ten different EEG signals.The vertical axis indicates the amplitude of the EEG signals.

Figure 6 .
Figure 6.Electroencephalogram (EEG) signals of length N = 1000 from ten channels.The markers from GP1 to GP10 correspond to the ten different EEG signals.The vertical axis indicates the amplitude of the EEG signals.

Figure 7 .
Figure 7. (a)The computation time of SpEn in the ten channels' EEG signals of length N = 1000 using the VDS method and the conventional method, the computation time of SpEn is significantly (p < 0.005) reduced using the VDS method; (b) The percentages of vectors determined to be dissimilar by the VDS decision, for the ten channels' EEG signals the percentages are 84.7%,85.8%, 85.6%, 86.6%, 87.5%, 84.3%, 82.8%, 86.3%, 86.5% and 86.6%, respectively, as determined by the VDS decision in the computation process.

Figure 7 .
Figure 7. (a)The computation time of SpEn in the ten channels' EEG signals of length N = 1000 using the VDS method and the conventional method, the computation time of SpEn is significantly (p < 0.005) reduced using the VDS method; (b) The percentages of vectors determined to be dissimilar by the VDS decision, for the ten channels' EEG signals the percentages are 84.7%,85.8%, 85.6%, 86.6%, 87.5%, 84.3%, 82.8%, 86.3%, 86.5% and 86.6%, respectively, as determined by the VDS decision in the computation process.

Figure 8 .
Figure 8.(a) The changes in computation times of SpEn in the ten channels' EEG signals over parameter β; (b) The mean computation time of SpEn in the ten channels' EEG signals increases with the increase of the value of parameter m.

Figure 8 .
Figure 8.(a) The changes in computation times of SpEn in the ten channels' EEG signals over parameter β; (b) The mean computation time of SpEn in the ten channels' EEG signals increases with the increase of the value of parameter m.

Figure 9 .
Figure 9. EEG signals from ten different subjects corresponding to electrode name Cz (numbering 11 in the records).The markers from Sb1 to Sb10 correspond to different experimental subjects, labeled from S001 to S010.

Figure 9 .
Figure 9. EEG signals from ten different subjects corresponding to electrode name Cz (numbering 11 in the records).The markers from Sb1 to Sb10 correspond to different experimental subjects, labeled from S001 to S010.

Figure 10 .
Figure 10.The computation time of SpEn in ten subjects' EEG signals by adopting the VDS method and conventional method.The computational parameters are N = 1000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each EEG signal.The computation time of SpEn is significantly (p < 0.005) reduced using the VDS method.

Figure 10 .
Figure 10.The computation time of SpEn in ten subjects' EEG signals by adopting the VDS method and conventional method.The computational parameters are N = 1000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each EEG signal.The computation time of SpEn is significantly (p < 0.005) reduced using the VDS method.

Figure 12 .Figure 11 .
Figure 12.The computation times of SpEn in EMG and ECG signals using the VDS and conventional method, for the EMG and ECG signals the computation times of the VDS method are all significantly (p < 0.005) reduced using the VDS method.(a) The mean computation times of SpEn by adoption of the VDS and conventional method are 0.396 s and 1.848 s, respectively; (b) the mean computation times of SpEn by adoption of the VDS method and conventional method are 0.218 s and 0.892 s, respectively.

Figure 12 .Figure 12 .
Figure 12.The computation times of SpEn in EMG and ECG signals using the VDS and conventional method, for the EMG and ECG signals the computation times of the VDS method are all significantly (p < 0.005) reduced using the VDS method.(a) The mean computation times of SpEn by adoption of the VDS and conventional method are 0.396 s and 1.848 s, respectively; (b) the mean computation times of SpEn by adoption of the VDS method and conventional method are 0.218 s and 0.892 s, respectively.

Figure 13 .
Figure 13.EEG signals from ten different subjects corresponding to electrode name Fz (numbering 34 in the records).The markers from Sb1 to Sb10 correspond to different experimental subjects labeled from S001 to S010.

Figure 14 .
Figure 14.The computation time of ApEn in the ten subjects' EEG signals by the algorithm exploiting the VDS method and conventional algorithm without exploiting the VDS method.The computation time of ApEn is significantly (p < 0.005) reduced by exploiting the VDS method.

Table 1 .
The measurement of SpEn in the ten channels' EEG signals.The computational parameters are N = 1000, m = 2, r = 0.2 × STD, where STD is the standard deviation of each EEG signal.The channel numbers from CH1 to CH10 correspond to the ten channels' EEG signals.

Table 1 .
The measurement of SpEn in the ten channels' EEG signals.The computational parameters are N = 1000, m = 2, r = 0.2 × STD, where STD is the standard deviation of each EEG signal.The channel numbers from CH1 to CH10 correspond to the ten channels' EEG signals.

Table 2 .
The mean computation time of SpEn in EEG signals with five kinds of lengths.The five lengths are: N = 500, N = 1000, N = 2000, N = 4000, N = 8000, respectively; the other computational parameters are m = 2 and r = 0.2 × STD, where STD is the standard deviation of each EEG signal.

Table 2 .
The mean computation time of SpEn in EEG signals with five kinds of lengths.The five lengths are: N = 500, N = 1000, N = 2000, N = 4000, N = 8000, respectively; the other computational parameters are m = 2 and r = 0.2 × STD, where STD is the standard deviation of each EEG signal.

Table 3 .
The mean computation time of time-shift multiscale entropy (TSME) in the signals of ten groups' signals using the VDS method and Pham's method.The mean computation time with time intervals from kmax = 1 to kmax = 8 are calculated by averaging the ten groups' EEG signals.The computational parameters are N = 8000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each EEG signal.

Table 4 .
For the given time interval kmax = 8, the TSME for each k, k = 1, 2, ..., kmax, in ten groups' EEG signals is shown.The computational parameters are N = 8000, m = 2 and r = 0.2 × STD, where STD is the standard deviation of each EEG signal.Finally, we further demonstrated the VDS method to accelerate the computation of ApEn in EEG signals.The EEG signals used were the ten subjects' EEG signals in Figure

Table 5 .
The measurement values of ApEn and SpEn in the ten subjects' EEG signals.The computational parameters are N = 1000, m = 2, r = 0.2 × STD, where STD is the standard deviation of each EEG signal.The computation time of ApEn in the ten subjects' EEG signals by the algorithm exploiting the VDS method and conventional algorithm without exploiting the VDS method.The computation time of ApEn is significantly (p < 0.005) reduced by exploiting the VDS method.