Research about the Characteristics of Chaotic Systems Based on Multi-Scale Entropy

The logistic chaotic system, as a classical complex phenomenon of nonlinear dynamic systems, has received extensive attention in the field of secure communication. It is generally believed that the characteristics of chaos are suitable for the needs of encryption systems. In this paper, a multi-scale entropy theory analysis and statistical analysis are carried out on the chaotic sequences produced by different parameters and different initial values of logistic systems. According to the simulation results, the complexity of the chaotic system represented by the logistic system is mainly decided by parameter μ. Not all characteristic parameters of the chaotic system depend on the initial values. It is possible to make a reasonable estimation and prediction of the chaotic system from a macroscopic level. A variance estimation method for the parameter μ is proposed and applied to a logistic system and to another chaotic system, which is equally effective.


Introduction
Chaos, as a classical complex phenomenon of a nonlinear dynamic system, is widely used in military and commercial spread spectrum communication systems with high demands for secrecy, because of its characteristics of wideband, quasi-noise, and sensitivity to initial state [1]. The chaotic system, a logistic chaotic map, has been paid much attention by researchers and has been studied extensively. In 1978, Feigenbaum [2] made a detailed analysis of its mathematical properties. The literature [3] has realized the bifurcation control of the multiple periods of the logistic model. The authors of [4] discussed the influence of delay time on the transition probabilities between the metastable state and the stable state of a logistic system. The authors of [5] applied the model to the evolution of biomolecular networks. The authors of [6,7] made a detailed description of the bifurcation and fractional dimension of the multiple complex logistic systems. The authors of [8] studied the chaotic fractal characteristics of C-K mapping, which is partially similar to logistic sequences. It can be seen that the logistic model has had a lot of research done on its local features, and these research work plays an important role in the field of information communication.
As a chaotic system, unpredictability and uncertainty are its most important features. From the point of view of information theory, to study the whole uncertainty of a function, in addition to studying its closed expression, we can also make a statistical analysis and a quantification of the discrete sequences generated by it. In information theory, the best way to quantify uncertainty is entropy. A modified permutation entropy was proposed in literature [9], in order to obtain a quantitative estimate of the Kolmogorov-Sinai entropy in hyper-chaotic models. The authors of [10] introduced a multivariate permutation entropy (MvPE) method and used it to quantify the complexity of chaotic systems. As an 1.
Set x 1 , x 2 , . . . , x L as a discrete-time sequence, including L points. Conduct coarse-graining conversion on the original time series, and divide the original time series into non-overlapping windows with a length of τ. Calculate the average value of each window, and obtain the new coarse-graining time series. Each new data point is derived from Equation (2), as follows: x(i), j = 1 L/τ.
In Equation (2), τ is the scale factor. Each coarse-graining time series has a length of L/τ. Equation (3) and Equation (4) are the methods for computing the coarse-graining of the time series, in which τ = 2 and τ = 3. When τ is 1, it is the original series. (3)

2.
Then, for a different τ, compute the SampEnon coarse-graining time series. SampEn originates from ApEn. However, the calculation of the approximate entropy also involves comparing its own data, which may induce certain errors. In order to reduce the errors, in the literature [28], for the given threshold value r, Richman calculated the ratio between the number of d Y (τ) (i), Y (τ) ( j) < r and the total distance N − m, denoted as C τ,m i (r).

3.
Calculate the average of all points, as follows: 4. Add dimension m by 1, repeat procedures 2-3, and obtain C τ,m+1 i (i) of scale τ, then recalculate the average C τ,m+1 i (i). In the actual calculation, use SampEn, as follows: MSE is as follows:

Macroscopic Characteristics of Logistic Chaotic Sequences
In order to measure the overall characteristics of the logistic chaotic sequence, all of the initial values and parameters (µ) that can be obtained for the sequence are taken. A simulation experiment is conducted in MATLAB R2010. Logistic sequences are generated after 10,000 iterations of Equation (1). The system accuracy utilizes the default double precision of MATLAB. In the MSE analysis, parameter r is generally 15% of the standard deviation (SD) and remains unchanged at all scales, and does not recalculate for each coarse-graining time series [29]. For SD, in the process of the initial normalization, the variance changed by coarse-graining is related to the time structure of the original time series, so it should be considered by the calculation of entropy. However, initial normalization may also prevent an MSE value of two different time series from being impacted by variance, and the impact comes from the organization of the sequence itself. The difference value of the two adjacent points in the same chaotic sequence is shown in the following Equation (9).
2µ (the width of this interval is 1/µ), it is mapped to x n+1 ∈ µ 2 −1 4µ , µ 4 (width of the interval is 1/4µ). Thus, it can be seen that after one iteration, previously wide area is mapped to the upper bound zone of the system, and the interval is decreased.
Substantial experiments with different parameters were carried out in order to obtain extensive results. More representative results were selected and are shown in Figures 1-3. N = 1000, 2000, and 5000 were chosen as the lengths for the chaotic sequences. Set the parameters of MSE as m = 2, and r = 0.15. The scale factor was τ = 2. The initial values were x 0 ∈ (0, 1). The step was 0.01; 100 points were utilized. The parameters were µ ∈ (3.5, 4), and the step was 0.005. Calculate the MSE of the logistic chaotic time series. Figure 1 shows the analysis results of MSE for the time series N = 1000, where Figure 1a is a 3D figure of the MSE results, Figure 1b is the means of different MSE values when µ is fixed, Figure 1c shows the variances of the MSE values, and Figure 1d is the coefficient variables of a different µ.      As shown in Figure 1a, Figure 2a and 3a, when μ is fixed, regardless of the data length or the initial value of the system, the MSE values of the logistic system remain unchanged. When μ increases, MSE increases as well. For a specific μ, some mutational points may show up when the entropy increases, and the logistic system maintains a bifurcated rather than chaotic status. As a result, the MSE values of the mutational points approach μ < 3.57 or even lower. Some information is summarized, as follows: 1. The 3D graph in Figure 3a appears smoother than Figure 2a and Figure 1a. In the same statistic conditions and threshold conditions, time series with longer lengths indicate a more stable statistical probability and lower degrees of dispersion. Hence, the length of the sequences has an impact on MSE. 2. In Figure 1c,d, the MSE value changes significantly. Because the length of the sequence is reduced by half during the process of coarse-graining. For the MSE method, although the number of points were reduced, the coarse-graining time series is not a subset of the original time series. On the contrary, the series includes all of the information about the original time series. Therefore, the error caused by the reduction of length of the coarse-graining time series is likely to be lower than that caused by the subset of the original time series. That is to say, coarse-graining time series on large time scales is likely more irregular (and is assigned a higher MSE value) than the original time series. 3. Figure 2c As shown in Figures 1a-3a, when µ is fixed, regardless of the data length or the initial value of the system, the MSE values of the logistic system remain unchanged. When µ increases, MSE increases as well. For a specific µ, some mutational points may show up when the entropy increases, and the logistic system maintains a bifurcated rather than chaotic status. As a result, the MSE values of the mutational points approach µ < 3.57 or even lower. Some information is summarized, as follows: 1.
The 3D graph in Figure 3a appears smoother than Figures 1a and 2a. In the same statistic conditions and threshold conditions, time series with longer lengths indicate a more stable statistical probability and lower degrees of dispersion. Hence, the length of the sequences has an impact on MSE. 2.
In Figure 1c,d, the MSE value changes significantly. Because the length of the sequence is reduced by half during the process of coarse-graining. For the MSE method, although the number of points were reduced, the coarse-graining time series is not a subset of the original time series. On the contrary, the series includes all of the information about the original time series. Therefore, the error caused by the reduction of length of the coarse-graining time series is likely to be lower than that caused by the subset of the original time series. That is to say, coarse-graining time series on large time scales is likely more irregular (and is assigned a higher MSE value) than the original time series. 3. Figure 2c,d indicate the variance and coefficient variables of a sequence at N = 2000. The mean value of the variances, excluding the maximum and minimum values, is about 0.0012. There are 23 coefficient of variable (CV) that exceed 5%, including the value near the parameter µ = 3.57, and the sequence is not in a completely chaotic status. Figure 3c,d indicate the variances and coefficient variables of sequences at N = 5000. The maximum variance is 2.5 × 10 −3 . The remaining points are not more than 0.1 × 10 −3 , other than the points near µ = 3.57. The range of coefficient variables is from 1% to 3%. In other words, the changes of the initial value do not change the MSE values of the series. Not all chaotic characters of the logistic system have dependency on the initial value. For this, it is possible to make a reasonable estimation and prediction on a chaotic system from macroscopic perspectives.

The Parameter Estimation of Chaotic Sequences
This section explains the mean variance method based on statistical analysis theory. Then, it is applied on a logistic chaotic sequence and tent chaotic sequence.

The Parameter Estimation of the Logistic Chaotic Sequence
In the field of secure communication, the sensitivity to the initial value is an important feature of the logistic chaotic sequence cipher. The initial value (x 0 ) and parameter (µ) are two important keys of sequence cryptography. Dubois [30] presented the exact closed solution when researching the recurrent generation of logistic chaos maps. It was indicated that the solution of the logistic growth model is not unpredictable, which is consistent with the experimental results mentioned in Section 4.
For a logistic chaotic cipher sequence with an adequate length, the key space of the chaotic sequence cipher can be reduced by taking the current iteration value of x as a key, without considering the initial value (x 0 ), as long as the reasonable value of parameter (µ) is estimated.
The method of estimating parameter µ is based on the theory of statistical analysis. The variance mean method is used to obtain parameters uniformly in each interval within the range of the initial value and the parameter µ, and to generate chaotic sequences according to logistic chaotic mapping. First, a data table based on the mean of variances is generated. When estimating the parameters of a logistic sequence, the variance of the sequence can be obtained, and then the value range of parameter µ can be estimated roughly by looking at the table. The calculation process is as follows.
The initial values range x 0 ∈ [0, 1], and the step is 0.01. The parameter is µ ∈ [3.6, 4.0], the step is 0.05, the length N = 10,000, and the variances of the series are shown in  Figure 4a indicates that different initial values for the same parameter (µ) have no impacts on variance distribution. Figure 4b shows the relationship between the means of variances and parameter µ. Figure 4c indicates that the maximum coefficient variable is not greater than 1%. Considering the relationship between the initial value and MSE value, the parameter µ of the series can be estimated when the logistic sequence has an adequate length. For the chaotic series, the key space of the time series password can be reduced. remaining points are not more than 0.1 × 10 , other than the points near = 3.57. The range of coefficient variables is from 1% to 3%. In other words, the changes of the initial value do not change the MSE values of the series. Not all chaotic characters of the logistic system have dependency on the initial value. For this, it is possible to make a reasonable estimation and prediction on a chaotic system from macroscopic perspectives.

The Parameter Estimation of Chaotic Sequences
This section explains the mean variance method based on statistical analysis theory. Then, it is applied on a logistic chaotic sequence and tent chaotic sequence.

The Parameter Estimation of the Logistic Chaotic Sequence
In the field of secure communication, the sensitivity to the initial value is an important feature of the logistic chaotic sequence cipher. The initial value (x0) and parameter (μ) are two important keys of sequence cryptography. Dubois [30] presented the exact closed solution when researching the recurrent generation of logistic chaos maps. It was indicated that the solution of the logistic growth model is not unpredictable, which is consistent with the experimental results mentioned in Section 4.
For a logistic chaotic cipher sequence with an adequate length, the key space of the chaotic sequence cipher can be reduced by taking the current iteration value of x as a key, without considering the initial value (x0), as long as the reasonable value of parameter (μ) is estimated.
The method of estimating parameter μ is based on the theory of statistical analysis. The variance mean method is used to obtain parameters uniformly in each interval within the range of the initial value and the parameter μ, and to generate chaotic sequences according to logistic chaotic mapping. First, a data table based on the mean of variances is generated. When estimating the parameters of a logistic sequence, the variance of the sequence can be obtained, and then the value range of parameter μ can be estimated roughly by looking at the table. The calculation process is as follows.
The initial values range  Figure 4a indicates that different initial values for the same parameter (μ) have no impacts on variance distribution. Figure 4b shows the relationship between the means of variances and parameter μ. Figure 4c indicates that the maximum coefficient variable is not greater than 1%. Considering the relationship between the initial value and MSE value, the parameter μ of the series can be estimated when the logistic sequence has an adequate length. For the chaotic series, the key space of the time series password can be reduced.  According to the previous study, the initial value x0 does not impact the MSE values of a chaotic series. If a chaotic series has adequate length, the current value of x is a key. Parameter μ is the only key involved. Figure 4b divided into four regions is shown in Figure 5. The relations of the four sections are arranged in Table 1, which are the variances and μ. The range of the parameter μ can be obtained by calculating the variance of a sufficiently long chaotic sequence. Despite the fact that the exact value of the parameter cannot be obtained, it reduces the key space of the sequence to some extent. When the variance of the sequence belongs to Regions 1 or 2, the corresponding range of μ is unique and does not overlap with the other. But if the variance is in Regions 3 or 4, because there are some overlaps in the two regions, the parameter μ is harder to determine. The specific variance of the corresponding parameter μ is shown in Table 1.
In Region 3, the range of μ is from 3.825 to 3.865. As shown in Figure 6, in this region, the chaotic sequences are periodical, while in other regions they are opposite. Figure 7 shows the logistic chaotic mapping at μ = 3.835. Figure 8 shows the logistic mapping at μ = 3.85. In general, the parameter μ of Region 3 is not selected, because of its periodical. If the variance of a sequence is in Regions 3 or 4, Region 4 will have a higher probability.  According to the previous study, the initial value x 0 does not impact the MSE values of a chaotic series. If a chaotic series has adequate length, the current value of x is a key. Parameter µ is the only key involved. Figure 4b divided into four regions is shown in Figure 5. The relations of the four sections are arranged in Table 1, which are the variances and µ. The range of the parameter µ can be obtained by calculating the variance of a sufficiently long chaotic sequence. Despite the fact that the exact value of the parameter cannot be obtained, it reduces the key space of the sequence to some extent. When the variance of the sequence belongs to Regions 1 or 2, the corresponding range of µ is unique and does not overlap with the other. But if the variance is in Regions 3 or 4, because there are some overlaps in the two regions, the parameter µ is harder to determine. The specific variance of the corresponding parameter µ is shown in Table 1.     In Region 3, the range of µ is from 3.825 to 3.865. As shown in Figure 6, in this region, the chaotic sequences are periodical, while in other regions they are opposite. Figure 7 shows the logistic chaotic mapping at µ = 3.835. Figure 8 shows the logistic mapping at µ = 3.85. In general, the parameter µ of Region 3 is not selected, because of its periodical. If the variance of a sequence is in Regions 3 or 4, Region 4 will have a higher probability.

The Parameter Estimated of the Tent Chaotic Sequence
The tent chaotic map is also one of the methods used to generate chaotic sequences in secure communication. As the topologically conjugate of the logistic, the sequence is also applied to the variance mean estimation method. The tent chaotic system is followed by Equation (10), and its system mapping is shown in Figure 9. In the three-dimensional distribution of variance, the CV of the variances for the same parameter μ are no more than 1.4%, which is shown in Figure 10c. That is, the chaotic sequence generated by the tent is not dependent on the initial value, and the sequence is more dependent on parameter μ. Table 2 was generated based on Figure 10b. It is an important reference to estimate the parameter μ of the tent chaotic sequence. There are five regions in Table 2, μ will be estimated when the variance of a tent chaotic sequence belongs to one of the five regions.

The Parameter Estimated of the Tent Chaotic Sequence
The tent chaotic map is also one of the methods used to generate chaotic sequences in secure communication. As the topologically conjugate of the logistic, the sequence is also applied to the variance mean estimation method. The tent chaotic system is followed by Equation (10), and its system mapping is shown in Figure 9.
where x 0 ∈ [0, 1] and parameter µ ∈ [1.0, 2.0]. After 10 3 iterations, 10,000 chaotic sequences with a length of 10 4 were selected. The three-dimensional distribution of the variance of each sequence is shown in Figure 10a. The formula for calculating the variance of different parameters is In the three-dimensional distribution of variance, the CV of the variances for the same parameter µ are no more than 1.4%, which is shown in Figure 10c. That is, the chaotic sequence generated by the tent is not dependent on the initial value, and the sequence is more dependent on parameter µ. Table 2 was generated based on Figure 10b. It is an important reference to estimate the parameter µ of the tent chaotic sequence. There are five regions in Table 2, µ will be estimated when the variance of a tent chaotic sequence belongs to one of the five regions.

The Parameter Estimated of the Tent Chaotic Sequence
The tent chaotic map is also one of the methods used to generate chaotic sequences in secure communication. As the topologically conjugate of the logistic, the sequence is also applied to the variance mean estimation method. The tent chaotic system is followed by Equation (10), and its system mapping is shown in Figure 9. In the three-dimensional distribution of variance, the CV of the variances for the same parameter μ are no more than 1.4%, which is shown in Figure 10c. That is, the chaotic sequence generated by the tent is not dependent on the initial value, and the sequence is more dependent on parameter μ. Table 2 was generated based on Figure 10b. It is an important reference to estimate the parameter μ of the tent chaotic sequence. There are five regions in Table 2, μ will be estimated when the variance of a tent chaotic sequence belongs to one of the five regions.

Conclusions
In this paper, through statistical theoretical analysis and mathematical simulation experiments, the MSE values of chaotic sequences under all of the available parameters are calculated. It is found that the complexity of the logistic chaotic sequences tends to be uniform and the density of periodic points is relatively weakened with the continuous increase of the logistic chaotic sequences. The

Conclusions
In this paper, through statistical theoretical analysis and mathematical simulation experiments, the MSE values of chaotic sequences under all of the available parameters are calculated. It is found that the complexity of the logistic chaotic sequences tends to be uniform and the density of periodic points is relatively weakened with the continuous increase of the logistic chaotic sequences. The initial value does not change the entropy value of the sequence too much, so not all of the characteristic parameters of the chaotic system have initial value dependence. As the chaotic sequence satisfies such a distribution characteristic, it is possible to make a reasonable estimation and prediction of the chaotic system from a macroscopic perspective. Considering that the security algorithm is public, only the key is secret, that is, the key involved in the chaotic sequence cryptography system is the initial value and parameter µ of the chaotic map. The statistical analysis and the variance mean method can reduce the key space and even achieve breakthroughs within a short period of time. Moreover, some system security engineers and mathematical cryptographers indeed give consideration to this influence when new solutions are proposed. The logistic chaotic system and the tent chaotic system were both verified in this paper, which proved that this method was effective.