Wide Range Multiscale Entropy Changes through Development

Abstract

How variability in the brain’s neurophysiologic signals evolves during development is important for a global, system-level understanding of brain maturation and its disturbance in neurodevelopmental disorders. In the current study, we use multiscale entropy (MSE), a measure that has been related to signal complexity, to investigate how this variability evolves during development across a broad range of temporal scales. We computed MSE, standard deviation (STD) and standard spectral analyses on resting EEG from 188 healthy individuals aged 8–22 years old. We found age-related increases in entropy at lower scales (<~20 ms) and decreases in entropy at higher scales (~60–80 ms). Decreases in the overall signal STD were anticorrelated with entropy, especially in the lower scales, where regression analyses showed substantial covariation of observed changes. Our findings document for the first time the scale dependency of developmental changes from childhood to early adulthood, challenging a parsimonious MSE-based account of brain maturation along a unidimensional, complexity measure. At the level of analysis permitted by electroencephalography (EEG), MSE could capture critical spatiotemporal variations in the role of noise in the brain. However, interpretations critically rely on defining how signal STD affects MSE properties.

1. Introduction

The brain operates at the “edge of criticality” between formation and dissipation of network configurations [1], displaying a spontaneous exploratory behavior which, in nonlinear systems, is critically fueled by noise [2,3,4]. Such a perspective challenges the intuitive notion of noise as a nuisance factor and has motivated a rapidly evolving literature seeking significance of brain signals beyond the traditional emphasis on mean responses into investigating their variability [5].

Development is assumed to progress towards an optimization of virtually all expressions of brain functions, and a plethora of changes have been detailed affecting all levels of organization. A system level perspective on development—including an understanding of variability changes—would be the natural framework to accommodate such global modifications, providing a unified account of the changes observed through broad, critical developmental periods such as that spanning childhood to adulthood.

Noise acts on many different spatial and temporal scales from thermal and molecular noise that modify ion channel densities, to neuronal population firing activity as seen in electroencephalography (EEG) recordings [6,7,8]. Given the nontrivial effects of noise, in theory differently affecting each of these multiple non-linear processes, it is difficult to predict how the variability in brain signals changes throughout development. Optimal levels of noise have been postulated—as in the context of stochastic resonance theory [5,9,10]. Since an identical increase of variability could either deteriorate neuronal communications or optimize metastable brain dynamics, the directionality of variability changes assumed to be beneficial to improvements in information processing critically depends on the level at the initial, immature condition.

The variability in brain signals conveys important information about the neural system dynamics [8]. Concepts and analysis techniques derived from nonlinear dynamics are applied under the assumption that the observed time series allows a reconstruction of the underlying multidimensional system [11]. The analytical challenge is deriving a measure that is able to capture key aspects of brain dynamics such as complexity. The term is used to refer to the degree of structured interactions observed in systems that exhibit a mixture of randomness and regularity. Accordingly, an ideal measure would yield optimal values for meaningful variability while being minimal for both completely regular and completely random systems [12,13,14,15].

Multiscale entropy (MSE [11,12]) has been shown to satisfy this requirement. It captures the richness of complex signals by computing predictability estimates at multiple time scales. The use at each scale of a regularity estimator (Sample Entropy, SampEn [16]) that can be applied to finite, relatively short time series further justified its application to complex biological systems and, foremost, the brain [17,18,19,20].

Despite its appeal, there have been relatively few studies assessing the maturation in MSE over the course of development. EEG and magnetoencephalography (MEG) studies consistently reported monotonic increases in MSE, starting from one month to five years [21], and progressing from infancy into adulthood [20,21,22]. Time scales up to 28 ms were explored and the emerging picture is a widespread increase of MSE estimates through development [4]. The relation to complexity—for which no optimal value and no upper limit is postulated—allowed interpreting findings unambiguously as a developmental improvement in information processing [18].

However, MSE offers consideration of a few interpretative issues. Particularly relevant to brain signal analysis are the scale dependency of findings and a special case of parameter dependency, the definition and normalization of the similarity criterion.

Initial guidelines for comparisons of MSE profiles take a unidimensional approach and suggest considering one signal to be more complex than another if its entropy estimates are higher for the majority of time scales [12,15]. However, scale specific findings have also been reported. For instance, within the same scale range explored throughout development, opposite changes have been observed at higher and lower scales in aging [18]. When a wider range was explored, scale-specific differences have also been reported in clinical conditions [19]. It is therefore possible that scale dependency can be missed if a narrow scale range is explored. One main goal of the present study was to explore developmental MSE differences over a broader scale range. It should be noted that the upper range limits are not theoretically motivated but strictly dependent on the length of the examined signal. All above-mentioned studies investigated the EEG/MEG responses during task execution. Differently from resting state analysis such paradigms forces the definition of relatively short time windows. In the current study, we examined age-related changes in resting state EEG in healthy individuals 8–22 years old.

How to define the criterion determining similarity is critical for SampEn computation [23]. The value of the similarity criterion (r) directly affects the magnitude of entropy estimates, and the very estimation can fail for r values that are too high or too low [24]. In theory, a principled choice should be guided by a formal definition of meaningful differences. In practice, it is convenient to normalize the tolerance by the standard deviation of the original time series. Since signal correlation properties contribute to entropy estimates, there is no straightforward relationship between signal distribution and entropy [15]. Accordingly, such normalization of r allows positioning of the definition of similarity within the amplitude distribution while allowing a complementary description. This is of critical importance in developmental studies, where the interaction of dramatic changes in volume conduction and neuronal generators requires r to be tailored to age-specific EEG/MEG amplitudes [22,25]. In MSE, the definition of boundaries is complicated by the assessment of similarity at different time scales. In general, these are adjusted to raw signal standard deviation and kept constant under the assumption that variance at higher scales contains information about the whole signal [15,24,26]. This reasoning is in contrast with the initial normalization procedure, where variance contribution to entropy differences is removed, and critically neglects the time scale dependency of variability, which, during development, is further complicated by the interaction of relative and global power changes [25]. There is, therefore, a fundamental ambiguity in the interpretation of entropy estimates at different scales, hindering the goal of gathering complementary information about development and understanding MSE changes over and beyond spectral differences. Given the importance of making the definition and normalization of similarity boundaries transparent, we gathered standard deviation estimates and assessed its relationship to developmental MSE trajectories.

2. Experimental Section

2.1. Participants

One hundred eighty-eight participants in the 8–22 years age range were recruited from the greater Allegheny County, Pennsylvania to participate in this study. Participants were carefully selected, uniformly sampled in the target age range and matched for demographic and general cognitive ability (IQ, Wechsler Abbreviated Scale of Intelligence [27]). For details about sample descriptions, see Cho et al. [28]. Briefly, any psychiatric or neurological confounding factor was excluded at enrollment. Seven of the enrolled participants were excluded based on participant withdrawal or technical issues. Thus, data from 181 participants were entered into the analyses. Written informed consent was obtained prior to testing in accordance with the Institutional Review Board at the University of Pittsburgh. For purpose of analysis subjects were binned in demographics matched bins in increments of three years (8–10, 11–13, 14–16, 17–19, 20–22 years). Sample full scale age-corrected IQ scores (104.3 ± 13.1), were in agreement with normative data (Z₁₈₁ vs. 100 ± 15 = 3.9, p < 0.001), and matched across bins (Kruskal–Wallis, χ²_4,180 = 5.3, p = 0.25).

2.2. EEG Acquisition and Preprocessing

EEG sessions were conducted in an electrically shielded, sound-attenuated room lit with low-level ambient light. EEG data were collected using a 128-channel Geodesic Sensor Net (EGI, Eugene, OR, USA): 250 Hz sampling rate, common reference at Cz, online elliptical bandpass hardware filter 0.01- to 100-Hz, electrode impedances were maintained at or below 50 kΩ. Subjects underwent an extensive EEG study including resting state acquisitions (70 s eyes open, 70 s eyes closed), cognitive tasks and auditory event related studies. Here, we present analysis only from the resting state, eyes open blocks. These were acquired at the beginning of each session.

EEG data were preprocessed using custom MATLAB scripts and exploiting the extended Infomax independent component analysis (ICA) algorithm [29] as implemented by EEGLAB [30]. Each participant’s EEG data were segmented into 1 s epochs. Prior to ICA, bad channels and epochs containing unique, non-stereotyped artifacts were removed. Components capturing stereotyped artifacts were excluded from reprojection [31]. Residual data segments containing artifacts were eliminated. All preprocessing was implemented through unsupervised algorithms that allowed blind and reproducible procedures and improved by exploiting EEG derived electro-oculogram traces [32].

2.3. Analysis

MSE captures predictability of time series at different time scales through calculation of Sample Entropy (SampEn [16]). First, each time scale level of analysis is achieved by averaging data points within non-overlapping windows of increasing length τ (i.e., the scale factor). For a discrete time series, x = {x₁, ..., x_i, ... , x_N}, such successive coarse-grained time series are {Y^(τ)}:

$$Y_{ j}^{\left(\mathsf{\tau }\right)}=\frac{1}{\mathsf{\tau }}{\displaystyle \sum }_{i=\left(j-1\right)\mathsf{\tau }+1}^{j\mathsf{\tau }}{x}_i,1\le j\le \frac{N}{\mathsf{\tau }}$$

(1)

SampEn is then calculated for each {Y^(τ)}. Taking two similar sequences of a defined pattern length m, SampEn provides an unbiased estimator of the conditional probability that they remain similar at the next point—i.e., their distance remains smaller than a matching criterion r. Consider m-long vectors u^m(i) and let $n{\left(r\right)}_i^m$ be the number of u^m(j) vectors whose Euclidean distance is smaller or equal to r, then:

$$\text{SampEn}\left(m,r,N\right)= ln\frac{{\sum }_{i=1}^{N-m} n{\left(r\right)}_i^m}{ {\sum }_{i=1}^{N-m}n{\left(r\right)}_i^{m+1}}, i\ne j, 1\le i\le N-m$$

(2)

The procedure is analytically described in Costa et al. [12,15]. A popular implementation [33] was used for consistency with other developmental studies.

Critically SampEn comparisons can only be done with identical m and r [15]. Estimates are assumed to be robust even for short and noisy time series [16], with stable values beyond a minimum number of time points [15,26,34]. Such minimum length is conservatively taken to be ~700 [35], but, more precisely, it is known depends on m (~10^m [36]). For instance, valid estimates for m = 2 have been shown for as little as 50 time points [22]. In the context of the MSE algorithm, the length of each coarse-grained time series at a given scale is N/τ (Equation (1)), where N is the length of the raw time series. Therefore, N and m define the upper limit of scale factors that can be reliably explored (τ_max).

One of the aims of the present study was to extend observations to higher scales. Accordingly, we used a small value for pattern length—m = 2, i.e., two consecutive data points were used for pattern matching. Such value is used in most developmental studies and allows an efficient use of N: τ_max = N/50.

We maximized N by computing MSE estimates on 4 s windows, achieved by joining each 4 consecutive 1 s epochs uninterrupted by bad quality epochs allowing an exploration of higher scale estimates while minimizing data loss for intervening artifacts. τ_max was therefore set to 1000/50 = 20. Note that the scale factor τ linearly relates to physical time as a function of the sampling rate sr, t = τ/sr: τ = 20 corresponds to t = 80 ms. Since scales over 28 ms were not explored in prior developmental studies, we considerably extended the explored scale range.

The normalized value of r = 0.5 was used here, i.e., a difference between data points equal or smaller than 50% of the signal standard deviation is considered a match. This value is at the higher end of values commonly used in EEG/MEG analysis, yet it has been largely used in developmental studies [20,21,22]. Larger r values have been associated better and accuracy and confidence of the SampEn [37]. However, as noted, an upper limit for r values exist [23]. We also verified the consistency of results for a smaller r = 0.2.

For each subject, MSE estimates, standard deviation (STD), absolute and relative power spectral densities were computed at each channel for all viable 4 s epochs. Given our focus on detailing the general trend of wide scale MSE changes, and, in line with the reported widespread spatial distribution of developmental differences [21], robust subject specific estimates were gathered by averaging mean values across all epochs for all good channels. This approach effectively reduced the number of comparisons and allowed a univariate approach. We therefore addressed differences across the five age bins by fixed effects analyses of variance (ANOVA) with age bin as a factor. MSE developmental trajectories were detailed through pairwise comparisons across age bins and curve-fitting regression analyses using age as a continuous predictor variable. Bonferroni correction for significance testing was applied to adjust for type-1 error inflation for comparisons across the twenty considered time scales and for pairwise age bin comparisons at each scale. Finally, we assessed the relation between MSE and STD through general linear modeling (GLM) and analysis of residuals to address age-dependent lack of fit of a simple linear model relating the two variables.

3. Results and Discussion

Individual MSE, STD and spectral analysis (PSD) were computed on an average of 13.1 (+/−2) 4 s epochs per participant across all age groups (ANOVA, F_4,181 = 1.2; non-significant (n.s.) for α = 5%. MSE estimates showed scale-dependent age differences (Figure 1). Development was accompanied by a general increase of entropy values at lower scales (8–20 ms; ANOVA at t = 8 ms, F_4,181 = 7.5, p < 0.001, Bonferroni corrected for comparisons at multiple time scales) and a general decrease at higher scales (68–72 ms; at t = 80 ms, F_4,181 = 5.64, p < 0.001). Pairwise comparisons (t-test, α = 5% Bonferroni corrected) revealed that, at lower scales (t = 8 ms), the two youngest age bins differed from each other and from all other age bins; at higher scales (t = 80 ms), all pairwise comparisons were significant aside those contrasting the two age bins within the youngest and oldest pairs. Such pattern is consistent with scale-dependent developmental trajectories whose shape has been further addressed by curve-fitting regression analyses. At lower scales, greater changes occur from ages eight to age 13—quadratic fitting performed better than linear fitting (R² = 0.31, p < 0.001 vs. R² = 0.20); conversely, at higher scales, a faster entropy decrease was observed from ages 13 to age 17—cubic better than linear fitting (R² = 0.35, p < 0.001 vs. R² = 0.21). Estimates for r = 0.2 differed in absolute values but showed overlapping relative, age-related differences (see Figure 1 inlay; ANOVA at t = 8 ms: F_4,181 7.1, p < 0.001; at t = 80 ms: F_4,181 5.5, p < 0.001).

Figure 1. MSE profiles through development. Age-specific MSE profiles are shown in the upper panel (* p < 0.001, ANOVA, Bonferroni corrected). These are generally consistent across the examined matching criteria (r = 0.2 in the inlay). Averages and standard errors for extreme scale values are shown in the bottom panels, where age bin color coding matches. Differences of opposite signs were observed for lower scales (up to ~20 ms) and higher scales (beyond ~50 ms). These are related to adults showing higher values in the lower scales range, with smaller growth gradient in this range and later inversion, determining a crossing of profiles in the intermediate scale values and relatively smaller entropy at higher scales. At lower scales, entropy estimates are in good agreement with existent studies, both in terms of age differences and numerical values (reviewed in [4]). Note that aside generally overlapping differences, r affects entropy magnitudes and relative gradients over scales, highlighting the potential dramatic impact of normalization choices.

Figure 1. MSE profiles through development. Age-specific MSE profiles are shown in the upper panel (* p < 0.001, ANOVA, Bonferroni corrected). These are generally consistent across the examined matching criteria (r = 0.2 in the inlay). Averages and standard errors for extreme scale values are shown in the bottom panels, where age bin color coding matches. Differences of opposite signs were observed for lower scales (up to ~20 ms) and higher scales (beyond ~50 ms). These are related to adults showing higher values in the lower scales range, with smaller growth gradient in this range and later inversion, determining a crossing of profiles in the intermediate scale values and relatively smaller entropy at higher scales. At lower scales, entropy estimates are in good agreement with existent studies, both in terms of age differences and numerical values (reviewed in [4]). Note that aside generally overlapping differences, r affects entropy magnitudes and relative gradients over scales, highlighting the potential dramatic impact of normalization choices.

EEG spectral properties showed typical global power differences (Figure 2). These were accompanied by dramatic STD developmental changes (ANOVA, F_4,181 = 25.3, p < 0.001). The STD trajectory roughly followed an opposite trend to what was observed for entropy estimates at lower scales. In fact, significant correlations between STD changes and entropy were observed up to the 64 ms scale, being maximal at the lowest scales (Figure 3). We further tested the covariance of STD and MSE by GLM: by using STD as a covariate, age differences vanished at lower scales (at t = 8 ms: F_4,181 = 0.44, n.s.), and survived at higher scales (at 80 ms: F_4,181 = 19.0, p < 0.001, Bonferroni corrected). Accordingly, the prediction of a linear model relating MSE and STD left unaccounted for an age-dependent distribution of values at higher scales, with residuals showing a pattern similar to what was observed in the dependent variable (at t = 8 ms: F_4,181 = 0.33, n.s.; at t = 80 ms: F_4,181 = 8.9, p < 0.001; Figure 3).

Figure 2. PSD and STD. Power spectral profiles followed typical age-related differences. Greater absolute power was observed in children, with development being accompanied by a decrease in total power and increase in the contribution of energy from higher frequencies. Global power differences are mirrored in STD changes. As the general 1/f relation observed in EEG signals remains largely unaffected, relative power differences are highlighted by PSD on signals divided by STD.

Figure 2. PSD and STD. Power spectral profiles followed typical age-related differences. Greater absolute power was observed in children, with development being accompanied by a decrease in total power and increase in the contribution of energy from higher frequencies. Global power differences are mirrored in STD changes. As the general 1/f relation observed in EEG signals remains largely unaffected, relative power differences are highlighted by PSD on signals divided by STD.

Our observations were made on average scalp values during resting state, no-task EEG. While source analysis studies have reported some anatomical heterogeneity in developmental trajectories, the overall pattern has been shown to be remarkably consistent across sources [20]. Accordingly, MSE estimates from electrode time series showed similar general agreement [22], and justified the use of average whole scalp measure to capture the general progression [21]. In the present study, we did observe a spatial variation of MSE profiles. However, channel profiles conserved the observed scale dependency and trajectory across ages, mainly differing in terms of the exact value in which age curves crossed. This is likely related to relative differences in values between higher and lower scales values that would not hinder the interpretation of general trajectories.

Figure 3. MSE and STD. The scale dependency of the association (correlation coefficient, R) between entropy estimates and STD is shown in the left panel. Higher STD—as observed in children—are related to lower entropy values particularly at lower scales. Consistently, a strong covariance of values was observed at lower scales (e.g., scatterplot in lower left inlay), while, at higher scales, adults showed lower entropy values even beyond the portion of variability explained by STD (e.g., scatterplot in the upper left inlay). This was evident in the age-dependent distribution of deviations from the fitted linear model $MSE=\alpha + STD\times \beta$ , i.e., residuals (e, shown in the right panel).

Figure 3. MSE and STD. The scale dependency of the association (correlation coefficient, R) between entropy estimates and STD is shown in the left panel. Higher STD—as observed in children—are related to lower entropy values particularly at lower scales. Consistently, a strong covariance of values was observed at lower scales (e.g., scatterplot in lower left inlay), while, at higher scales, adults showed lower entropy values even beyond the portion of variability explained by STD (e.g., scatterplot in the upper left inlay). This was evident in the age-dependent distribution of deviations from the fitted linear model $MSE=\alpha + STD\times \beta$ , i.e., residuals (e, shown in the right panel).

MSE patterns have been reported to be task dependent [20] and all prior developmental studies have investigated EEG/MEG during task execution. Task effects have been shown to be far smaller than age-related changes and interactions between age and task is also relatively small or null [18,21]. While resting-state is a poorly defined condition, it should be noted that task execution can also introduce performance related confounds. This is particularly true in a correlative study where a causal relation between MSE and performance cannot be conclusively established. In fact, task-related findings appealed to a theoretical framework modelling non-linear resting state interactions at a systems level [2,3,4,8]. Accordingly, the reported developmental MSE changes can be interpreted to capture robust and widespread changes that are potentially related to stable differences in the neuronal context that support integrated stimulus processing.

Scale dependency of MSE differences complicates their interpretation as maturation along a unidimensional, complexity measure. At the observation level allowed by EEG, time-scale-dependent differences have been interpreted to map into spatially distinct processes, relating lower scales to more anatomical confined patterns and higher scales to more global patterns of activity and long range connections [18,38]. Therefore, MSE profiles could rather capture spatiotemporal variations in brain noise and relate to complexity in a scale-dependent way by appealing to general principles of segregation and integration [14]. Accordingly, in the immature system brain, an increase in noise at lower scales would optimize metastable brain dynamics [5,20] while deteriorating neuronal communications at higher scales. Brain maturation appears to be accompanied by a reduction of this latter variability, and, interestingly, we observed that changes are particularly marked through adolescence, when critical neurodevelopmental events occur.

Such interpretations commonly juxtapose MSE temporal scales with the frequency domain framework. However, the relation between spectral properties and MSE estimates is multifaceted. While a correlation between the two measures is stably shown, they are generally intended as complementary descriptors of brain signals [18]. We noted that an STD-based definition of the similarity criterion—which leaves unaccounted the varying contribution of variance and signal autocorrelation to sample entropy estimates—makes the comparison of sample entropy estimates across different scales problematic and complicates their interpretation.

A complementary MSE account is possible if spectral properties do not directly affect its computation. Absolute and low frequency EEG power drive STD values. Therefore, the relative magnitude of the similarity criterion to the amplitude of high frequency activity is related to spectral profile differences and critically affects estimates. Given the signal smoothing that occurs during coarse graining, the effect is confined to lower scales. The directionality of this effect is captured by the observed correlation between STD and entropy estimates. The net result is a relative underestimation of the high frequency contribution to lower scale entropy estimates for children.

4. Conclusions

We assessed age-related changes in maturation of MSE estimates on EEG signals in a large sample of participants aged eight to 22 years. We replicated prior findings showing increases in MSE entropy estimates at time scales up to ~20 ms. By extending observations to higher scales up to 80 ms, we report for the first time a time-scale dependency of developmental changes. At scales higher than ~60 ms, entropy estimates appeared to decrease through development. This pattern was robust and generally insensitive to normalized r changes from 0.5 to 0.2. Age-related differences in signal STD, mainly driven by global power and low frequencies changes appeared to be associated with lower scale differences. A substantial covariation over age between signal STD and lower scale estimates appeared to nullify the age-related MSE changes when accounting for such changes in signal variability.

Such original findings inform a system level understanding of development by suggesting that optimal information processing is hindered in the immature brain by noise at higher temporal scales. This view complements current conceptualizations of maturation, which is believed to improve metastability increases of lower scale variability.

The sensitivity to temporal variations in brain noise makes the relationship of MSE to complexity not straightforwardly unidimensional, but more akin to the notion of segregation and integration which neither in isolation translates into optimal information processing [14]. However, as is made evident by applying MSE in the context of the rich spectral changes that characterize development, a clear interpretation of MSE profiles depends on understanding how signal STD affects MSE properties when the similarity criterion is normalized to it. This in general makes the relative contribution of variance to entropy scale-dependent and could qualitatively alter assessments of MSE differences across development and clinical populations.