Spatial Hurst–Kolmogorov Clustering

Definition

The stochastic analysis in the scale domain (instead of the traditional lag or frequency domains) is introduced as a robust means to identify, model and simulate the Hurst–Kolmogorov (HK) dynamics, ranging from small (fractal) to large scales exhibiting the clustering behavior (else known as the Hurst phenomenon or long-range dependence). The HK clustering is an attribute of a multidimensional (1D, 2D, etc.) spatio-temporal stationary stochastic process with an arbitrary marginal distribution function, and a fractal behavior on small spatio-temporal scales of the dependence structure and a power-type on large scales, yielding a high probability of low- or high-magnitude events to group together in space and time. This behavior is preferably analyzed through the second-order statistics, and in the scale domain, by the stochastic metric of the climacogram, i.e., the variance of the averaged spatio-temporal process vs. spatio-temporal scale.

1. Introduction

The main themes of the encyclopedia entry are the Hurst–Kolmogorov (HK) clustering behavior and its stochastic analysis in the scale domain.

1.1. HK Clustering

Clustering in nature was first identified by Hurst [1] (Figure 1a) while studying the long-term behavior in a variety of scales of the discharge timeseries of the Nile River to develop engineering projects in its basin. Particularly, Hurst discovered a tendency of high-discharge years to cluster into high-flow periods, and low-discharge years to cluster into low-flow periods. This behavior, also known as the Hurst phenomenon or Joseph effect [2], is characterized by long-term persistence (LTP; also called long-range dependence), which leads to high unpredictability on large scales due to the clustering of events as compared to the purely random process (i.e., white noise) or other short-range dependence models (e.g., Markovian).

The mathematical description of the Hurst phenomenon is attributed to A.N. Kolmogorov (Figure 1b) who developed it while studying turbulence in 1940 [3]. Based on this, Koutsoyiannis [4] coined the term Hurst–Kolmogorov (HK) dynamics (Figure 2), so as to give credit to both contributing scientists and to expand it from the narrower Gaussian LTP processes (e.g., fractional Gaussian noise [5]). The HK dynamics also incorporate short-range dependence (e.g., fractal-type behavior [6]) and intermediate-scale behavior often observed in global-scale hydrological-cycle and turbulent processes [7].

The HK dynamics are also linked to the entropy maximization principle, and thus, to robust physical justification [8]. It is worth noting that the stochastic simulation of the HK dynamics is still a mathematical challenge [9] since it requires the explicit preservation of high-order moments in a vast range of scales [10,11], affecting both the intermittent (fractal) behavior in small scales [12] and the dependence in extremes [13].

The 1D clustering behavior has been identified in many scientific fields (see reviews in [7,15,16,17]). The 2D spatial clustering behavior has also been explored in many fields such as hydrology (e.g., [18,19,20], and references therein), biology and ecosystems (e.g., [21,22]), life sciences (e.g., [23,24,25,26]), networks (e.g., [27,28,29]), urban structures (e.g., [30,31]), rock formation (e.g., [32]), turbulence (e.g., [7,33]), art (e.g., [34,35,36]), landscape analysis (e.g., [37,38]), simulated evolution of the universe [39] and many others (e.g., [40]). A unified approach for the quantification of the 2D spatio-temporal clustering in terms of variability in the scale domain (instead of in the common lag and frequency domains) can be found in the applications of the current entry, where a stochastic methodology is presented that quantifies clustering in 2D spatial fields by analyzing the spatial structures over time, and by exploring how the HK dynamics highly increase the induced uncertainty in terms of spatio-temporal variability in the scale domain.

1.2. Stochastic Analysis in the Scale Domain

The most common stochastic metrics in literature are based on the dependence structure of the second-order statistics (see discussions in [5,7,13,41]), which are commonly estimated through the autocovariance function, c(h), or the autocorrelation function, i.e., c(h)/c(0), where h is the continuous time lag in time units, or through the power spectrum, whose definition is based on c(h), i.e., $s\left(w\right):=2{{\displaystyle \int }}_{-\infty }^{\infty }c\left(h\right)\cos \left(2\mathsf{\pi }wh\right)\mathrm{d}h$, where $w$ is the continuous frequency in reverse time units. Common estimators of the latter are the so-called periodogram and the autocovariance (or autocorrelation) [42,43]. However, the latter estimators are based only on the domains of lag and frequency. Beran [44], inspired by the empirical work of Smith [45] in agricultural crops and by similar works in other fields [46,47], applied a similar estimator but to the scale domain (i.e., plot in logarithmic axes of the variance of the accumulated or aggregated process vs. scale [48,49]). Nevertheless, this estimator was later abandoned, since it was characterized as a “bad estimator” of the long-term persistence, mainly due to the large estimation bias [43,50]. Another similar estimator for the identification of long-term persistence, introduced by Hurst [1] and known as rescaled range (R/S), was also found to have similar or even worse issues than the aggregated variance method [43,51,52,53,54,55].

The identification of the stochastic structure in the scale domain is revisited much later and compared to the lag domain (i.e., through the correlogram estimator of the autocovariance/autocorrelation function) and the frequency domain (i.e., through the power spectrum), but after theoretically defining the expressions, estimators and bias expressions for all three metrics in the three domains [56].

The aggregated variance was also found to be a misnomer of this method in the scale domain (i.e., the variance is not aggregated but rather the scale is [57]), and a single name for this method did not exist (as, for example, for the periodogram or the correlogram methods); hence, Koutsoyiannis [4] suggested to use the term climacogram to emphasize the graphical representation and the link of the concept to scale (i.e., climax in Greek), and so as not to be confused with the already established term of scale(o)gram. It is noted that the climacogram is explicitly linked to the autocovariance c(h), i.e., $c\left(h\right)=\frac{1}{2}{\mathrm{d}}^2\left(h^2\gamma \left(h\right)\right)/\mathrm{d}{h}^2$, and thus, also to the power spectrum [5].

Through the above works, the climacogram could now be adjusted for estimation bias [10,12,14,56,58,59], as well as the autocovariance and power spectrum for estimation and discretization bias. (See next section for the bias expressions of the multi-dimensional second-order climacogram, while for the autocovariance and power spectrum, the bias expressions can be found in [40,56].) Thus, after eliminating the main criticized limitations of this estimator, Dimitriadis and Koutsoyiannis [56] illustrated how the climacogram outperforms the other two estimators. Particularly, the climacogram is shown (a) to exhibit a smaller estimation bias than the more commonly used tools of autocorrelation and power spectrum (i.e., autocovariance-based metrics); thus, it can robustly identify from fields analysis both the short-term structure and the long-range dependence behavior, with the latter highly linked to the clustering effect, which emerges specifically in complex systems as opposed to the simplest yet uncommon behavior of white noise (absence of dependence) or Markov structure (i.e., short-range dependence); (b) to be free of discretization bias as opposed to the autocovariance-based metrics, in which the transformation of the continuous to discrete representation of the spatio-temporal field is involved due to the double integration calculations; and (c) to be well defined and with a monotonic behavior, as compared to the autocovariance-based metrics, whose double logarithmic slopes (e.g., used to identify the long-term persistence) exhibit a non-monotonic behavior. Thus, a stochastic process with short-term fractal and long-term persistent structures should be analyzed in the scale domain instead of in the lag and frequency domains.

Based on this scientific boost, the climacogram (rather than autocovariance-based metrics) was found preferable for the identification, model building and simulation of a stochastic process. Since then, the interest in the scale domain has been highly increased, and the climacogram has been used to identify the LTP behavior in various scientific studies. For a review of such studies, see [7,13] and references therein, where also a massive global-scale analysis in the scale domain is included for several hydrological-cycle processes (i.e., near-surface air temperature, dew point, humidity, atmospheric pressure, near-surface wind speed, streamflow and precipitation) and microscale turbulent processes (such as grid turbulence and turbulent jets). Alternative scientific fields, where the analysis is performed in the scale domain and by using the climacogram, include studies of rock formations [32], landscapes [37,38], water–energy nexus [60,61], time-irreversible processes [62,63], multilayer perceptron [64] and many others [65] as shown in the applications of the entry. It is again emphasized that this entry focuses on the multi-dimensional spatio-temporal stochastic metrics in the scale domain, as presented in the next sections.

2. Methodology

For the exploration of change and estimation of the spatio-temporal variability in the scale domain (instead of in the traditional lag and frequency domains), we use the stochastic metric of the climacogram, as thoroughly discussed in the previous section.

Particularly, we expand the 1D scale domain to an $L$-dimensional ($L$D) one. We denote $\underset{\_}{x}\left(\mathit{t}\right)$, the continuous-state random variable of a stochastic stationary and isotropic process in L dimensions, with $\mathit{t}$, a vector of $L$ variables, and $l$, an index varying from 1 to $L$, i.e., $\mathit{t}≔\left\{t_1,\dots ,t_L\right\}$, that is used to describe each dimension of the process (e.g., $t_1$ can be a temporal variable, $t_2$ a spatial one etc.). Note that the L dimensions are considered independent of each other. Discretized processes are subject to a sampling frequency $\mathit{D}≔\left\{D_1,\dots ,D_M\right\}$ and a response time $\mathbf{\Delta }≔\left\{{\Delta }_1,\dots ,{\Delta }_L\right\}$ as in the 1D case. Both $\mathit{D}$ and $\mathbf{\Delta }$ have the same units with the corresponding variable $\mathit{t}$ (e.g., if $t_1$ is a temporal variable measured in seconds then $D_i$ and ${\Delta }_i$ will be measured also in seconds). Here, we focus only on the case of $\mathit{D}$ = $\mathbf{\Delta }$ > 0. Finally, n denotes the total amount of data in the $L$D field. Thus, the discretized stochastic process ${\underset{\_}{x}}_{i_1,i_2,\dots ,i_L}^{\left({\Delta }_1,{\Delta }_2,\dots ,{\Delta }_L\right)}$, for $\Delta$ > 0, can be estimated from the continuous stochastic process $\underset{\_}{x}\left(t_1,t_2,\dots ,t_L\right)$ as [40]:

$${\underset{\_}{x}}_{i_1,i_2,\dots ,i_L}^{\left({\Delta }_1,{\Delta }_2,\dots ,{\Delta }_L\right)}=\frac{{{\displaystyle \int }}_{\left(i_1-1\right){\Delta }_1}^{i_1{\Delta }_1}{{\displaystyle \int }}_{\left(i_2-1\right){\Delta }_2}^{i_2{\Delta }_2}\dots {{\displaystyle \int }}_{\left(i_L-1\right){\Delta }_L}^{i_L{\Delta }_L}\underset{\_}{x}\left({\xi }_1,{\xi }_2,\dots ,{\xi }_L\right)\mathrm{d}{\xi }_1\mathrm{d}{\xi }_2\dots \mathrm{d}{\xi }_L}{{\Delta }_1{\Delta }_2\dots {\Delta }_L}$$

(1)

where $i_1\in \left[1,n_1\right], i_2\in \left[1,n_2\right],\dots , i_M\in \left[1,n_L\right]$ are indices of each variable in $\mathit{t}$.

In

Table 1. Climacogram definition and expressions for a 2D continue process, a discretized one, a common estimator for the climacogram and the estimated value, based on this estimator.

Type	2D Climacogram
continuous	$\gamma \left(k_1,k_2\right):=\frac{\mathrm{Var}\left[{{\displaystyle \int }}_{t_1}^{t_1+k_1}{{\displaystyle \int }}_{t_L}^{t_L+k_L}\underset{\_}{x}\left({\xi }_{1,}{\xi }_2\right)\mathrm{d}{\xi }_1\mathrm{d}{\xi }_2\right]}{{\left(k_1{k}_2\right)}^2}$	(T1-1)
discretized	${\gamma }^{\left(\Delta \right)}\left({\kappa }_1{\Delta }_1,{\kappa }_2{\Delta }_2\right):=\gamma \left(k_1,k_2\right)$	(T1-2)
typical estimator	$\underset{\_}{\widehat{\gamma }}\left({\kappa }_1{\Delta }_1,{\kappa }_2{\Delta }_2\right)=\frac{{\kappa }_1{\kappa }_2}{n_1{n}_2}{\displaystyle {\displaystyle \sum }_{r_2=1}^{[n_2/{\kappa }_2]}}{\displaystyle {\displaystyle \sum }_{r_1=1}^{[n_1/{\kappa }_1]}}\left(\frac{1}{{\kappa }_1{\kappa }_2}\right({\displaystyle {\displaystyle \sum }_{i_2={\kappa }_2(r_2-1)+1}^{{\kappa }_2{r}_2}}{\displaystyle {\displaystyle \sum }_{i_1={\kappa }_1(r_1-1)+1}^{{\kappa }_1{r}_1}}{\underset{\_}{x}}_{i_1,i_2}^{\left({\Delta }_1,{\Delta }_2\right)})-\frac{{{\displaystyle \sum }}_{i_2=1}^{n_2}{{\displaystyle \sum }}_{i_1=1}^{n_1}{\underset{\_}{x}}_{i_1,i_2}^{\left({\Delta }_1,{\Delta }_2\right)}}{n_1{n}_2}),\mathrm{where} [n_l/{\kappa }_l] \mathrm{is} \mathrm{the} \mathrm{floor} \mathrm{of}{n}_l/{\kappa }_l$	(T1-3)
expected value of estimator	$\mathrm{E}\left[\underset{\_}{\widehat{\gamma }}\left({\kappa }_1{\Delta }_1,{\kappa }_2{\Delta }_2\right)\right]=\gamma \left({\kappa }_1{\Delta }_1,{\kappa }_2{\Delta }_2\right)-\gamma \left(n_1{\Delta }_1,n_2{\Delta }_2\right)$	(T1-4)

, we provide all necessary definitions and equations for the true continuous, true discrete and most common estimators and estimations for the expected climacogram for a $2$D process (for the autocovariance, variogram and power spectrum, see [40]). As shown in Table 1, the climacogram estimator is expressed through the classical estimator of variance, but other and higher-order moments can be used (e.g., [13]).

For example, the second-order climacogram estimator in two dimensions (i.e., called 2D-C) is shown for the estimation of spatial variance vs. scale, where the spatial scale is defined as the ratio of the area of k × k adjacent cells (i.e., scale k), whose values are averaged to form the (scaled) spatial field. The 2D-C is defined as (see also [13]):

$$\underset{\_}{\widehat{\gamma }}\left(\kappa \right)=\frac{1}{n^2/{\kappa }^2}{\displaystyle \sum }_{i=1}^{n/\kappa }{\displaystyle \sum }_{j=1}^{n/\kappa }{\left({\underset{\_}{x}}_{i,j}^{\left(\kappa \right)}-\overline{\underset{\_}{x}}\right)}^2$$

(2)

where the ‘^’ over γ denotes estimate, κ is the dimensionless spatial scale, ${\underset{\_}{x}}_{i,j}^{\left(\kappa \right)}$ represents the local average of the space-averaged process at scale κ and at grid cell (i, j) and upper-lined x is the global average, while underlined variables represent random variables as opposed to regular ones. Note that Δ = 1 and the maximum available scale for this estimator is n/2, but estimation beyond n/10 is not reliable [56].

As previously mentioned, an important stochastic process that characterizes the variability over small and large scales is the Hurst–Kolmogorov (HK) behavior. For example, the general case of an LD HK process can be described by the equation:

$$\left({\underset{\_}{x}}_i^{\left(\kappa \Delta \right)}-\mu \right){=}_{\mathrm{d}}{\kappa }^{2L\left(1-H\right)}\left({\underset{\_}{x}}_j^{\left(\Delta \right)}-\mu \right)$$

(3)

where μ is the mean of the process ${\underset{\_}{x}}_i^{\left(\Delta \right)}$, and ${\underset{\_}{x}}_j^{\left(\kappa \Delta \right)}$ is the same process at scale κ.

The $L$D climacogram in the continuous-scale domain can be expressed as:

$$\gamma \left(k\right)≔\frac{\lambda }{{\left(k/\alpha \right)}^{2L\left(1-H\right)}}$$

(4)

where $k$ is now the geometric mean of scales $k_1$, $k_2$,…, $k_M$, i.e., $k=\sqrt{k_1{k}_2\dots k_L}$, α is a scale parameter in units of $k$, so that $\gamma (\alpha )=\lambda$, and H is the Hurst parameter (0 < H < 1). For 0 < H < 0.5, the HK process exhibits an anti-persistent behavior, while H = 0.5 corresponds to the white noise process, and for 0.5 < H < 1, the process exhibits persistence, a common behavior exhibited in nature.

The long-term persistence behavior can be quantified by the Hurst parameter as described by the previous equation. The problem with this process is that it yields an infinite variance as scale tends to zero. One way to remedy this is by introducing a modified and also generalized form of the HK model, which was found to be efficient in geophysical processes [7], and as also shown in the applications, as it can simulate both short-term and long-term dependence structures. This can be expressed in a standardized form through the climacogram expression (for additional models, see [7,13]) as:

$$\gamma (k)={(1+{(k/\alpha )}^{2M})}^{L\left(H-1\right)/M}$$

(5)

where 0 < M < 1 is a fractal parameter indicative of the roughness of the process. For M < 0.5, the process is characterized as rough, and M > 0.5 as smooth, whereas for M = 0.5, there is neutrality, corresponding to the so-called generalized (GHK) model [6].

Finally, since the clustering behavior is of particular interest for this entry, we show a simple example of how a cluster in a timeseries can be easily identified through the second-order climacogram (i.e., plot of variance of the averaged process vs. scale in logarithmic axes). In Figure 3a,c, the cases of two 1D data series are depicted, each with a different change in magnitude at the beginning and at the end of the series. The data series with the larger change in magnitude Figure 3a corresponds to the climacogram with the higher log-log slope (Figure 3b), which is an indication of a stronger clustering behavior of the 1D data series. Similarly, the weaker clustering behavior, although still stronger than in the white noise one, corresponds to the smaller log-log slope of the climacogram (Figure 3d) and the data series with the smaller change in magnitude (Figure 3a).

3. Illustrative Applications

3.1. One-Dimensional Turbulence

Here, we present the analysis of a massive dataset comprising 40 timeseries of the longitudinal velocity behind a grid provided by the John Hopkins university [66]. Each timeseries comprises 36 × 10⁶ records, and after a data-homogenization procedure, we were able to analyze them as if they were recorded at the same location and with similar initial conditions (for more details on the homogenization, see [12]). In Figure 4, we show the standardized second-order metrics of the climacogram, autocovariance and variogram, i.e., c(h) − c(0), and power spectrum, where it can be seen that the variogram and the climacogram exhibit the smallest uncertainty at the long-range scales and, in particular, on the double logarithmic slope that is attributed to the strength of the long-range dependence (i.e., the magnitude of the Hurst parameter). However, since the variogram has the limitation that the double logarithmic slope tends to zero, only from the climacogram is it possible to robustly estimate the Hurst parameter (for comparison between the two metrics, see [32]).

3.2. Benchmark Analysis of Two-Dimensional Art Paintings

Here, we show an application of the stochastic metric 2D-C in a benchmark white noise process without (Figure 5a) and with (Figure 5b) cluster areas, and we compare this to a famous art painting (Figure 5c). Particularly, in Figure 6, we show how the 2D-C can effectively be used for comparison of the magnitude of the variabilities among images. Specifically, it illustrates how much larger and smaller is the variability vs. the scale of the benchmark image when including a few cluster areas, as compared to that of the white noise process and to that of the art painting, respectively.

3.3. Two-Dimensional Rock-Formations

Here, we show the application of the 2D-C metric at images of sandstones depicted at different spatial scales of 8 orders of magnitude (Figure 7 and Figure 8). We see how all the different samples can be both easily compared to each other, in terms of the variability, and unified under the HK dynamics, when analyzed through the scale domain (for more information on the samples and the derived Hurst parameter, see [32]).

3.4. Spatio-Temporal Wind Speed of a Hurricane

Next, we apply the 2D-C metric to a spatial image of wind speed magnitude from Hurricane Sandy (www.nhc.noaa.gov/data), observed in October 2012 (Figure 9). As the starting point for the estimation of the climacogram, we chose the center of the image, which has approximately 0 wind speed as it is the eye of the hurricane [40]. In Figure 10, we estimate the climacogram of the hurricane images in several time frames (observed climacogram), and we fit, for illustration, the GHK model (denoted as true climacogram) by adjusting for bias (that is extremely high in this example), a procedure that is usually neglected in stochastic analysis.

3.5. Spatio-Temporal Evolution of Clustering

The tool called 2D-C quantifies the variability of images through the variance of the brightness intensity in grayscale. By a careful selection of images with spatial information, we can quantify how clustering alters over time, which can be useful for either characterizing spatial variability, such as in urbanization, and thus enabling tracking of their spatio-temporal evolution, or for revealing spatial hidden patterns through scales that may be invisible in the first glance of the image. A range of such applications is presented in [39], with focus in (a) natural sciences, in terms of the evolution of the universe as suggested by cosmological simulations and of the ecosystems, such as forests and lakes, and in (b) social sciences dealing with social structures, as revealed by the evolution of worldwide cropland data, satellite images of night lights and spatial data on urban land cover.

Here, the 2D-C is implemented for the estimation of spatial variance vs. scale in temporal sequence of images (after being processed by filtering or enhancing techniques [65]). The pixels analyzed are represented by numbers denoting their color intensity in grayscale (white = 1, black = 0). Figure 11 presents images from three timeframes of the evolution of the universe as generated by a cosmological model of evolution [67]: (a) 500 million years after Big Bang, an image with faint clustering; (b) 1000 million years after Big Bang, an image with mild clustering; and (c) 10,000 million years after Big Bang, an image with intense clustering. Figure 12 depicts the scaled images at scales k = 2, 4, 8, 16, 20, 25, 40, 50, 80, 100 and 200, which are used to calculate the 2D climacogram.

The presence of clustering is reflected in the climacogram (Figure 13a,b), which shows a marked difference as the scale increases. Specifically, the variance of the images is notably higher at the scales with higher clustering, indicating a greater degree of variability of the process at these scales.

For the integration of all information contained in the 2D climacogram of each image, we evaluate the cumulative areas underneath each one for all scales, i.e., the climacogram integral ${{\displaystyle \int }}_{\mathsf{\Delta }}^k\gamma \left(x\right)/x \mathrm{d}x$, where Δ and k are the minimum and maximum scale, and we divided by x in order for the integral to converge for arbitrarily high k ($k\to \infty$). In the discrete case, this can be approximated as:

$$\mathrm{CI}(k)={\displaystyle \sum }_{i=1}^{n\left(k\right)-1}\left(\gamma \left(x_{i+1}\right)+\gamma \left(x_i\right)\right) \frac{x_{i+1}-x_i}{x_{i+1}+x_i}$$

(6)

where n(k) is the number of integration intervals for scale k. We evaluate $\mathrm{CI}(k)$ at the maximum available spatial scale, so that it is the best approximation of the limit $\mathrm{CI}(\infty )$.

In Figure 14a,b, each climacogram is represented by its corresponding integral, permitting the evaluation of the rate of the alteration of clustering through time.

One more example is given in Figure 15 and Figure 16, where we evaluate the evolution of clustering in (randomly picked) zoomed areas from the original images of the evolution of the universe of clustering through time.

This method enables the comparison of clustering patterns at various spatial scales and among different time frames, and it is a meaningful metric for understanding and quantifying the change in clustering through time.

4. Conclusions

The Hurst–Kolmogorov dynamics describe a commonly observed structure in multidimensional spatio-temporal natural processes, which includes a short-term (fractal) structure in small scales and a long-range dependence (or persistence) behavior in large ones. The latter attribute is linked to the clustering effect of a spatio-temporal stationary stochastic process with an arbitrary marginal distribution function and a power-type dependence structure in large spatio-temporal scales yielding a high probability of low- (or high-) magnitude events to group together in space and time.

The analysis in the scale domain through the climacogram (instead of in the common lag and frequency domains, through the autocovariance and power spectrum, respectively) can effectively be used for identification and simulation of the Hurst–Kolmogorov behavior in a vast range of scales. Moreover, through spatio-temporal information, we can quantify how clustering alters over time, which can be useful for either characterizing spatial variability, thus enabling tracking of their spatio-temporal evolution, or for revealing hidden patterns through scales that may be difficult to trace with conventional methods.

The exploration of different topics in the stochastic nature of natural processes on a multidimensional spatio-temporal scale domain can be numerous and related to different scientific fields, as illustrated in the literature review and the presented applications of the current entry.