A Simple Overview of Complex Systems and Complexity Measures

Abstract

Defining a complex system and evaluating its complexity typically requires an interdisciplinary approach, integrating information theory, signal processing techniques, principles of dynamical systems, algorithm length analysis, and network science. This overview presents the main characteristics of complex systems and outlines several metrics commonly used to quantify their complexity. Simple examples are provided to illustrate the key concepts. Speculative ideas regarding these topics are also discussed here.

1. Introduction

This article discusses how complexity measures reflect the nontrivial architecture and/or the irregular behavior of a complex system, highlighting the importance of both structural and dynamical aspects. The title of this article, featuring the words “simple” and “complex”, is a rhetorical device used to emphasize the difficulty of defining an adjective without relying on another adjective with either the same or the opposite meaning. For instance, dictionaries often define “big” as something “large” (a synonym) or as something “not small” (an antonym). Similarly, “complex” is described as “intricate” or as “not simple”.

A helpful way to begin understanding the meaning of the adjective “complex” in the context of “complex system” might be to consider how this word is employed as a noun in a wide range of diverse realities. For instance, the terms guilt complex and superiority complex are used in psychology, the industrial complex and the residential complex in civil engineering, and the vitamin B complex and the Golgi complex in biology. In these six examples, there is a diversity of interconnected entities (thoughts, buildings, and organic substances). In the expression “complex system”, the adjective “complex” conveys this same idea: a network of many elements with distinct characteristics [1,2,3,4]. However, the possibility of systems with very different natures being considered complex imposes a difficulty: there is no single definition of a complex system [1,2,3,4]. And if there is no single definition, there is no single way to measure its complexity [1,2,3,4]. A factor contributing to this lack of consensus is the domain-specific dependency: a feature that is relevant in one study may be irrelevant in another. Despite these issues, there is broad agreement that biomes [5], neural circuits [6], protein interactions [7], power grids [8], networks of air transport [9] and maritime transport [10], Earth’s climate [11], the spread of contagious diseases and rumors [12], the global economy [13], and galaxy clusters [14] are complex systems. Recent examples include the COVID-19 pandemic [15], the cryptocurrency transaction recording network [16], the diffusion process of electric vehicles [17], and large language models, such as ChatGPT (Generative Pre-trained Transformer) [18]. Recognizing that a system is complex is often easier than defining what it means to be complex.

In the examples mentioned above, the collective behavior of the systems goes beyond the sum of the individual behaviors of their parts [1,2,3,4]. Indeed, the function of the brain cannot be understood by examining the firing of a single neuron; the performance of the worldwide economic network cannot be predicted by analyzing the actions of a single bank. As a consequence, modeling a complex system is a difficult task because models necessarily involve simplifications, which lead to incomplete descriptions of the reality being studied [1,19,20,21,22]. Neglecting seemingly minor aspects of a complex system in a theoretical study can significantly undermine the reliability of the model’s predictions [1,2,3,4]. The traditional reductionist method, which examines components separately, has been criticized by classical authors on complexity, such as L. von Bertalanffy [23] and E. Morin [24], who advocate for more holistic and integrative approaches.

Although complexity is typically associated with multi-agent interactions, simply structured systems exhibiting chaotic or random behavior are sometimes described as complex due to their irregular dynamics. Usually, the complexity of both isolated and interconnected systems is assessed based on the variability of the time series that they display, changes in their qualitative dynamical behaviors, the length of the algorithms capable of generating them, and the topological properties of the graphs representing them. Here, these topics are explored in Section 2, Section 3, Section 4 and Section 5. In Section 6, the main conclusions and some speculations are presented.

A remark: under a mean-field approximation, complex systems composed of many elements can be roughly described by low-dimensional dynamical systems [25,26,27,28,29,30,31,32]. It is clear that such an approximation implies a loss of complexity. However, for simplicity, all examples examined here are first-order systems.

2. Complexity as Information

From the perspective of telecommunication engineering, the complexity of a system can be quantified based on the variability of its time series. In 1948, C. E. Shannon proposed a mathematical formula to quantify the information contained in a message sent from a source through a communication channel [33,34]. Consider that the probability of the k-th message being transmitted is $p_k$. For Shannon, its information content $h_k$ is related to $p_k$ by $h_k=log(1/p_k)$. Thus, if $p_k=1$, then $h_k=0$ (messages reporting events that are certain to occur lack both variability and novelty); if $p_k\to 0$, then $h_k\to \infty$ (messages reporting rare events carry a lot of information). Consider also that there are A alternative messages (or symbols) that can be transmitted; thus, ${\sum }_{k=1}^A{p}_k=1$. The average information content per message sent is defined as H and is obtained from [33,34]:

$$H=\sum _{k=1}^A{p}_k{h}_k=\sum _{k=1}^A{p}_{k}log{\displaystyle \frac{1}{p_k}}=-\sum _{k=1}^A{p}_{k}log{p}_k$$

(1)

This weighted average of $h_k$ is called Shannon entropy or information entropy. An analogue formula, called thermodynamic entropy S, was introduced in studies on statistical mechanics conducted by J.W. Gibbs about 70 years before Shannon’s work. Thermodynamic entropy reflects the amount of disorder associated with the microscopic states of a physical system [35,36]. In Quantum Mechanics, J. von Neumann proposed a similar expression in 1927 [37] to quantify the uncertainty of a quantum system’s state.

In Equation (1), the choice of base for log is arbitrary. If base 2 is taken, H provides the average number of bits needed to represent the information content of the message. For instance, if $A=2$ and $p_1=p_2=1/2$, then $H=1$ bit; that is, one bit is enough to specify which of two equally probable events has occurred (for instance, if 0 denotes heads and 1 denotes tails, then one bit is enough to describe the result of a coin toss).

The value of H (in base 2) of a source represents the average number of bits needed to encode its messages in an optimal coding scheme [33,34]. The maximum value of H is obtained for $p_k=1/A$ for $k=1,2,\dots ,A$ [33,34]. In this case, $H=H_{max}=logA$. Therefore, the entropy is maximized if all messages are equally likely. This case represents the highest uncertainty and the greatest average information gain per message transmitted through the channel. In statistical mechanics, the maximum entropy is given by $S=k_{B}lnW$, in which $k_B$ is the Boltzmann’s constant and W is the number of accessible microstates corresponding to a given macroscopic state [35,36]. Thus, W is the number of microscopic configurations in which the system can be arranged that result in the same macrostate. The formula $S=klogW$ is inscribed on the tombstone of L. E. Boltzmann, who derived it in 1877. The nontrivial relation between information and thermodynamics was evidenced by the thought experiment known as Maxwell’s demon, which suggests that the thermodynamic entropy can be reduced by manipulating information [38].

The information entropy H has been used to evaluate the complexity of time series, based on the variability of its symbols (or values). In these applications, $p_k$ represents the probability of the k-th symbol appearing in the time series. For instance, the LMC (López-Ruiz, Mancini, Calbet) complexity measure $C_{LMC}$ is defined as follows [39]:

$$C_{LMC}=H\sum _{k=1}^A{\left(p_k-{\displaystyle \frac{1}{A}}\right)}^2$$

(2)

and the SDL (Shiner, Davison, Landsberg) complexity measure as follows [40]:

$$C_{SDL}={\left({\displaystyle \frac{H}{H_{max}}}\right)}^a{\left(1-{\displaystyle \frac{H}{H_{max}}}\right)}^b$$

(3)

in which a and b are positive constants (here, $a=b=1$). Observe that $C_{LMC}=0$ and $C_{SDL}=0$ for $H=0$ (perfect order) and for $p_k=1/A$, which implies $H=H_{max}$ (absolute disorder). Therefore, for these authors, complexity lies between full predictability and pure randomness. These measures have been employed to analyze, for instance, the sunspot cycle [41], pseudorandom bit generators [42], and the Brazilian exchange rate [43].

Note that these complexity measures require a finite number A of symbols, but a time series may consist of a long sequence of real numbers. The conversion of real numbers into symbols through a quantization process affects the value of H. The smaller the partition, the more precise the symbolic representation; however, it may also increase sensitivity to noise. For instance, it is intuitive to see that random noise of high magnitude (when compared to the original signal) increases H because it adds uncertainty to the system, leading to a more uniform probability distribution and greater unpredictability.

Information entropy was originally conceived to address stochastic sources of information; however, it, along with its derivative complexity measures, has also been used to characterize the complexity of deterministic sources of apparent randomness [39,40,44,45,46]. Figure 1 illustrates four dynamical behaviors obtained for the sine map given by [47,48]:

$$x(t+1)=F\left(x\right(t\left)\right)=qsin\left(\pi x\right(t\left)\right)$$

(4)

If $0\le q\le 1$, then $0\le x\left(t\right)\le 1$. The dynamics of this deterministic discrete-time dynamical system have already been studied [49,50,51,52]. For $t\to \infty$, $x\left(t\right)$ tends toward zero for $0\le q<0.318$, it approaches a non-zero steady-state for $0.318<q\le 0.720$, and it tends toward either an oscillatory or a chaotic solution for $0.720<q\le 1$ (because, in this parameter range, windows of periodic behavior emerge, embedded within regions characterized by chaotic dynamics). Figure 1 shows that $x\left(t\right)$ tends toward oscillation (with period 2) for $q=0.8$ and toward a chaotic regime for $q=1$. Chaos implies high sensitivity to initial conditions [53,54]. Thus, starting from two arbitrarily close initial conditions ($x\left(0\right)=0.8$ and $x\left(0\right)=0.801$ in this plot), the system generates two bounded and aperiodic solutions that diverge (exponentially) from each other (in the corresponding state space). Since, in practical situations, the initial condition cannot be known with infinite precision, a small deviation in its value can lead to a solution that is radically different from the one observed in the actual chaotic system. The characteristic timescale associated with the divergence of two nearby chaotic solutions is inversely proportional to the system’s largest (positive) Lyapunov exponent [48,55].

The complexity of the solutions shown in Figure 1 can be assessed by computing H, $C_{LMC}$, and $C_{SDL}$, as presented in Figure 2. In the plot of H, downward peaks are observed in the low-period periodic windows, which are reflected as upward peaks in the plots of $C_{LMC}$ and $C_{SDL}$. To generate this figure, a sine map was iterated for 2000 time steps, starting from the initial condition $x\left(0\right)=0.8$; then, the first 1200 iterations were discarded to remove transients. Also, interval $[0,1]$ was divided into A equally sized subintervals (here, $A=100$). In this context, $p_k$ corresponds to the relative frequency with which the k-th subinterval is visited by the remaining 800 iterations. Therefore, in this quantization process, a time series of real numbers is represented by a sequence of symbols, with each symbol corresponding to a distinct subinterval. For instance, for $q<0.720$, only a single subinterval is visited; consequently, $p_k=1$ for this subinterval and $p_k=0$ for all others, implying $H=0$ bits. For $q=0.8$, $x\left(t\right)$ oscillates between 0.475 and 0.798 (after discarding transients). Thus, for the two subintervals containing these values, $p_k=1/2$; for the others, $p_k=0$. Therefore, for $q=0.8$, $H=[(1/2)\times {log}_{2}2]+[(1/2)\times {log}_{2}2]=1$ bit, $C_{LMC}=1\times [{((1/2)-(1/100))}^2+{((1/2)-(1/100))}^2]=0.480$ bits, and $C_{SDL}=[1/\left({log}_{2}100\right)]\times [1-(1/\left({log}_{2}100\right))]=0.128$. For $q=1$, the 100 subintervals are visited with approximately equal frequencies; that is, $p_k\approx 1/100$ for the 100 symbols. Hence, $H\approx {log}_{2}100=6.644$ bits, $C_{LMC}\approx 0$ bits, and $C_{SDL}\approx 0$. The chaotic behavior of maps with sinusoidal nonlinearity has been applied, for instance, in image encryption [56].

A variety of entropy measures inspired by information entropy have been proposed [57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75]. Permutation entropy [61,65,76,77] is one of them. It evaluates the temporal structure of a time series by analyzing the relative ordering of its values within short windows. For instance, the first seven values obtained by iterating the sine map from $x\left(0\right)=0.8$ for $q=1$ are $\{0.800,0.588,0.962,0.118,0.364,0.910,0.280,\dots \}$. By sliding a window of size $d=3$ across this time series (without skipping any numbers), five three-element sequences are formed: $\{0.800,0.588,0.962\}$, $\{0.588,0.962,0.118\}$, $\{0.962,0.118,0.364\}$, $\{0.118,0.364,0.910\}$, and $\{0.364,0.910,0.280\}$. By assigning 0 to the lowest value, 2 to the highest value, and 1 to the intermediate value, these five sequences can be rewritten as follows: $\{1,0,2\}$, $\{1,2,0\}$, $\{2,0,1\}$, $\{0,1,2\}$, and $\{1,2,0\}$. Recall that by permuting the numbers 0, 1, and 2, six possible sequences can be formed: ${\sigma }_1=\{0,1,2\}$, ${\sigma }_2=\{0,2,1\}$, ${\sigma }_3=\{1,0,2\}$, ${\sigma }_4=\{1,2,0\}$, ${\sigma }_5=\{2,0,1\}$, and ${\sigma }_6=\{2,1,0\}$. Let ${\rho }_i$ be the relative frequency of the i-th sequence. In this example, ${\rho }_1={\rho }_3={\rho }_5=1/5$, ${\rho }_4=2/5$, and ${\rho }_2={\rho }_6=0$. The permutation entropy P is computed from the following:

$$P=\sum _{i=1}^{d!}{\rho }_{i}log{\displaystyle \frac{1}{{\rho }_i}}$$

(5)

in which $d!$ is the maximum number of permutations. For this example, by taking base 2, $P=3\times (1/5)\times {log}_{2}5+(2/5)\times {log}_2(5/2)=1.922$ bits. The maximum permutation entropy is $P_{max}=logd!$ (in this example, $P_{max}={log}_{2}3!=2.585$ bits), which can be used to normalize P. Evidently, other size windows can be considered in this calculation (in this example, for $d=2$, $P=1$ bit). A low value of P implies regularity; a high value indicates irregularity (due to randomness or chaos). Figure 3 presents P as a function of q for the sine map for $d=2$ and $d=3$ with $0.9\le q\le 1$. In this figure, the same 800 iterations per value of q, as used in Figure 2, were considered. Observe that d strongly affects P.

Permutation entropy is generally robust to noise, as small fluctuations may not alter the ordinal patterns within the windows [61]. For the time series with seven numbers shown above, a 10% noise does not affect the relative ordering in the three-element sequences.

Another well-known alternative entropy measure was proposed by C. Tsallis [57]. It is written as follows:

$$T\left(s\right)=\kappa \left({\displaystyle \frac{1-{\sum }_{k=1}^A{\left(p_k\right)}^{\alpha }}{\alpha -1}}\right)$$

(6)

in which $\alpha$ and $\kappa$ are constants. The Tsallis entropy $T\left(s\right)$ of the system s reduces to the Shannon entropy H (or to the thermodynamic entropy S) in the limit $\alpha \to 1$ (by applying L’Hôpital’s rule). Hence, it can be considered a generalization of the standard entropy. Let the Tsallis entropy of the systems $s_1$ and $s_2$ be obtained from the following:

$$\begin{array}{ccc}\hfill T\left(s_1\right)& =& \kappa \left({\displaystyle \frac{1-{\sum }_{k=1}^A{\left(p_k\right)}^{\alpha }}{\alpha -1}}\right)\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill T\left(s_2\right)& =& \kappa \left({\displaystyle \frac{1-{\sum }_{j=1}^B{\left(p_j\right)}^{\alpha }}{\alpha -1}}\right)\hfill \end{array}$$

(8)

with ${\sum }_{k=1}^A{p}_k=1$ and ${\sum }_{j=1}^B{p}_j=1$, in which A and B are the numbers of available symbols (or microstates) of the systems $s_1$ and $s_2$, respectively. Assume that these systems are statistically independent; that is, $p_{kj}=p_k{p}_j$. The Tsallis entropy of the combined system $s_1+s_2$ can be written as follows [78]:

$$\begin{array}{ccc}\hfill T(s_1+s_2)& =& \kappa \left({\displaystyle \frac{1-{\sum }_{k=1}^A{\left(p_k\right)}^{\alpha }{\sum }_{j=1}^B{\left(p_j\right)}^{\alpha }}{\alpha -1}}\right)\hfill \\ & =& T\left(s_1\right)+T\left(s_2\right)+(1-\alpha )T\left(s_1\right)T\left(s_2\right)/\kappa \hfill \end{array}$$

(9)

Therefore, the Tsallis entropy is purely additive only when $\alpha =1$ (in fact, the information entropy and the thermodynamic entropy are additive) [57]. For $\alpha \ne 1$, $T\left(s\right)$ is non-additive, suggesting the presence of nontrivial correlations. Under this condition, the whole is not equal to the sum of its parts, which is a common feature of complex systems. Equation (9) shows that $\alpha <1$ implies super-additivity, while $\alpha >1$ implies sub-additivity [79]. Tsallis entropy has been employed in numerous studies related to complexity [78,79,80,81,82,83].

There is no single universal definition of entropy that applies optimally to all types of data or systems. For instance, Shannon entropy does not take into account the sequential structure within data. Permutation entropy translates a time series into a symbolic sequence that captures its temporal structure. Shannon entropy does not distinguish the complexity of two discrete probability distributions that are similar, potentially missing subtle temporal correlations. Tsallis entropy generalizes Shannon entropy by modifying how probabilities contribute to the overall measure through the inclusion of a tunable parameter $\alpha$, making it suitable for systems with non-additive behavior.

Entropy measures have also been used to deal with qubits (quantum bits, existing in a superposition of both 0 and 1), particularly when analyzing quantum entanglement and the uncertainty associated with quantum states [84,85,86,87,88].

In this section, only discrete probability distributions have been considered up to now; however, the continuous domain has also been analyzed. Shannon defined information entropy for the one-dimensional continuous case, denoted here by $H^c$, as follows [33,34]:

$$H^c=-k{\int }_{L_1}^{L_2}p\left(y\right)logp\left(y\right)dy$$

(10)

in which y is a random variable such that $y\in [L_1,L_2]$. The normalization condition for the probability density function $p\left(y\right)$ is expressed as ${\int }_{L_1}^{L_2}p\left(y\right)dy=1$. In the case of a n-dimensional distribution, n integrals are required in Equation (10).

The LMC complexity measure for the one-dimensional continuous case, denoted here by $C_{LMC}^c$, is determined from the following [89]:

$$C_{LMC}^c=e^{H^c}{\int }_{L_1}^{L_2}{\left(p\left(y\right)-p_u\right)}^{2}dy$$

(11)

in which $p_u$ represents the uniform probability distribution over the domain $L_1\le y\le L_2$; therefore, $p_u=1/(L_2-L_1)$ (for $L_1\to -\infty$ and $L_2\to +\infty$, $p_u=0$). Similar to the discrete formulation, this complexity measure is expressed as the product of a term associated with disorder and another term quantifying the deviation from the equiprobable distribution. It is important to note, however, that unlike the discrete case, in which H is always a non-negative quantity, $H^c$ can assume negative values [90]. Hence, to ensure positivity, as well as invariance under translation and rescaling transformations [89], the exponential form $e^{H^c}$ is used instead of $H^c$. This complexity measure has been applied in various contexts, including the analysis of the periodic table [91] and the structure of neutron stars [92].

Another relevant information measure for the continuous case was proposed by R.A. Fisher in 1925 [93]. For the one-dimensional probability density function $p(y;\theta )$, the Fisher information I is a measure of how much information an observable random variable x carries about an unknown parameter $\theta$ of $p(y;\theta )$ ($\theta$ can be, for instance, the mean or variance of $p(y;\theta )$). The greater the value of I, the more precisely the parameter $\theta$ can be estimated from y. Fisher proposed calculating I as follows [93]:

$$I={\int }_{L_1}^{L_2}{\left({\displaystyle \frac{\partial lnp(y;\theta )}{\partial \theta }}\right)}^{2}p(y;\theta )dy$$

(12)

Since Fisher information can quantify the degree of structure or predictability in a complex system, it has been used in complexity analyses. For one-dimensional systems, the Fisher–Shannon complexity is defined as $IJ$, in which $J=e^{2H_c}/\left(2\pi e\right)$, with $IJ\ge 1$ [94,95,96]. This metric has been computed in studies on different topics, such as atomic and molecular physics [96,97], orthogonal polynomials [98], and soil degradation [99].

3. Complexity as Bifurcations and Chaos

It is common to read that a complex system is more than the sum of its parts [1,2,3,4,23,24]. The failure of the superposition principle to hold is a characteristic of nonlinear systems [100,101].

Nonlinearity is essential for the occurrence of bifurcation, which happens when there is a change in the qualitative dynamical behavior of a system (and/or in its phase portrait) due to the variation in the value of one or more parameters [100,101]. A system that can exhibit a range of qualitatively different dynamical behaviors may be considered complex, even if it is isolated or simply structured.

Bifurcations can cause, for instance, the transition from a steady-state solution to a periodic one, or from a periodic solution to a chaotic one [100,101]. Figure 4 exhibits the bifurcation diagram of the sine map for $0\le q\le 1$. The vertical axis represents the values of $x\left(t\right)$ obtained for $t\to \infty$ (that is, after removing transients). This figure shows that for $q=0.2$ and $q=0.6$, $x\left(t\right)$ converges to a steady-state; for $q=0.8$, $x\left(t\right)$ tends to oscillate between two values; for $q=1$, $x\left(t\right)$ takes on many distinct values in the interval $[0,1]$. Clearly, these results are compatible with those presented in Figure 1. Observe that, as q increases, the attractor qualitatively changes. Each change corresponds to a bifurcation.

The sine map for $0\le q\le 1$ and the logistic map [54], written as $x(t+1)=G\left(x\right(t\left)\right)=\mu x\left(t\right)(1-x(t\left)\right)$ with $0\le \mu \le 4$ and $0\le x\left(t\right)\le 1$, exhibit similar plots for H, $C_{LMC}$, $C_{SDL}$, and the bifurcation diagram. In fact, both are unimodal maps; specifically, both $F=qsin\left(\pi x\right)$ and $G=\mu x(1-x)$ are continuous functions that have a single maximum for $x\in [0,1]$ and map this interval to itself. Also, both undergo a period-doubling route to chaos [100,101]. From the perspective of dynamical system theory, a system that exhibits a cascade of bifurcations and chaos can be viewed as complex.

Figure 5 shows the bifurcation diagram of the sine map for $0\le q\le 5$. The plot for $q>1$ cannot be reproduced by the logistic map. Thus, the sine map and the logistic map present equivalent complexity for $0\le q\le 1$ and $0\le \mu \le 4$, respectively; however, the sine map can be considered more complex for $q>1$ due to the richness of its behaviors. In contrast, the dynamics of the logistic map for $\mu >4$ are trivial: as $t\to \infty$, $x\left(t\right)$ diverges to $-\infty$.

A bifurcation usually indicates the boundary between different dynamical regimes, and it can be responsible for the emergence or disappearance of spatiotemporal patterns. From the perspective of dynamical systems theory, self-organization can be linked to a bifurcation. Self-organization, a concept popularized by I. Prigogine [102], is often associated with complexity and it refers to the creation of ordered structures through local interactions among the elements of a system, without any external input or central control. Thus, order may spontaneously arise when the value of a parameter is either above or below a critical threshold. For instance, in a fluid heated between two metallic plates, convection cells form spontaneously when the temperature of the plates surpasses a critical value, leading to a self-organized fluid motion. This example is paradigmatic in studies of climate models, in which the fluid represents the Earth’s atmosphere, as considered in the work of E. N. Lorenz [53]. A pattern that persists thanks to the dissipation of energy in an open system is commonly known as a dissipative structure [102]. Other typical examples [103,104,105,106,107,108] include the Turing instability [109], which leads to the emergence of spatial patterns in reaction–diffusion systems; traveling waves, which preserve their shape while propagating through space over time; and spatiotemporal chaos, characterized by a lack of periodicity in time and irregularity across space.

Studies on bifurcation and chaos have been presented in numerous papers and books on dynamics [47,48,53,54,100,101,102,110,111,112,113,114,115,116].

4. Complexity as Algorithm Length

From the perspective of computer science, A. N. Kolmogorov [117,118] (along with others, like R. J. Solomonoff [119,120] and G. J. Chaitin [121,122]) proposed that the complexity of a piece of information (or a system) is the length of the shortest algorithm capable of generating it when executed on a universal Turing machine [123,124]. Observe that this formulation of algorithmic complexity depends on the theoretical model of computation introduced by A. M. Turing [125,126]. For instance, the string “11111” can be generated by the program “print 1 five times”; however, the program used to generate the string “8h5m1” would be “print 8, then h, then 5, then m, then 1”. The longer the code, the higher the Kolmogorov complexity. Hence, repeating data have lower Kolmogorov complexity than incompressible data.

The Kolmogorov complexity of a time series obtained for $0\le q\le 1$ from the sine map (given by Equation (4)) can be computed as follows. First, discard transients (as in Figure 2, Figure 3, Figure 4 and Figure 5). Then, convert the remaining real-valued sequence $x\left(t\right)$ into a binary sequence $b\left(t\right)$, in which $b\left(t\right)=0$ if $0\le x\left(t\right)\le 0.5$, and $b\left(t\right)=1$ if $0.5<x\left(t\right)\le 1$. After compressing $b\left(t\right)$, the size of the compressed sequence provides a rough upper bound for the Kolmogorov complexity of $b\left(t\right)$.

Compression is the process of encoding a sequence into a more compact form by eliminating redundancy and repeating patterns [127,128]. The compressed sequence must retain the same information as the original and require fewer bits to represent it. For instance, a crude compressed version of 00000111111110 can be written as (5, 0), (8, 1), (1, 0), in which the first number in each pair is the count of the second number [127,128]. The idea of assessing complexity through data compression underlies the metrics derived from the works of A. Lempel and J. Ziv [129,130,131].

A Lempel–Ziv complexity measure can be computed by applying the $LZ78$ algorithm (introduced in their paper [131], published in 1978). This measure, denoted here as $LZ$, serves as a practical (computable) approximation of the Kolmogorov complexity. It quantifies the complexity of a binary sequence by identifying patterns, which are used to build a dictionary [132]. To determine $LZ$, start with an empty dictionary. The original sequence must be read from left to right. At each step, find the shortest subsequence that has not been recorded in the dictionary yet, and then add it to the dictionary. A cut is made after identifying this new subsequence, and the reading of the original sequence must resume from the next bit immediately following the cut. The $LZ$-complexity is the number of distinct subsequences recorded in the dictionary. These subsequences can be used to reconstruct the original binary sequence. For instance, consider the string 01001. The first cut is made after the first 0, yielding $0|1001$, and 0 is added to the dictionary. The second cut is made after the first 1, resulting in $0\left|1\right|001$, and 1 is added to the dictionary. The third cut is made after the third 0 in order to create a new substring; that is, $0\left|1\right|00|1$. Note that the final digit, 1, is already in the dictionary. Thus, the dictionary contains three substrings: $(0,1,00)$; therefore, $LZ=3$. For the string 0000011101, $LZ=5$ (because it is parsed as $0\left|00\right|001\left|1\right|10|1$).

Consider two sequences with the same number of bits. The sequence with a higher $LZ$ has more subsequences in the dictionary than the one with a lower $LZ$. The higher the $LZ$ of a sequence, the harder it is to compress. Figure 6 shows $LZ$ as a function of q for the sine map. This plot is based on the same data used in Figure 2, after converting $x\left(t\right)$ for each q into a binary sequence $b\left(t\right)$ as described above. As expected, the $LZ$ value for $q=1$ is greater than that for $q=0.8$.

The $LZ$-complexity has been computed in a wide range of studies [133,134,135,136]. Its computation provides a model-free measure of both the information content and structural complexity of finite sequences.

5. Complexity as Connectivity

Since complex systems are usually made up of many interacting elements (which can be cells, molecules, computers, living beings), it is relevant to characterize the topological properties of the connections among these elements. Hence, graph theory has been employed to analyze complex systems [108,137,138,139,140,141,142]. For instance, a complex network approach has been applied to examine the progression of Alzheimer’s disease [143] and to evaluate the underlying dynamics of electroencephalogram (EEG) signals for diagnosing epilepsy [144]. There are methods for converting time series into graphs [145,146,147,148], enabling the application of graph theory concepts to analyses of temporal patterns.

Simplistically, a complex network can be defined as a graph with a connectivity structure that is neither completely regular nor entirely random [108,137,138,139,140,141,142]. In undirected graphs, this structure can be characterized by the degree k of a node, which is the number of links connected to that node; the degree distribution $D\left(k\right)$, which represents the percentage of nodes with degree k; the shortest path length ℓ between two nodes, which is the minimum number of edges connecting them; and the clustering coefficient c of a node, which is defined as the ratio of the number of actual links among its neighbors to the maximum possible number of such links. Additionally, the average degree $\langle k\rangle$, the average shortest path length $\langle \ell \rangle$, and the average clustering coefficient $\langle c\rangle$ are computed by averaging k, ℓ, and c over all N nodes in the network [108,137,138,139,140,141,142].

The two most commonly used models of complex networks in academic research are known as small-world [149] and scale-free [150]. Both models have strengths and weaknesses [108,151,152,153]. A small-world graph is created by starting with a regular ring network with a fixed number of nodes, in which each node is connected to m neighbors on the right and m neighbors on the left, and then randomly rewiring some of the m regular connections on the right [149]. The rewired graph can present a high average clustering coefficient and a low average shortest path length, as observed in several real-world networks [108,137,140,149]; however, it is formed from an artificially constructed starting point and exhibits a bell-shaped degree distribution, which is rather uncommon in open real-world networks. A scale-free graph can be created by considering that the number of nodes increases as time progresses. In this model, the probability of a new node connecting to an existing node is directly proportional to the degree of that existing node [150]. This graph has a low average shortest path length and a scale-free degree distribution, as found in several real-world networks [108,138,140,150]; however, it has a low average clustering coefficient, and the rule used to grow the network assumes that the new node is aware of the degrees of all nodes in the growing network, which is unrealistic. Despite these criticisms, both models share an important feature: the graphs that they generate are based on simple probabilistic rules that are easy to describe. Also, they inspire the development of models that incorporate topological features found in both small-world and scale-free graphs [154,155,156,157,158,159,160,161].

A simple way of characterizing the complexity of a graph is through Shannon entropy H. Accordingly, the degree structure entropy of a graph is computed from Equation (1), in which $p_k$ represents the percentage of nodes with degree k [108,141,162,163,164]. In this context, H quantifies the heterogeneity in the degree distribution $D\left(k\right)$. As an example, consider the graphs shown in Figure 7. In (a), all seven nodes have $k=2$; hence, $p_2=1$ and $p_k=0$ for $k\ne 2$, resulting in $H=0$ bits. In (b), five nodes have $k=2$, one node have $k=1$, and one node have $k=3$; thus, $p_1=1/7$, $p_2=5/7$, and $p_3=1/7$, leading to $H=1.149$ bits. Thus, (b) is a more complex graph than (a), since it presents a more irregular connection pattern reflected in a higher value of H. From the perspective of network science, complexity signifies an intricate pattern of connectivity.

Several methods for evaluating graph complexity, grounded in node connectivity and entropy measures, have been reported in the literature [83,165,166,167,168,169].

6. Discussion and Outlook for Future Research

When examining the time evolution of a system, complexity refers to symbol variability (Section 2 and Section 4) and/or changes in qualitative behavior (Section 3); when examining the physical structure of a system, complexity refers to nontrivial connectivity (Section 5).

In my point of view, complexity measures should explicitly increase with the number of nodes/edges comprising the networked system and with the symbol repertoire of a signal source. For a complex graph, I propose that the node-edge complexity $C_{NE}$ of a graph be calculated simply as $C_{NE}=N\times E$, in which N is the number of the nodes and E is the number of edges. By taking this formula into account, the node-edge complexity of the human brain and of the GPT-3 can be quantified and compared. For a human brain with $1\times {10}^{11}$ neurons [170] and $1\times {10}^{15}$ synapses [171], $C_{NE}=1\times {10}^{26}$. For the GPT-3 [172], $N=96\times 12288$ (96 layers with 12288 hidden units per layer) and $E=175\times {10}^9$ parameters (synaptic weights); therefore, $C_{NE}=2\times {10}^{17}$, which is comparable to the complexity of the zebrafish brain [173] (by considering $1\times {10}^3$ synapses per neuron). Other functions relating $C_{NE}$ to N and E could be suggested (for instance, $C_{NE}=\gamma N+\delta E$, $C_{NE}=\gamma Nlog(\delta E+1)$, and $C_{NE}=N^{\gamma }{E}^{\delta }$, in which $\gamma$ and $\delta$ are constants). The proposed node-edge complexity may yield more meaningful results in networks exhibiting spatial uniformity; that is, in the absence of characteristic spatial subscales, where the average node degree varies substantially with the position within the network. This may not constitute an original idea; nevertheless, I have not found it in the literature.

Computing the variability of a real-valued series is tricky, because it depends on the quantification process: the smaller the partition, the greater the number of symbols A; however, there is no upper limit to A. Hence, it is more convenient to compute a complexity measure when dealing with binary series. I propose that the LZH complexity of a binary series be simply calculated as $C_{LZH}=LZ\times H$, in which $LZ$ is the Lempel–Ziv complexity and H is the Shannon entropy; thus, both the number of essential substrings and the relative frequency of bits are taken into account. The originality of this simplistic proposal is also uncertain.

Evidently, the value obtained by computing any complexity measure usually has no standalone meaning. Such measures are primarily useful for comparing the complexity of time series or graphs in relative terms.

This overview began with a list of the uses of the word “complex” as a noun in three fields of knowledge: psychology, civil engineering, and biology. In these fields, and in all others, there are paradoxical situations. Here are examples for each field: Can the attempt to suppress negative thoughts actually strengthen them, due to the hard-to-understand feedback loops of human cognition? Does the construction of additional roads to reduce traffic actually lead to more congestion by encouraging more people to drive, since the roads become initially less congested? Can the introduction of a new species, which would increase biodiversity, destabilize the system and lead to the extinction of species, thereby reducing biodiversity? Complexity often permeates paradoxes.

This overview ends with a list of speculative (and rather unoriginal) questions about complex systems. Is there a relationship between the structural complexity of tourist complexes and their profits? Can the fluctuations in cryptocurrency values predict economic crises? To what extent has the average genome size of living species influenced, and been influenced by, Darwinian evolution? Is there a relationship between functional complexity and structural complexity in proteins? Is there a minimum node-edge complexity of a neural system that makes the emergence of consciousness possible? How can a signal sent by extraterrestrial intelligent life, potentially much more complex than our own, be recognized? The study of complex systems, which relies on interdisciplinary approaches, is crucial for advancing our understanding of nature and society.