Towards a Universal Measure of Complexity

Recently, it has been argued that entropy can be a direct measure of complexity, where the smaller value of entropy indicates lower system complexity, while its larger value indicates higher system complexity. We dispute this view and propose a universal measure of complexity that is based on Gell-Mann’s view of complexity. Our universal measure of complexity is based on a non-linear transformation of time-dependent entropy, where the system state with the highest complexity is the most distant from all the states of the system of lesser or no complexity. We have shown that the most complex is the optimally mixed state consisting of pure states, i.e., of the most regular and most disordered which the space of states of a given system allows. A parsimonious paradigmatic example of the simplest system with a small and a large number of degrees of freedom is shown to support this methodology. Several important features of this universal measure are pointed out, especially its flexibility (i.e., its openness to extensions), suitability to the analysis of system critical behaviour, and suitability to study the dynamic complexity.


Introduction
Analysis of the concept of complexity is a non-trivial task due to its diversity, arbitrariness, uncertainty, and contextual nature [1][2][3][4][5][6][7][8][9][10]. There are many different levels/scales, faces, and types of complexity, researched with very different technologies/techniques and tools [11][12][13] (and refs. therein). In the context of dynamical systems, Grassberger suggested [14] that a slow convergence of the entropy to its extensive asymptotic limit is a signature of complexity. This idea was materialized [15,16] further by information and statistical mechanics techniques. It generalizes many previous approaches to complexity, unifying physical ideas with ideas from learning and coding theory [17]. There also exists a connection of this approach to algorithmic or Kolmogorov complexity. The hidden pattern can be the essence of complexity [18][19][20][21]. Techniques adapted from the theories of information and computation have led physical science (in particular, the region extended between classical determinism and deterministic chaos) to discover hidden patterns and quantify their dynamic structural complexity [22]. The above approaches are not universal-they only capture small fragments of the concept of complexity.
We must remember that complexity also depends on the conditions imposed (e.g., boundary or initial conditions), as well as the restrictions adopted. This creates a challenge for every complexity study. It concerns the complexity that can appear in the movement of a single entity and collection of entities braided together. These entities can be irreducible or straightforward, simple systems, but they can also be complex systems.
When we talk about complexity, we mean irreducible complexity, which can no longer be divided into smaller sub-complexities. We refer to this as a primary complexity. Considering the primary complexity here, we mean one that can be expressed at least in an algorithmic way-it is an effective complexity if it also contains a logical depth [23][24][25][26][27]. We should take into account that our models (analytical and numerical) and theories describing reality are not fully deterministic. The evolution of a complex system is potentially multi-branched and the selection of an alternative trajectory (or branch selection) is based on decisions taken randomly.
One of the essential questions concerning a complex system is the problem of its stability/robustness and the question of the stationarity of its evolution [28]. Moreover, the relationship between complexity and disorder on the one hand, and complexity and pattern on the other is an important question-especially in the context of irreversible processes, where non-linear processes, running away from the equilibrium, play a central role. Financial markets can be a spectacular example of these processes [29][30][31][32][33][34][35][36][37][38][39].
The central question of whether entropy is a direct measure of complexity is one we answer in the negative. In our opinion, based on the Gell-Mann concept of complexity, the measure of complexity is appropriately, non-linearly transformed entropy. This work is devoted to finding this transformation and examining the resulting consequences.

Definition of a Universal Measure of Complexity and Its Properties
In this Section, we translate the Gell-Mann general qualitative concept of complexity into the language of mathematics, and we present the consequences of this.

The Gell-Mann Concept of Complexity
The problem of defining a universal measure of complexity is urgent. For this work, the Gell-Mann concept [23,40] of complexity is the inspirational starting point. We apply this concept to irreversible processes, by assuming that both fully ordered and fully disordered systems cannot be the complex. The fully ordered system essentially has no complexity because of maximal possible symmetry of the system, but the fully disordered system contains no information as it entirely dissipates. Hence, the maximum of complexity should be sought somewhere in between these pure extreme states. This point of view allows for the introduction of a formal quantitative phenomenological complexity measure based on entropy as a parameter of order [29,41]. This measure reflects the dynamics of the system through the dependence of entropy on time. The vast majority of works analyzing the general aspects of complexity, including its basis, are based on information theory and computational analysis. Such an approach requires supplementing with a provision allowing a return from a bit representation to physical representation-only this will allow physical interpretations, including understanding of the causes of complexity.
We define the phenomenological partial measure of complexity as a non-linear function of entropy S of the order of (m, n), CX(S; m, n) def.
where S min and S max are minimal and maximal values of entropy S, respectively, S arit = S min +S max 2 , and the entropic span Z def.
= S max − S min , whereas m and n are natural numbers (an extension to real positive numbers is possible but this is not the subject of this work). They define the order (m, n) of the partial measure of complexity CX. Let us add that this formula is also applicable at a mesoscopic scale. In other words, complexity appears in all systems for which we can build entropy. Notably, S max does not have to concern the state of thermodynamic equilibrium of the system. It may refer to the state for which entropy reaches its maximum value in the observed time interval. However, in this work, we are only limited to systems having a state of thermodynamic equilibrium. Below, we discuss the Equation (1), indicating that it satisfies all properties of the measure of complexity. Of course, when m = 0 and n = 1 then CX simply becomes S − S min , i.e., the entropy of the system (the constant is not important here). However, when m = 1, n = 0, we obtain the information contained in the system (constant does not play a role here). Equation (1) gives us a lot more-showing this is the purpose of this work (helpful features of CX are shown in Appendix A).
The partial measure of complexity given by Equation (1) is determined with the accuracy of the additive constant of S, i.e., this constant does not contribute to the measure of the complexity of the system.
Using Equation (1), we can also enter the partial measure of specific complexity, as follows, where N is the number of entities that make up the system and specific entropy s = S/N. As one can see, the partial measure of specific complexity cx is independent of N for an extensive system. Specific entropy and specific complexity are particularly convenient when comparing different extensive systems and when we do not examine the complexity dependence on N. However, the extraction of an additional multiplicative constant (e.g., particle number) to have s independent of N often presents a technical difficulty, or may even be impossible, especially for non-extensive systems. Then it is more convenient to use the entropy of the system instead of the specific entropy. It is also important to realize that determining extreme entropy values (or extreme specific entropy values) of actual systems can be complicated and it requires additional dedicated tools/technologies, algorithms, and models.
The partial measures of complexity are enslaved by entropy in every order (m, n) of complexity. However, the kind of entropy we use in Equation (1) depends on the specific situation of the system and what we want to know about the system, because our definition of complexity does not specify this. From our point of view, relative entropies formulated in the spirit of Kullback-Leibler seem to be the most appropriate (this is referred to in Appendix B). Using the Kullback-Leibler type of entropy, one can express both ordinary entropies and conditional entropies, in particular one can describe the entropy rate increasingly used in the context of complexity analysis.
The entropy here can be both additive (the Boltzmann-Gibbs thermodynamic one [42], Shanon information [17], Rényi [43]), and non-additive entropy (Tsallis [44]). The measure CX(S) is a concave (or convex up) function of entropy S, which disappears on the edges at points S = S min and S = S max .
It has a maximum | S=S < 0. The quantity S max CX is a characteristic also because it is a weighted average. The quantity CX max is well suited to global universal measurements of complexity, because (at a given order (m, n)), it only depends on the entropy span Z. The quantity cx max def. = CX max /N m+n might also be a good candidate for measuring the logic depth of complexity.

The Most Complex Structure
The question now arises about the structure of the system corresponding to entropy S max CX given by Equation (4). The answer is given by the following constitutive equation, where Y is the set of variables and parameters (e.g., thermodynamic), on which the state of the system depends. However, Y = Y CX max is a set of such values of these variables and parameters that are the solution of Equation (5). This solution gives the entropy value S = S max CX that maximizes the partial measure of complexity, that is CX = CX max . Hence, with the value of Y CX max , we can finally answer the key question: what structure/pattern is behind CX max or what the structure of maximum complexity looks like.
There are a few comments to be made regarding the constitutive Equation (5) itself. It is a (non-linear) transcendental equation in the untangled form relative to the Y. This equation should be numerically solved, because we do not expect it to have an analytical solution for maximally complex systems. An instructive example of a specific form of this equation and its solution for a specific physical problem is presented in Section 3. However, this will help us to understand how our machinery works.
Equation (4) legitimizes the measure of complexity we have introduced. Namely, its maximum value falls on the weighted average entropy value, which describes the optimal mixture of completely ordered and completely disordered phases. To the left of S, we have a phase with dominance of order and to the right a phase with dominance of disorder. The transition between both phases at S is continuous. Thus, we can say that the partial measure of complexity that we have introduced also defines a certain type of phase diagram in S and CX variables (phase diagram plain). Section 2.5 provides more detailed information.

Evolution of the Partial Measure of Complexity
Differentiating Equation (1) over time t, we obtain the following non-linear dynamics equation, where the entropic S-dependent (non-linear), susceptibility is defined by and dS(t) dt can be expressed, for example, using the right-hand side of the master Markov equation (see Ref. [45] for details). However, we must realize that the dependence of entropy on time can, in general, be non-monotonic, because real systems are not isolated (cf. the schematic plot in Figure 2). One can see how the dynamics of complexity is controlled in a non-linear way by the evolution of the entropy of the system.
In concluding this Section, we state that Equations (1)-(6) together provide a technology for studying the multi-scale aspects of complexity, including the dynamic complexity. However, it is still a simplified approach, as we show in Section 4.

Significant Partial Measure of Complexity
We consider the partial measure of complexity to be significant when the entropy of the system is located between two inflection points of the CX(S; m, n) curve, i.e., in the range S − ip ≤ S ≤ S + ip . This case occurs for n, m ≥ 2. We then obtain see Figure 1d for details. There are two different cases where a single inflection point is present. Namely, and In Figure 1b, we present the case defined by Equation (9), while that defined by Equation (10) is shown in Figure 1c.
For n = m = 1, the curve CX(S; m, n) vs. S has no inflection points and it looks like a horseshoe (cf. Figure 1a).
Notably, we can equivalently write Let us consider the span Z ip = S + ip − S − ip of the two-phase area. From Equation (8), or equivalently from Equation (11), we obtain As one can see, the span Z ip depends linearly on the span Z and in a non-trivial way on the exponents n and m. Thus, with the Z set, only Z ip 's non-trivial dependence on the order (m, n) of measure of complexity CX occurs, which is different from CX max dependence. In other words, Z ip is less sensitive to complexity than CX max . The significant partial measure of complexity ranges between the two inflection points only for the case n, m ≥ 2 (cf. Figure 1d). Indeed, a mixture of phases is observed in this area. For areas where S min ≤ S < S − ip and S + ip < S ≤ S max , we have (practically speaking) only single phases, ordered and disordered, respectively (see Section 2.5 for details). The case defined by Equation (8), and equivalently by Equation (11), is the most general, while taking into account the fullness of complexity behaviour as a function of entropy. Other cases impoverish the description of complexity. Therefore, we will continue to consider the situation, where n, m ≥ 2.
The choice of any of the CX(S; m, n) forms (i.e., exponents n and m) is a somewhat arbitrary function of the state of the system as it depends on the function of the state, that is on the entropy. In our opinion, the shape of the CX(S; m, n) measure vs. S we present in Figure 1d is the most appropriate, because only then the significant complexity is ranging between non-vanishing inflection points S − ip and S + ip .
In generic case we should, however, use the series of partial measures defined by Equation (1). Then, we define the order of the partial complexity using the pair of exponents (n, m). The introduction of the order of the partial complexity is in line with our perception of the existence of multiple levels of (full) complexity.
We are able to discover the nature of the CX measure, i.e., its dynamics and, in particular, its dynamical structures, when we analyze the entropy dynamics S(t) (see Figure 2 for details).  The measurability of the partial measure of complexity is necessary for characterizing it quantitatively and to be able to compare different complexities. Following Gell-Mann [40], we must identify the scales at which we perform the analysis and thus determine coarse-graining to define the entropy. Its dependence on complexity cannot be ruled out.
However, the question of direct measurement of the partial measure of complexity in an experiment (real or numerical) remains a challenge.

Remarks on the Partial Entropic Susceptibility
An essential tool for studying phase transitions is the system susceptibility-in our case, the partial entropic susceptibility of the partial measure of complexity. Here, it (additionally) plays the role of the partial order parameter.
The plot of susceptibility χ CX (S; m, n) vs. S is presented in Figure 3. Four phases, already marked in Figure 1, are visible (also numbered 1 to 4).
Plot of the partial entropic (non-equilibrium) susceptibility χ CX (S; m, n) of the partial measure of complexity vs. S given by Equation (7) at fixed order (m = 2, n = 2). The finite susceptibility value at the S − ip and S + ip phase transition points (cf. Figure 1) may be considered to correspond to finite susceptibility value in absorbing the non-equilibrium phase transition in the model of direct percolation at a critical point in the presence of an external field [21]. However, the situation presented here is richer, because susceptibility changes its sign, smoothly passing through zero at S = S max CX . At this point, the system is exceptionally robust and, therefore, is poorly affected by data artefacts, because its susceptibility vanishes there.
Phase number 1 is almost entirely ordered-the disordered phase input is residual. At point S − ip , there is a phase transition to the mixed-phase marked with number 2, still with the predominance of the ordered phase. At the S − ip inflection point, the entropic susceptibility reaches a local maximum. By further increasing the entropy of the system, it enters phase 3 as a result of phase transition at the very specific S max CX transition point. At this point, the entropic susceptibility of the partial measure of complexity disappears. This mixed phase (number 3) is already characterized by the advantage of the disordered phase over the ordered one. Finally, the last transition, which occurs at S + ip , leads the system to the dominating phase of the disordered phase-the input of the ordered phase is residual here. At this transition point, the susceptibility reaches a local minimum. Intriguingly, entropic susceptibility can have both positive and negative value passing smoothly through zero at S = S max CX , where the system is exceptionally robust. The presence of phases with positive and negative entropic susceptibility is an exceptionally intriguing phenomenon. The phases discussed above, together with the above-mentioned inflection points, are also marked in Figure 1d. Let us add that the location of the phases mentioned above, i.e., the location of the inflection points, depends on the order (m, n) of the partial measure of complexity. This is clearly seen in Figures 4 and 5.
The values of local extremes of the entropic susceptibility of the partial measure of complexity are finite here and not divergent, as in the case of (equilibrium and non-equilibrium) phase transitions in the absence of an external field. We use this definition to describe the critical behaviour of a system that we demonstrate in Section 2.7, where it requires an explicit dependence on N.

Universal Full Measure of Complexity
The full universal measure of complexity X is a weighted sum of the partial measures of complexity CX(S; m, n) for individual scales. That is, where w(m, n) is a normalized weight, which must be given in an explicit form. This form is to some extent imposed by the power-law form of partial complexity. Namely, we can assume which seems to be particularly simple because independently of m and n.
As one can see, Equation (13), supported by Equation (15), is the product of the sums of two geometric series, If both series converge for any S min ≤ S ≤ S max , which is the case if and only if the condition Z(= S max − S min ) < M is met, then we directly obtain In other words, the M parameter can always be chosen, so that the sums of both series in Equation (21) diverge for all S values. Thus, m 0 , n 0 ≥ 1 is the natural lower limit of m 0 , n 0 , satisfying the condition of X(S; m 0 , n 0 ) disappearing for S = S min , S max . We still assume more strongly that m 0 , n 0 ≥ 2, which has already been explained above.
For extensive systems, Equation (17) can be presented in a form that clearly shows the dependence of the X complexity on the number of entities N, simply replacing S entropy by Ns, where s is already N-independent specific entropy. Subsequently, We emphasize that X does not scale with N, as opposed to partial measures of complexity. In Figures 6 and 7, we show the dependence of X on N (on the plane) and on N and s (in three dimensions), respectively. We obtained the singularities of full complexity, N cr j (s), j = 1, 2, as a result of the zeroing of denominators in the Equation (17) at nonzero numerators.
Note that, for M Z, both measures of complexity have approximate values X(S; m0, n0) ≈ CX(S; m0, n0). Important differences between these two measures only appear for Z/M close to 1, because only then does the denominator in Equation (17) play an important role. Of course, M is a free parameter, and possibly its specific value could be obtained from some additional (e.g., external) constraint.
In Figure 4, we compare the behaviour of the partial (black curve) and full (orange curve) measures of complexity, where we used the entropy instead of the specific entropy. Whether CX lies below or above X depends both on M parameter (determining the weight at which individual measures of partial complexity enter the full measure of complexity), and on the Z/M ratio.   (1) and full measure of complexity X(S; m 0 = 2, n 0 = 2) given by Equation (17), for instance, for the symmetric case of m = n = m 0 = n 0 . In addition, we assume that S min = 0, S max = 8 and M = 10. Vertical dashed lines indicate inflection points: black for the CX curve, orange for the X curve, while S max CX = S max X = 4. Notably, S max X maximizes X (here at a given ratio Z/M = 0.8). Vertical dashed lines mark the locations of inflection points on both curves.
We continue to determine the full entropic susceptibility of the full measure of complexity, where S max CX is given here by Equation (4) but for m = m 0 and n = n 0 . Notably, for the symmetric cases m = n and/or m 0 = n 0 , we have S max CX = S max X = S arit , which are independent of m, m 0 . Similarly to the partial entropic susceptibility of a partial measure of complexity, we obtain the full entropic susceptibility of a full measure of complexity, where s max CX = S max CX /N, s arit = S arit /N, s min = S min /N, and s max = S max /N. The progression of susceptibility χ X (S; m 0 , n 0 ), depending on S, for selected parameter values is shown in Figure 5. This progression course is similar to the analogous one that is presented in Figure 3.
Plot of the full entropic susceptibility χ X (S; m 0 , n 0 ) of the full measure of complexity vs. S given by Equation (19), at arbitrary fixed order (m 0 = 2, n 0 = 2). As expected from the comparison with Figure 3, the turning points of CX (cf. Figure 4) lie within the S interval bounded by inflection points of X.
Thus, the evolution of X is governed by an equation that is analogous to Equation (6), except that χ CX present in that equation should be replaced by χ X given by Equation (19). Therefore, we have The relationship between measures of complexity and time is implicit in our work-complexity indirectly depends on time through the dependence of entropy on time. It should be emphasized that the dependence of entropy on time is external in our approach-it can be taken into account based on additional modelling that is dedicated to specific real situations. We have already signalled this when discussing Equation (6).

Criticality in Extensive Systems
By using Equation (17), we show when the universal full measure of complexity diverges and, thus, the system enters a critical state. We assume that we are dealing with an extensive system, i.e., that Equation (17) can be represented as where entropy densities s(= S/N), s min (= S min /N), s max (= S max /N) are (at most) slowly varying functions of the number N of elements making up the system and special entropy span z = s max − s min . As one can see, the measure X is divergent in two critical points N max cr (s) = M s max −s and N min cr (s) = M s min −s , where s min < s < s max . Moreover, the susceptibilities given by Equations (19) and (20) diverge at the same points where measures of complexity given by Equations (17) and (18) diverge, which underlines the self-consistency of our approach. Equation (22) can now be written in a form that explicitly includes both critical points (both physical and non-physical): where critical exponents assume the mean-field values β max = β min = 1. In this case, we could speak of two-criticality were it not for the fact that one of these criticalities is unphysical. Figure 6 shows dependence X(Ns; m 0 , n 0 ) vs. N at fixed s = 0.8. The values of parameters are shown there, while the specific entropy s is chosen so that the condition s − s min < s max − s is satisfied (this is equivalent to a condition s < s arit ). This means that s is closer to s min than s max . The existence of these divergences is a signature of criticality. However, the situation for borderline cases s = s min or s = s max changes rapidly-it is a different consideration.
Critical numbers of entities in the system N max cr (s) and N min cr (s) are determined by the ratio of the M parameter characterizing the hierarchy/cascade of scales in the system and the distance between entropy density s and its extreme values s min and s max . The construction of these critical numbers resembles the canonical critical temperature structure for the Ising model in the mean-field approximation, where β c Jz = 1 (here β c = 1/k B T c and k B is the Boltzmann constant). In our case, the role of the inverse temperature β c is played by N max cr and N min cr , the role of the coupling constant J is 1/M, while the role of the mean coordination number z is played by s max − s and s − s min , respectively.
The hierarchy is the source of criticality here. Criticality is an immanent feature of our full description of complexity. Nevertheless, in this work, we do not specify the sources of this hierarchy-it could be self-organized criticality or due to some other sources.
For the sake of completeness, note that the dependence on N of the partial measure of complexity is given by Equation (2). This means that for extensive systems this measure increases powerfully depending on N. Therefore, only the weighted infinite sum of these measures generates the existence of singularity. Let us now consider in more detail the behaviour of X(Ns; m 0 , n 0 ) depending on N and s. A three-dimensional plot of Figure 7 will be helpful here. One can see how the mutual location of the singularities of N max cr (s) and N min cr (s) changes with the increase of s. From the situation of s < s arit , in which N max cr (s) > N min cr (s), through the situation when s = s arit in which N max cr (s) = N min cr (s), up to the situation in which N max cr (s) > N min cr (s) for s > s arit . It must be clearly stated that the area physically accessible is the one in front of the first singularity, which is further emphasized in Figure 7 by blue curves. Let us emphasize that the N range in which criticality occurs is sufficient to cover the corresponding values of N discussed in the literature to date, especially the Dunbar numbers [46][47][48][49] (e.g., N = 5, 15, 50, and N = 150). However, it should be noted that our view of complexity is complementary to that presented in the literature.  (23), for m 0 , n 0 ≥ 1. Notably, the full measure of complexity and its susceptibility have singularities at the same points N max cr (s) and N min cr (s). We are dealing here with complexity barriers separating the phases/states of the system and the small and large number of entities that form them. The parameters we adopted here are, as follows: M = 30, s min = 0, s max = 2, s = 0.8, m 0 = n 0 = 2. These are the same parameters that we used to construct the plain plot in Figure 6.

Finger Print of Complexity in Simplicity
Let us consider a perfect gas at a fixed temperature, which is initially closed in the left half of an isolated container. The partition is next removed, and the gas undergoes a spontaneous expansion. Here we are dealing (practically speaking) with an irreversible process, even for a small number of particles (at least the order of 10 2 ).
Let us recall the definition of 'perfect gas'. It is a gas of particles that cannot 'see' each other, i.e., there are no interactions between them. Thus, from a physical point of view, it is a dilute gas at high temperature. We further assume that all of the particles have the same kinetic energy. A legitimate question is whether such a gas will expand after the partition is removed. We notice that the thermodynamic force is at work here, being roughly proportional to the difference in the number of particles in the right and left parts of the container. This force causes the expansion process. Thus, we are dealing with the simplest paradigmatic irreversible process [50]. The particles remain stuck in the final state and will not leave it (with accuracy subject to slight fluctuations in the number of particles in the right half of the container). Such a final state of the whole system is referred to as the equilibrium state. The simple coarse-grain description of the system allows us to introduce here the concept of configuration entropy.
Note that the macroscopic state of the system (generally, the non-equilibrium and non-stationary/relaxing one) can be described by the instantaneous number of particles in the left (N L (t)) and right (N R (t)) parts of the container, with N = N L (t) + N R (t), where N is the fixed total number of particles in the container (isolated system). It allows for one to define the weight of the macroscopic state Γ(N L (t)), also called thermodynamic probability. This is the number of ways to arrange the N L (t) particles in the left part of the container and N R (t) = N − N L (t) in the right. Hence, Here we do not distinguish permutations of particles inside each part of the container separately. We only take into account permutations of particles located in different halves of the container. This is because our resolution here is too small to observe the location of particles inside each container separately. Such a coarse-graining creates an information barrier: more information can mask the complexity of the system. We will not be able to see the complexity, because we will not be able to construct entropy. This creates a paradoxical situation: the surplus of information makes the task difficult and does not facilitate obtaining the insight into the system. Here we have an analogy with chaotic dynamics, where chaos is only visible in the Poincaré surface cross-section of the phase space and not in the entire phase space.
The configuration entropy at a given time t we define, as follows, where Γ(N L (t)) is given by Equation (24). The above expression can be used both for the equilibrium and non-equilibrium states. It can be demonstrated using the Stirling formula that for large N, entropy S is reduced to the BGS form, where p J (t) def.
= N J (t) N , J = L, R, and s(t) is a specific entropy. The law of entropy increase Equation (A8) is also fulfilled here, as expected.
We now prepare the equation for determining N CX max L , i.e., the number of particles in the left part of the container that maximizes the partial complexity measure CX. To this end, we assume, for instance, the symmetric partial measure of complexity of the order of (m = 2, n = 2). Next, we substitute N L = N CX max L into the both sides of Equation (25) and according to constitutive Equation (5), we equate Equation (25) to S max CX . Hence, we obtain a constitutive equation for the relaxing perfect gas, where N CX max L is our sought quantity. Now, we need to independently determine S max CX . Recall that the number of N L particles that maximize entropy is the number of N eq L particles in the statistical/thermodynamic equilibrium state of the system. This number is equal to half of all particles in the container, i.e., N eq L = N/2. It can still be assumed (without reducing the general considerations) that S min = 0. Therefore, However, from Equation (A5), we know that S max CX = S max /2. By using it, we transform Equation (27) into the form, Equation (27) is an example of the general constitutive Equation (5), where N CX max L plays the role of Y CX max . This equation has the following explicit form, Just deriving Equation (30) (see Appendix C for details) is the primary purpose of this example. This is a transcendental equation of which the exact analytical solution is unknown. When deriving Equation (30), we used the initial condition for the entropy that is, S(t = 0) = S min = ln Γ(N L = N) = 0, which follows from Equations (24) and (25). Even for such a simple toy model, determining the partial measure of complexity is a non-trivial task, also because N L is different from N/2 (as we show below).
The numerical solutions of Equation (30) Figure 9. That is, in a log-log scale, their slopes equal 1. However, in linear scale, the directional coefficients of these straight lines equal 0.11 and 0.89, respectively. This is clearly shown in Figure 9. Only the solution with orange circles is realistic, because the chance that 89% of particles will pass in a finite time to the second part of the container (as indicated by the solution marked with blue circles) is negligibly small. The black solid tangent straight line indicates a reference case N CX max L = N/2. Having the N CX max L dependence on N, we can obtain the dependence of the partial measure of specific complexity cx max def. = CX max /N m+n on N order (m = 2, n = 2). We can write as in our case s max = s(N/2) equals the logarithm of the right-hand side of Equation (30) divided by N. Notably, Equation (31) is based on Equation (A12).
In Figure 10, we present the dependence of cx max on N. Quantity cx max is a non-extensive function-it reaches the plateau for N 1. For N ≈ 10 4 the plateau is achieved with a good approximation. This is important for researching complexity. Namely, systems can attain complexity already on a mesoscopic scale. Although the absolute value of the complexity measure is relatively small, it is evident and possesses a structure that is related to the current inflection point there (near N = 10).  Figure 10. Dependence of cx max on N given by Equation (31). As one can see, cx max is a non-extensive function-it reaches the plateau for N 1. For N ≈ 10 4 the plateau is achieved with a good approximation. This is an important issue for researching complexity. Namely, systems can attain complexity already on a mesoscopic scale. It can be said that the curve's inflection point (located near N = 10) marks the beginning of the complexity stabilization region. This example shows that even such a simple arrangement of non-interacting objects may have non-equilibrium non-stationary complexity. A necessary (but not sufficient) condition is the possibility of constructing entropy and the presence of a time arrow.

Concluding Remarks
In many recent publications [5,8,9,51] it is argued that entropy can be a direct measure of complexity. Namely, a smaller value of entropy indicates more regularity or lower system complexity, while its larger value indicates more disorder, randomness and higher system complexity. However, according to Gell-Mann, more disorder means less, and not more, system complexity. These two viewpoints are contradictory-this is a serious problem, which we have addressed.
Our motivation in solving the above problem was based on Gell-Mann's view of complexity. This is because we fail to agree that the loss of information by the system as it approaches equilibrium increases its complexity; notably, ∆I(p eq , p eq ) (see Appendix B for detail) takes its minimum value then, and complexity must decrease.
In addition, the differences between entropies in Equation (1) eliminate the useless dependence of complexity on the additive constant that may appear in the definition of entropy. It can be said that the system state with the highest complexity is the state most distant from all of the states of the system of lesser or no complexity.
Thus, in the sense of Gell-Mann, the measure of complexity should supply complementary information to the entropy or its monotonic mapping.
Therefore, in this work, we have presented a methodology which allows building a universal measure of complexity as a function of a system state based on non-linearly transformed entropy. This is a non-extensive measure. This measure should meet a number of conditions/axioms, which we have indicated in this work. A parsimonious example, of the simplest system with a small and a large number of degrees of freedom, is presented in order to support our methodology. As a result of this approach, we have shown that (generally speaking) the most complex are optimally mixed states consisting of pure states, i.e., of the most regular and most disordered, which the space of states of a given system allows. This also applies to the distinctive examples outlined in Appendixes D and E (although this requires a redefinition of some variables and parameters).
We should pay attention to an essential issue regarding the definition of the phenomenological partial measure of complexity that is given by the Equation (1). This definition is open in the sense that if the description of complexity requires, for example, one additional quantity, then the Equation (1) takes on an extended form, whereby E min ≤ E ≤ E max this new quantity is marked. This definition has still an open character. Specifically, this definition also allows (if the situation requires) the replacement of one quantity with another, e.g., entropy with free energy, or considering some derivatives (e.g., of the type ∂S ∂E ). Openness and substitutability should be the key features of the measure of complexity. Moreover, exponents m j , n j , j = 1, 2, determine the order of complexity, i.e., its level or scale. We emphasize that the measure of complexity introduced can describe isolated and closed systems (although in contact with the reservoir), as well as open systems that can change their elements.
From Equations (13) and (32), we get the phenomenological universal full measure of complexity in the form, which extends Equation (17), The full measure of complexity is a weighted sum of partial measures of complexity across all complexity scales. As one can see, this full measure may contain singularities. They are the necessary signatures of criticality existing in the system. This meets the expectations presented in the literature.
Definitions of measures of complexity Equations (1) and (17) and their possible extensions are universal and useful. It is due to entropy that is associated not only with thermodynamics (Carnot, Clausius, Kelvin) and statistical physics (Boltzmann, Gibbs, Planck, Rényi, Tsallis), but also with the information approach (Shannon, Kolmogorov, Lapunov, Takens, Grassberger, Hantschel, Procaccia), and with the approach from the side of cellular automata (von Neumann, Ulam, Turing, Conway, Wolfram, et al.), i.e., with any representation of the real world using a binary string. Today, we already have several very effective methods for counting entropy of such strings, as well as other macroscopic characteristics sensitive to organization and self-organizing systems, as well as to their synchronization (synergy, coherence), competition, reproduction, adaptation-all of them sometimes having local and sometimes global characters.
Our definition of complexity also extends to meet research into the complexity of the biologically active matter. In this, especially research on the consciousness of the human brain can derive a fresh impulse. The point is that most researchers believe that the main feature of conscious action is a maximum complexity or even a critical complexity [52]. In our approach, it would be CX max and X(N cr 1 s). We hope that our approach will enable: (i) the universal classification of complexity, (ii) the analysis of a system critical behaviour and its applications, and (iii) the study of dynamic complexity. All of these constitute the background to the science of complexity.

Appendix A. Properties of the Partial Measure of Complexity
It is worth paying attention to Equations (1)-(4). For a fixed span of Z and the order (m, n), there may still be systems of different complexities. The complexity description only using CX max is insufficient, because there can be many systems with the same span and order. However, we assume systems to be equivalent, i.e., belonging to the same complexity class (Z, m, n), if they have the same span at a given order. We can still distinguish them as, in general, they differ in the location of S max CX . We can say that a given class has a greater partial measure of complexity if it has a larger CX max . In a given class, the system has a larger complexity if the system stays closer to CX max , i.e., its current entropy S is closer to S max CX . For a given CX with Equation (1), we obtain (for each order (m, n)) two S solutions: one on the left and the other on the right of CX max (except when S = S max CX ). Division into classes allows us to introduce an additional classification of complexity.
A distinction should be made between two cases of measuring complexity: (i) Z < m + n and (ii) Z > m + n. This is particularly evident when we consider the ratio of both types of complexity measures for m + n > 1, where Z i belongs to case (i), while Z ii to case (ii). Thus, the greater the exponent m + n, the greater the difference between CX max (Z ii ) and CX max (Z i ).
The alternate form of Equation (1), where deviation ∆ = ∆(t) = S(t) − S max CX makes the operating of the CX coefficient easier in the vicinity of S max CX , where the parabolic expansion is valid. We then have: that is a Gaussian form, which has variance σ 2 = nm (n+m) 2 Z 2 . Only in the narrow range of S around S max CX is the measure of complexity CX symmetrical regardless of the order (m, n).
In fact, only the location of the maximum of CX(S; m, n) is determined (for a given range of S) by the ratio of m to n. However, to have dependence of coefficient CX on entropy in the entire entropy range S min ≤ S ≤ S max , it is necessary to determine two extreme values of entropy (S min and S max ) and two exponents (n and m). In general, finding these parameters and exponents is still far from trivial because they have a contextual (and not a universal) character.
However, in a particular situation, when the maximum complexity is symmetrical, i.e., when m = n, we obtain and Equation (1) of the partial measure of complexity applies both to single-and multi-particle problems because entropy can also be built even for a very long single-particle trajectory. Moreover, Equation (1) emphasizes our point of view that any evolving system for which one can introduce the concept of entropy and which has a state of thermodynamic equilibrium (for which entropy reaches a global maximum) contains at least a signature of complexity. For systems of negligible complexity, i.e., for which S ≈ S min or S ≈ S max , the measure CX(S; m, n) is close to zero. This does not mean, however, that we cannot locate S max CX near S min or S max . It is then sufficient to have strongly asymmetric situations when n m or n m, respectively.
where p(t) = {p j (t)} and p eq = {p eq j }. Furthermore, entropy S(t) is closely related to partition function. Therefore, in this approach, the entropy is a base function.
Most often the function f (x) is selected in the form [45,56], coupled with a constant S 0 = 1 α−1 , where α can converge to 1. With these choices the entropy given by Equation (A6) is called Tsallis (relative) entropy and the entropy given by Equation (A7) Rényi (relative) entropy. Usually, the entropic index α is denoted by q in the case of Tsallis entropy.
Entropies given by Equations (A6) and (A7) converge, with the help of Equation (A10), to Kullback-Leibler entropy [9] when entropy index α → 1. However, the Rényi and Tsallis entropies are essentially different for α = 1. The Rényi entropy is an additive function describing extensive systems, while the Tsallis is not. It is a non-additive function describing non-extensive systems.
Using relative entropy in the definition of complexity measures is productive. It is because other types of entropy can be derived from it, such as ordinary entropy (or Boltzmann-Gibbs-Shannon one) and conditional entropy.

Appendix C. Derivation of the Constitutive Equation for Perfect Gas
The derivation of constitutive Equation (30) comes down to presenting both sides of Equation (29) in explicit form, shortening of common factors, proper organization and presentation. Accordingly, the left side of Equation (29) takes the form, where the number of factors in the numerator and denominator is the same and equals N − N CX max L . As for the right side of Equation (29), we present it in an explicit form, Comparison of Equations (A11) and (A12) just leads to Equation (30).

Appendix D. Entropies of Time Series-A Sketch
The entropy study of various time series is a crucial issue in system dynamics. The point is that the activity of the systems is perceived precisely through time series. The study of nonlinear time series is particularly important. Below we outline two essential methodologies for constructing entropy (including a multi-scale one). Then we show how to connect our complexity measure with these methodologies.

Appendix D.1. Entropy of Embedded Time Series
Various time series from stock exchanges or Forex quotations are the central sources of empirical data available from financial markets. The main question is about the entropy of the time series and hence about the measure of time series complexity. Following Li et al. [51], we present a method of constructing entropy for a finite time series.
We consider the time series {x j } N j=1 consisting of N elements x j , j = 1, 2, . . .. From this we select 1 ≤ i ≤ N − l + 1 sub-series indexed by i. Each sub-series consists of l components defined as follows, = x(i + k). As one can see, m here means the embedding dimension.
Moreover, two subsequent sub-series characterized by the same l have l − 2 elements in common. In the collection of sub-series, which create m-dimensional vector space, one can enter a topology defined by the metric d ij defining the distance between arbitrary sub-series i and j. Then one can build a distribution (histogram), p(d ij ), of distances between vectors. Of course, the question of how to choose the embedding dimension l is fully justified. For example, one could assume that this dimension is equal to the correlation dimension [7]. However, we treat l as a free parameter, and we do not impose any additional restrictions on it. With the probability distribution, p(d ij ), one can build the entropy (for example, S(l) = − ln p(d ij ) ) and hence the complexity of the time series (see Section 2.1 and Appendix B for details).
Similarly to the example with expanding gas in Section 3 and based on Equation (5), we can formulate the constitutive equation in the form where S(l max ) = S max and S(l min ) = S min . The transcendental Equation (A13) should be solved numerically due to the value of the unknown l CX max sought. However, first, the values of l min and l max must be found (also numerically), which leads to the determination of S min and S max , respectively. It allows us to determine CX max . In the final step, one can (similar to that presented in Appendix B) find informative relationships of the above quantities l CX max , l CX max /N and CX max from N. This is all possible if one has sufficiently useful statistics, i.e., when N − l + 1 l ≡ N−1 2 l.

Appendix D.2. Multi-Scale Entropy
We can now proceed to define multi-scale complexity, but first we need to define multi-scale entropy or the hierarchy of entropies. For this purpose, we prepare the coarse-grained scheme. The primary time series consists of N elements. We divide it into n = Int[N/τ] non-overlapping intervals, where τ is the time horizon/scale, i.e., the number of time steps that we use to separate the elements of the first time series. Now we can build a new time series, of which the non-overlapping elements are defined as arithmetic means in subsequent intervals of τ, More can be said about the choice of τ using the (bilinear) autocorrelation function if we are dealing with a stationary and a long time series. Then time τ = τ c can be considered, e.g., as the half-life of this function, i.e., AC(τ c ) ≈ 1 2 AC(0). Other choices for τ can also be considered [43]. Note that the time series {y τ j } can consist of τ > τ c with statistically independent elements.
With time series dependent on the time scale τ, we can build scale-dependent entropy S τ (t) and the corresponding complexity CX(S τ ) by the corresponding methods presented in Appendix D.1. Thanks to this, for each scale τ separately (i.e., for each n separately), we can find quantities such as n CX max , n CX max /N and CX max .

Appendix D.3. Elements of Deterministic Chaos: A Cyclically Kicked Damped Rotor
Deterministic chaos can be an example of the complexity of single-particle motion (i.e., the complexity of its phase space). It is caused by instability due to initial conditions. A typical example of this is a cyclically kicked damped rotor [20] or a damped pendulum with cyclic driving force [19]. Here we sketch this example.
The starting point is Newton's equation for a rotor motion in the presence of viscous drag, which takes the form where φ(t) is a time-dependent rotation angle of the rotor, γ is the viscous drag coefficient, and the moment of inertia is equal to the unit. The exact solution of Equation (A16) is based on the time-dependent exponential function. Next, we enrich this equation with a non-linear impulse forcing force, where f (φ) is a non-linear function of φ and κ is its amplitude, while T is the period of this force. Hence, in the stroboscopic variables (or in the Poincaré representation) we obtain the Poincaré map, where ω = dφ dt . The above set of recursive equations allows us to examine on the Poincaré surface both dissipative deterministic chaos (e.g., logistic or Henon mapping) and conservative (i.e., Chirikov mapping). This depends on the values of γ and κ parameters and the form of the function f .
Belief in the complexity of the phase space {ω, φ} of the system presented above is common. The complexity requires a knowledge of entropy. The total entropy of the system is the entropy of its long phase trajectory. However, we consider the complexity of the phase space structure mapped to the Poincaré surface, i.e., based on narrow entropy describing only this mapped structure. Obviously, this entropy depends on initial conditions and parameters assumed. Constructing entropy first requires defining a long time series. We have this time series as one consisting of N two-component elements {ω n , φ n } N−1 n=0 . Indeed, the approach presented in Appendix D.1 can be used to calculate entropy. The goal here is to have the entropy value S max CX at which the complexity measure reaches the maximum value CX max . It comes down to finding the dimension of the embedded subspace l max CX for which the entropy S = S max CX . It is l that is the optimized parameter (in Section 3 it was the dynamic variable N L ). It can be expected that l max CX is neither too small nor too large compared with the dimension N of the base space. Of course, a computer analyzes quotations of all companies on this stock exchange that are in the range mentioned above. Two vertical red straight lines mark the scanning window, and its centre is marked by a blue line. The current window location, i.e., the beginning and end, are given dates just below the DAX and MOL plots. The same scanning window applied to the MOL indicator. The middle plot at the top shows the course of the widely used mean-occupation layer (MOL) [32]. The upper plot on the right shows the distribution of degrees of the vertices of the network shown below. The red circle on this plot, which stands out significantly from the power-law distribution, concerns the centre of the star (the most prominent red circle). The coloured circles in the legend and the abbreviations of the names mean the companies presented in the graph. The figure was taken from our publication [32].
The structure dynamics of the dragon-king is presented in Figure A2. That is, we presented there the dependence of the degree of this node on time, yielding a very meaningful λ peak characterizing here a structural condensate (for details see [32]).
In Figure A3 we compare empirical degree entropies S(t) = − ∑ k P(k) ln P(k), where P(k) defines the empirical degree distribution, versus time in the presence and absence of SZG company on the network. In Figure A1, the upper plot on the right shows an example of the (non-normalized) distribution, P(k), where k means the degree of a node. Various nodes are marked here with coloured circles defined in the legend. As one can see, the presence of the SZG node significantly changes the structure of the MST network. It is worth determining the complexity of such a network.  Figure A2. The temporal SZG vertex degree, k SZG , vs. time. It forms the so-called λ-peak marked with a red vertical dashed straight line. It shows temporary edge condensation on the SZG node. The span of this peak is marked by the last blue vertical dashed straight lines. These lines are also plotted in Figure A3 below. The equilibrium scale-free networks are placed outside this area. The centre of this peak has been extrapolated on Thursday 25 January 2007-a deviation of two trading days from the result shown in Figure A1 is irrelevant here. The plot was taken from the publication [32] with the consent of the editors.  Figure A3. Degree entropy (defined in [32] by degree distributions) vs. time. The solid line marked entropy in the presence of the SZG vertex but dotted after removing this node. One can see the crucial role of this node in preparing the temporary network structure. The well-defined local absolute minimum of degree entropy is placed on Thursday, 25 January 2007. The plot was taken from the publication [32] with the consent of the editors.
Substituting these numbers into Equations (A4) and (A5) we get the entropy of the order of (m = 2, n = 2), and complexity measure of the same order CX max = 2.769 10 −4 .
From the plot data: 'Entropy with SZG 'in Figure A3, dates corresponding to the entropy given by Equation (A30) can be read. There are many of them-we chose three particular ones: 6 January 2006, 21 December 2007, 15 April 2008. All of them concern the most complex networks. Figure A4 shows the network corresponding to the second date. One can see how much it differs from the least complex shown in Figure A1. This Figure shows a mixture of few mini stars with a degree not greater than about 20 (but no central one with degree almost 100, as shown in Figure A1). It is indicated by the degree distribution on the right-hand side of the plot, besides several skeletons and developed cascades arranged there in a somewhat disordered way. It is a richer and more disordered structure-more complex than that shown in Figure A1. Figure A4.
The snapshot picture of the Frankfurt Stock Exchange complex network on 21 December 2007. This network represents the most sophisticated state of the stock market, that is, CX max given by Equation (A31). One can see how much this network differs from that shown in Figure A1-e.g., the dominant central star has ceased to exist. The network structure is now skeletal.

Conflicts of Interest:
The authors declare no conflict of interest.