Extracting Metasystem: A Novel Paradigm to Perceive Complex Systems

Abstract

Abundant evidence shows that there is a core component within a complex system, referred to as the metasystem, that fundamentally shapes the structural and dynamical characteristics of a complex system. The limitations of existing techniques for analyzing complex systems have made it increasingly desirable to extract metasystems for modeling, measuring, and analyzing complex phenomena. However, the methods of extracting metasystems are still in their infancy with various shortcomings. Here, we propose a universal framework based on divide and conquer to extract fine-grained metasystems. The method comprises three stages performed in sequence: partitioning, sampling, and optimizing. It can decompose a complex system into interconnected metasystem and non-metasystem components, providing a lightweight perspective for studying complex systems: essential insights can be gained by merely examining the internal mechanisms of each component and their interaction patterns.

1. Introduction

It is common for natural and artificial systems to sustain their fundamental functions even when some of their components fail. For instance, a damaged Internet can maintain connectivity and continue to operate even in the event of massive cascading failures [1,2]. Social systems are also self-holding even when the interaction patterns among social actors are in constant flux [3,4]. Similarly, although gene knockout can propagate to the metabolic network through the failure of enzymes, a dysfunctional regulatory network is seldom observed [5,6,7]. Such instances are quite common, leading to an increasing belief that there must exist a specific component capable of controlling the structure, behavior, and realization of a complex system [8,9,10,11,12].

Such components are typically referred to as the metasystem or the system of systems [13,14,15]. Although metasystems have yet to be precisely defined and measured so far, extensive efforts have been made to extract metasystems over the past decades [16,17], resulting in various techniques such as super graphs [18,19], graph limits [20,21], graph sparsification [22,23], graph coarsening [24,25], and higher-order networks and graph minor [26,27]. In general, the components extracted by these techniques are treated as coarse-grained metasystems incapable of substantially enhancing the modeling, measuring, and analysis of complex systems. More seriously, most of them are only theoretically, but not practically, feasible. Therefore, it is necessary to devise robust methods to extract fine-grained metasystems.

2. Methods

Graph limit theories demonstrate that every graph can be well approximated by the union of a constant number of random-like bipartite graphs [28], which theoretically demonstrates the existence of metasystems and implies that ‘divide-and-conquer’ is the most proper way to extract them. Following the idea, we propose a multiple-stage method to extract metasystems. The first stage partitions an original system into subgraphs termed basic sampling units (BSUs). The second stage samples BSUs, beginning from the core and extending toward the periphery, to construct a preliminary metasystem. Finally, the preliminary metasystem is optimized using the genetic algorithm (GA) to yield a fine-grained metasystem. These three stages are sequentially referred to as partitioning, sampling, and optimization and are integrated elaborately as a whole.

2.1. Stage 1: Partitioning

Stage 1 aims to partition an original system into BSUs, and the basic process is as follows:

(1)

Given a graph $\mathrm{G}=(V,E)$ and its Laplacian matrix $\mathrm{L}\left(\mathrm{G}\right)$, the eigenvector–eigenvalue pairs of $\mathrm{L}\left(\mathrm{G}\right)$ are solved, and the eigenvalues are labeled in ascending order: ${\lambda }_1\le {\lambda }_2\le \cdots \le {\lambda }_{\mathrm{n}}$. The second eigenvector $V_2$ (the Fiedler vector [29]) of the Laplacian eigenvalue set is employed to partition the original system into 2-way cuts.

(2)

The second eigenvector $V_2$ is decomposed into two subsets, with its median as the dividing line. Correspondingly, all nodes in the original system are decomposed into left and right subgraphs following the one-to-one correspondence between eigenvectors and nodes.

(3)

The modularity [30] of the partitioned subgraphs is estimated to determine whether further partitioning is required. Specifically, if the modularity is below a predefined threshold (critical value = 0.4), it indicates the original system is sufficiently decoupled; otherwise, the above process will continue.

Figure 1 simulates the partitioning process of an original system. By using the Fiedler vector, the 2-way partition is implemented hierarchically to build a binary tree, setting the bottom leaf nodes as the basic sampling units applied in Stage 2. The decomposition depth depends on the structural complexity of the original system; for our purpose, thorough partitioning is unnecessary, so the criterion of modularity is set to a relatively high upper limit in practice.

2.2. Stage-2: Sampling

Stage 2 aims at constructing a preliminary metasystem. The idea of the delayed rejection sampling method (DRSM) [31] is drawn on, and the basic process is as follows.

(1)

The PageRank of each node in a BSU is estimated, and the node with the maximum value is selected as the initial seed to guide the sampler. In terms of the implication of PageRank, we can conclude that the node with the maximum value certainly positions at the core area of the BSU.

(2)

The DRSM is applied to select appropriate edges within the BSUs from the core toward the periphery. All selected samples are then incorporated to construct a preliminary metasystem, an intermediate version of the final metasystem.

(3)

To capture the overall structure within a BSU, the DRSM terminates as soon as all the edges within the BSU are visited at least once. This is necessary because the elements belonging to a metasystem may be distributed across both the core and the periphery areas.

2.3. Stage 3: Optimizing

The preliminary metasystem is further optimized using the GA to generate the final metasystem. The critical operators are described as follows.

(1)

Selection operator: Figure 2 describes the selection operator of the GA, in which S1–S100 represent all the edges in $S$. The elements in the blue frame (S1–S80) represent the retained sequences, whereas those in the red frame (S81–S100) represent the new sequences selected from the candidate set $C$ to replace the eliminated sequences within $S$. The rule of the selection operator is fairly straightforward: the top 80% of the sequences in $S$ are preserved, and the others are replaced by new sequences from $C$.

(2)

Crossover operator: Any two sequences in $S$ are randomly selected, and a classic crossover approach, the partially mapped crossover [32], is applied to generate two new sequences to replace the original sequences.

Figure 3 illustrates the fundamental idea behind the crossover operator, in which $C1$ and $C2$ $(C1<C2)$ denote control parameters randomly generated between [1, 0.5 ∗ (length (feasible solution)] and [0.5 ∗ (length (feasible solution) + 1, length (feasible solution)]. After the crossover positions are determined using the control parameters, the elements are separately identified from the selected sequences and swapped with each other to yield new feasible solutions. Note that the crossover operator is not always valid, with an incidence of validity ranging from 5% to 7%.

(3)

Mutation operator: The mutation operator employs a common method, the simple mutation method. Specifically, elements randomly selected within sequence $F$ are replaced by an equal number of elements from the candidate set $C$. In Figure 4, two random numbers $M1$ and $M2$ ($M1<M2$) are first generated as control parameters to set the mutation field. Then, the elements within the mutation field in sequence $F$ are replaced by new elements from the original system not chosen in the previous steps. Like the crossover operator, the mutation operator has a low incidence of validity of 3–5%. Moreover, the total number of mutation bits is finite, up to 50% of the entire length of the sequence $F$.

As the GA reaches convergence, the final metasystem is generated. Note that the elements processed by the GA come from two parts: the preliminary metasystem and the supplementary set $C$. That is, during the optimization process, the elements of the preliminary metasystem are likely to be eliminated using the selection, crossover, and mutation operators to extract the final metasystem as accurately as possible.

Our proposed multi-stage method is developed based on divide-and-conquer. Its standard time complexity satisfies an equation: $O\left(N\right)=a\ast S\left(B\right)+f\left(T\right)$, where $T$ = size of input, $a$ = number of subproblems in the recursion, and $B$ = size of each subproblem. Since the proposed method employs spectrcum bistiction to partition a graph, all subproblems are approximately the same size. $f\left(T\right)$ = cost of the work performed outside the recursive task, comprising two parts: the cost of stage 1 (partitioning) and stage 3 (optimizing).

Given a graph $\mathrm{G}(\mathrm{V},\mathrm{E})$, we first estimate the worst-case time complexity. The time complexity of the standard spectral partitioning is $\mathrm{O}\left({\mathrm{N}}^3\right)$, where $N$ is the number of vertices in the graph. For the GA, a common estimation is $(\mathrm{G}\ast \mathrm{L}\ast \mathrm{N})$, where population size $\left(\mathrm{N}\right)$, number of generations ($\mathrm{G}$), and chromosome length $L$. That is, $f\left(T\right)=(\mathrm{G}\ast \mathrm{L}\ast \mathrm{N})$. The time complexity of $T$ is $O\left(\right|\mathrm{V}|/\mathrm{a})$, as is well known, a graph at most comprises N * (N − 1) edges (a complete graph), and thus $S\left(B\right)=\mathrm{O}\left(\right(\mathrm{N}\ast \mathrm{N}-1)/\mathrm{a})$. In view of this, $O\left(N\right)$ is determined by $f\left(T\right)$.

In general, a real graph is typically sparse and spectral bisection is applied in stage 1, thereby decreasing the complexity of $f\left(T\right)$ can be decreased to $O\left(N\ast \mathrm{l}\mathrm{o}\mathrm{g}\mathrm{N}+\mathrm{G}\ast \mathrm{L}\ast \mathrm{N}\right).$ For a large complex network, $N\gg G\ast P$, so the overall time complexity can be relaxed to $O\left(N\ast \mathrm{l}\mathrm{o}\mathrm{g}\mathrm{N}+\mathrm{a}\ast \mathrm{G}\ast \mathrm{L}\ast \mathrm{N}\ast (1-\mathrm{a}\right))$ ($a\in [0,1]$ a is the sampling ratio in stage 2). Due to $a\ast \left(1-\mathrm{a}\right)\le 1/4$, So the final complexity has two possible outcomes: $O\left(N\right)=N\ast \mathrm{l}\mathrm{o}\mathrm{g}\mathrm{N},\mathrm{n}\to \mathrm{\infty }$; otherwise, $O\left(N\right)=N\ast \mathrm{l}\mathrm{o}\mathrm{g}\mathrm{N}+\mathrm{G}\ast \mathrm{L}\ast \mathrm{n}/4$.

3. Results

We comprehensively compare the differences between original systems and their metasystems extracted from various types of original systems, with the critical indicators listed in

Table 1. The differences in key dimensions between the original system (scale-free network) and its metasystem.

	I1	I2	I3	I4	I5	I6	I7	I8	I9	I10	I11
	Interactivity				Connectivity				Aggregation
Original System	−0.0493	13.95	0.0071	0.026	16	10	6.096	0.164	0.0145	0.0102	10.936
Metasystem	−0.0445	2.88	0.00284	0.017	32	18	12	0.090	0.0064	0.0048	10.025

and

Table 2. the differences in key dimensions between the original system (random network) and its metasystem.

	I1	I2	I3	I4	I5	I6	I7	I8	I9	I10	I11
	Interactivity				Connectivity				Aggregation
Original System	0.00038	57.59	0.00477	0.041	44	35	31	0.03419	0.004347	0.00435	23.885
Metasystem	0.00019	10.95	0.00119	0.078	87	63	47	0.02553	0.002457	0.00258	21.913

. The eigenvalue-centrality-entropy in Table 1 and Table 2 evidence that the indicator of metasystems approximates those of the original systems with a similarity of up to 91%. Given the high degree of similarity of the indicator and the fact that a metasystem is necessarily a subgraph isomorphic to the original system (a metasystem is extracted following the interaction structure of an original system), we can conclude that an original system and its metasystem essentially share the same interaction mode.

Note that the similarity here does not imply that a metasystem aligns with the original system in eigenvalue-centrality; on the contrary, there are completely distinct between corresponding nodes. Nevertheless, the large difference in eigenvalue-centrality does not interfere with the interaction mode among nodes. Spectral graph theories illustrate that the nodes with lower degree typically feature lower eigenvalue-centrality, exerting only a minor influence on entropy distribution. Therefore, we can extract a metasystem based on the intrinsic relation mode within an original system, even with large differences in edge density.

Of course, there are clear differences between original systems and their metasystems. We first discuss these differences in terms of network density. The network densities of the metasystems in both tables are all lower than those of their original systems, implying that the ratio of nodes to edges in the original systems is higher than that of the metasystems. A complex system constitutes a collection of nodes and edges; therefore, the lower network density of metasystems indicates that although some new nodes are added to the metasystem during the evolution from a metasystem to an original system, the more significant change is the large increment of relationships between nodes. Thus, we can conclude that an original system evolves following the architecture of the metasystem, and the metasystem is the fundamental component of the original system.

Similarly, the network efficiency, diameter, and radius of the metasystems presented in Table 1 and Table 2 indicate that these metasystems perform less effectively than the original systems in terms of connectivity, interactivity, and cohesion. The main reason is due to the difference in node degree, and the chain reaction is as follows: the larger the average degree, the denser the network (average degree and network density are positively correlated); the denser the network, the higher the connectivity; the higher the connectivity, the better the interactivity.

Interactivity {I₁: assortativity coefficient, I₂: average Degree, I₃: network density, I₄: normalized average degree}

Connectivity {I₅: diameter, I₆: radius, I₇: average-path-length, I₈: network efficiency}

Aggregation {I₉: clustering coefficient, I₁₀: transitivity, I₁₁: eigenvalue-centrality-entropy}.

Figure 5 depicts the evolution process of four critical indicators from the intermediate metasystem to the final metasystem. Each curve can be roughly categorized into evolutionary and stagnating sections, indicating that all indicators have different degrees of sensitivity to introduced feasible solutions. The direct reason for the stagnation section can be attributed to the continuous rejection of the newly generated candidate solutions during the execution of the GA. However, the rejection of candidate solutions is entirely independent of the GA. Accurately, only feasible solutions that capture the architecture of the metasystem can simultaneously influence various indicators. Thus, the unique organizational capability of the metasystem achieves disproportionate performance improvement across multiple indicators, even at a relatively low network density.

Regarding the influence of metasystems on original systems, two opposite results can be observed straightforwardly. The assortativity coefficients of metasystems and original systems (scale-free networks) in Table 1 tend to converge with a slight difference of about 5–8%. However, the result is opposite in Table 2 when extracting the metasystem from random networks. In systems theory, the assortativity coefficient is one of the key indicators to measure orderliness [33]. By repeating the experiment and examining the results, we find that the orderliness of an original system is closely related to that of its metasystem. A metasystem exhibits largely consistent orderliness with the original system if the assortativity coefficient exceeds a specific threshold; otherwise, the opposite result is observed.

Why do metasystems impose different degrees of impact on the orderliness of original systems? The answer is the node-degree. In Table 1, the normalized-average-degrees of the metasystems extracted from scale-free networks are considerably larger than those of the original, which implies that the elements within a metasystem may be tightly coupled even with a relatively low network density and thus outperforms its original system in terms of interactivity. This result states that the evolution of a complex system is closely related to its metasystem. Specifically, the more orderly the metasystem, the stronger its influence on the formation of an original system; conversely, the evolution of an original system is likely to be uncontrolled if its metasystem is not ordered enough.

Figure 6 provides a schematic of an original system and its metasystem. The black dots denote the nodes in the original system, and the red solid and blue dashed lines represent edges within the metasystem and the original system, respectively. As a subset of the original system, the metasystem shares common nodes and edges with it. Unexpectedly, elements in the metasystem are scattered throughout the original system, contradicting the speculation that elements of a metasystem are mainly positioned at the core of an original system. Indeed, a metasystem is an independent component that is also composed of the core and the peripheral part, and its scattered nature reflects the underlying structure of an original system. Consequently, any evolution occurring throughout a complex system will certainly be influenced by its metasystem.

4. Discussion

(1)

key points in stage 1

Three critical questions arise in stage 1. First, is it necessary to decompose the original system? The answer is yes for two reasons. First, Szemerédi’s regularity lemma theoretically demonstrates that a large-scale graph can be approximated by some partitioned subgraphs, implying that divide and conquer is the most appropriate strategy for extracting metasystems [34,35]. Second, the division of an original system can substantially decrease structural complexity to facilitate subsequent steps.

Second, do different graph partitioning strategies affect the follow-up stages differently? Graph partitioning techniques, as known, are broadly classified into balanced and unbalanced partitioning. Typically, a complex system is composed of communities, large and small. Partitioning large-scale graphs using the unbalanced partitioning method naturally yields numerous small basic sampling units (BSUs) with few nodes and edges, posing tricky problems for Stage 2 because serious sampling errors are inevitable when sampling small-scale BSUs. Thus, the spectral-bisection method is more advantageous for follow-up steps.

Finally, what is the most appropriate indicator to determine whether an original system is decomposed sufficiently? The consensus is modularity, which is ideal for assessing the structural complexity of a large-scale graph [30]. Practical tests show that it is fairly effective at measuring the aggregation of BSUs.

(2)

key points in stage 2

Existing graph sampling algorithms suffer from the over-selection of high in-degree nodes, frequently resulting in biased and tightly aggregated modules. To resolve the issue, we propose the DRSM (Algorithm 1) that can reasonably improve the selection probability of low in-degree nodes to construct the overall aggregation structure of BSUs. The DRSM (Algorithm 1) redesigns the probability transition kernel based on the Metropolis-Hastings algorithm, and it is theoretically defined in (1), where $Proc$(1) is the likelihood ratio, $Proc$(2) is the ratio of proposal probabilities, and $Proc$(3) is the ratio of complementary probabilities acceptance probabilities.

$$Proc\left(\alpha ,{\beta }_1,\dots {\beta }_i\right)=\left[Proc\left(1\right)\right]\ast \left[Proc\left(2\right)\right]\ast \left[Proc\left(3\right)\right]$$

(1)

$$Proc\left(1\right)=[{\displaystyle \frac{\pi \left({\beta }_i\right)}{\pi \left(\alpha \right)}}]$$

$$Proc\left(2\right)=[{\displaystyle \frac{q_1\left({\beta }_i,{\beta }_{i-1}\right)\dots q_i\left({\beta }_i,{\beta }_{i-1,}\dots ,\alpha \right)}{q_1\left(\alpha ,{\beta }_1\right)\dots q_i\left(\alpha ,{\beta }_1,\dots ,{\beta }_i\right)}}]$$

$$Proc\left(3\right)=[{\displaystyle \frac{[1-P_1\left({\beta }_i,{\beta }_{i-1}\right)]\dots [1-P_{i-1}\left({\beta }_i,{\dots \beta }_1\right)]}{\left[1-P_1\left(\alpha ,{\beta }_1\right)\right]\dots [1-P_{i-1}\left(\alpha ,\dots {\beta }_{i-1}\right)]}}]$$

As the name suggests, the DRSM (Algorithm 1) is launched when a candidate generated from the proposal is rejected. That is, whenever a candidate is rejected, instead of proposing a new candidate using the Metropolis-Hastings random walk (MHRW), the DRSM uses the current rejected candidate (hence the term “delayed rejection”) as a seed to propose a new candidate with the certain probability (2) (one of the directly immediate neighboring nodes of the rejected candidate is proposed).

There are two critical points of interest in the DRSM (Algorithm 1). The first is its optimal depth of delayed rejection. Previous studies show that the optimal value is 2, 3, or 4. The general principle is that the larger the variation in the node degree of the BSU, the bigger the value of the delayed rejection. In our study, the depth of delayed rejection is set to 2. The second point of interest is identifying the initial seed of the DRSM (Algorithm 1), which is critical because the sampler of the DRSM (Algorithm 1) needs to visit all elements within a BSU along the core toward the periphery. We adopt PageRank for this purpose. Similar indicators such as authority code, HITS score, and centralized eigenvector would also suffice, as they are strongly positively correlated and conceptually very similar to PageRank.

Algorithm 1 DRSM

Input: Basic sampling units $\left(S(u,v)\right)$

1:    //$EC$ is the exit criterion of the DRSM           While ($EC$) { 2:     //Select a node with maximum PageRank as an initial seed.               IS = GetSeed ($\mathrm{S}\left(\mathrm{u},\mathrm{v}\right)$) 3:     //Select a candidate element using the original MHRW                                           $Proc\left(IS,CE\right)=1\wedge [{\displaystyle \frac{\pi \left(CE\right)}{\pi \left(IS\right)}}][{\displaystyle \frac{q_1\left(CE,IS\right)}{q_1\left(IS,CE\right)}}]$ 4:    //If $CE$ is rejected, delayed rejection is launched. one of the immediate neighbors                          upon the current candidate rejected will be proposed as a new candidate                          with the following new probability      $Proc\left(IS,{CE}^1,{CE}^2\right)=1\wedge [{\displaystyle \frac{\pi \left({CE}^2\right)}{\pi \left(IS\right)}}][{\displaystyle \frac{q_1\left({CE}^2,{CE}^1\right)}{q_1\left(IS,{CE}^1\right)}}][{\displaystyle \frac{[1-q_1\left({CE}^2,{CE}^1\right)]q_2\left({CE}^2,{CE}^1,IS\right)}{{[1-q_1\left(IS,{CE}^1\right)]q}_2\left(IS,{CE}^1,{CE}^2\right)}}]$ 5:    //The edges or nodes unselected in previous steps will be add into the preliminarily metasystem           PMe = Add(elements)           }

Output: A preliminary metasystem ${M(u}^{,},v^{,})$

(3)

key points in stage 3

As it is a typical combinatorial optimization problem to select elements from the supplementary set $C$ to construct the final element system, a genetic algorithm is adopted preferentially. For our purpose, two critical questions deserve careful investigation.

First, how do we construct the fitness function of the GA? The fact that metasystems cannot be mathematically defined prevents the description of the fitness function of the GA in a functional expression. Several insightful works have pointed out that the dynamic and structural features of a complex system are primarily determined by its interactivity [36,37]. Consequently, the normalized average degree and the assortativity coefficient are selected as central variables to control the generation of a metasystem.

In addition, two correction parameters (diameter/radius, transitivity/clustering coefficient) are designed to fine-tune the expansibility and aggregation of a metasystem. Specifically, the fitness function of the GA is constructed using the Standardized Euclidean distance composed of the four parameters [38], as defined in (2), where $\mathrm{X}$ and $\mathrm{Y}$ denote the measurements of the metasystem and the original system, respectively. Guided by the distance, candidate solutions are either retained or eliminated until a coarse-grained metasystem gradually approaches a fine-grained metasystem [39].

$$\mathrm{T}\mathrm{h}\mathrm{e} \mathrm{S}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{r}\mathrm{d}\mathrm{i}\mathrm{z}\mathrm{e}\mathrm{d} Euclidean Distance\left(X,\mathrm{Y}\right)=\sqrt{{\sum }_{k=1}^{k=n}{\left({\displaystyle \frac{x_{1k}-x_{2k}}{s_k}}\right)}^2}$$

(2)

Second, how do we scope the feasible convergence set? The feasible solution set in our proposed method consists of two parts: one with all elements of the current intermediate metasystem and the other derived from the supplementary set $\mathrm{C}$. What we propose for feasible solution sets is theoretically classified as a global optimization [40]. With the strategy, existing elements of an intermediate metasystem will likely be eliminated during the follow-up optimization process, ensuring that the most appropriate elements can be injected into the final metasystem while removing unessential elements as much as possible.

In summary, our proposed method provides a lightweight perspective to investigate complex systems. It decomposes a complex system into a metasystem and a non-metasystem component; they are connected and thus subject to influences from each other. The essential insight can then be obtained by merely studying the internal mechanisms of each component and the interaction mode between them. Compared to existing techniques [41,42,43], the introduction of metasystems enables us to handle a complex system at a higher level without sinking into non-critical details, thus significantly reducing complexity.

5. Conclusions

Our proposed method has advantages in terms of usability. First, it applies to various complex systems with dynamical behaviors intertwined at multiple scales. Second, it is highly scalable and can be flexibly customized simply by introducing additional control parameters to tailor the fitness function of the GA. Of course, common drawbacks of evolutionary algorithms, such as pseudo-convergence, inefficiency, and indeterminacy, occur occasionally. Fortunately, many targeted methods previously developed can address these drawbacks. Finally, it is easily parallelized and independent of specific big data technologies.

In our plan, large model technology will be incorporated into the future improvement strategy. We anticipate that the future upgraded method can proactively interact with users and guide them based on their inherent mechanisms to improve usability as much as possible. Also, the current bottleneck remains in the GA. Similarly to traditional combinational optimization problems, the optimization measure must be closely aligned with the characteristics of application scenarios to effectively improve the fitness function of the GA.