Graph Metrics for Network Robustness—A Survey

: Research on the robustness of networks, and in particular the Internet, has gained critical importance in recent decades because more and more individuals, societies and ﬁrms rely on this global network infrastructure for communication, knowledge transfer, business processes and e-commerce. In particular, modeling the structure of the Internet has inspired several novel graph metrics for assessing important topological robustness features of large complex networks. This survey provides a comparative overview of these metrics, presents their strengths and limitations for analyzing the robustness of the Internet topology, and outlines a conceptual tool set in order to facilitate their future adoption by Internet research and practice but also other areas of network science.


Introduction
The Internet has become a part of our everyday lives and is used by more and more individuals, firms and organizations worldwide [1]. Internet disruptions can entail major risks and huge costs [2], in particular for the growing number of e-businesses. Being cut off from the Internet for even the shortest period of time affects customer confidence and could lead to severe business losses. Motivated by these risks, a new field of research has evolved that investigates the robustness of the Internet with respect to random failures and targeted attacks, based on mathematical graph models [3][4][5]. This fundamental abstraction enables analysis and quantitative evaluation of the main function of large and complex networks such as the Internet: their ability to provide-preferably short and fast-communication paths between entities. Assessing the robustness of the Internet structure and corresponding traffic flows has recently inspired a large number of novel robustness metrics. As of yet, however, there is no comprehensive review of this field. The present survey aims to fill this gap in order to foster future research. It presents important analysis strategies and metrics that are proposed in the literature, along with major results.
Most of the metrics discussed are based on graph properties of network models, but not any implementation details or specific protocols of the Internet. Since the application of graph theory and network science is growing in many other research fields, the present work could therefore also be of benefit to those domains, whenever the robustness of networks needs to be investigated.
This article is structured as follows. First, Network Robustness for communication networks is introduced from an intuitive perspective; then, layer and graph models of the Internet as well as network generators are presented, and challenges and aspects of robustness are introduced. The main categories of metrics are introduced, and the corresponding article structure is motivated. Then, important general features of metrics are compared and discussed, followed by a list of symbols, concepts and mathematical notations.
In the main part of the article, the actual metrics and methods for robustness analysis are presented and discussed, using categories that reflect their main idea: Adjacency, Clustering, Connectivity, Distance, Throughput, Spectral Methods, and Geographical Metrics. This is followed by a discussion section that compares the suitability of the metrics for measuring major robustness aspects and outlines a conceptual tool set of metrics for further research on the robustness of the Internet and of networks in other research domains. Finally, a summarizing table in the Appendix A provides an overview of the main metrics and their features.

Network Robustness
Network Robustness can be intuitively defined as the ability of a network to maintain its function (e.g., its service of transferring data between pairs of nodes in the case of the Internet) in a timely manner under challenges. In general, the interpretation of robustness and its relationship to the more general concept of resilience varies to some extent in different research communities [3,6]. In this article, we discuss the Internet as our motivating use case for studying network robustness. However, the focus is always placed on topological robustness properties and corresponding topological robustness metrics or graph metrics, which mainly abstract from technical or organizational details of the Internet by the use of mathematical graph theory and particular graph models. Therefore, even if we often use the term Internet Robustness from now on, many of the presented metrics can be adopted when studying other types of networks.

Layer Models and Graph Models of the Internet
For the Internet, there are the well-known ISO/OSI and TCP/IP models that focus on technical aspects of communication and are described by a hierarchy of (sometimes optional) layers, such as physical and local connections, Internet routing, transport control and application-layer protocol connections, including the Hypertext Transfer Protocol (HTTP) for the Web but also protocols for exchanging routing information (such as BGP, the Border Gateway Protocol, see RFC 4271), for email exchange, or peer-to-peer (P2P) overlay networks. For instance, in [6], a technically-layered model of Internet Resilience is constituted by a Physical (P), Link+MAC (L), Topology (To), Routing (R), Transport (Tr), Application (A) and End User (U) layer. Each layer provides the base for the next higher one and can be characterized by layer-specific metrics, including non-topological performance measures.
On the other hand, there are organizational and even political perspectives on the exchange, forwarding, controlling and filtering of data on the Internet, for instance, models that distinguish the layers of Internet Protocol (IP) interfaces, routers, points-of-presence (PoP), autonomous systems (AS), Internet Service Providers (ISPs) or other organizations, and countries. As an example, in [7], the authors construct an organizationally-layered family of graph models derived from empirical measurement data.
For both perspectives and most layers, there are historical examples of attacks and failures with impact on connectivity [5,8]. Some attacks on higher layers, such as BGP misconfiguration, affect the robustness of AS-level and IP-level graphs without damage to the underlying physical infrastructure and interconnections of the lower layers. Other failures and attacks take place on the Physical Layer of the Internet, but can cause faults on higher abstraction levels, such as routing or AS layers.
Correspondingly, it is useful to model and analyze entities and their interconnections as graphs for studying its topology and robustness at any particular layer. Traditionally, the complex-network community has studied Internet Robustness at single layers [3][4][5][9][10][11], and this is the main focus of our study; but there are several recent approaches to model the Internet and assess its Robustness using layered hierarchies. Most graph metrics that are discussed in the following sections can be relevant for organizational as well as for technical graph models, and can be applied at every layer where an exchange or relation between different entities is of interest.
Each layer can be modeled by several different graph types [11]. A simple graph is a strong abstraction as it is constituted solely of two entity types: nodes (e.g., routers or ASes) and undirected edges (e.g., links or routing connections), but it has the advantage that fewer information is needed for analysis. In edge-weighted graphs, transmission speed or capacity constraints are also modeled [12]. A directed graph accounts for routing policies between the ASes, e.g., denoting a customer-provider relationship by an edge that points from the former to the latter; peer-to-peer ASes are connected through two links, one in either direction [13,14]. Nodes can only communicate if both have a directed path towards a common node, e.g., if they share a common upstream provider. A weighted directed graph can also be taken into consideration. Further important modeling aspects are the amount of traffic flow between nodes [15] and their Euclidean distance [16].
Concerning properties of complex networks, Internet graphs on both router and AS-levels have been shown to feature scale-free (SF) degree distributions [17] and the smallworld property [9,11], in particular, shorter paths and higher clustering than expected within random networks with a given degree distribution [18].
Not all network-generating models exhibit these characteristics: The Erdös-Rényi (ER) model [19] exhibits none of them, the Watts-Strogatz (WS) model [18] lacks the scale-free degree distribution, and the clustering coefficient of the Barabási-Albert [20] (BA) model is slightly too high. The Klemm-Eguíluz (KE) model [21] and the HOT model [22] of the router level of the Internet [15] seem to be suitable in this sense. However, there is currently no final consensus on a most accurate model, in particular also with respect to the highly dynamic and evolutionary character of the Internet [10,23]. Therefore, in this article, metrics are generally discussed independently of these models.

Challenges and Aspects of Robustness
Even with an abstract and topological approach, the question whether an Internet graph is robust should be considered with respect to the requirements of the particular service that is to be delivered; usually, this is a multidimensional problem, which requires several metrics measuring different robustness aspects that should remain in an acceptable state even under severe challenges.
Two main variants of topological robustness are often discussed in complex network literature: The robustness of a network against random failures of entities, and its robustness against targeted attacks [3][4][5]. These challenges can be further motivated ( [6]): Failures of entities are random and can be caused by software, hardware or human errors. Furthermore, unexpected non-malicious traffic increases in parts of the network can be caused by flash crowds. Not least, geographically extended faults can occur due to earthquakes or power outages.
Targeted and correlated attacks, such as distributed denial of service (DDoS) and malware, aim at destroying crucial entities, e.g., those identified with graph metrics such as highest Degree or Betweenness. Not specifically investigated in our study is the topic of network epidemics, which is relevant for self-disseminating malware such as Internet worms and their propagation strategies, but only indirectly affects network robustness, e.g., by malicious payloads or network saturation. Interactions of malware with topological network properties could be the focus of a dedicated survey on network epidemics.
Many metrics can be used with both random failure and targeted attack strategies. Local or non-adaptive attack strategies calculate the metric values of all entities once and take them down in decreasing order. In contrast, global or adaptive strategies recalculate metrics after every deletion and tend to have a more severe impact while also needing real-time and global information access [24]. In this article, if an attack is analyzed without further specification of its strategy, it always refers to an adaptive attack on highestdegree nodes.
Loosely corresponding to the challenges, there are major aspects of topological robustness. Disconnection Robustness of a graph is measured by metrics that assess path redundancy, detect and evaluate topological bottlenecks, and analyze the severity of graph disconnections. It is defined as the ability of the network to maintain a connection between all (or most) of the node pairs when challenged. Another crucial feature to take into consideration is the small-world property of the Internet, which is associated with the inverse of the average minimum hop distance between node pairs. Transmission Speed Robustness is the ability to keep the decay of this measure low, even under stress.
Another aspect, Traffic Robustness of a network, is defined as the ability to endure and transmit highly dynamic traffic demands without becoming overloaded.
All of these aspects are important for the Internet Backbone, which forms the core of the Internet, providing shortest paths through high-capacity links and routers. Corresponding distance concepts focus, for example, on the number of hops a data package has to traverse in order to reach the target node, implicitly assuming that the longer the path, the slower the transmission speed. Throughput takes into account that capacities for information forwarding are limited, and calculates the maximum workload of network entities or their likelihood to become overloaded when other entities fail.
However, apart from the above mentioned exceptions, the study of layer interactions and their implications for Internet robustness is still an emerging research field, whose tools and methods are not yet broadly established. The same holds for the study of robustness of interacting networks [25] such as electricity and communication networks. For the broader and multi-facet concept of Resilience, we refer the reader to related work [6].
As a baseline for future extensions, our study aims to survey the most relevant metrics for single-layer Internet Robustness analyses, with emphasis on the Topology and Routing Layer and above. To what extent the surveyed metrics are suitable for studying these mentioned aspects, is a complex research question in itself, which will be discussed in more detail in Section 10.

Metric Selection and Article Structure
The selection of metrics is based on literature research, mainly conducted via Google Scholar using diverse topic-related keywords, and inverse search using the references in publications already identified. The selection of metrics for this survey, out of the vast number of general graph metrics in the literature [26], is conducted based on the following criteria: The metric has to be applicable to at least one graph type and either directly measure an aspect of network robustness or at least a clearly related characteristic.
Exclusively assigning the extensive amount of metrics relevant for Internet robustness to non-overlapping groups would prove to be infeasible since some metrics are relevant for several aspects. Therefore, the article is structured as follows: the presentation of metrics and corresponding results for Internet graphs is conducted by dividing them into six major groups that indicate their respective main task. Three of them, Adjacency, Clustering and Connectivity build on each other and describe the general structure of graphs. The Distance and Throughput categories focus on the concepts that are crucial for communication networks, approximating the concrete Internet routing processes via shortest-paths. Throughput accounts for Internet-specific link capacity restrictions. Aspects of all these groups are also regarded in Spectral Methods with the help of random walks. The more sophisticated matrix-calculation schemes involved justify their bundling into a dedicated chapter. Afterwards, a separate section assesses the suitability and interactions of the metrics with respect to the different Robustness Aspects.

Features of Metrics
Many metrics are used with different names in the literature. In order to ensure consistency of notation, only one name is selected and used throughout the article; the others are provided in brackets. At the outset of every section, general features of the metric are displayed.
First, the technical layers as defined above in Section 2.2, following [6], on which the metric can be used particularly well (but often not exclusively), are indicated by their respective abbreviations given in Section 2.2. This categorization should be understood as a preliminary step to combine the traditional single-layer graph analysis of the Internet with a more detailed technical layer model. The next feature indicates the graph types to which the metric has been applied in the literature. Whenever an application to further graph types is straightforward, by substituting simple-graph elements by their directed or weighted counterparts, this will be listed in brackets. For example, the entry "(un)weighted (directed)" indicates that the metric is applied to simple and weighted graphs, but can be modified for directed ones. In order to highlight metrics that can be applied to every graph type, "all" is used.
The third entry states whether the metric is local-either measuring the individual node's impact on network robustness or its liability to faults of connectivity to the rest of network-or if it is global, capturing features about the entire graph [27]. Local metrics often depend on a smaller information set, provide a more detailed insight and need less calculation time, while global metrics are more meaningful for assessing the state of the entire network and also allow for comparison of different graphs.
A metric is denoted static if it is applied without simulating faults-taking a snapshot of characteristics that influence robustness-or dynamic if it assesses network behavior under arbitrary removal strategies; the entries failures or worst-case indicate metrics only suitable for these scenarios.
Next, the codomain is provided, providing a quick impression of the value range of a metric-for instance, if the metric ranges from zero to one. Its dependency on parameters such as the network size indicates whether a direct comparison of graphs that differ in these aspects is possible. If it is set to '-', the presented method is not a metric in the narrower sense, but a general algorithm, e.g., for finding cluster structures and minimum cuts.
Finally, an efficiently computable metric is denoted by a check mark, otherwise its exact calculation is NP-hard or NP-complete. If calculation orders O of (approximating) algorithms are provided in the literature, they are stated in brackets. A non-calculable metric is not practical for robustness analysis of large graphs, but still can provide interesting ideas and a base for further heuristics.

List of Symbols
The following notations are used throughout the article: G(V, E) Graph G with a set of vertices V and set of edges E. v = |V| Number of nodes in the graph. e = |E| Number of edges in the graph. V i set of nodes that are neighbors of node i in a simple graph x Arithmetical mean of x y for all nodes/edges/etc. y.
By substituting the following elements of simple graphs with their weighted and/or directed equivalents, various metrics can be enhanced with the additional information contained in more advanced graph models.
Adjacency matrix; a ij = 1 if an edge from node i to j exists, a ij = 0 else. A w = [w ij ] Weight matrix; w ij is the edge weight between nodes i and j. k Minimum hop distance from node m to node n (Section 6). d w (m,n) Minimum weighted distance from node m to node n (Section 6).

Adjacency
The assessment of node adjacency is one of the first and easiest approaches for investigating network robustness. The intuition is that a vertex with many edges, i.e., a high-degree node, could be more important to the overall graph structure than one with a low degree. The average node degree is a first indicator for the overall network robustness, and the node-degree variance for the resilience against high-degree attacks .
3.1. Node Degree (Degree Centrality) The Node Degree [28] is used in most of the classical results on Internet robustness. The directed version of this metric (Equation (2)) is given in [12].
Notably, in [29], the simple-graph version k i is used to build an attack algorithm that first deletes the nodes with the highest degrees. The authors show that this attack algorithm affects ER, KE and WS networks more than random node failure in terms of Global Network Efficiency E i (Section 6.2) and Number of Nodes in the Largest Component |V L | (Section 5.12). However, at each step, the removal of a different node than the one with the highest Node Degree could cause even larger global damage. Therefore, the Degree of a node has some, but also a limited, influence on global Disconnection and Transmission Speed Robustness, beyond assessing the local connectivity of a node to direct neighbors.
Hence, relying on the degree alone for estimating the importance of a node could be misleading. In [30], a high correlation between Node Degree and Betweenness Centrality (Section 7.1) was discovered at the AS-level of the Internet in particular for higher degree nodes, as well as in a WS, BA, ER, and clustered scale-free networks.
The global version of this metric is the Degree-Frequency Distribution, P(k) = |V k |/v, with |V k | defined as the number of nodes with degree k. Its codomain is [0, 1]. A classical result of [17] states that the Internet belongs to the class of scale-free networks with P(k) ∝ k −γ . This distribution indicates that few highly connected nodes (hubs) exist that are very important for the network connectivity. More precisely, the authors found γ ∈ [2.15, 2.2] for the Internet on the AS-level between 1997 and 1998, whereas for the router level from 1995, γ = 2.48. However, in [15], is is challenged that the Internet exhibits a hub-like core. The authors claim that a more suitable Internet topology with the same P(k), the HOT network, exhibits a rather different topology with hubs on the periphery and hence not as a crucial Backbone. Their removal would hence only have local effects.

Strength
Strength is the adaptation of Node Degree for weighted graphs [12,31]. Quantifying the amount of information that would have to be redirected in the case of a failure of node i, it can be used as a first indicator for its impact on Transmission Speed and Traffic Robustness. Moreover, the largest-capacity Backbone nodes are indicated by highest Strength. Its global version [32] ranges from zero to one: P(s) = |V s |/v, with V s defined as the set of nodes with strength s. It gives an overview over the heterogeneity of local node importance.
Similar to the Degree-Frequency Distribution, a more uniform Strength Distribution leads to a network that is more robust against attacks, but more vulnerable to random failures.

Entropy
Entropy H measures the homogeneity of the Degree-Frequency Distribution P(k) (Section 3.1) The maximum of this metric is H max = log(v − 1). Wang et al. [33] find that optimizing an SF-network's Disconnection Robustness to random failures in terms of Percolation Threshold p c (Section 5.9) is equivalent to maximizing its Entropy-and thus homogeneity of P(k)while maintaining the SF-structure and keeping the average Node Degree constant.

Skewness
Skewness Sk ≈ 0 indicates a very preferential or heterogeneous network, Sk ≈ 1 a very uniform one. Sk u is the Skewness of a network with uniform degree k. The lowest node rank r i = 1 is given to the node with the highest degree, r i = 2 to the one with the second highest, and so on. Each rank is assigned only once; if there is more than one node with the same degree, the order in which they receive the respective ranks is randomly chosen [34].
Ghedini and Ribeiro [29] show that a more homogeneous Degree-Frequency Distribution P(k) (Section 3.1), as measured by Skewness Sk, contributes to higher Transmission Speed and Disconnection Robustness in terms of Global Network Efficiency E i (Section 6.2) and Number of Nodes in the Largest Component |V L | (Section 5.12) against highest-Degree attacks, but leaves the graph more vulnerable in this respect to random failures. In the preceding and current sections, contrasting conclusions about the impact of P(k) on Disconnection Robustness to random failures are drawn. It must be noted, though, that p c measures a worst-case scenario at high failure rates and thus a different facet of Disconnection Robustness than |V L |, which evaluates graph behavior at lower failure rates. In this light, the apparent conflict between the metrics is solved.

Vulnerability Function
The Vulnerability Function v σ (G) [35,36] aims at measuring the vulnerability of a graph with an axiomatic approach, only based on information contained in the Node Degree Distribution D(k) = P(k) · v. In the formula, σ is the standard deviation of D, and P(k) is the Degree-Frequency Distribution (Section 3.1). A high value of v σ (G) indicates a vulnerable network, a low value a robust one.
Similarly to Entropy (Section 3.3) and Skewness (Section 3.4), the Vulnerability Function assesses the impact of the homogeneity of P(k) on robustness, additionally taking the relation of numbers of nodes to edges into account-it is designed not to simply increase when edges are added. For graphs with the latter two parameters fixed, the Vulnerability Function thus shares the implications of the other two metrics on Disconnection and Transmission Speed Robustness.

Assortative Coefficient
The Assortative Coefficient [37] calculates whether the network is assortative, r > 0, meaning that most nodes are connected to nodes with a similar degree, or disassortative, r < 0, as the Pearson correlation coefficient of node degrees at both ends of an edge. Similar to Average Neighbor Connectivity (Section 3.7), this metric summarizes the JDD by calculating the probability that a hub node is connected to another hub, but the information loss is even bigger. The advantage is the clear codomain, with r > 0 indicating an assortative network, which is thus robust against failures and attacks, and r < 0 a disassortative one. If r = 0, there is no correlation between the node degrees of an edge.
In [38], assortative networks are found to exhibit a higher Disconnection Robustness concerning the Number of Nodes in the Largest Component |V L | (Section 5.12) against both failures and attacks and the disassortativity of the Internet is explained by high-degree connectivity providers and low-degree clients. Furthermore, Liu et al. [39] find assortative networks to be more Disconnection Robust with respect to Natural Connectivity λ (Section 8.7) and stability of |V L | under attacks, but less so in terms of Algebraic Connectivity λ 2 (Section 8.4). The latter metric is thus in contrast with the former two. As it only assesses how easily any set of nodes can be disconnected, irrespective of its size, more weight should be put on |V L |, which conveniently addresses this problem. Still, further analysis of these three metrics is necessary to finally decide if they indeed measure different facets of this robustness aspect. Only then, an unambiguous interpretation of the impact of r on Disconnection Robustness would be possible.

Average Neighbor Connectivity
Average Neighbor Connectivity calculates the average neighbor degree of k-degree vertices [37], summarizing the Joint Degree Distribution JDD(k, k ) = P(k, k ), which measures the probability that a randomly chosen edge points from a kto a k -degree node [40]. It thus assesses the level of network assortativity and its Disconnection Robustness as measured by Number of Nodes in the Largest Component |V L | (Section 5.12) in a more detailed but also more complicated way than Assortative Coefficient (Section 3.6). According to Costa et al. [12], it holds that k nn (k) ≈ k 2 /k for all k if there is no correlation between k nn (k) and k. If k nn (k) is increasing in k, the investigated graph is assortative; if it is decreasing, the graph is disassortative.
For weighted networks, Barrat et al. [32] propose the following formulae: The local weighted average of the nearest-neighbor degree is calculated according to the normalized weight of the connecting edges w ij /s i . Similarly, k w nn,i measures the effective affinity to connect with high-degree or low-degree neighbors, considering the magnitude of interactions. The weighted version of Average Neighbor Connectivity, k w nn (k), measures the average weighted neighbor degree of a degree-k node. When applying both the unweighted and weighted versions of this metric to a weighted graph, then k w nn,k > k nn (k) indicates that edges with larger weights point to neighbors with larger degree, while k w nn,k < k nn (k) shows that such edges are connected to lower-degree neighbors. This procedure can thus help to identify whether hubs are located at the high-capacity meaning high-strength Internet Backbone core or at the periphery (Section 3.1).

Rich-Club Connectivity (Rich-Club Coefficient)
The Rich-Club Coefficient φ(k) measures the interconnectedness of hubs by calculating the ratio of existing edges between nodes with a degree higher than k to the maximum possible number of edges [41]. Its codomain is [0, 1] with zero indicating no direct interconnections, and one for a fully connected cluster. Uncorrelated networks that are neither assortative nor disassortative usually have a non-zero value of φ. Therefore, φ(k) has to be normalized by φ ran (k) in order to assess with Rich-Club Connectivity ρ null if there is a rich-club behavior (ρ null > 1), or not (ρ null < 1).
The Internet graph investigated in [41] lacks rich-club ordering, which is explained by the Backbone hubs gaining their importance through high bandwidths and traffic capacities between each other rather than by high numbers of interconnections. Along with [15], it is suggested that hubs solely provide connectivity to local networks on the periphery without redundant connections between themselves. The weighted Rich-Club Coefficient φ w (s) thus measures the weighted interconnectedness of high-strength nodes, with the same codomain as above and again ρ w null > 1 indicating a rich club behavior as well as increased Transmission Speed and Traffic Robustness. For more details, see [42].

Recapitulation
The adjacency metrics presented in this chapter are descriptive and provide a static snapshot of basic characteristics of a graph. The Degree-Frequency Distribution, its summarizing metrics Entropy, Skewness, and Vulnerability Function and Strength Distribution indicate whether a network is vulnerable to Disconnection through attacks or random failures and to Transmission Speed reduction due to attacks. In order to attain the same goal, the easyto-handle Assortative Coefficient can be used, which also gives a first indicator whether the graph is assortative and thus more Disconnection robust against both failures and attacks. The Average Neighbor Connectivity metric uses a similar measuring approach, but provides more details about the network and has a counterpart for weighted graphs, which makes it very suitable for Internet robustness analysis. Another very interesting metric is the weighted Rich-Club Connectivity. It evaluates the Backbone structures that provide many alternative transmission ways if one of the most important nodes fails. Since this is the only metric measuring this aspect, it should be part of the methodological repertoire for robustness analysis.

Clustering
Clustering metrics aim to provide a detailed overview of the community structure of the Internet in order to gain an understanding of its dynamical evolution and organization [12].

Clustering Coefficient (Transitivity)
The Clustering Coefficient (CC) is commonly applied to measure the interconnectedness of nodes with the same neighbor. This metric has many different versions. In all of them, the coefficient ranges from zero to one, with a high value indicating high clustering. The frequently used local version C i for simple graphs compares e(V i ), defined as the number of edges among the neighbors V i of node i, to the maximum possible number [18,43]: A high C i denotes a node against whose removal the network is Disconnection and Transmission Speed robust, as many alternative routes between its neighbors exist [44].
C i is not defined for nodes having degree of one, which is problematic as the global CC is defined asĈ = C i [45]. In [44], the Internet is evaluated from 2002 to 2010 using AS-level graphs based on snapshots of BGP routing-table dumps. The authors discover that most structural metrics are time-invariant. An exception isĈ, which decreases while maintaining its distribution, possibly due to a flattening of net growth of transit providers in 2005 [46] as they are substituted by an increasing number of unobservable peer-to-peer links.
An enhanced version includes only nodes of degree higher than one [47]: Generally, many real-world networks exhibit a high global C [47]. Another approach to deal with the bias problem uses the following definition [48]: ∑ k>j>i (a ij a ik + a ji a jk + a ki a kj ) (23) As this approach normalizes by the sum of the possible edges between the neighbors of a node, the nodes of degree zero or one can be considered without further precautions. It gives the same weight to each triangle in the network. As they are usually involved in a larger number of triangles than low-degree vertices, hubs tend to have a higher weight in the calculation than with the other two global versions that weigh each node equally. A comparison thus reveals whether clustering occurs relatively more on hubs (C < C ∆ ), or on low-degree vertices (C > C ∆ ).
In [47], the average C i for nodes with degree k, C(k), is found to be a decreasing function of vertex degree because of the disassortative nature of Internet graphs (Section 3.6), where high-degree vertices mainly interconnect subgraphs consisting of smaller-degree vertices with few inter-subgraph connections. It holds that C(k) ∝ k −α . Ravasz and Barabási [49] find a hierarchical exponent α = 0.75 on the AS-level and a C(k) independent of k on the router-level.
In [47], a CC, is introduced that is not correlated to the Assortative Coefficient: where , takes into account that not all the k i − 1 excess edges are available at the neighbors of i.c i is obtained by a rewiring algorithm. This metric has the following characteristics: If all neighbors of a vertex have degree one, the CC is undefined,c i ≥ C i holds for all i, and if all neighbors of i have a degree larger than or equal to k i , it follows thatc i = C i . Furthermore, two different global versions are proposed: For disassortative networks such as the Internet, C and C ∆ give highly contrasting results, whereasC andC ∆ are similar, andc i does not depend as strongly on the node degrees but remains constant or decays logarithmically with increasing vertex degree.
Comparing these simple-graph CCs, a high C or C ∆ as such indicates a graph of greater Disconnection and Transmission Speed Robustness as more alternative routes with hardly lengthened paths exist around the failed node. An assessment whether clustering rather occurs on hubs C < C ∆ or on low-degree vertices C > C ∆ , has direct implications for Internet resilience: Doyle et al. [15] (contrasting Albert et al. [3]) state that the Internet Backbone is formed by a low-degree mesh graph, and the access networks by hub ring-, staror tree-like graphs located at the network periphery [50]. This implies that C > C ∆ should hold. An increased clustering of low-degree vertices (C) enhances global Transmission Speed and Traffic Robustness of the network as a whole; locally though, a higher clustering of hubs (C ∆ ) significantly enhances the Disconnection Robustness of single peripheral clusters and end-users. Generally, a node of high C i has a low local impact in terms of these aspects, while one of very low C i is a local bottleneck. If graphs of equal Degree-Frequency Distribution P(k) (Section 3.1) and varying C or C ∆ are investigated, though, the interpretation becomes more complex: the higher C, the more intra-connected the clusters and the less interconnected. Thus, Disconnection and Transmission Speed Robustness against the removal of inner cluster entities is increased, but decreased against faults of community-peripheral bridge entities. To determine the position of a node in a cluster, the Participation Coefficient (Section 4.5) can be used.
AsC andC ∆ calculate how many of available neighbor ties are used for clustering, thus not as community-connecting bridges, its interpretation corresponds to the one of altered C with constant distribution P(k). Properly addressing this problem with one single metric,C andC ∆ are very suitable to discover a too-pronounced community structure that bears the risk of fragmenting Disconnections. It could also cause severe Transmission Speed delays and Overload increases in the whole network in worst-case scenarios of cutting crucial bridges as presented in Section 5.
In [32], the following local version is proposed for weighted networks: where the global versionĈ w is again obtained by averagingĈ w i . Here, connections between nodes with high-weighted connections to node i get a higher weight, regardless of their own weight. When comparing C andĈ w ,Ĉ w > C indicates a network where interconnected triplets are more likely to be formed by edges with larger weights. ForĈ w < C, the opposite holds: clustering is less important for the traffic flow and thus also for network organization [51].
This metric is only able to assess the individual impact of node i on Disconnection Robustness on a mere topological level. As the weight of neighbor-interconnecting edges is not taken into account, meaning the ability to redirect possibly large traffic flows through them when i fails, it is not suitable for assessing local impacts on Transmission Speed and Traffic Robustness. A better approach in this sense is introduced in [52]: C w i decreases smoothly with the weight of the neighbor-interconnecting edge, as opposed toĈ w i , which first stays constant, but abruptly drops when the weight turns zero. In [53], a weighted equivalent to the global C ∆ is introduced: The triplet value τ can be defined as either the arithmetic mean of the weights of links that form the triplet-which is severely influenced by extreme values of weights-or as the less sensitive geometric mean, or the maximum, respectively, minimum value of edge weights. The latter two options, given two triplets with the same average weight, apply a higher/lower value to the one with larger weight dispersion, thus making C τ with the minimum definition of τ very useful for global Transmission Speed and Traffic Robustness.
Furthermore, the definition of Wassermann and Faust [54] is used to make Equation (28) applicable to directed graphs: the nodes i, j, k form a triplet if an edge points from i to j and one from j to k, and a closed triplet if, additionally, a direct link from i to k exists, which preserves the path if j fails. This definition can thus be used to measure all three Robustness Aspects on the AS level.

Edge Clustering Coefficient
In this definition of the Edge Clustering Coefficient [55], T∆ (i,j) denotes the number of triangles the edge (i, j) belongs to. According to Newman [56], the +1 in the numerator is added in order to not overestimate the importance of edges that do not belong to triangles and connect to a low-degree vertex. C (i,j) is not defined for edges that connect degree-one nodes.
More complex loops, such as squares, can be also taken into account. Then, low values identify edges that connect different communities and are bottlenecks not only on a local scale as in Equation (29), with high impact on Disconnection and Transmission Speed Robustness. As stated in [12], this holds only for networks with a high average CC, which is the case for the Internet.

Modularity
Determining the clusters or communities in large graphs is of particular significance for detecting vulnerable points and assessing the network structure. To this end, Modularity [57] measures the quality of a given global network division into k communities, where E is a k × k matrix whose elements e ij are defined as the fraction of all links in the graph that interconnect communities i and j. Consequently, Tr E gives the total fraction of inner-community edges. ∑ j e ij is the fraction of edges that connect to vertices in community i. In a graph where edges connect vertices randomly without considering their community, e ij = ∑ i e ij ∑ j e ij holds, thus Q = 0. Therefore, a low Q indicates a bad division into communities, and Q = 1 a graph where every community represents one disconnected cluster.
The following two algorithms are the most suitable out of the ones provided in [56,58] for Internet graphs since they apply to real networks without homogeneous community sizes and knowledge of the number of communities, using minimal calculation time:

Modularity Matrix
This bisection algorithm [58] maximizes the value of Modularity [12]. The Modularity Matrix B is constructed with its elements defined as where δ ij is the Kronecker Delta. The entries of the eigenvector corresponding to the largest positive eigenvalue divide the nodes into a group with positive eigenvector values, and one with negative values. Existing edges between them are not removed as this would change the degrees of the nodes and hence bias further calculations. The calculation is repeated within the groups until the Modularity Matrix eigenvalues are all zero or negative, which indicates that no further useful division is possible. The advantage of this algorithm compared to other bisection methods is that the sizes of the communities, into which the network finally splits, do not have to be homogenous or known. The disadvantage is that at any step only a division into exactly two communities is possible. As stated in [56], dividing a network into two parts and then dividing one of those again does not yield the best possible result for a division into three communities.

Edge Betweenness Partitioning Algorithm
The Edge Betweenness Centrality metric (Section 7.1) used in this algorithm is the ratio of shortest paths between two nodes that pass through the considered edge, summed up over all node pairs. The idea behind it is that communities are characterized by having scarce inter-community edges that act as bottlenecks, obtaining high Edge Betweenness Centrality values [56,59]. The algorithm repeatedly deletes the highest valued links and recalculates the Modularity and the Edge Betweenness Centrality after each removal. Finally, the state with the highest Modularity is chosen.
The implementation of this algorithm does not require homogeneity of community sizes or knowledge about their number, and as opposed to the Modularity Matrix, the graph can be split into any number of communities. Still, the whole process is very costly since after every edge removal, the Edge Betweenness Centrality of each remaining edge must be calculated again, which takes O(ev) every time. In total, this can take O(v 3 ) on a sparse graph or O(e 2 v) on a non-sparse graph in the worst case. In order to reduce the calculation time and to introduce a stochastic element, Tyler et al. [60] propose to only sum over a random subset of vertices, giving the partial Edge Betweenness Centrality scores for all edges as a Monte Carlo estimation, which provides good results with reasonably small sample sizes.

Z-Score of the Within Module-Degree
Z-Score of the Within Module-Degree [61] calculates the rank of a node within its community, which is higher for a larger z i . e i (K i ) is the number of connections that node i has within its community, e(K i ) is the average number of intra-community connections of the nodes in K i , and σ e(K i ) the corresponding standard deviation. This is a normalized version of the Node Degree (Section 3.1) if just the community as a subgraph is considered. This approach of finding inner-community hubs is sufficient by itself for detecting vertices of high impact on inner-cluster Disconnection and Transmission Speed Robustness, as by definition inside of communities no bottlenecks exist.

Participation Coefficient
L, To, R simple (weighted) local impact and liability static [0, 1] is the number of links from node i to community K j . The Participation Coefficient [61] measures how equally distributed the connections of i are among all communities. P i = 0 indicates that i is only connected to nodes in its own community, while P i = 1 shows that its edges are uniformly distributed among all communities. P i serves as an indicator of the impact of a node, as a bridge, on the Robustness against community Disconnection or decreased Transmission Speed. Furthermore, the removal of a small share of neighbors of a node with high P i can degrade the node's Transmission Speed severely, if the removed neighbors belong to the same community.

Recapitulation
Knowledge about the community structure of a graph provides insights into its hierarchical characteristics and gives first indicators of bottlenecks. The algorithms presented with Modularity are useful tools for community detection, and a resulting high value of the metric itself or of the CC of available neighbor edges,C, serves as an indicator of a too pronounced clustering structure that is prone to severe Disconnections, Transmission Speed decreases and Overloads. In the presence of such a structure, the role of a node inside its communities as determined by Z-Score of the Within Module-Degree and Participation Coefficient is crucial; especially the latter detects vulnerable points.
In contrast toc i , the local C i by itself measures the ability to locally redirect information flows if node i fails, thus its local impact on Disconnection and Transmission Speed Robustness.
The importance of clustering for network flows can be estimated by the comparison ofĈ w and C. As indicators for Transmission Speed and Traffic Robustness, in a weighted graph C w i serves best by accounting for weights of neighbor-interconnecting edges; in a weighted directed network, C τ with the minimum-value triplet definition can be used.

Connectivity
Clustering in a network contributes to its local robustness but entails the presence of fewer bridges between these clusters, and hence a reduced global Disconnection Robustness. In the subsequent sections, the fundamental and NP-complete task [62] of finding a graphseparating Minimum Cut Set and approximating bi-partitioning algorithms are discussed. After this worst-case approach, metrics to assess Connectivity under multiple random failures are presented.

Vertex-, Edge-and Conditional Connectivity (Cohesion, Adhesion and P-Connectivity)
The Vertexκ(G) and Edge-Connectivity µ(G) of a graph are defined as the minimum number of nodes and edges whose removal disconnects the graph, respectively, [36,63]. A severe flaw is that the resulting component sizes are not accounted for, making it unsuitable for robustness analysis: on sparse graphs such as an Internet graph containing stub nodes, the return will equal the minimum Node Degree. On networks with a high minimum Node Degree and severe bottlenecks, this problem is NP-complete [64].
A more useful, though also NP-complete concept is the Conditional Connectivity κ(G, P). It is the smallest number of vertices (links) whose removal disconnects the graph, with every component having property P [65]. In order to address the above-mentioned flaws, P can be set to the minimum of the resulting component sizes. Further metrics that address this problem are presented in the following sections.

Sparsity
To, R simple graph Here, A, X and A are three non-empty sets of nodes that together form the entire graph G. X is the set of vertices that, if deleted, causes a separation from A and A . Therefore, a low value for Q X indicates a network of low Disconnection Robustness, and a high value a robust one.
Calculating the minimum of this metric is NP-hard [66]. In [67], this problem is addressed by considering every bi-partition of the graph into A and A , defining c( A) as node subset in A adjacent to A , and c( A ) analogously, and taking Interestingly, the inverse of this metric is the minimum possible average Betweenness Centrality (Section 7.1) of the nodes in X, as every shortest path from A to A must pass through X.

Cheeger Constant (Isoperimetric Number)
This metric [62] is similar to the Sparsity bisection approach, but focuses on edges instead of nodes. It calculates the minimum ratio of edge cuts necessary for graph bisection to nodes in the resulting smaller cluster. Small cut sets and large impacts in terms of Reachability (Section 5.12) are rewarded. h(G) = 0 holds for a disconnected graph, and h(G) = v/2 for a fully connected one.
In [68], bounds for the NP-complete Cheeger Constant are derived using the calculable Algebraic Connectivity λ 2 (Section 8.4): . However, they are not very tight and thus of little use due to the scale-free nature of the Internet. With the subsequently presented Network Partitioning Algorithm, the Cheeger Constant can be approximated by setting the balancing criterion to m/v = 0.05, repeatedly running the algorithm for a = |A|/v = {0.05, 0.15, ..., 0, 45}, respectively, and taking the minimum resulting h(G).

Minimum m-Degree
Very similar to the Cheeger Constant (Section 5.3), the Minimum m-Degree denotes the smallest number of edges that must be removed to split the network into two components, one of which contains exactly m nodes [69]. Since for estimating Disconnection Robustness the exact component size is not crucial, this restriction can be relaxed as follows.

Network Partitioning Algorithm
This algorithm finds a graph bi-partitioning Minimum Edge Cut Set E min (m/v) of size ξ(m) = |E min (m/v)| with splitting ratios of approximately m/v. First, a balancing criterion of c nodes is set, by which each partition may deviate at most from the splitting ratio m/v. A separation then bisects the graph, denoting the resulting inter-partition links as cuts. Afterward, the following pass is executed: the gain of all nodes, initially denoted as unlocked, is calculated as the decrease in the number of cuts if the node is moved to its opposite partition. The largest (possibly negative) gain node is moved, as long as the balancing criterion is not violated, and denoted as locked. All node gains are recalculated, and the process is repeated until either all nodes are locked or the balancing criterion prevents further moves. The split with the fewest cuts is executed. Then, all nodes are unlocked again, and the procedure is repeated until the cut set cannot be further reduced. The computation takes O(e) per pass and the convergence occurs quite fast. In the worst case, O(e) passes are needed [70].
Wang et al. [71] studied Internet models both on AS and router levels and find that approximately 6.5% of the links have to be removed in order to obtain a splitting ratio of 50%. For a splitting ratio up to 30%, the disconnected parts are highly fragmented pieces in the router model, whereas in the AS model a large cluster contains 20-40% of disconnected nodes. Afterward, in both models, the vertices glue together, which enables them to communicate. The largest cluster in a scale-free network separated this way remains scale-free and functions well.
Furthermore, it is analyzed if the Minimum Edge Cut Set for randomly distributed nodes, which can be a connected cluster, can be significantly reduced by this algorithm compared to just cutting off all the links adjacent to the target nodes. The results show that this is only possible if the target nodes are connected and only at the expense of cutting off many non-target nodes with them.

Ratio of Disruption
Another metric that assesses edge cuts is the Ratio of Disruption where [.] denotes the floor function [72]: Since this metric takes the size of the other (in most cases larger) component V − A into account, it is more likely to result in more equally-sized graph cuts, but at the expense of bigger cut-set sizes |∂A| than the Cheeger Constant. This metric is thus designed to find cutsets which have considerably more severe impacts on a graph's Reachability (Section 5.12). Unfortunately, it is NP-complete and cannot be conveniently approximated by the Network Partitioning Algorithm (Section 5.4.1). Therefore, this important concept is currently not applicable to large Internet graphs.

Local Delay Resilience (Resilience)
To, R, Tr simple, directed local liability/global worst- G i (h) is the subgraph induced by nodes within the h-hop environment of node i. The local Local Delay Resilience R i (h) measures the size of its Minimum Cut Set E min of splitting ratio 0.5, assessing its proneness to local Disconnection or severe Transmission delays when entities around it fail. The global Local Delay Resilience R is a function of the hop-radius h, but as the number of vertices within it is higher in graphs with a high Expansion (Section 6.5), R is presented as a function of v(h), denoting the average number of vertices within h hops. Karypis and Kumar [73] present a partitioning heuristics for this NP-hard problem.
In [14], the investigated AS and router-level graphs are stated to have a high Local Delay Resilience for any h. Their directed adaptation, which takes policy-based routing into account, yields substantially lower values than the simple one. Now, G i (h) is induced by nodes of policy-compliant paths no longer than h hops. Thus, in AS-level graphs, only paths that do not violate provider-customer relationships are considered; in order to determine a router-level policy path, first the corresponding AS-level path is computed and then shortest paths within the ASes.
The name of this metric given above is an adapted version of the one found in the literature. It needs to be emphasized that it only measures one aspect of Resilience, and could by no means aggregate all facets of this complex concept (Section 2.3) into one single-valued number.

Toughness, Integrity, Scattering Number
To, R simple graph global worst-case N/A NP-complete The previously presented Connectivity metrics concentrate on detecting worst-case bisections, but do not conveniently assess the damage of partitioning the network into more than two components. Toughness T S [74], Integrity I [75] and Scattering Number s [76] aim to fill this gap by finding cut sets S (either the set of removed nodes V or edges E), which minimize or maximize certain parameters.
They account for important characteristics of the disconnected graph, namely cut set size S, number of nodes in the largest component |V L | and the number of resulting components ω G−S , but every metric omits one of these. An inner similarity of these metrics can be ascertained by the results they provide: knowing the Scattering Number and some basic information about the graph, a lower or upper bound for the other metrics can be derived [77].

Tenacity
To, R simple graph global worst-case N/A NP-complete As opposed to the metrics presented in the previous section, Tenacity T(G), Edge Tenacity T e (G) and Mixed Tenacity T m (G) [78] take all of the three important parameters into account for evaluating the Disconnection Robustness of a graph.
Here, V and E represent the sets of removed nodes and edges, respectively, |V L | X and |E L | X the resulting number of nodes and edges in the largest component of graph X, and ω X its number of components.
A study on the behavior of Tenacity, or an approximating algorithm, applied to Internet graphs is currently still due. As stated in [79], a large set of graph classes exists that are edge-tenacious, meaning that the minimum of the term measured by T e (G) is obtained by cutting the entire set of edges in the graph. As the same presumably holds for Internetrepresenting graphs, this metric would become useless.

Percolation Threshold
To, R simple graph global Percolation Threshold p c measures the threshold of uniform node failure probability after which the network disintegrates, meaning that no giant component of graph size order exists [80]. For SF-networks with exponent γ ≈ 2.5 like the Internet, lim v→∞ p c = 1 for infinitely large graphs. For finite Internet networks, the Largest Component Size (Section 5.12) decreases with increasing node failure probability p, but the largest component persists for a p of nearly 100%.

Reliability Polynomial
To, R simple graph global The K-Reliability Polynomial Rel(K) measures the probability that, given a uniform edge-reliability p and thus edge failure probability 1 − p, all nodes of a subset K are connected. The number of K-connecting subgraphs with j edges is denoted by a j . This is the most general definition, denoted K-Reliability. If for all p ∈ (0, 1), Page and Perry [81] rank the importance of edge i higher than the one of edge h based on purely topological characteristics. Here, G * E i corresponds to graph G, where the nodes adjacent to edge i are contracted to being one vertex. The 2-Terminal Reliability, |K| = 2, calculates the probability that a message can be transmitted between both members of K.
By calculating the All-Terminal Reliability, K = G, before and after removing single entities, their importance to the topological structure can be ranked for a given p [81]. The All-Terminal Reliability contains the size of minimum edge cutsets, which disconnect the graph (corresponding to Edge-Connectivity, Section 5.1) as well as their number. According to the authors, the Number of Spanning Trees (Section 8.6) is correlated to All-Terminal Reliability. The algorithm provided in [82] is not practical for large Internet graphs, but can be used to measure the Disconnection Robustness of its Backbone.
Interestingly, the coefficients of the polynomial indicate the Number of Spanning Trees, Size of Smallest Edge-Cutset and Number of Smallest Edge-Cutsets. The latter two metrics could be useful for analysing the Backbone, if the Size is not determined by the node with minimum degree. An algorithm for Reliability Polynomial calculation is presented in [82].

Partition Resilience Factor (Resilience Factor)
To, R simple graph global Again, an adapted metric name is used here to distinguish it from the much broader Resilience concept. The Partial i-Connectivity k(i) denotes the ratio of node failure sets of size i, which disconnect the graph. A low Partition Resilience Factor R F thus indicates a wellconnected network [83]. It attempts to capture subtle Disconnection Robustness differences, but it does not consider the severity of disconnection measured by Reachability (Section 5.12), and is computationally very costly. Only an application to formerly identified Backbone graphs seems appropriate due to much smaller network sizes and the high importance of every single vertex.

Analysis of Disconnected Components and Reachability (Flow Robustness)
To, R simple graph global dynamic different codomains In order to assess the impact of an attack or failure, it is necessary to take some characteristics of the disconnected components into account. According to Sun et al. [84], these could be the Total Number of Isolated Components N ∈ [1, v], the Fraction of Nodes in the In [85], it is stated that the latter is not suitable for an application to the Internet as the disconnected component size seems to be heavy-tailed, always resulting in a low Average Size of Isolated Components. In [86], investigating an AS-level network measured in 2000, less than 3% of the highest-degree nodes have to be removed to reach |V L |/v ≤ 0.05.
For large graphs, the Distribution of Component Class Frequency [85] assigns disconnected components of similar size to disjunct classes c ∈ [1, 2, ..., C], whose numbers indicate the decimal logarithm of the upper bound size of the class, and counts the nodes |V c | in each of them. Another related metric is the Distribution of the Relative Number of Nodes per Class [85], which is calculated as f (c, r) = |V c |/v ∈ [0, 1]. It gives the probability that a node can communicate with at most a given number of nodes for a given percentage r of removed nodes.
All these aspects are covered by Reachability (or Flow Robustness) R, being the reason why Disconnection Robustness is best evaluated in terms of the resulting decrease in R. It is defined as R = 1 v(v−1) ∑ i =j∈G R ij , with R ij = 1 if a path exists between node i and j and R ij = 0 otherwise [45]. It calculates the fraction of node pairs which can still communicate, thus R ∈ [0, 1]. Confirming the findings in (Section 3.1), a positive correlation is found between the exponent γ of an SF network and its Disconnection Robustness in terms of R under node attacks [45].
In [87], the R of the highest level of a multilevel AS graph after faults on lower levels is investigated. The multilevel graph consists of L levels of simple graphs; for any pair of them, the set of all nodes in the higher level is a subset of the node set in the lower level. A connecting path between two nodes on a higher level can only exist if such a path also exists on the lower level. A partition of the tier-1 ISPs is found to be unlikely when the logical link layer is attacked. An increase in L, a switch from adaptive to non-adaptive attacks, as well as from static routing on the original shortest-path to perfect dynamic routing resulted in a significantly higher Reachability.

Recapitulation
The fundamental task of calculating exact Minimum Cuts is NP-complete. Cohesion and Adhesion are not useful, as resulting component sizes are not considered (contrary to the other worst-case cut finders). A fundamental drawback of those is that suitable approximating algorithms, exist so far only for bisections: Sparsity and Network Partitioning Algorithm are both apt for global Disconnection Robustness analysis. Yet, their calculation requires predetermined sizes of the resulting components, contrary to the NP-complete Cheeger Constant, which could be conventiently approximated by the repeated execution of the Network Partitioning Algorithm. For evaluating general cuts, Reachability in combination with the fraction of removed entities is a convenient tool, as it implicitly accounts for number and sizes of all components.
Nodes close to local bottlenecks are found by Local Delay Resilience, which needs little calculation time relying only on subgraph G i (h).
Of the measures presented in Sections 5.7 and 5.8, only the Tenacity metrics consider all features of graph partitions of more than two components. However, as they probably yield trivial results for Internet graphs, and as all metrics in these sections are NP-complete, none is suitable for robustness assessment.
Percolation Threshold suitably assesses minimum Connectivity requirements after multiple failures, namely, at least a large part of the network is still connected and functioning. Reliability Polynomial and Partition Resilience Factor are more precise failure evaluation measures, but only useful for application to the Backbone due to their computational cost.

Distance
Distance metrics consider the path length between node pairs. Its simple or directed version d (m,n) is defined by the minimum number of hops it takes from node m to n. Its weighted counterpart, d w (m,n) = min a,b,...,z∈V {w −1 ma + w −1 ab + ... + w −1 zn } [31], further takes the speed and capacities of the connections into account.
The local Average Shortest Path Length (ASPL) is useful for finding central nodes that can play a significant role in forwarding information. The global ASPL d characterizes the average Transmission Speed with which node pairs communicate [84]. Its degradation due to Challenges hence serves as an indicator for comparing Transmission Speed Robustness of networks that display differing characteristics as measured by other metrics. An implementation of Dijkstra's algorithm is used in [88].
In SF networks, ASPL tends to remain unchanged even when up to 5% of the nodes fail [3]. However, after a certain threshold of attacks, ASPL is no longer suitable as it approaches ∞ when a node is disconnected. To address this issue, only d (m,n) = ∞ or just nodes in the giant connected component [84] could be taken into account. These versions, however, can obtain low values for disintegrated graphs, suggesting one with redundant connections.
Diameter-Inverse-K (DIK) addresses this problem with the infinite paths excluding d [89]: where |Ψ| is the number of node pairs and |Π| the number of pairs connected by a path. It increases continuously as the graph disconnects.
As Distance Distribution Width, the standard deviation of path lengths d (m,n) [37], is small, d is only slightly smaller than the Diameter D [44,90], whose codomain is [1, ∞] if infinite paths are taken into account, and [1, v − 1] otherwise.
Due to these findings, d, DIK and D overlap considerably and can be used interchangeably in connected graphs, while DIK is preferable in disconnected ones.
The formulae above refer to simple networks. For weighted networks, the following adaptation proposed in [31] is applicable to every metric containing d (m,n) : The weighted shortest path between two nodes is the path for which the sum of the inverted weights of the traversed edges is minimal, and is independent of the number of traversed edges, which can be very useful as the way that a data package takes depends highly on the speed and capacities of the connections.

Global Network Efficiency (Harmonic Centrality/Closeness Centrality)
Tr As with the metrics presented in the previous section, the degradation of Global Network Efficiency indicates the Transmission Speed Robustness of a network under Challenges. By summing up the reciprocal of the distances, the problem of disconnected graphs is solved [91,92]: for the local version, E i = 0 only holds for a totally disconnected node i, while E i = 1 indicates that node i is directly connected to every other node. Analogously, E global only becomes zero for a totally disconnected network and one for a network where every node is directly connected to all others.
As opposed to ASPL and DIK (Section 6.1), which weigh a one-hop increase equally for any path length, the Global Network Efficiency is considerably more affected by one-hop increases in short than in long paths. This is interesting if, as assumed in the work of Gkantsidis et al. [93], traffic is not homogeneously distributed between all node pairs, but is to a certain extent concentrated on topologically close nodes. In this case, Global Network Efficiency would be more suitable for assessing Transmission Speed than ASPL.
However, for connected SF-graphs that do not exhibit highly improbable structures, there is a high correlation between the reciprocal ASPL [94] and this metric according to Rochat [91]. Since it most conveniently assesses the Speed, with which a node or entire graph can communicate, and also works in disconnected graphs, Transmission Speed Robustness against Challenges is best evaluated, of all metrics presented so far, in terms of Global Network Efficiency decrease.
In [87], multilevel Internet networks as described in Section 5.12 are analyzed by simulating non-adaptive attacks against highest-E i nodes. After a certain number of deletions, the resulting Reachability (Section 5.12) and thus Disconnection Robustness is higher than for the case of random failures. The reason is suggested to be the non-adaptive strategy, but it could also be due to mere unsuitability for detecting nodes with a high impact on this aspect as well-interconnected Backbone vertices tend to exhibit the largest E i . The Internet graph investigated by Latora and Marchiori [92] showed a relatively high E global .
Another version is the Harmonic Mean of Geodesic Distances [12]: Its codomain is [1, ∞], a higher h indicating a less efficient network. Both E global and h allow for graph comparison, but the former is more useful due to its codomain.

Local Network Efficiency (Cyclic Coefficent)
Tr simple (all) local impact and liability/global The Local Network Efficiency [92] calculates the average of inverse shortest-path lengths between the neighbors V i of a node i after it failed. The global version of this metric is found to show relatively high values for the assessed Internet graph. E(V i ) = 0 indicates a node in a tree branch, and E(V i ) = 1 a node that is connected to all others. It follows that bottleneck vertices, which are the only connectivity providers for their branches to the giant cluster (thus with crucial impact on Disconnection Robustness), always exhibit E(V i ) ≈ 0. At the same time, this indicates that only a few of i's edges can lead to the giant cluster, making itself prone to Disconnection when these links fail. For slightly larger but still low E i values, the detected node acts as a bridge between communities, thus having a high impact on the Transmission Speed between them.
In [95], a very similar metric called Cyclic Coefficient is introduced, which in its local version calculates the average of the smallest loop sizes of node i with two of its neighbors: In the analysis of the Internet AS-level topology in 1999 by Kim and Kim [95], treestructure nodesÊ(V i ) = 0 dominate, while most of the other nodes' neighbors have a small average distance to each other.
A disadvantage of this metric is the codomain, [0, 1/3], and a less intuitive interpretation. Thus, to measure this aspect, the Local Network Efficiency is more appropriate.

Characteristic Path Length
Tr labeled (all) local impact static Characteristic Path Length (CPL) [96] is an adaption of the global ASPL (Section 6.1) for labeled graphs. Equation (64) is used for calculations between nodes of the same label Q i , Equation (65) for different labels Q i and Q j . This allows detailed investigations, e.g., inspecting the Transmission Speed Robustness between customer AS to failures of higher hierarchy nodes.
Trpevski et al. [96] show that large ISP exhibit the lowest CPL of 1.5 between themselves, confirming that the Backbone is almost a full-mesh. Its value between IXPs and large or small ISPs, respectively, is also small, while the one between customer ASes and between universities is substantially larger (L = 4). Removal of all small ISPs results in a remaining of all large ISPs in the giant component, while 20 to 30% of nodes of other labels are disconnected. It is shown though, that attacks based on highest-Betweenness (Section 7.1) and even more on highest-Degree (Section 3.1), irrespective of their label, are substantially more damaging in terms of Fraction of Nodes in the Largest Component (Section 5.12) than the same strategies, when only nodes of a certain type are removed. These metrics are thus more suitable for assessing a node's impact on Disconnection Robustness than its label.

Expansion
Expansion calculates the fraction of vertices reachable within h hops. In a tree-like and in a router-level Internet graph, local and global Expansion follows a power-law E(h) (i) ∝ h p (i) [14,97]. Palmer et al. [97] find a strong negative correlation between p i and Effective Eccentricity (Section 6.6), suggesting that highest-p i nodes are likely to form the Backbone. Furthermore, in a router-level graph of 1999, the vulnerability to Disconnection and Transmission Speed decreases in terms of Reachability (Section 5.12); p degradation is reported to be highest for largest-Degree attacks, followed by highest-p i and then random failures. This supports the findings of Section 6.1, that Transmission Speed providing Backbone vertices might not have the largest impact on network Transmission Speed Robustness since, when removed, traffic is easily redirected through their neighbors.
The directed version inspects only customer-provider compliant AS paths. A minimum cut set for halving the local node data Transmission Speed in terms of E(h) i is calculated by Local Delay Resilience (Section 5.6).

Effective Eccentricity and Effective Diameter
Effective Eccentricity and Diameter measure exactly the same as the previously presented Expansion, but from another perspective: they count the minimum hop number h required to reach a certain percentage r of the nodes reachable from node i, and on average in the graph, respectively [97]. N(i, h) is the number of nodes within the h-hop environment of node i. Effective Diameter is a global metric that calculates the minimum number of hops required to enable communication between a fraction r of node pairs that are connected through a path.
As described in the previous section, Ecc eff (i, r) finds Backbone nodes with some impact on Disconnection and Transmission Speed Robustness. Palmer et al. [97] analyze an Internet router-level graph from 1999 with an approximating algorithm, obtaining D e f f (0.9) = 10, and a skewed distribution of Ecc e f f (0.9) ∈ [5,19], with most routers having an Ecc e f f (0.9) = 8.

Recapitulation
Though less intuitive to interpret than ASPL, due to its applicability to disconnected graphs, Global Network Efficiency is most appropriate for measuring and comparing the average Transmission Speed of graphs and for finding distance-central Backbone vertices. Those are the main providers of Transmission Speed, but they do not have the largest impact on this aspect and on Disconnection when removed, as they are embedded in a mesh-like neighborhood that can resume the traffic-forwarding tasks.
In contrast, very low Local Network Efficiency values indicate bottlenecks with high local impact on and liability to Disconnection Robustness, and low values suggest a local impact on Transmission Speed. Distance metrics that address the difficulty of providing unbiased results even in partially-disconnected networks can be divided into two groups: One group uses adaptations of the ASPL, the other the Global Network Efficiency. In the first group, the Diameter Inverse K metric seems most appropriate to take disconnected areas into account. The advantage of the metrics in the second group is their codomain [0, 1], which allows the expression of dynamic changes after failures as percentage rates. A disadvantage is the less intuitive interpretation of the results as they do not correspond to a certain number of hops. Therefore, if a quick and comparable abstract evaluation of distances in a network is conducted, metrics of the second group are preferable. For a concrete transmission cost or speed analysis, the ASPL-based group provides more suitable measurement tools. The local version of the Local Network Efficiency can detect local bottleneck nodes. Their Expansion or, equivalently, Effective Eccentricity further indicates whether they are peripheral or constitute critical bottlenecks closer to or in the Backbone. Characteristic Path Length is useful for assessing the path structure and Transmission Speed Robustness of different node types, such as ISPs or customer ASes.

Throughput
Evaluating path redundancy and distance increases is not enough for assessing robustness: traffic loads induced by flows along shortest paths and their capacity constraints must also be considered.

Betweenness Centrality and AS Hegemony
Betweenness Centrality B u sums up over all node pairs (i, j), the fractions of shortest paths between i and j that pass through u, σ(i, u, j), relative to the total number of shortest paths σ(i, j) between i and j [98]. B u quantifies the number of flows which have to be redirected if u fails, thus how much control u has over network traffic [99]. The first codomain holds for u as a vertex, the second for u as a link. Brandes and Fleischer [88] present a corresponding algorithm. For the average Edge Betweenness Centrality B e (G) = 1 |E| ∑ u∈E B u , it holds that B e (G) = v(v−1) 2e d, with global ASPL d [100]. Furthermore, in [31], a weighted Betweenness Centrality version is presented where σ counts the number of shortest weighted paths as defined in Section 6.
Ghedini and Ribeiro [29] find that highest Betweenness entities have a huge impact on network Transmission Speed and Disconnection Robustness in terms of Global Network Efficiency E global (Section 6.2) and Fraction of Nodes in the Largest Component |V L |/v (Section 5.12). These metrics decrease faster for highest-Betweenness than highest-Degree attacks in WS and KE models, while the difference in E global is moderate and the one in |V L |/v crucial. The average Local Network Efficiency E local (Section 6.3) decreases faster for highest-Degree removals, thus leading to a more bottleneck-dependent graph. In a WS network, largest-Betweenness attacks on up to 40% of the nodes even increase E local of the graph, as the removed nodes often constitute community-interconnecting bridges with low Local Network Efficiency.
In contrast, the investigation of an AS-level representing network in [101] finds that both attack strategies are similarly harmful for the Internet with respect to E global and |V L |/v. This is explained by their high correlation, which according to Holme et al. [101] especially holds for high-Degree nodes. Furthermore, a weaker correlation between the Edge Degree, which can be defined as k (i,j) =k i k j or k (i,j) = min(k i k j ), and the Edge Betweenness Centrality is observed, which suggests that the existing edges between hubs play a crucial role as Transmission Speed providers. Supporting the argumentation in Section 4.1, for a network with fixed number of edges a higher CC C i (Section 4.1) is found to increase the vulnerability to bottleneck-removing Betweenness attacks.
These findings about correlations contrast those of Doyle et al. [15], who state that ISP Backbones are never represented by hubs as predicted in common SF-graph approaches. According to their study, high-Degree nodes are mainly found in local networks at the periphery of the Internet, and thus have low Betweenness values. However, they investigate the Internet on the router level but not the AS level, which could be a reason for the substantial differences in the findings.
As Betweenness Centrality can be used as an efficient attack strategy, an increase in its variance and maximal value substantially decrease network Robustness in all of the three Aspects.
Concerning further developments, Mahadevan et al. [37] investigate the AS-level topology and state that Edge Betweenness is not a link-centrality measure, as the edges adjacent to peripheral one-degree nodes obtain medium-ranked, thus disproportionally high values. Fontunge et al. [102] attribute these findings to the non-random sampling method used to collect BGP data rather than the metric itself. Due to the selection of only a few viewpoints, the Betweenness values of ASes close to them are disproportionally high and the ones of those further away too low. To obtain an unbiased measure in this respect, they propose AS Hegemony, BC (j) (i) is the Betweenness value computed with paths from only one viewpoint j, arranged in ascending order and [.] is the floor function. σ(i, u, j) denotes the number of paths from i to j that pass through u, and S is the total number of paths. It uses only unbiased viewpoints for calculation, disregarding the 2α viewpoints with the highest/lowest number of paths passing through u, respectively. With this approach, they obtain centrality values for the ASes from only a few viewpoints, which are consistent with the ones calculated with all viewpoints. They find that the Hegemony distribution does not to change over time and the one in the US seems most balanced.

Central Point Dominance
Tr simple (all) global static [0, 1] The Central Point Dominance (CPD) [98] calculates the average Betweenness Centrality difference of the most central node to all the other vertices in the network: CPD quantifies the extent to which a network is decentralized (CPD → 0) or concentrated, indicating low Disconnection and Transmission Speed Robustness against their removal. Along with the correlation of Betweenness Centrality to Node Degree, it provides an indicator for determining if high-Betweenness attacks should be investigated as an alternative to high-Degree attacks.

Effective Load
Tr simple (all) local impact and liability dynamic The Effective Load λ(u) Ω calculates the expected number of shortest paths passing through u when a communicating node set Λ, containing a fraction of A nodes, is chosen randomly [103]. Contrary to Betweenness Centrality, it aims at modeling real flows and measuring their loads, accounting for the fact that not all node pairs communicate at the same time.
In [103], an edge is overloaded and removed, if its capacity λ max (u) is exceeded, which leads to the possibility of congestion after an arbitrary Challenge. After each removal, the Effective Load is recalculated, allowing for breakdown avalanches during one time step.
By defining Ω to be the ensemble of a small number of randomly chosen Λ, a stochastic element can be introduced. For repeated calculations with different Ω, a load distribution for a given edge can be estimated, and hence the probability of a congestion-induced failure.
For an appropriate modeling of the router-capacities vector C = [C u ] considering their economic costs, Motter and Lai [104] propose to measure node Betweenness Centrality B u (Section 7.1) in the unrestricted network and set C u = (1 + α)B u . Consequently, it depends on the choice of parameter α if avalanches of failures are limited or take down the entire network. In [105], the critical threshold for a BA graph is α c = 0.15 and in an Internet graph the number of node failures during an avalanche process s follow a power law: p α c =0.15 ∼ s −τ with τ ≈ 1.8.

Performance
Tr Performance [15] measures the maximum aggregate network throughput of 'gravity flows' [106]. Such a two-way-traffic flow x ij is exchanged between all end vertex pairs (i, j) with bandwidth demands y i and y j . The global constant ρ is to be maximized, given the restrictions on the right-hand side of Equation (74), and thus describes the worst capacities to demand share of a pair: The shortest-path routing matrix R = [R r f ] entries are set to R r f = 1 if flow f passes through router r, and R r f = 0 otherwise, and the vector x of all flows x ij is indexed to match R. Their product must be consistent with the router bandwidth capacities vector b.
A high Performance-constant ρ value indicates a network that is robust against flow congestions. In particular, a high ρ > 1, which maximizes the term in Equation (74), indicates that in every node there are unused capacities which can at least carry ρ − 1 percent more flows. ρ < 1 indicates that an overloaded node exists, which is requested to carry 1 − ρ percent more flows than it is capable of. In contrast to Effective Load (Section 7.3), breakdowns caused by overloads are not modeled.
The router-level model used in [15] is the HOT model, and the graph used aims to represent a small segment of the Internet, while taking all hierarchical layers into account. In comparison to SF router-level models of the same Degree-Frequency Distribution (Section 3.1), HOT performs significantly better with respect to Traffic Robustness in terms of Performance, with a difference of more than two orders of magnitude. This is explained by the hubs in SF-networks becoming saturated and thus severe bottlenecks. In the HOT topology, a modestly loaded mesh-like Backbone provides enough alternative high-capacity paths if core nodes fail.

Elasticity
Tr simple (all) global worst-case [0, 1] Elasticity E(n) is a dynamic adaptation of Performance (Section 7.4) with link-instead of router-bandwidth restrictions [107,108]. It measures the gradual degradation of graph throughput during the successive removal of nodes up to a fraction n, which does not disconnect the graph.
This normalized graph throughput T G (n) is calculated by the normalized maximum sum of homogeneous flows x ij (n) between all node pairs (i, j). They are restricted by the requirement that the product of shortest-path (e × v 2 ) routing matrix R(n) = [R l f ] and flow vector x(n) = [x(n) ij ] is not to exceed link bandwidth capacities vector b, which is set to be the unit vector; thus an unweighted graph is observed. The constant ρ varies flow proportions in the network and the normalization parameter α is set so that T G (0) = 1. A benefit of Elasticity is that it takes the Traffic Robustness during the entire node-removal process into account by calculating the integral of T G (n). Its flaws are that it does not model congestion failures, and that unrealistically all flows are reduced equally if one capacity restriction is reached, thus assessing a worst-case scenario.
In [107], Elasticity, results under attacks on high-Degree nodes in a HOT and an SF topology are compared to those of Performance of Doyle et al. [15]. They are found to be consistent, validating the simplifying assumptions of homogeneous flows and capacities. Again, HOT is more resilient than the SF model to Overloads in terms of Elasticity for a node-removal fraction of 80%. Furthermore, other Internet topologies with mesh-like Backbone and negative Assortative Coefficient r (Section 3.6) are shown to feature a high E. Still, the relationship between E and r is not found to be clear-cut.
The upper bound for E(1) is 1/3 in a full-mesh network [108]. Furthermore, on average, highest-Betweenness vertices are reported to have the most impact on Traffic Robustness in terms of removal induced Elasticity degradation, followed by highest-Degree and then random failures. Elasticity is found to be negatively correlated to Heterogeneity [109], defined as the fraction of standard deviation of node degrees to the average node degree, for highest-degree and highest-betweenness attacks, and also to the global ASPL (Section 6.1) for all attack types.

Vulnerability Impact Factors
Tr labeled weighted (directed) local impact and liability/global dynamic [0, ∞]/[0, 1] Hariri et al. [110] present an agent-based framework for monitoring attacks designed to consume all system resources. Examples include TCP SYN, Smurf, ICMP flood, Ping of Death for server attacks and Distributed denial-of-service (DDoS) for router attacks.
The following Vulnerability Impact Factors are calculated by all agents a ∈ {1...A} in real time for different fault scenarios FS k and agent-specific measures M: The local Component Impact Factor CIF(a, FS k ) calculates the share of a fault-induced change |M norm − M f ault | in M to the change that makes the agent operate in an abnormal state |M norm − M min |. Its global counterpart, System Impact Factor SIF a (FS k ), indicates the share of agents in this abnormal state, thus those whose CIF j surpassed the normal operation threshold d, with Θ denoting the Heaviside step-function. Example measures M include data transfer rates for clients, buffer utilization, number of flows open or in process, request-processing rates for routers and connection queue lengths for servers as agents.
Hariri et al. [110] simulate single and multiple-router failures on a topology of clients, servers and routers, with determined bandwidth, using file transfer (TCP/IP) and open shortest path first routing protocol. The former induce drastic increases of CIF j of other routers into abnormal states, and breakdown avalanches with failure rates of up to SIF Clients = 25%. In a similar way, agents with highest local impact on and liability to Traffic Robustness can be determined.

Survivability Function, Failures
Molisz [111] introduces a measure for Traffic Robustness of Backbone networks of limited size for random occurrences of node or link failure (ROF), taking into account traffic demands, capacity constraints modeled by link weights, and effects of routing protocols: The Survivability Function S(ζ) is defined as the probability function of the Share of Delivered Data Flow X(ζ) after a (set of) v D node or e D link failure(s) ζ occurred.
Investigating a broad-band network of 12 nodes and 34 edges, the author finds an Expected Survivability E[X] = ∑ x xS(x) of 89.60% for a total load l = 90% and uniform link failure probability e −1 , given a routing based on minimum number of hops. In this network, an increase of l from 60% to 90% does not have a large impact on S(x) for node failures. For l = 200%, though, congestion causes a dramatic degradation to E(x) = 62%.

Recapitulation
Betweenness Centrality measures the traffic load of an entity when all node pairs communicate. Its highest values indicate bottlenecks that render the network vulnerable in all three Robustness Aspects. The ongoing discussion about the amount and nature of its correlation with Node Degree provides crucial insights into the Internet graph structure. Furthermore, Betweenness Centrality can be used to define capacity restrictions for nodes and edges. In AS-level graphs, AS Hegemony is independent of the viewpoints adapted, and thus a more convenient metric.
Effective Load can assess the probability of nodes and edges to become overloaded, and also the whole network behavior during possible breakdown avalanches. The Performance constant measures how much flow increase in percent could be carried or by how much it would have to decrease to match capacity restrictions, in the relatively most loaded node of the graph. It does not model congestion-induced failures, just like Elasticity, which provides a worst-case flow-reduction scenario in the absence of congestion: It assumes that all flows are reduced equally if the capacity of one single edge is reached, and measures its gradual degradation.
The Vulnerability Impact Factors are technical metrics, which allow the detailed investigation of the exact cause of congestion-induced avalanche breakdowns after a fault scenario. The Survivability Function for failures conveniently assesses the Traffic Robustness on the highest level, the Application Layer against random failures on the Physical or Routing Layer, given that the occurrence probabilities are known. For capturing the important aspect of data flow, Betweenness Centrality is the most prevalent metric and has been studied extensively.

Spectral Methods
The following metrics cover aspects of all previous chapters. Their calculation relies on random walks, which provide a useful tool to estimate entity importance as their role in alternative, non-shortest backup-paths is accounted for. The assumption of shortest-path routing seems in general more suitable for approximating entity loads and distances in the majority of the classical Internet Backbone [112]. However, the potential of mobile adhoc networks (MANETs) in particular at the Internet edges has increased, and with it the possibility of a more random character of routing [113]. For this network type, the presented loads and distance metrics can be particularly convenient.
Since all sections in this chapter rely on matrix algebra, its basic concepts and notation are introduced subsequently, based on [93,114].
The adjacency matrix A, its derivatives and corresponding v eigenpairs (λ i , u i ) are key for all calculations. The eigenvalues are sorted by size: λ 1 ≤ λ 2 ≤ ... ≤ λ v . In undirected graphs, the largest eigenpairs of the symmetric adjacency matrix are most important, and the Spectral Gap is defined as ∆λ = λ v − λ v−1 . For any k ∈ N, the entries of A k stand for the number of paths between the corresponding nodes that have length k, where revisiting the same nodes and edges in the same path is allowed. This feature is useful to estimate path redundancy, both locally and globally. Golnari et al. [115] use the random walk matrix R(A) for simple and directed connected graphs; in the latter case, A is not symmetric and D is the diagonal matrix of out-degrees. The expected number of visits at a transit node m of a random walk with source node s and a target node t is counted by the fundamental t-matrix F t = [F t sm ] = (I − R(A \tt )) −1 . A \tt denotes the adjacency matrix without line and column t and I is the (v − 1) × (v − 1) identity matrix. From this, a fundamental tensor F = [F smt ] is defined, whose entries equal F t sm if s, m = t, and 0 otherwise. The metrics presented in the last three sections of this chapter aim at measuring distance and load aspects, thus implicitly assuming that random walks can be used to describe routing behavior. The majority of routing on the router-level, though, relies on shortest paths [112]. On the AS-level, restricting routing policies can be explicitly modeled by directed graphs, and the different costs of forwarding information through different neighbors can be conveniently accounted for by link weights. As cost minimization is the main parameter with which routing decisions are determined, finding a shortest path on this weighted graph is most suitable to model routing behavior.

Eigenvector Centrality (Significance)
To, R simple (all) local impact and liability static [0, 1] The Eigenvector Centrality [116] of a node i is given by the ith entry of the eigenvector of the adjacency matrix A corresponding to the largest eigenvalue. It can be calculated using the following algorithm provided by [117]. Initially, all nodes have equal Centrality values. At each step t, the value of each node is set to the sum of the values of its neighbors V i . Then all values are normalized so that their sum is one and the process is repeated until it converges.
The vector entries reflect how many and how important the neighbors of a given node are, implicitly accounting for the entire topology. Unsurprisingly, Eigenvector Centrality highly positively correlates with Degree (Section 3.1) [93] and shortest path stochastic traffic loads [93], hence possibly also with Betweenness Centrality (Section 7.1), and negatively with Effective Eccentricity (Section 6.6) [117], thus conceivably marking Backbone entities.

Symmetry Ratio
To, R simple graph global In this equation for the Symmetry Ratio SR [118], # λ is the number of distinct eigenvalues of the adjacency matrix, and D = max m,n∈V d (m,n) the Diameter (Section 6.1). For lowsymmetry networks, SR ∈ [8,15] holds and for high-symmetry networks, SR ∈ [1,3]. In the latter case, all nodes are similarly important, and the network is more Traffic Robust against attacks as measured by the Share of Delivered Data Flow X(ζ) (Section 7.7), which then have a similar impact as failures.
An analysis of different graphs [119] finds that networks with smaller average Node Degree (Section 3.1) tend to be more symmetric, but also more vulnerable to attacks. It has to be noted, though, that in [118] SR is related to robustness after the average Node Degree has been controlled for.

Spectral Cluster Identification
L, To, R simple, directed both static - The main goal of Spectral Cluster Identification is the detection of cluster sets S of nodes that have a large ratio of links between themselves to the total sum of their edges, or of clusters of edges that connect two topologically or geographically remote regions. In [93], these clusters are found by sorting the nodes according to their weights in the eigenvectors of the largest eigenvalues of A, and bisecting the node set with special preference to sharp weight jumps. According to [120], the largest eigenvalues of this non-normalized adjacency matrix follow a power law as a result of the power-law degree distribution, with an exponent between 0.45 and 0.5 in inter-domain and 0.18 in router graphs [17]. This method is thus considered to mainly show trivial clusters concentrating on hubs such as ISPs.
Non-trivial groupings of ASes with clear semantic proximity, such as geography and business interests, are obtained for a directed AS graph where edges point from customers to providers by constructing N(A) or N(SI M(A)). Generally, high eigenvalues of the normalized matrices indicate the existence of clusters in the underlying topology, which are found not to be located in the core. Both in the core and the entire topology, North America exhibits less clustering than Europe and Asia. Furthermore, from 1997 to 2001 Internet graph spectra did not change substantially.
In order to discover edge clusters of high importance, a similar strategy can be applied: In an undirected graph, implement shortest paths between all node pairs v 2 and construct the v 2 × |E| traffic matrix T, where each row corresponds to a node pair that has a 1 in the column if the corresponding link is used by the path. Applying the SVD method to gain eigenvalues and eigenvectors in non-square matrices, the link importance and cluster membership can be derived. In [93], the resulting crucial link clusters are very intuitive, one example being cross-Atlantic and east-coast to west-coast connections in the US.
Spectral methods are also used in [93] to model data traffic more realistically: Instead of assuming uniform traffic between every node pair, a certain percentage is switched to run between nodes in the same cluster. The resulting change in the maximum link load is crucial. For the extreme case of 100% 'intra-cluster' traffic, the maximum load measured by the Edge Betweenness (Section 7.1) is reduced by more than 40%, which can only partially be explained by the reduction in the global ASPL of approximately 15%. If, in fact, traffic distribution on the Internet turns out to exhibit this behavior, Throughput metrics should be adapted accordingly as the importance of formerly identified bottleneck entities could turn out to be substantially lower.

Algebraic Connectivity (Laplacian Spectral Gap)
To, R simple graph global static [0, v v−1 min{k i |i ∈ V}] The larger the Algebraic Connectivity, which in [121] is defined as the second-smallest eigenvalue of the Laplacian matrix λ 2 (and thus the Laplacian Spectral Gap), the more difficult it is to cut a graph into independent components [122]. Its codomain [123] implies that, for a complete graph, λ 2 = v holds and for a disconnected one λ 2 = 0. The multiplicity of eigenvalues that are zero corresponds to the number of connected components. The metrics Cohesion κ(G) and Adhesion µ(G) (Section 5.1) follow Fiedler's inequality λ 2 ≤ κ(G) ≤ µ(G).
In [124], it is shown that the minimum edge cut set ∂A between two disjoint node sets A and V − A is bounded by the Laplacian eigenvalues: [68] derives bounds for the NP-complete Cheeger Constant (Section 5.3): λ 2 2 ≤ h(G) ≤ λ 2 (2k max − λ 2 ) . As discussed in [122], a disadvantage of Algebraic Connectivity is that it becomes zero for disconnected networks. Furthermore, in contrast to Natural Connectivity (Section 8.7), it does not always distinguish between networks with intuitively different robustness values, for instance, given a graph and the same graph with a removed edge. Furthermore, it cannot distinguish between cuts that separate single nodes, and those which separate large components.

Good Expansion
To, R simple, weighted global A graph is a Good Expansion (GE) network of factor α, if and only if any subset of nodes S with less then half of the nodes of the graph in it has a neighborhood |N S | that is larger than |S| multiplied by α. N S is defined as the set of vertices with one endpoint in S and the other in S = V\S. The global exponent α is known as the Expansion Parameter, its computation is NP-hard. GE networks are desirable because the minimum cut set of vertices necessary to disconnect a node set exceeds a certain fixed lower bound, which means that there are no crucial bottlenecks [125].
A large Spectral Gap ∆λ = λ v − λ v−1 of A is a necessary, but not sufficient condition for GE [125], as in some cases it fails to distinguish between networks with obvious bottleneck structures and others that are relatively well-connected.
The author also presents a method to determine whether a network is GE-in that case, holds. u v (i) denotes the i-th component of the v-th eigenvector of the adjacency matrix A.
The local odd Subgraph Centrality SC odd (i) = ∑ ∞ k=0 µ 2k+1 (i) (2k+1)! stands for the weighted sum of closed walks µ l (i) = (A l ) ii of different but odd lengths l = 2k + 1 starting and ending at node i, where re-visiting the same edges and nodes is allowed. Thus, for a GE-network, a linear regression of log[u v (i)] on log[SC odd (i)] must approximately exhibit the form of Equation (84). Regarding the resulting plot for all nodes, tightly connected clusters will be close to a linear trend while nodes that are connected to the main node cluster through a bottleneck lie substantially below the curve, whose number reveals the importance of the bottleneck. Nodes that are more remote from the main cluster exhibit a larger negative bias from the curve. Although this method cannot estimate the exact parameter α, it is a very helpful tool for both global and local Disconnection Robustness analysis. A weighted graph can also be analyzed, which is particularly interesting for detecting capacity bottlenecks.
In [125], two Internet graphs at the AS-level from 1997 and 1998 are found to be GE.

Number of Spanning Trees
To, R simple graph global static L is the Laplacian Matrix L with one row and column deleted; its dimension is thus [126]. A Spanning Tree is defined as a subgraph without loops where all nodes are connected. Thus, an increased Disconnection Robustness to random failures would be expected if the Number of Spanning Trees N ST rises because it indicates enhanced Path Redundancy. However, Zhang et al. [127] show that scale-free networks exhibit significantly lower N SF than regular lattices with the same number of nodes and edges while being more robust against failures. This metric is also not suitable to measure robustness against attacks, as it does not account for bottlenecks. Further investigations are therefore necessary before an unambiguous conclusion about the effect of an increased Number of Spanning Trees on network robustness can be drawn.

Natural Connectivity (Natural Eigenvalue)
To, R simple graph Here, the global Subgraph Centrality SC(G) [128] stands for the weighted sum of numbers of closed walks µ l of length l in the graph, where higher weights are assigned to shorter walks. In contrast to the Algebraic Connectivity (Section 8.4), λ does not become zero for disconnected graphs. Furthermore, it yields more intuitive results when applied to a BA-network: The Natural Connectivity λ [122] increases strictly monotonically with the addition of edges, capturing subtle differences in path redundancy. This contributes to a higher Disconnection Robustness. However, the same values can be obtained for graphs with severe bottlenecks which are else well-connected, and a medium-densely connected network without this flaw. According to Wu et al. [129], for SF networks with sufficiently large Spectral Gaps, λ SF ≈ λ 1 − ln v holds.

Random Walk ASPL
The random walk (rw) equivalents for the distance d (s,t) between a source s and target node t and the local Random Walk ASPL d rw t are provided in [115] as follows: They count the expected number of hops between a node pair or towards a target node on a random walk. Note that d rw (s,t) is not symmetric: A stub node has a random-walk distance d (s,t) = 1 to its neighbor with a larger degree, while when starting from that neighbor, wandering off into the network is possible, thus increasing the expected distance d (t,s) > 1.
In the presence of random routing, the degradation of this metric under Challenges indicates the Transmission Speed Robustness of the network.
In [87], multilevel Internet networks as described in Section 5.12 are analyzed under attack strategies targeted at nodes of highest Current-Flow Closeness or local Average Shortest Path Length (Section 6.1). They find that after a certain number of deletions, Disconnection Robustness in terms of Reachability (Section 5.12) is higher than for random failures. This is ascribed to the fact that only non-adaptive attacks are performed, but could also be due to a unsuitability of these metrics for detecting nodes with a large impact on Disconnection Robustness.

Random Walk Betweenness
Tr The Random Walk Betweenness B rw i counts the times a flow passes through a node on any random walk in a connected network [115,130], where at each time step the transition probability through the outgoing links is proportional to their weights. It is proportional to the Network Criticality τ (Section 8.12), and, in undirected networks with a stationary probability π i = s i 2e , to Strength s i (Section 3.2). Thus, in undirected networks it does not provide additional information about the network. In a directed network, however, it shares the implications of its shortest-path equivalent Betweenness Centrality (Section 7.1) on robustness for MANETs.

Current-Flow Betweenness
The Current Flow-Betweenness B c i is introduced by [99]: The running time to calculate this metric is O(v 3 ) on sparse graphs [99], but a faster approximating algorithm is presented in [88]. The Current Flow-Betweenness has two interpretations: In an electrical network, it calculates the amount of current that flows through node i, averaged over all sources s and sinks t. Regarding random walks in general networks from any s to any t, their averaged net flow through an edge is calculated as the difference between the number of times a flow passes in one direction, to the one they pass the other way round. Only connected graphs are regarded.
This variant of the metric thus differs from Random Walk Betweenness Centrality B rw i in that moving back and forth through a vertex does not increase its B c i . Still, its random walk nature has the advantage of assigning a non-zero value to nodes that lie on a path between an st-pair, which is not the shortest one. Thus, the importance of alternative paths is accounted for.
Current Flow-Betweenness is shown to highly correlate with its shortest-path equivalent (Section 7.1) , and also with Node Degree (Section 3.1) [99]. Due to these correlations, it is unsurprising that Çetinkaya et al. [87] find that adaptive and non-adaptive attacks targeted at high B c i nodes have a greater impact on the Disconnection Robustness in terms of Reachability (Section 5.12) than random failures.
Generally, like the previously presented metric, it shares the implications of its shortestpath equivalent Betweenness Centrality (Section 7.1) on robustness for MANETs.

Network Criticality (Effective Resistance)
Tr L + is the Moore-Penrose inverse of the Laplacian L of a connected networks, and u st = [0...1... − 1...0], with 1 and −1 in the s th and t th position, and K(G) the Kirchhoff Index [115]. The random walk begins at source node s and ends at destination node t. At each step, at node i it chooses to travel to the adjacent node j with probability p = w ij s i . If the weights are interpreted as capacities, the Network Criticality τ globally quantifies the relative entity load, and thus the risk of using them: Node and Edge Criticality, n(k) and n ij , are defined as the share of the entity's Random Walk Betweenness Centrality B rw to the Strength or Weight, respectively, and are constant for all nodes and edges and proportional to τ.
In [130], its suitability to capture Disconnection Robustness against topological changes as node failures and Traffic Robustness against variation in link capacities, changes in active sources and sinks, and traffic demand shifts is expounded: As its bounds are determined by Algebraic Connectivity a(G) and vice versa, a lower τ leads to a lower upper bound for a(G), which makes it a (weak) indicator for Disconnection Robustness. Furthermore, the authors show that the maximum of average input rate λ and thus critical load, which a network with link capacities w ij can transmit, is achieved by minimizing τ by altering the capacity matrix W = [w ij ], max W λ = 2v(v − 1)/ min W τ. Thus, for a given topology, the effect of the capacity distribution on Traffic Robustness can be assessed by τ. Additionally, it is proportional to average path cost ϕ = 1/2τ ∑ ij w ij z ij to traverse a network with link weights w ij and traverse costs z ij on random walks. A possible interpretation of the traversal costs is the message delay; thus, the degree of increase of this metric under Challenges in special networks governed by random-walk routing indicates their Transmission Speed Robustness.
With the Traffic-Aware Network Criticality (TANC), another version of this metric is provided in [130], which also relies on random walks and shares its above-mentioned implications about all Robustness Aspects: where γ and is the total traffic and γ st the traffic between source s and destination t.
The results of Tizghadam and Leon-Garcia [130] are not applicable to directed graphs.

Recapitulation
Spectral metrics aim at globally assessing a graph's topology with random walks. Although relatively resource-consuming, they promise to be highly accurate and to provide solutions for the difficult detection of communities and minimum-cut sets.
The Eigenvector Centrality is more suitable for capturing the importance of a single node than the local Node Degree (Section 3.1) as it can discriminate between hubs that are mostly connected to one-degree-nodes and hubs that are neighbors to other hubs close to the network Backbone. The Symmetry Ratio assesses how much the Traffic Robustness against failures and attacks diverges. The methods Spectral Cluster Identification and Good Expansion detect community structures, similar to the algorithms in Modularity (Section 4.3), and estimate the severity of community-interconnecting cut sets, thus having similar goals as the Minimum Cut algorithms. However, they are more practical as no parameter prefixing is necessary. For simple graphs, the GE procedure should be used as it provides a very detailed insight into the network topology; for directed networks, the Spectral Cluster Identification is more suitable. Natural Connectivity and Number of Spanning Trees can capture subtle differences in path redundancy, but not necessarily graph Disconnection Robustness if applied by themselves.
Random Walk ASPL and Current-Flow Closeness assume random-walk routing and thus mainly serve for analyzing mobile adhoc networks. In contrast, the Current-Flow Betweenness can also be used in networks relying on shortest-path routing to assess the importance of entities as their role in crucial backup-paths is accounted for. As for Network Criticality, its traffic-aware version TANC is most convenient for assessing Transmission Speed and Traffic Robustness in mobile networks.

Geographical Metrics
This section discusses metrics that take geographical aspects into account.

Distance Strength and Outreach
Distance Strength D i and Outreach O i [31] are variations of Node Degree and Strength (Sections 3.1 and 3.2) that account for the geographical distance g ij between node i and its neighbors V i . High values indicate crucial bridges such as submarine cables or links connecting large ISPs.

Survivability Function, Geographical
In [50,131], a measure for Disconnection Robustness against geographically extended failures is presented with the Survivability Function: It measures the probability that the Fraction of Nodes in the Largest Component |V L |/v (Section 5.12) is x, given the probabilities P(e) of geographically extended failure events e that result in this outcome. Due to the limited size of possible sets e, this metric can also be calculated in large graphs. From this, the Expected Survivability and Worst-Case Survivability s 0 = min P[|V L |/v=x]>0 s can be derived.
Pointwise Vulnerability is a dynamic metric that calculates the relative drop in the Euclidean Network Efficiency E eucl caused by removing node i [16,132]. It assigns higher weights to connections between node pairs of high geographical distance g ij , which often constitute scarce bottleneck edges such as submarine cables. U i = 1 indicates a very vulnerable highest-position vertex, U i = 0 a low-position vertex of little importance.
The global metrics Global Vulnerability U g = max i U i and the Relative Variance of comparable [132]. Its codomain is [0, ∞], a higher h indicating a more heterogeneous network in terms of the nodes' importance for the overall Euclidean Network Efficiency, thus a higher vulnerability to attacks that select targets based on this feature.

Effective Geographical Path Diversity
Effective Geographical Path Diversity (EGPD) [133] estimates the liability of the communication of node s with d to geographically extended failures, e.g., caused by natural disasters.
D min (P i ) = min(D g (P i , P 0 )) (101) P is a path vector that contains all links and intermediate nodes from source s to destination d, P a and P b paths with the same s and d, and D min the minimum geographical distance between any member node of P a and any node of P b . The area of the polygon, whose borders are formed by the paths P a and P b , is denoted by A, and ω ∈ [0, 1] is a weighting factor, which is set to 0.5. The shortest path between nodes s and d is denoted by P 0 , and λ is an experimentally determined constant, which scales the impact of k sd . For a node pair that is not geographically path-diversified, EGPD converges to zero and for a highly diversified node pair, to one.
The Total Geographical Graph Diversity TGGD is calculated as the average of EGPD over all node pairs. cTGGD is the corresponding normalized global metric, called the Compensated TGGD: The normalization by ρ ≈ 0.05 eliminates the penalty to a dense network for a given size of the physical region. The codomain is again [0, 1], with a high value indicating a welldiversified network. In [133], first the sets of geographically close nodes are identified that are not well path-diversified. Each of them is then successively taken down in a simulated natural catastrophe, and the flow robustness of the remaining network is calculated with an alternative routing algorithm.

Recapitulation
Outreach and Pointwise Vulnerability use different approaches to detect crucial bottleneck nodes that connect geographically remote regions, for instance landing stations of submarine cables. The latter metric is more sophisticated and more likely to yield better results, especially when a robustness assessment instead of a mere topological analysis is conducted, but also has a substantially higher computational cost. Effective Geographical Path Diversity measures the vulnerability to natural catastrophes in an elaborative way. For robustness analyses of networks in areas prone to such disasters, this metric is very useful. If the extent and probability of possible future incidents can be estimated, the geographical Survivability Function yields the most reliable and exact results for a Disconnection Robustness assessment of geographically extended failures.

Comparison and Discussion
Considering the diversity of goals and methods of the presented metrics, it becomes clear that an isolated application of a single metric or metrics of the same family would not be sufficient for properly studying Internet Robustness. In the following, we outline a conceptual tool set that accounts for Backbone identification and all previously defined Robustness Aspects. Besides providing methodological insights, such a tool set can also be used to improve practical software for graph-based analysis of connectivity risks, such as the CORIA framework [134].

Backbone Identification and Evaluation
In order to identify the Backbone, the metrics Node Degree, Strength, weighted In-Degree, Eigenvector Centrality, Outreach, Betweenness Centrality and AS Hegemony (Sections 3.1, 3.2, 8.1, 9.1 and 7.1) and their respective value distributions can be applied. An alternative metric to Node Degree, which yet highly correlates with it, is Eigenvector Centrality. It does not only take the direct neighbors of a node into account, but also the number of all nodes that can be reached with a random walk and hence can be considered as a better measure for node centrality. Depending on the selected network-generating model, the Node Degree may highly correlate [101] or not correlate at all [15] with Betweenness Centrality and the similar AS Hegemony, which is better suited for AS-level graphs. If the Backbone is thought of as a well-interconnected set of high-capacity nodes, the findings of Colizza et al. [41], indicating that hubs are usually not very well-interconnected (but high-Strength nodes are), reinforce the claims of Doyle et al. [15] and cast doubt on the idea that Node Degree is a suitable tool for detecting Backbone nodes. Contrary to this, Strength, weighted In-Degree and Outreach are appropriate for Backbone detection, regardless whether the weights are defined as loads or capacities. Low ASPL and high Effective Eccentricity or Effective Diameter (Sections 6.1 and 6.6) can also serve as indicators for Backbone nodes. They should not be applied in isolation since a one-degree node adjacent to a central node would be wrongly perceived as a Backbone node.
After conducting community detection (Section 4), local Backbones can be found using the Z-Score of the Within Module-Degree and Participation Coefficient (Sections 4.4 and 4.5). Note that both metrics are subgraph equivalents of the Node Degree, but they do not share its drawbacks as they are applied to communities where, per definition, no internal bottlenecks should exist.
To evaluate the Backbone robustness, weighted Rich-Club Connectivity (Section 3.8) can be applied to measure how easily data can be redirected through other high-capacity nodes. Average distances between Backbone node types such as large ISPs or IXPs are measured by CPL (Section 6.4). The Reliability Polynomial and Partition Resilience Factor (Sections 5.10 and 5.11) are computationally too costly for large Internet graphs, but in the smaller Backbone, they could conveniently assess the damage of random failures. The same holds for the Survivability Function (Section 7.7), which is an important indicator for Traffic Robustness on the highest level, the Application Layer, against random failures on the Physical or Routing Layer, given that the occurrence probabilities are known.
Both, the entire network and the Backbone can be analyzed with respect to the different aspects of robustness as expounded in the following three sections.

Disconnection Robustness
The scale-free nature of the Internet, which entails a heterogeneous Degree Distribution (Section 3.1), as measured by Entropy, Skewness or Vulnerability Function (Sections 3.3-3.5), makes it vulnerable to Disconnection by high-Degree attacks, but less so by failures. Its disassortativity (Section 3.6), though, contributes to a larger vulnerability to both in terms of a fast decline in giant component size and reduced path redundancy.
Path redundancy is a crucial prerequisite for Disconnection Robustness. Spectral methods for quantifying this aspect by a single number are Number of Spanning Trees (Section 8.7) and Natural Connectivity (Section 8.7), which is computationally more costly but for which an approximation method exists.
However, path redundancy can also be displayed in networks that feature serious bottlenecks. A high CCC orC ∆ (Section 4.1) can indicate that a network is prone to such weak points because of its pronounced clustering structure. In this case, most available edges of a vertex are "used up" for connections inside of its cluster, so that no links remain to interconnect communities. As opposed to this metric, whose high values indicate a structural or global vulnerability to disconnection, large values of their counterparts C or C ∆ indicate that node removals locally have no large impact due to the high degree of interconnections between neighbors. C ∆ is more critical for Disconnection Robustness of peripheral structures than C as it focuses more on the clustering of hubs, which, according to [15], are more involved in ring-, star-or tree structures at the network periphery. They are thus more prone to partitions than the mesh-like Backbone, which is dominated by lowdegree nodes.
Due to its small-world property, the clustering in Internet graphs is relatively high. The first step is thus to find local topological bottlenecks and to determine whether they are central and important or peripheral. The calculation of the local CC (Section 4.1) is fast since it only relies on a local information set. In simple graphs, C i is most appropriate, and for a weighted networkC w i , as the quality of the alternative path is taken into account with the weight of the neighbor-interconnecting edge. For a (weighted) directed network, C τ with the triplet definition by Wassermann and Faust [54] is most suitable. Better results can be obtained with the Local Network Efficiency (Section 6.3) since in addition to direct connections, it measures shortest paths of all lengths between the neighbors of the concerned node and weighs them accordingly. For links, the Edge Clustering Coefficient (Section 4.2) can be used for different loop sizes. If information about the geographical position of the nodes is available, Outreach and Pointwise Vulnerability (Sections 9.1 and 9.3) can be applied to detect important bridge nodes such as landing stations of submarine cables.
In order to estimate the importance of the detected bottlenecks, this can be combined with the local version of the Expansion (Section 6.5) that measures the fraction of nodes that can be reached within a certain number of hops. This approach, however, entails the risk of detecting only locally important or peripheral k-degree nodes that connect k-1 stub nodes to the rest of the network and that are close to well-interconnected clusters. Another possible combination is with Betweenness Centrality (Section 7.1), on router-level or AS Hegemony on AS-level graphs. Its random-walk equivalent Current-Flow Betweenness (Section 8.11) might be even more suitable, as it assigns nodes in crucial, but non-shortest backup paths a non-zero value. Since it relies on a global information set, the substitution of Edge Betweenness by the Edge Clustering Coefficient as applied by Radicci et al. [55] could be suitable, if their strong negative correlation also holds for disassortative networks. The nodes of highest Strength (Section 3.2) coincide with the most central ones, and the detected bottlenecks can be expected to be non-trivial. The simple CC, C i , (Section 4.1) is a further important detector for local bottlenecks.
Despite their drawbacks, combinations of these metrics are most convenient for local bottleneck detection as they are applicable to all graph types and can be normalized to the codomain [0, 1]. The combination of Local Network Efficiency and Betweenness Centrality, respective AS Hegemony seems most promising. The next step could be finding topologically-close minimum link sets, the removal of which locally disconnects subgraphs, with the local Local Delay Resilience (Section 5.6). By using it on small to medium-size overlapping subgraphs in the whole graph or around locally detected bottlenecks, a cut set that globally disconnects the graph could emerge in less calculation time than with global disconnection algorithms.
Other substantially more time-consuming approaches are global network partitioning or cluster-identifying algorithms. Sparsity and the Network Partitioning Algorithm (Sections 5.2 and 5.4.1) do not rely on spectral methods and use bisections that require a predetermination of the resulting network sizes and are only applicable to simple graphs. Therefore, spectral methods, such as Modularity Matrix, Spectral Cluster Identification and Good Expansion procedure (Sections 4.3, 8.3 and 8.5) are more suitable. The advantage of the Modularity Matrix is that it maximizes the value of Modularity, a well-defined measuring tool for the quality of a graph division. A major disadvantage is that it is only applicable to simple graphs, as opposed to the Spectral Cluster Identification, which is also applicable to directed graphs, and the Good Expansion approach, which can be used in weighted networks. The latter is particularly appealing since it is the only metric that does not necessarily create a bisection and indicates which nodes are in the giant cluster and which are affected by bottlenecks.
Further analysis of communities by the Participation Coefficient (Section 4.5) finds the most important nodes that interconnect communities. This strongly depends on the quality of the division into communities, but offers an easy-to-calculate approach for bottleneck detection. For the analysis of a disconnected network, the metrics in Section 5.12 can be applied, especially Reachability summarizes the network state accordingly. Effective Geographical Path Diversity (Section 9.4) detects geographical bottlenecks of non-diversified areas that are particularly vulnerable to natural catastrophes.

Transmission Speed Robustness
A very convenient definition of the network-transmission speed for a simple graph is based on the inverse of the number of hops that a flow has to take on its path from one node to another. In a weighted network, the sum of the inverse weights of the edges that constitute this path is used (Section 6). These definitions match exactly the Global Network Efficiency (Section 6.2), which, in contrast to the global ASPL (Section 6.1), can also be applied in disconnected networks. It seems likely to correlate with Effective Eccentricity (Section 6.6) and Expansion (Section 6.5), which calculates the fraction of nodes a vertex can reach in a certain amount of time, and its Expansion Exponent p i . Although more damaging than failures, attacks on nodes of high Expansion Exponent, counter-intuitively, may not yield the highest decreases of global Expansion Exponent. This indicates that vertices of high Transmission Speed are not the most vulnerable points in terms of Transmission Speed Robustness. The reason for this can be their position in the Backbone, as suggested by Palmer et al. [97], and its mesh-like structure [15] that can redirect traffic easily. This argument is supported by the very short CPL for large ISPs (Section 6.4).
In MANETs, the degradation of Random Walk ASPL, Current-Flow Closeness and, most suitably, the Average Path Cost derived with the Traffic-Aware Network Criticality (Sections 8.8, 8.9 and 8.12) under Challenges measure their Transmission Speed Robustness. They are only applicable in connected networks.
Vulnerable points with respect to Transmission Speed Robustness can be discovered with Node Degree (Section 3.1) and Betweenness Centrality on router-or AS Hegemony on AS-level graphs (Section 7.1). A more uniform distribution of Degree [29], as measured by Entropy, Skewness or Vulnerability Function (Sections 3.3-3.5), leads to an increase in Transmission Robustness against the respective attacks, but also to a higher vulnerability to failures. The removal of bottlenecks does not only involve a threat of Disconnection, but also of severe Transmission Speed decrease. Inside communities, nodes with a high Z-Score of the Within Module-Degree (Section 4.4), and between them, those of large Participation Coefficient (Section 4.5) are relevant providers of short-cuts. The most convenient metrics for estimating the hop increase of local traffic redirections are Local Network Efficiency (Section 6.3) in case of a node removal, and the generalized Edge Clustering Coefficient (Section 4.2) in case of a link removal.
If the focus is on Transmission Speed Robustness between geographically remote nodes, which are often vulnerable in this sense, Pointwise Vulnerability (Section 9.3) can be applied. Its relative variance indicates whether particularly weak spots do exist. Outreach (Section 9.1) identifies crucial bridge nodes for Transmission Speed by taking into account their weights (thus bandwidths or capacities) and the geographical distance of their neighbors.
The local version of Local Delay Resilience (Section 5.6) calculates the minimum edge cut set in the vicinity of a node for halving the number vertices reachable within a certain amount of time, and thus its Transmission Speed liability to other network failures.
Special attention must be given to the routing policies and the general distribution of flows [15,93]. Bandwidth demands of single nodes can vary highly, and thus also the number of data transmissions between node pairs. Geographically close vertices in the same cluster may communicate more intensively with one another than with others. This needs to be considered by modeling non-uniform communication flows between nodes since it influences the measured Global Network Efficiency. Furthermore, the routing behavior is not only determined by shortest paths, but by routing policy constraints, which can be accounted for by applying a directed graph.

Traffic Robustness
In this section, finite capacity and its implication of a maximum data flow are considered. As routing policies and flow distribution crucially influence Traffic Robustness [15], they must be modeled properly.
The worst ratio of capacity to traffic-load demand is calculated by the Performance constant (Section 7.4). Its advantage is that it allows for modeling different bandwidth demands. However, it does not reflect the fact that in practice vertices usually do not communicate equally with all all other vertices, but often with geographically close nodes. Similar benefits and drawbacks hold for Elasticity (Section 7.5), which dynamically measures the uniform reduction of data flow in the entire graph when only one node becomes overloaded. Here, it is unrealistically assumed that every node has access to global information about the load state of the other nodes. In fact, it only serves as a worst-case scenario assessment in the absence of congestions, but captures the gradual performance degradation in one number.
Possible cascading overloads can be assessed by explicitly modeling capacity constraints and calculating loads with Betweenness Centrality, respectively, AS Hegemony, or its counterpart metric Effective Load (Sections 7.1 and 7.3). While the assumption of a uniform flow distribution with Betweenness Centrality and AS Hegemony seems unrealistic, Effective Load at least accounts for the fact that not every pair of nodes communicates at the same time. If only a small subset of possible node-pair communications are repeatedly averaged, a load distribution can be calculated, which together with the edge capacity determines its risk of failure. After attacking the nodes with highest loads, the risk increase of congestion failure in the entities passed by the redirected flows can be calculated. The more technical Vulnerability Impact Factors (Section 7.6) pursue the same goal and allow the detailed investigation of the exact cause of congestion-induced avalanche breakdowns after a fault scenario.
For networks of fixed average Node Degree (Section 3.1), a low value of the Symmetry Ratio (Section 8.2) indicates a network whose Share of Delivered Data Flow is more robust against attacks.
In the presence of random routing, the Traffic-Aware Network Criticality (Section 8.12) is a suitable global parameter which assesses the load to capacity relation, and thus its proneness to overloads. Furthermore, it serves to find an optimal capacity allocation for a given topology and traffic demand and compare it to the one in the investigated network.
The weighted Good Expansion procedure (Section 8.5) finds clusters that share ties of higher weights between themselves than with the rest of the graph. Based on this, the identification of the largest-strength bridge tie is easy. Such bridges represent heavily loaded bottlenecks. Their deletion and successive redistribution of flows to other, lowercapacity links could make those prone to overload.

Ambiguity and Correlations of Metrics
The interpretation of metrics can be ambiguous and their implications for network robustness may be inconclusive without further investigation. Three main challenges present themselves most prominently in this context.
First, the individual impact of a characteristic as measured by a metric cannot be estimated without controlling for other metrics that correlate with it. One example for this statement is the Clustering Coefficient C (Section 4.1). If estimated alone, a graph with a higher value would be considered more densely connected and featuring fewer bottlenecks than a graph with a lower value. If both networks have the same number of links, however, it becomes apparent that the graph with a higher C exhibits a pronounced community structure, leading to local Transmission Speed and Disconnection Robustness inside of the communities, yet a more vulnerable global graph structure between communities.
These ambiguous conclusions are related to the positive correlation of the CC with the number of links in a graph; if more links exist in total, some will interconnect neighbors and increase the CC. Therefore, it is of high importance to further study metric interactions and in particular correlations on the various graph types used for modeling the Internet.
With the help of this information, the individual impact of the assessed graph features on robustness can be evaluated correctly.
Second, many of the presented metrics overlap as they measure similar features that indicate or influence one or several Robustness Aspects. The investigation of these groups of overlapping metrics shows that some may lead to conclusions about robustness that seem to be incompatible with each other. An example is Entropy and Skewness (Section 3.4): Both measure the same feature, namely the heterogeneity of the Degree Distribution, which however is found to have different effects on Disconnection Robustness against random failures, measured by Percolation Threshold p c (Section 5.9) and Number of Nodes in the Largest Component |V L | (Section 5.12), respectively. In the former case, robustness increases for more homogeneous networks, and in the latter case, it decreases. It must be noted though, that p c measures the threshold failure probability at which the worst-case scenario, a disintegration without giant component, occurs, while |V L | measures the size of just this giant component at lower failure rates. They thus assess different facets of the same aspect. Another example is provided by Liu et al. [39] who report that the Assortative Coefficient r is positively correlated with Natural Connectivity and the stability of the Largest Component Size under attacks, and indicates a more Disconnection Robust graph. At the same time, though, it is negatively correlated to Algebraic Connectivity, and thus to another measure of Disconnection Robustness.
Third, there could be possible conflicts between robustness goals in certain graphs. For example, as recently shown by Liu et al. [39], the optimization of a BA network with respect to Natural Connectivity and limiting the decrease of the Largest Component Size caused by removals, can lead to a significant increase in ASPL. Hence, given certain constraints such as a fixed edge number, path redundancy and Disconnection Robustness are in conflict with Transmission Speed.
The last two problems cannot be solved generally. The question if a graph is robust or not must always be answered depending on the adapted point of view-is the global graph robustness of interest or the one of a certain set of nodes-and the services that are to be delivered and their special demands. These demands, expressed by certain metric values which are to be maintained at an acceptable level, may stand in direct conflict to each other. In this case, a careful evaluation of their relative importance is to be conducted.
These and many similar problems need to be addressed in future research on the way to a profound and general understanding of Internet robustness.

Transfer to Other Research Fields
The current article uses research on Internet robustness as a guiding use case for applying graph metrics to study network robustness. However, since the application of graph theory and network science [11] (Chapter 1) is growing in many other research fields such as natural sciences [135], social networks (including networks of human actors like friends, customers, influencers, but also criminals) [136], organizational networks (such as logistics [137,138], public transportation networks [139], information and process flows) and further information technology and service networks (including application-layer communication, cloud connectivity [8], peer-to-peer systems, and blockchain transactions [140]), the present survey could also be consulted by researchers in those and further domains, whenever the robustness of networks needs to be investigated. A more detailed transfer and interpretation of metrics for further networks is left for-potentially very interesting and fruitful-future research in those fields.

Conclusions
This survey discusses an extensive set of robustness metrics in six major categories, their advantages, drawbacks and the main results obtained so far on Internet graphs. Furthermore, a comparison and assessment of their suitability for detecting crucial backbone structures and for measuring important robustness aspects is presented. Thus, we provided an outline for the conceptual tool set that will support future research in this field. Before selecting metrics for an individual robustness assessment, it should be very carefully analyzed which graph type is investigated and which network-generating model is applied. For each research goal, the accuracy-cost trade-off should be evaluated and the limitations of the metrics selected should be taken into account. To enable even more accurate and efficient robustness assessments, an investigation of metric interactions and correlations, extending for example the research of [12,31] on the various graph types representing the Internet would constitute a fundamental future milestone in this research area. However, also many further disciplines that adopt methods of network science could use this article as a starting point for studying network robustness and interpreting it for the particular phenomena of their given fields.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A. Summarizing Table
The following table summarizes all discussed metrics and their main characteristics. The notation of graph types is as follows: simple: s, directed: d, (edge)-weighted: w, nodeweighted:w, labeled: l, and geographical: g. Letters without brackets stand for required graph types, letters within brackets for further possible types. In order to highlight metrics that can be applied to every graph type, -allis used. Graph types, to which the metric has been applied in the literature, are listed in black. Other types, for which it could be calculated after simple modifications, in gray.
In the column Scope, the following abbreviations apply: local liability: l, local impact: i and global: g. The entry l and i/g indicates that a local metric version, which simultaneously assesses aspects about the liability and impact of an entity, and a global version exist andboththat both local and global aspects are treated by one measure. In Static, the features are displayed as follows: static: s and dynamic: d; in the latter case, if only applicable to worst-case scenarios: w; or failures: f. Next, • indicates that the codomain is independent of network parameters as v and bounded; else, they are listed and unbounded codomains are marked by ∞. The entry N/A indicates that the codomain could not be found in the literature, and '-' that this column does not apply. Metrics that directly measure one or several of the Robustness Aspects, Disconnection: d, Transmission: t, Traffic: o, or evaluate the Backbone: b, show the respective letters. If they are just an indirect indicator for these aspects, the letters are put into brackets. The column Comp. is checked if the metric is efficiently computable, and NP-h. or NP-c. if the computation is known to be NP-hard or NP-complete. If an exact or approximating algorithm is provided in the literature, its calculation order is displayed instead.

Section and Metric
Graphs Scope Static Codo. Robustness Comp.