Grand Canonical Ensembles of Sparse Networks and Bayesian Inference

Abstract

Maximum entropy network ensembles have been very successful in modelling sparse network topologies and in solving challenging inference problems. However the sparse maximum entropy network models proposed so far have fixed number of nodes and are typically not exchangeable. Here we consider hierarchical models for exchangeable networks in the sparse limit, i.e., with the total number of links scaling linearly with the total number of nodes. The approach is grand canonical, i.e., the number of nodes of the network is not fixed a priori: it is finite but can be arbitrarily large. In this way the grand canonical network ensembles circumvent the difficulties in treating infinite sparse exchangeable networks which according to the Aldous-Hoover theorem must vanish. The approach can treat networks with given degree distribution or networks with given distribution of latent variables. When only a subgraph induced by a subset of nodes is known, this model allows a Bayesian estimation of the network size and the degree sequence (or the sequence of latent variables) of the entire network which can be used for network reconstruction.

1. Introduction

Networks [1,2] have the ability to capture the topology of complex systems ranging from the brain to financial networks. Network models are key to have reliable unbiased null models of the network and to explain emergent phenomena of network evolution. Network model can be classified in two major classes: equilibrium maximum entropy models [3,4,5,6,7,8,9,10,11,12,13,14,15] and growing network models [1,16,17,18]. While growing network models have a number of nodes that increases in time, maximum entropy models are used so far only for treating networks of a given number of nodes N. In this paper we are interested in extending the realm of maximum entropy network models to networks of varying network size N.

Maximum entropy network ensembles are the least biased ensembles satisfying a given set of constraints. As such maximum entropy ensembles are widely used as null models and for network reconstruction starting from features associated to the nodes of the network. Given the profound relation between information theory and statistical mechanics [19,20], maximum entropy network ensembles can be distinguished between microcanonical ensembles and canonical ensembles [3,21,22] similarly to the analogous distinction traditionally introduced in statistical mechanics for ensembles of particles. Microcanonical network ensembles are ensembles of networks of N nodes satisfying some hard constraints (such as the total number of links, or the given degree sequence). Canonical network ensembles instead are ensembles of networks of N nodes satisfying some soft constraints, (such as the expected total number of links or the expected degree sequence). The canonical ensembles with expected degree sequence can be also formulated as latent variable models where the latent variables can be associated to the nodes [5,23].

Maximum entropy models have been very successful in solving challenging inference models [6,8,24,25,26], however they have the limitation that they only treat networks with a given fixed number of nodes N. Indeed in several scenarios, the number of nodes might not be fixed or might not be known. In this context an important problem is to compare networks of different network sizes. For instance in brain imaging one might choose a finer grid or a coarser grid of brain regions and an outstanding problem in machine learning is how to build neural networks that can generalize well when tested on network data with different network size than the network data in the training set [27,28].

In order to have network ensembles that can treat networks of different size, here we introduce the grand canonical network ensembles in which the number of nodes can vary. A well-defined grand-canonical network ensemble necessarily needs to be exchangeable [29], i.e., needs to be invariant under permutation of labels of the nodes of the network, so that removing or adding a node has an effect that is independent of the particular choice of the node added or removed.

The research on exchangeable networks is currently very vibrant. The graphon model [30] is the most well established exchangeable network model. However this model is dense, i.e., the number of links scales quadratically with the number of nodes while the vast majority of the network data is sparse with a total number of links scaling linearly with the network size. In other words most of the real world networks have constant average degree. However popular models for sparse networks such as the configuration model [31] and the exponential random graphs [4] are not exchangeable. In fact these models treat networks of labelled nodes with given degree or with given expected degree sequence. Therefore the network ensemble is not invariant under permutation of the node labels, except if all the degrees of all the expected degrees of the network are the same (for a more diffused discussion of why these networks are not exchangeable see discussion in ref. [32]). Several works have been proposed exchangeable network models in the when the average degree of the network diverges sublinearly with the network size [33,34,35,36,37,38]. Only recently, in ref. [32], a framework able to model sparse exchangeable networks in the limit of constant degree, has been proposed. The model is very general and has been extended to treat generalized network structures including multiplex networks [39] and simplicial complexes [40]. However the model is well defined only for finite networks of large but finite number of nodes N as exchangeable sparse networks need to obey the Aldous-Hoover theorem [41,42] according to which infinite sparse exchangeable networks must vanish. An alternative strategy for formulating exchangeable ensembles is to consider ensembles of unlabelled networks for which several results are already available [43].

Here we build on the recently proposed exchangeable sparse network ensembles [32] to formulate hierarchical grand-canonical ensembles of sparse networks. The proposed grand-canonical ensembles are hierarchical models [25,44] with variable number of nodes N and with given degree distribution or alternatively given latent variable distributions. The grand canonical approach provides a way to circumvent the limitations imposed by the Aldous-Hoover theorem because in this framework one considers a mixture of network ensembles with finite but unspecified and arbitrary large network sizes. In this paper we define the grand-canonical ensembles and we characterize them with statistical mechanics methods, evaluating their entropy, the marginal probability of a link and proposing generative algorithms to sample networks from these ensembles. [Note that the proposed grand canonical ensembles differ from the ensembles proposed in refs. [45,46], as in our case we consider networks with undetermined number of nodes, while in refs. [45,46] is the total sum of weights of weighted networks that is allowed to vary. From the statistical mechanics perspective our approach is fully classical while in refs. [45,46] networks ensembles are treated as quantum mechanical ensembles where the particles are associated to the links of the network and the adjacency matrix elements play the role of occupation numbers.].

Finally, we use the gran-canonical network ensembles to solve an inference problem. We consider a scenario in which the entire network has an unknown number of nodes, and we have only access to a subgraph induced by a subset of its nodes. In this hypothesis we use the grand-canonical network models to perform a Bayesian estimation of the true parameters of the network model (given by the network size and the degree sequence or the sequence of latent variables). This a posteriori estimate of the parameters can then be used to reconstruct the unknown part of the network.

2. The Grand Canonical Network Ensemble with Given Degree Distribution

We consider the hierarchical grand canonical ensemble of exchangeable sparse simple networks where we associate to every network $G=(V,E)$ with $N=\left|V\right|>N_0$ nodes the probability

$$\mathbb{P}\left(G\right)=P\left(N\right)P\left(\mathbf{k}\right|N\left)P\right(G|\mathbf{k},N)$$

(1)

where $P\left(N\right)$ indicates the probability that the network G has N nodes, $P\left(\mathbf{k}\right|N)$ indicates the conditional probability that the network has degree sequence $\mathbf{k}$ given that the network has N nodes, and $P\left(G\right|\mathbf{k},N)$ indicates the probability of the network G with adjacency matrix $\mathbf{a}$ given that the network has N nodes and degree sequence $\mathbf{k}$ (see Figure 1 for a schematic representation of the model).

Figure 1. Schematic representation of the hierarchical grand canonical ensemble of exchangeable sparse simple networks. The proposed ensemble is a hierarchical model of networks in which first the total number of nodes N is drawn from a $P\left(N\right)=\pi \left(N\right)$ distribution, then a given degree sequence $\mathbf{k}=\{k_1,k_2,\dots k_N\}$ is drawn from the distribution $P\left(\mathbf{k}\right|N)$ among all the degree sequence with the total number of nodes N; finally a network G with adjacency matrix $\mathbf{a}$ drawn from the distribution $P\left(G\right|\mathbf{k},N)$ among all the networks with a given total number of nodes N and degree sequence $\mathbf{k}$. Panel (a) describes the hierarchical nature of the model, panel (b) provide an example of subsequent draw of the total number of nodes, the degree sequence and the adjacency matrix of the network, panel (c) is a visualization of the construction of a network according to the proposed model.

To be specific we consider the following model giving rise to the hierarchical grand canonical ensemble of exchangeable simple models:

(1)

Drawing the total number of nodes N of the network. Let us discuss suitable choices for the distribution of the number of nodes N with N greater or equal than some minimum number of nodes $N_0$. We indicate the distribution $P\left(N\right)$ as

$$\begin{array}{c}\hfill P\left(N\right)=\pi \left(N\right),\mathrm{for}N\ge N_0.\end{array}$$

(2)

While a statistical mechanics approach would suggest to take a distribution $\pi \left(N\right)$ with a well defined mean value (such as the exponential distribution)

$$\begin{array}{c}\hfill \pi \left(N\right)=C{e}^{-\mu N}\mathrm{for}N\ge N_0,\end{array}$$

(3)

where C is a normalization constant and $\mu >0$, in the context of network science it might actually be relevant to consider also broad distributions $\pi \left(N\right)$ such as power-law distributions

$$\begin{array}{c}\hfill \pi \left(N\right)=D{N}^{-\nu }\mathrm{for} N\ge N_0,\end{array}$$

(4)

where D is a normalization constant and $\nu >1$.

(2)

Drawing the degree sequence of the network. In order to obtain a sparse exchangeable network ensemble with given degree distribution $p\left(k\right)$ having finite average degree $⟨k⟩$, minimum allowed degree $\widehat{m}$ and maximum allowed degree K we consider the following expression for the probability of a given degree sequence given the total number of nodes

$$\begin{array}{ccc}\hfill P\left(\mathbf{k}\right|N)& =& \prod _{i=1}^N\left[p\left(k_i\right)\widehat{\theta }(K-k_i)\theta (k_i-\widehat{m})\right]\delta \left(\sum _{i=1}^N{k}_i,⟨k⟩N\right),\hfill \end{array}$$

(5)

where $\widehat{\theta }\left(x\right)$ indicates the Heaviside function $\widehat{\theta }\left(x\right)=1$ if $x\ge 0$ and $\widehat{\theta }\left(x\right)=0$ otherwise and where we used the notation $⟨k⟩={\sum }_{k}kp\left(k\right)$. In the following we will indicate with L the total number of links of the network given by $L=⟨k⟩N/2$. Note that $P\left(\mathbf{k}\right|N)$ is independent of the labels of the nodes, i.e., all the degree sequences that can be obtained by a permutation of the node labels of a given degree sequence have the same probability $P\left(\mathbf{k}\right|N)$.

(3)

Drawing the adjacency matrix of the network. The probability of a network G with adjacency matrix $\mathbf{a}$ given the total number of nodes N of the network and the degree sequence $\mathbf{k}$ is chosen in the least biased way by drawing the network from a uniform distribution, i.e., the conditional probability $P\left(G\right|\mathbf{k},N)$ is equivalent to the probability of a network in the microcanonical ensemble. Therefore, by indicating with $\mathcal{N}\left(\mathbf{k}\right|N)$ the total number of networks with N nodes and degree sequence $\mathbf{k}$ and with ${\sum }_N\left(\mathbf{k}\right)=ln\mathcal{N}\left(\mathbf{k}\right|N)$ the entropy of the ensemble we can express $P\left(G\right|\mathbf{k},N)$ as

$$\begin{array}{c}\hfill P\left(G\right|\mathbf{k},N)=\frac{1}{\mathcal{N}\left(\mathbf{k}\right|N)}=e^{-{\sum }_N\left(\mathbf{k}\right)}\end{array}$$

(6)

Note that for sparse networks of $N\ge N_0$ nodes the entropy ${\sum }_N\left(\mathbf{k}\right)$ obeys the Bender-Canfield formula as long as the network has a structural cutoff $K_S$, i.e., as long as $k_i\ll K_S=\sqrt{⟨k⟩N_0}$ [3,21,22,47]

$$\begin{array}{c}\hfill {\sum }_N\left(\mathbf{k}\right)=ln\left(\frac{\left(2L\right)!!}{{\prod }_{i=1}^N{k}_i!}\right)+o\left(N\right)\end{array}$$

(7)

where in Equation (7) we indicate with $\mathbf{k}=\{k_1,k_2,\dots ,k_N\}$ the degree sequence with $k_i$, the degree of node i, given by $k_i={\sum }_{j=1}^N{a}_{ij}$.

It follows that the hierarchical grand canonical ensemble for exchangeable sparse networks can be cast into an Hamiltonian ensemble with probability $\mathbb{P}\left(G\right)$ given by

$$\begin{array}{c}\hfill \mathbb{P}\left(G\right)=\frac{1}{Z}{e}^{-H\left(G\right)}\delta \left(⟨k⟩N/2,\sum _{i<j}{a}_{ij}\right)\widehat{\theta }\left(K-\underset{i=1}{\overset{N}{max}}{k}_i\right)\widehat{\theta }\left(\underset{i=1}{\overset{N}{min}}{k}_i-\widehat{m}\right),\end{array}$$

(8)

with Hamiltonian $H\left(G\right)$ given by

$$\begin{array}{c}\hfill H\left(G\right)=-ln\pi \left(N\right)-\sum _{i=1}^{N}ln\left[p\left(k_i\right)k_i!\delta \left(k_i,\sum _{j=1}^N{a}_{ij}\right)\right]+ln(\left(⟨k⟩N\right)!!).\end{array}$$

(9)

This Hamiltonian is global and is invariant under permutation of the node labels, therefore this hierarchical grand canonical ensemble is exchangeable. Indeed we have that the probability of a network $\mathbb{P}\left(G\right)$ given by Equation (8) obeys

$$\begin{array}{c}\hfill \mathbb{P}\left(G\right)=\mathbb{P}\left(\tilde{G}\right)\end{array}$$

(10)

where $\tilde{G}$ is any network obtained from network G under a generic permutation $\sigma$ of the labels of the nodes. Moreover we note that for $\pi \left(N\right)=\delta (N,\overline{N})$, i.e., when the network size is fixed this model reduces to the exchangeable model for sparse network ensemble proposed in ref. [32].

3. The Grand Canonical Network Ensemble with Given Distribution of the Latent Variables

The grand canonical formalism can also be easily extended to treat network models with latent variables $\mathbf{\theta }$ associated to the nodes of the network $G=(V,E)$. Note that here and in the following we assume that the latent variables take discrete values. To this end we can consider the soft grand canonical hierarchical model associating to each network with $N=\left|V\right|>N_0$ nodes, latent variables $\mathbf{\theta }$ and adjacency matrix $\mathbf{a}$ the probability

$$\begin{array}{c}\hfill P(G,\mathbf{\theta },N)=P\left(N\right)P\left(\mathbf{\theta }\right|N\left)P\right(G|\mathbf{\theta },N)\end{array}$$

(11)

with

$$\begin{array}{c}\hfill P\left(N\right)=\pi \left(N\right),\end{array}$$

(12)

where $\pi \left(N\right)$ is an arbitrary prior on the number of nodes in the network defined for $N\ge N_0$. Typical examples of the distribution $\pi \left(N\right)$ are given by Equations (3) and (4). The probability of the latent variables is chosen to be exchangeable and given by

$$\begin{array}{c}\hfill P\left(\mathbf{\theta }\right|N)=\prod _{i=1}^{N}p\left({\theta }_i\right)\end{array}$$

(13)

where $p\left({\theta }_i\right)$ is the probability distribution of each latent variable. The distribution $p\left(\theta \right)$ can be chosen arbitrarily, as long as the expectation of $\theta$ is finite. The probability of the network given the network size and the latent variables is chosen to be derived by a Bernoulli variable for each link, with probability of observing a link between node i and node j conditioned on the value of their latent variables given by $p_N({\theta }_i,{\theta }_j)$, i.e.,

$$\begin{array}{c}\hfill P\left(G\right|\mathbf{\theta },N)=\prod _{i<j}{p}_N{({\theta }_i,{\theta }_j)}^{a_{ij}}{(1-p_N({\theta }_i,{\theta }_j))}^{1-a_{ij}}.\end{array}$$

(14)

To be concrete we consider the following expression for the probability $p_N({\theta }_i,{\theta }_j)$ which is the general expression of the marginal probability of a link in canonical network ensembles (or equivalently exponential random graph models),

$$\begin{array}{c}\hfill p_N({\theta }_i,{\theta }_j)=\frac{{\theta }_i{\theta }_j/N}{1+{\theta }_i{\theta }_j/N}.\end{array}$$

(15)

The advantage of taking this expression for the probability $p_N({\theta }_i,{\theta }_j)$ is that $p_N({\theta }_i,{\theta }_j)$ is always smaller or equal to one for every value of the latent variables. Therefore in this model we do not need to impose a structural cutoff on the latent variables. In summary the grand canonical network ensemble with given latent variable distribution is a hierarchical network model in which given the network size and latent variables the network is drawn according to a canonical ensemble of networks. In this ensemble the probability of a network G can be written in Hamiltonian form as

$$\begin{array}{c}\hfill \mathbb{P}\left(G\right)=\frac{1}{Z}{e}^{-H\left(G\right)}\end{array}$$

(16)

with Hamiltonian $H\left(G\right)$ given by

$$\begin{array}{c}\hfill H\left(G\right)=-ln\pi \left(N\right)-\sum _{i=1}^N{p}_N\left({\theta }_i\right)-\sum _{i<j}\left\{a_{ij}ln{p}_N({\theta }_i,{\theta }_j)+(1-a_{ij})ln[1-p_N({\theta }_i,{\theta }_j)]\right\}.\end{array}$$

(17)

This Hamitonian is invariant under permutation of the node labels, therefore this model is exchangeable.

4. The Entropy of Grand Canonical Ensembles

In this paragraph we show that the entropy S [3,48] of the two proposed grand canonical network ensembles, defined as

$$\begin{array}{c}\hfill S=\sum _G\mathbb{P}\left(G\right)ln\mathbb{P}\left(G\right),\end{array}$$

(18)

can be decomposed into contributions that reflect the uncertainty related to an increasing number of hierarchical levels of the model. In order to show this results we discuss separately the entropy of the two proposed grand canonical ensembles.

4.1. Entropy of the Grand Canonical Ensemble with Given Degree Distribution

The entropy S of the ensemble fixing the degree distribution can be decomposed into the entropy of the model at different levels of the hierarchy according to the following expression,

$$\begin{array}{c}\hfill S=S_{\pi \left(N\right)}+{⟨S_{p\left(k\right)}⟩}_{\pi \left(N\right)}+{⟨{\sum }_N\left(\mathbf{k}\right)⟩}_{\pi \left(N\right),p\left(k\right)}\end{array}$$

(19)

where $S_{\pi \left(N\right)}$ is the entropy associated to the number of typical choices of the total number of nodes N, ${⟨S_{p\left(k\right)}⟩}_{\pi \left(N\right)}$ is the entropy associated to the choice of the degree sequence averaged over the distribution $\pi \left(N\right)$ and ${⟨{\sum }_N\left(\mathbf{k}\right)⟩}_{\pi \left(N\right),p\left(k\right)}$ is the average of the Gibbs entropy [3] of the networks with given degree sequence averaged over the distribution $\pi \left(N\right)$ and $P\left(\mathbf{k}\right|N)$. In other words we have

$$\begin{array}{ccc}\hfill S_{\pi \left(N\right)}& =& -\sum _{N>N_0}\pi \left(N\right)ln\pi \left(N\right),\hfill \\ \hfill {⟨S_{p\left(k\right)}⟩}_{\pi \left(N\right)}& =& \sum _{N>N_0}\pi \left(N\right)\left[-N\sum _{k}p\left(k\right)lnp\left(k\right)\right],\hfill \\ \hfill {⟨{\sum }_N\left(\mathbf{k}\right)⟩}_{\pi \left(N\right),p\left(k\right)}& =& \sum _{N>N_0}\pi \left(N\right)\sum _{\mathbf{k}}P\left(\mathbf{k}\right|N){\sum }_N\left(\mathbf{k}\right).\hfill \end{array}$$

(20)

4.2. Entropy of the Grand Canonical Ensemble with Given Latent Variable Distribution

Similarly to the previous case, it is easy to show that the entropy of the ensemble fixing the distribution of the latent variables can be decomposed into the entropy of the model at different levels of their hierarchy, according to the following expression

$$\begin{array}{c}\hfill S=S_{\pi \left(N\right)}+{⟨S_{p\left(\theta \right)}⟩}_{\pi \left(N\right)}+{⟨S_N\left(\mathbf{\theta }\right)⟩}_{\pi \left(N\right),p\left(\theta \right)},\end{array}$$

(21)

where $S_{\pi \left(N\right)}$ is the entropy associated to the number of typical choices of the total number of nodes N, ${⟨S_{p\left(\theta \right)}⟩}_{\pi \left(N\right)}$ is the entropy associated to the choice of the latent variable distribution averaged over the distribution $\pi \left(N\right)$ and ${⟨S_N\left(\mathbf{\theta }\right)⟩}_{\pi \left(N\right),p\left(k\right)}$ is the average of the Shannon entropy [3] of the networks with given sequence of latent variables averaged over the distribution $\pi \left(N\right)$ and $P\left(\mathbf{\theta }\right|N)$. In other words we have

$$\begin{array}{ccc}\hfill S_{\pi \left(N\right)}& =& -\sum _{N>N_0}\pi \left(N\right)ln\pi \left(N\right),\hfill \\ \hfill {⟨S_{p\left(\theta \right)}⟩}_{\pi \left(N\right)}& =& \sum _{N>N_0}\pi \left(N\right)\left[-N\sum _{\theta }p\left(\theta \right)lnp\left(\theta \right)\right],\hfill \\ \hfill {⟨S_N\left(\mathbf{\theta }\right)⟩}_{\pi \left(N\right),p\left(\theta \right)}& =& \sum _{N>N_0}\pi \left(N\right)\sum _{\mathbf{\theta }}P\left(\mathbf{\theta }\right|N)S_N\left(\mathbf{\theta }\right),\hfill \end{array}$$

(22)

where the Shannon entropy $S_N\left(\mathbf{\theta }\right)$ of the network given the sequence of latent variables and the network size N can be expressed as

$$\begin{array}{c}\hfill S_N\left(\mathbf{\theta }\right)=-\sum _{i<j}\left[p_N({\theta }_i,{\theta }_j)ln{p}_N({\theta }_i,{\theta }_j)+(1-p_N({\theta }_i,{\theta }_j))ln(1-p_N({\theta }_i,{\theta }_j))\right].\end{array}$$

(23)

5. Marginal Probability of a Link

5.1. The Case of the Grand Canonical Ensemble with Given Degree Distribution

The grand canonical ensemble of exchangeable sparse network ensembles is an ensemble in which the total number of nodes is not specified. If we consider the networks of this ensemble having a given number of nodes N, the model reduces to the exchangeable sparse network ensemble proposed in ref. [32] whose marginal probability of a link $(i,j)$ is given by

$$\begin{array}{c}\hfill {\tilde{p}}_{ij}=\sum _{k}p\left(k\right)\sum _{k^{\prime }}p\left(k^{\prime }\right)\frac{k{k}^{\prime }}{⟨k⟩N}.\end{array}$$

(24)

Since the grand-canonical ensemble of sparse exchangeable networks with given degree distribution can be interpreted as a mixture of the exchangeable sparse models proposed in ref. [32] with different size N, it is immediate to show that the marginal probability of a link between node i and node j in the grand canonical ensembles is given by the exchangeable expression,

$$\begin{array}{c}\hfill p_{ij}=\sum _{N>N_0}\pi \left(N\right)\sum _{k,k^{\prime }}p\left(k\right)p\left(k^{\prime }\right)\frac{k{k}^{\prime }}{⟨k⟩N}=\sum _{N>N_0}\pi \left(N\right)\frac{⟨k⟩}{N}.\end{array}$$

(25)

Moreover the probability that two nodes are connected given that they have degree k and $k^{\prime }$ is given by

$$\begin{array}{c}\hfill p_{ij|k_i=k,k_j=k^{\prime }}=p\left(k,k^{\prime }\right)=k{k}^{\prime }\sum _{N>N_0}\frac{\pi \left(N\right)}{⟨k⟩N}.\end{array}$$

(26)

Finally the probability that two nodes are connected given that they have degree k and $k^{\prime }$ and the actual size of the network is N is given by the uncorrelated network expression

$$\begin{array}{c}\hfill p_{ij|k_i=k,k_j=k^{\prime },N}=p_N(k,k^{\prime })=\frac{k{k}^{\prime }}{⟨k⟩N}.\end{array}$$

(27)

From these expressions of the marginal probability of a link it is possible to appreciate how the hierarchical grand canonical ensemble of sparse exchangeable networks circumvents the difficulties arising form the Aldous-Hoover theorem without violating it. Indeed the marginal probability $p_N(k,k^{\prime })$ of a link conditioned on the degrees of the two linked nodes and the number of nodes N of the network vanishes in the limit $N\to \infty$, however if the number of nodes of the network is arbitrarily large but unknown the marginal probability of the link remains finite (as both $p_{ij}$ and $p(k,k^{\prime })$ are finite).

5.2. The Case of the Grand Canonical Ensemble with Given Latent Variable Distribution

For the grand canonical ensemble with given latent variable distribution $p\left(\theta \right)$ we have that the marginal probability of a link is given by

$$\begin{array}{c}\hfill p_{ij}=\sum _{N>N_0}\pi \left(N\right)\sum _{\theta ,{\theta }^{\prime }}p\left(\theta \right)p\left({\theta }^{\prime }\right)p_N(\theta ,{\theta }^{\prime }).\end{array}$$

(28)

The probability of the link given the latent variable of the nodes is given by

$$\begin{array}{c}\hfill p(\theta ,{\theta }^{\prime })=\theta {\theta }^{\prime }\sum _{N>N_0}\pi \left(N\right)\frac{1}{N+\theta {\theta }^{\prime }};\end{array}$$

(29)

The probability of a link given the network size and the latent variables is given by

$$\begin{array}{c}\hfill p_N(\theta ,{\theta }^{\prime })=\frac{\theta {\theta }^{\prime }/N}{1+\theta {\theta }^{\prime }/N}.\end{array}$$

(30)

As we discussed in the case of the grand canonical ensemble with given degree distribution also for the grand canonical ensemble with given latent variable distribution the grand canonical approach allows to circumvent the Aldous-Hoover theorem without violating it as the marginal probability of a link in an arbitrarily large network of unknown size is finite.

6. Generating Single Instances of Grand-Canonical Network Ensembles

In this section we describe two algorithms to generate single instances of the proposed grand canonical ensembles. In particular we will discuss a Metropolis-Hastings ensemble to generate single instances of networks drawn from the grand canonical ensemble with given degree distribution and a Monte Carlo algorithm to generate single instances of networks drawn from the grand canonical ensemble with given distribution of latent variables.

6.1. Metropolis-Hastings Algorithm for the Grand-Canonical Ensemble with Given Degree Distribution

The grand-canonical exchangeable ensemble of sparse networks can be obtained by implementing a Metropolis-Hastings algorithm using the network Hamiltonian given by Equation (9).

(1)

Start with a network of N nodes having exactly $L=⟨k⟩N/2$ links and in which the minimum degree is greater of equal to $\widehat{m}$ and the maximum degree is smaller or equal to K.

(2)

Perform the Metropolis-Hastings algorithm for exchangeable sparse networks with N nodes (defined below);

(3)

Propose to change the number of nodes to $N^{\prime }=N+1$ (addition of one node) or $N^{\prime }=N-1$ (removal of one node) with equal probability and accept the move with probability $max\left(1,\pi \left(N^{\prime }\right)/\pi \left(N\right)\right)$ as long as $N^{\prime }>N_0$. If the move is accepted change the number of nodes adding or removing a node, set the number of links to $L=⟨k⟩N/2$ and ensure that each node has minimal degree at least $\widehat{m}$ and maximum degree less than K. In particular if a node is added ensure it has at least $\widehat{m}$ links by rewiring randomly the existing links of the networks and adding a number of links so that the total number of links is the integer that better approximates $⟨k⟩N/2$. Instead, if a node needs to be removed, choose a random node of the network remove it and rewire/remove links in order to enforce that the total number of links is the integer that better approximates $⟨k⟩N/2$.

The Metropolis-Hastings algorithm for the exchangeable sparse networks with N nodes is the same algorithm used in Ref. [32] for exchangeable networks with finite size N and is indicated below.

(1)

Start with a network of N nodes having exactly $L=⟨k⟩N/2$ links and in which the minimum degree is greater of equal to $\widehat{m}$ and the maximum degree is smaller or equal to K.

(2)

Iterate the following steps until equilibration:

(i)

Let $\mathbf{a}$ be the adjacency matrix of the network;

(i)

Choose randomly a random link $\ell =(i,j)$ between node i and j and choose a pair of random nodes $(i^{\prime },j^{\prime })$ not connected by a link.

(ii)

Let ${\mathbf{a}}^{\prime }$ be the adjacency matrix of the network in which the link $(i,j)$ is removed and the link $(i^{\prime },j^{\prime })$ is inserted instead. Draw a random number r from a uniform distribution in $[0,1]$, i.e., $r\sim U(0,1)$. If $r<\max (1,e^{-\Delta H})$ where $\Delta H=H\left({\mathbf{a}}^{\prime }\right)-H\left(\mathbf{a}\right)$ and if the move does not violate the conditions on the minimum and maximum degree of the network, replace $\mathbf{a}$ by ${\mathbf{a}}^{\prime }$.

The Metropolis-Hastings algorithm can be used to sample the space of networks with variable number of nodes and given (stable) degree distribution (see Figure 2).

Figure 2. Results of the Metropolis-Hastings algorithm for generating grand canonical ensembles with given degree distribution. The number of nodes $N\left(t\right)$ as a function of time t in the Metropolis-Hastings simulation of an exponential networks (panel (a)) and networks with more general degree distribution (panel (c)) are shown together with the average degree distribution of the networks that is stable as the number of networks varies (symbols of panel (b) and (d)). The solid lines in panel (b) and panel (d) indicate the target degree distributions $p\left(k\right)=C{e}^{-k/m}$ with $m=5$ (for panel (b)) and $p\left(k\right)=C{(3+k)}^{-\gamma }$ with $\gamma =3.4$ (for panel (d)). The prior on the number of nodes is taken to be exponential $\pi \left(N\right)=C{e}^{-N/\overline{N}}$ with $\overline{N}=1000$ with $N_0=500$ and $K=16$.

6.2. Monte Carlo Generation of Grand Canonical Network Ensemble with Given Latent Variable Distribution

A single instance of the grand canonical model with given latent variable distribution can be obtained by performing the following algorithm:

1

Draw the network size N from the $\pi \left(N\right)$ distribution;

2

Draw the latent variable ${\theta }_i$ of each node i independently from the latent variable distribution $p\left(\theta \right)$.

3

Draw each link $(i,j)$ of the network with probability $p_N({\theta }_i,{\theta }_j)$.

7. Bayesian Estimation of the Network Parameters Given Partial Knowledge of the Network

In this section we will use the grand canonical network ensembles for calculating the posterior distribution of the network parameters given partial information of a network $G=(V,E)$. In particular let us assume that we only know the subgraph $\widehat{G}(\widehat{V},\widehat{E})$ induced by a set of nodes $\widehat{V}\subset V$ of $\widehat{N}=\left|\widehat{V}\right|$ nodes and of adjacency matrix $\widehat{\mathbf{a}}$ and we do not have access to the full network G with adjacency matrix $\mathbf{a}$. Without loss of generality let us label the nodes of the network in such a way that the labels i with $1\le i\le \widehat{N}$ indicate the nodes in $\tilde{V}$ (denote as sampled nodes) and the labels i with $i>\widehat{N}$ indicate the nodes in $V\backslash \widehat{V}$ (denoted also as unsampled or unknown nodes). We indicate with $\mathbf{\kappa }$ the degree sequence of the sampled network $\widehat{G}$. Our goal is to make a Bayesian estimation of the network size N and the true network parameters given the observed subgraph $\widehat{G}$. These a posteriori estimates of the true parameters of the network can then be used to reconstruct the unknown part of the network G.

7.1. Inferring the True Parameters with the Grand Canonical Ensemble with Given Degree Distribution

In this paragraph we will use the grand canonical ensemble with given degree distribution to find the posterior probability distribution of the network parameters. For convenience we will indicate with $k_i$ the true degree of the sampled nodes $1\le i\le \widehat{N}$ and we will indicate $q_i$ the true degree of the remaining unsampled $N-\widehat{N}$ nodes $\widehat{N}+1\le i\le N$. To this end, using the Bayes rule we get the following expression for the posterior distribution of the network parameters given the observed subgraph $\widehat{G}$

$$\begin{array}{c}\hfill P(N,\mathbf{k},\mathbf{q}|\widehat{G})=\frac{P\left(N\right)P(\mathbf{k},\mathbf{q}|N)P\left(\widehat{G}\right|\mathbf{k},\mathbf{q},N)}{P\left(\widehat{G}\right)}\end{array}$$

(31)

where

$$\begin{array}{ccc}\hfill P\left(N\right)& =& \pi \left(N\right),\hfill \\ \hfill P(\mathbf{k},\mathbf{q}|N)& =& \prod _{i=1}^{\widehat{N}}p\left(k_i\right)\prod _{i=1+\widehat{N}}^{N}p\left(q_i\right),\hfill \\ \hfill P\left(\widehat{G}\right|\mathbf{k},\mathbf{q},N)& =& e^{-{\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })},\hfill \end{array}$$

(32)

with ${\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })$ given by

$$\begin{array}{c}\hfill {\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })={\sum }_N\left(\mathbf{k},\mathbf{q}\right)-{\widehat{\sum }}_N\left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa }).\end{array}$$

(33)

Here ${\sum }_N\left(\mathbf{k},\mathbf{q}\right)$ indicates the entropy of the network fo size N with degree sequence $[\mathbf{k},\mathbf{q}]$ whose expression is given by the Bender-Canfield formula [3,21,22,47] (Equation (7)) which reads in this case

$$\begin{array}{c}\hfill {\sum }_N\left(\mathbf{k},\mathbf{q}\right)=\left(2L\right)!!{\left[\prod _{i=1}^{\widehat{N}}{k}_i!\prod _{i=1+\widehat{N}}^N{q}_i!\right]}^{-1}.\end{array}$$

(34)

Moreover ${{\displaystyle \widehat{\sum }}}_N\left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })$ indicates the logarithm of the number of networks of N nodes having $\widehat{G}$ (with adjacency matrix $\widehat{\mathbf{a}}$ and degree sequence $\mathbf{\kappa }$) as induced subgraph between the $\widehat{N}$ sampled nodes.

Moreover in Equation (31) $P\left(\widehat{G}\right)$ indicates the evidence of the data given by

$$\begin{array}{c}\hfill P\left(\widehat{G}\right)=\sum _N\sum _{\mathbf{k},\mathbf{q}}\pi \left(N\right)\prod _{i=1}^{\widehat{N}}p\left(k_i\right)\prod _{i=1+\widehat{N}}^{N}p\left(q_i\right)e^{-{\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })}.\end{array}$$

(35)

Calculating the entropy ${{\displaystyle \hat{\sum }}}_N\left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })$ using statistical mechanics methods including the use of a functional order parameter (see Appendix A), we derive the following expression:

$$\begin{array}{ccc}\hfill {\widehat{\sum }}_N\left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })& =& ln\left[\frac{M!(Q-M)!!}{{\prod }_{i=1}^{\widehat{N}}(k_i-{\kappa }_i)!{\prod }_{i=1+\widehat{N}}^N{q}_i!}\left(\begin{array}{c}Q\\ M\end{array}\right)\delta (Q+M,2L+2\widehat{L})\right]\hfill \end{array}$$

(36)

where M indicates the number of links between the sampled nodes and the unsampled nodes and Q indicates the sum over all the degrees of the unsampled nodes, i.e.,

$$\begin{array}{ccc}\hfill M& =& \sum _{i=1}^{\widehat{N}}(k_i-{\kappa }_i),\hfill \\ \hfill Q& =& \sum _{i=1+\widehat{N}}^N{q}_i,\hfill \end{array}$$

(37)

where M and Q need to satisfy the constraint enforcing that the total number of true links is given by $L=⟨k⟩N/2$. Therefore, indicating with $\widehat{L}={\sum }_{i=1}^{\widehat{N}}{\kappa }_i/2$, we must impose

$$\begin{array}{c}\hfill Q+M=2L-2\widehat{L}.\end{array}$$

(38)

The expression obtained for the entropy ${{\displaystyle \widehat{\sum }}}_N(\mathbf{k},\mathbf{q}|\mathbf{\kappa })$ implies that the asymptotic expression for the number of networks with N nodes, degree sequence $[\mathbf{k},\mathbf{q}]$ having $\widehat{G}$ as a subgraph is given by (see Appendix A for the derivation)

$$\begin{array}{c}\hfill \mathcal{N}(\mathbf{k},\mathbf{q}|\mathbf{\kappa },N)=e^{{{\displaystyle \widehat{\sum }}}_N(\mathbf{k},\mathbf{q}|\mathbf{\kappa })}=\frac{M!(Q-M)!!}{{\prod }_{i=1}^{\widehat{N}}(k_i-{\kappa }_i)!{\prod }_{i=1+\widehat{N}}^N{q}_i!}\left(\begin{array}{c}Q\\ M\end{array}\right)\delta (Q+M,2L+2\widehat{L}).\end{array}$$

(39)

This expression admits a simple combinatorial interpretation. In fact the networks with degree sequence $[\mathbf{k},\mathbf{q}]$ having as subgraph $\widehat{G}$ can be constructed by adding (unsampled) links to the graph $\widehat{G}$. The unsampled part of the network can be constructed by assigning to each node i with $1\le i\le \widehat{N}$ a number of stubs given by $k_i-{\kappa }_i$ and to each node i with $i>\widehat{N}$ a number of stubs given by $q_i$. The unsampled networks can then be obtained by matching the stubs pairwise with the constrains that the stubs of the first $\widehat{N}$ nodes can be only matched with the stubs of the unsampled nodes $i>\widehat{N}$. Therefore the reconstructed part of the network is formed by a bipartite network between the sampled and the unsampled nodes with a number of links given by M and a simple network among the unsampled nodes with number of links given by $(Q-M)/2$. The number of matchings of the M links of the bipartite network is given by $M!$ the number of matching of the stubs of the simple network among unsampled nodes is $(Q-M)!!$. In order to get the number of distinct networks G with degree sequence $[\mathbf{k},\mathbf{q}]$ having as subgraph $\widehat{G}$ we need to divide by the number of permutations of the stubs belonging to the same nodes and we need to multiply by Q choose M indicating the number of ways in which we can choose the M stubs of the unsampled nodes to be matched with the stubs of the sampled nodes.

Given the expression for ${{\displaystyle \widehat{\sum }}}_N\left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })$ provided by Equation (36), we can deduce the explicit expression for ${\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })$:

$$\begin{array}{ccc}\hfill {\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })& =& ln\left[\prod _{i=1}^{\widehat{N}}\frac{k_i!}{(k_i-{\kappa }_i)!}\frac{M!(Q-M)!!}{(⟨k⟩N)!!}\left(\begin{array}{c}Q\\ M\end{array}\right)\delta (Q+M,2L-2\widehat{L})\right].\hfill \end{array}$$

(40)

It follows that the describe Bayesian inference assigns a probability to the model parameters a probability

$$\begin{array}{c}\hfill P(N,\mathbf{k},\mathbf{q}|\widehat{G})\propto \pi \left(N\right)\prod _{i=1}^{\widehat{N}}p\left(k_i\right)\prod _{i=1+\widehat{N}+1}^{N}p\left(q_i\right)e^{-{\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })},\end{array}$$

(41)

with ${\Delta }_N\sum \left(\mathbf{k},\mathbf{q}\right|\mathbf{\kappa })$ given by Equation (40). From this expression, imposing with a delta function that $M={\sum }_{i=1}^{\widehat{N}}(k_i-{\kappa }_i)$, expressing the delta in integral form and using the saddle point to evaluate the integral, we can calculate the marginal probability $P\left(k_i\right|\widehat{G},\omega )$ that a sampled node i with $1\le i\le \widehat{N}$ has true degree $k_i\ge {\kappa }_i$ given M and Q, i.e.,

$$\begin{array}{c}\hfill P\left(k_i\right|\widehat{G},\omega )\propto p\left(k_i\right)\frac{k_i!}{(k_i-{\kappa }_i)!}{e}^{-\omega k_i}\widehat{\theta }(k_i-{\kappa }_i)\end{array}$$

(42)

where $\omega$ is related to M by

$$\begin{array}{c}\hfill M=\sum _{i=1}^{\widehat{N}}\frac{{\sum }_{k}p\left(k\right)k\frac{k!}{(k-{\kappa }_i)!}{e}^{-\omega k}}{{\sum }_{k^{\prime }}p\left(k^{\prime }\right)\frac{k^{\prime }!}{(k^{\prime }-{\kappa }_i)!}{e}^{-\omega k^{\prime }}}.\end{array}$$

(43)

In Figure 3 we show the difference between an exponential prior distribution $p\left(k\right)$ on the degree of the nodes and the posterior marginal probability of the true degree of the sampled nodes $P\left(k\right|\widehat{G},\omega )$ plotted for different values of the sampled degree $\kappa$ of the same node. Finally, we can calculate the a posteriori probability $P\left(N\right|\widehat{G},M)$ that the real networks has N nodes, conditioned to M and to the sampled subrgraph $\widehat{G}$. To this end we sum Equation (41) over all the possible values of the degrees $\mathbf{k}$ and $\mathbf{q}$ such that Equation (37) are satisfied. Therefore, by inserting Equation (40) into Equation (41), enforcing Equation (37) with Kronecker deltas and integrating over all the possible values of $\mathbf{k}$ and $\mathbf{q}$ we get

$$\begin{array}{c}\hfill P\left(N\right|\widehat{G},M)\propto \pi \left(N\right)\widehat{\theta }(N-\widehat{N})C_{M,N}{I}^{\left(k\right)}\left(M\right)I^{\left(q\right)}(M,N),\end{array}$$

(44)

where

$$\begin{array}{ccc}\hfill C_{M,N}& =& \frac{M!(Q-M)!!}{(⟨k⟩N)!!}\left(\begin{array}{c}Q\\ M\end{array}\right),\hfill \\ \hfill I^{\left(k\right)}\left(M\right)& =& \sum _{\mathbf{k}}\left[\prod _{i=1}^{\widehat{N}}p\left(k_i\right)\frac{k_i!}{(k_i-{\kappa }_i)!}\delta \left(M,\sum _{i=1}^{\widehat{N}}{k}_i\right)\right],\hfill \\ \hfill I^{\left(q\right)}(M,N)& =& \sum _{\mathbf{q}}\left[\prod _{i=1+\widehat{N}}^{N}p\left(q_i\right)\delta \left(Q,\sum _{i=1+\widehat{N}}^N{q}_i\right)\right],\hfill \end{array}$$

(45)

where $Q=⟨k⟩N-2\widehat{L}-M$. By expressing the Kronecker deltas in an integral form according to the expression

$$\begin{array}{c}\hfill \delta (x,y)=\frac{1}{2\pi }{\int }_{-\pi }^{\pi }{e}^{\mathrm{i}\omega (x-y)},\end{array}$$

(46)

performing a Wick rotation and evaluating the integrals at the saddle point, we can express $I^{\left(k\right)}\left(M\right)$ and $I^{\left(q\right)}(M,N)$ as

$$\begin{array}{ccc}\hfill I^{\left(k\right)}\left(M\right)& =& \frac{1}{2\pi }\left[\prod _{i=1}^{\widehat{N}}\sum _{k>{\kappa }_i}p\left(k\right)\frac{k!}{(k-{\kappa }_i)!}{e}^{-{\omega }^{\star }k}\right]e^{{\omega }^{\star }M},\hfill \\ \hfill I^{\left(q\right)}(M,N)& =& \frac{1}{2\pi }{\left[\sum _{q}p\left(q\right)e^{-{\overline{\omega }}^{\star }q}\right]}^{N-\widehat{N}}{e}^{{\overline{\omega }}^{\star }Q},\hfill \end{array}$$

(47)

with ${\omega }^{\star }$ and ${\overline{\omega }}^{\star }$ fixed by the saddle point equations

$$\begin{array}{ccc}\hfill M& =& \sum _{i=1}^{\widehat{N}}\frac{{\sum }_{k>{\kappa }_i}p\left(k\right)\frac{k!}{(k-{\kappa }_i)!}k{e}^{-{\omega }^{\star }k}}{{\sum }_{k>{\kappa }_i}p\left(k\right)\frac{k!}{(k-{\kappa }_i)!}{e}^{-{\omega }^{\star }k}},\hfill \\ \hfill Q& =& (N-\widehat{N})\frac{{\sum }_{q}p\left(q\right)q{e}^{-{\overline{\omega }}^{\star }q}}{{\sum }_{q}p\left(q\right)e^{-{\overline{\omega }}^{\star }q}}.\hfill \end{array}$$

(48)

Figure 3. Marginal posterior probability for the true degree and of the true latent variable of a sampled node. The posterior probability $P\left(k_i\right|\widehat{G},\omega )$ (panel (a)) of the true degree of a sampled nodes depends on the degree $\kappa$ of the nodes in the sampled network $\widehat{G}$ and is non-zero only for $k\ge \kappa$. The posterior probability $P\left(\theta \right|\widehat{G},\overline{\theta })$ of the latent variable of a sampled node (panel (b)) can be non-zero on the entire range of $\theta$ values allowed by the prior. Here we have plotted $P\left(k_i\right|\widehat{G},\omega )$ and $P\left(\theta \right|\widehat{G},\overline{\theta })$ for different values of $\kappa$ and we have chosen $\omega =2$ and $\overline{\theta }=0.6$. The dashed lines indicate the exponential prior on the degrees (panel (a)) and on the latent variables (panel (b)).

In Figure 4 we display the marginal a posteriori distribution $P\left(N\right|\widehat{G},M)$ as function of M demonstrating that the sampled network can modify significantly the prior assumptions on the total number of nodes in the network.

Figure 4. Marginal posterior probability for the true number of nodes in the grand canonical ensemble with given degree distribution and in the grand canonical ensemble with given latent variable distribution. The posterior probability $P\left(N\right|\widehat{G},M)$ in panel (a) of the true number of nodes depends on the total number M of true but not observed links of the sampled nodes and on the total number of sampled links $\widehat{L}$; the posterior probability $P\left(N\right|\widehat{G})$ in panel (b) depends instead only on the degree $\kappa$ of the nodes in the sampled network $\widehat{G}$. We took $N_0=100$ and the priors given by $\pi \left(N\right)\propto e^{-N/\widehat{N}}$, $p\left(k\right)\propto e^{-k/m}$, $p\left(\theta \right)\propto e^{-\theta /m}$ with $\widehat{N}=200$, and $m=7$. In panel (a) we have plotted $P\left(N\right|\widehat{G},M)$ for different values of $M=(⟨k⟩-n)\widehat{N}$ with $n=1,2,3,4$ and $\widehat{L}=\widehat{N}/2$; in panel (b) we have plotted $P\left(N\right|\widehat{G})$ assuming that $\widehat{G}$ is regular with all sampled nodes having sampled degree $\kappa =1,2,3,4,5$. The dashed lines indicate the exponential prior $\pi \left(N\right)$ on the number of nodes.

7.2. Inferring the True Parameters with the Grand Canonical Ensemble with Given Latent Variable Distribution

In this section we treat the problem of Bayesian estimation of the parameters of the true network G given the sampled network $\widehat{G}$ using the grand canonical model with given latent variable distribution. Let us indicate with ${\theta }_i$ the latent variables of the sampled nodes $1\le i\le \widehat{N}$ and with ${\varphi }_i$ the latent variables of the unsampled nodes $i>\widehat{N}$. Using Bayes rule we have

$$\begin{array}{c}\hfill P(N,\mathbf{\theta },\mathbf{\varphi }|\widehat{G})=\frac{P\left(N\right)P(\mathbf{\theta },\mathbf{\varphi }|N)P\left(\widehat{G}\right|\mathbf{\theta },\mathbf{\varphi },N)}{P\left(\widehat{G}\right)},\end{array}$$

(49)

where $P\left(\widehat{G}\right|\mathbf{\theta },\mathbf{\varphi },N)$ is independent of $\mathbf{\varphi }$, i.e., $P\left(\widehat{G}\right|\mathbf{\theta },\mathbf{\varphi },N)=P\left(\widehat{G}\right|\mathbf{\theta },N)$ and where

$$\begin{array}{ccc}\hfill P\left(N\right)& =& \pi \left(N\right),\hfill \\ \hfill P(\mathbf{\theta },\mathbf{\varphi }|N)& =& \prod _{i=1}^{\widehat{N}}p\left({\theta }_i\right)\prod _{i=1+\widehat{N}}^{N}p\left({\varphi }_i\right),\hfill \\ \hfill P\left(\widehat{G}\right|\mathbf{\theta },N)& =& \prod _{i<j|i,j\in \widehat{V}}{p}_N{({\theta }_i,{\theta }_j)}^{{\widehat{a}}_{ij}}{(1-p_N({\theta }_i,{\theta }_j))}^{1-{\widehat{a}}_{ij}}\hfill \end{array}$$

(50)

with $p_N({\theta }_i,{\theta }_j)$ given by Equation (15) and with $\widehat{\mathbf{a}}$ indicating the adjacency matrix of the sampled subgraph $\widehat{G}$. In Equation (49) $P\left(\widehat{G}\right)$ indicates the evidence of the data given by

$$\begin{array}{c}\hfill P\left(\widehat{G}\right)=\sum _N\sum _{\mathbf{\theta }}\pi \left(N\right)\prod _{i=1}^{\widehat{N}}p\left({\theta }_i\right)P\left(\widehat{G}\right|\mathbf{\theta },N).\end{array}$$

(51)

Since, as we have observed previously, $P\left(\widehat{G}\right|\mathbf{\theta },\mathbf{\varphi },N)$ is independent of $\mathbf{\varphi }$ the Bayesian estimation of the parameters $\mathbf{\varphi }$ reduces simply to the prior in this case. Therefore we focus here only on the Bayesian estimate of the latent variables $\mathbf{\theta }$, i.e., we consider

$$\begin{array}{c}\hfill P(N,\mathbf{\theta }|\widehat{G})=\frac{P\left(N\right)P\left(\mathbf{\theta }\right|N)P\left(\widehat{G}\right|\mathbf{\theta },N)}{P\left(\widehat{G}\right)},\end{array}$$

(52)

with $P\left(N\right),P\left(\widehat{G}\right|\mathbf{\theta },N),P\left(\widehat{G}\right)$ having the same definition as above and

$$\begin{array}{c}\hfill P\left(\mathbf{\theta }\right|N)=\prod _{i=1}^{\widehat{N}}p\left({\theta }_i\right).\end{array}$$

(53)

Using the explicit expression of $p_N({\theta }_i,{\theta }_j)$ given by Equation (15), we can express the likelihood $P\left(\widehat{G}\right|\mathbf{\theta },N)$ of the sampled network as

$$\begin{array}{ccc}\hfill P\left(\widehat{G}\right|\mathbf{\theta },N)& =& \prod _{i=1}^{\widehat{N}}{\theta }_i^{{\kappa }_i}\prod _{i<j|i,j\in \widehat{V}}{\left(1+\frac{{\theta }_i{\theta }_j}{N}\right)}^{-1}\frac{1}{N^{\widehat{L}}},\hfill \end{array}$$

(54)

where $\widehat{L}$ is the number of links of the sampled network $\widehat{G}$. In the limit $N\gg 1$ we can approximate this expression as

$$\begin{array}{c}\hfill P\left(\widehat{G}\right|\mathbf{\theta },N)\simeq \int d\overline{\theta }\prod _{i=1}^{\widehat{N}}\left[{\theta }_i^{{\kappa }_i}{e}^{-{\theta }_i\overline{\theta }/2}\right]\frac{1}{N^{\widehat{L}}}\delta \left(\overline{\theta },\sum _{j=1}^{\widehat{N}}{\theta }_j/N\right)\end{array}$$

(55)

with this approximation we get that the posterior probability $P(N,\mathbf{\theta }|\widehat{G})$ is given by

$$\begin{array}{cc}\hfill P(N,\mathbf{\theta }|\widehat{G})\propto & \pi \left(N\right)\frac{1}{N^{\widehat{L}}}\int d\overline{\theta }{\prod }_{i=1}^{\widehat{N}}\left[p\left({\theta }_i\right){\theta }_i^{{\kappa }_i}{e}^{-{\theta }_i\overline{\theta }/2}\right]\delta \left(\overline{\theta },{\sum }_{j=1}^{\widehat{N}}{\theta }_j/N\right).\end{array}$$

(56)

Calculating the marginal posterior probability of a single latent variable conditional of $\overline{\theta }$ we get

$$\begin{array}{c}\hfill P\left({\theta }_i\right|\widehat{G},\overline{\theta })=p\left({\theta }_i\right){\theta }_i^{{\kappa }_i}{e}^{-{\theta }_i\overline{\theta }/2}.\end{array}$$

(57)

In Figure 3 we show the difference between an exponential prior distribution $p\left(\theta \right)$ on the latent variables of the nodes and the posterior marginal probability of the true latent variables of the sampled nodes $P\left(\theta \right|\widehat{G},\overline{\theta })$ plotted for different values of the sampled degree $\kappa$ of the same node.

Stating from Equation (56) we can also calculate the posterior distribution $P\left(N\right|\widehat{G})$ of the true number of nodes $N>\widehat{N}$. To this end we express the delta function in an integral form and we sum over all possible latent variables $\mathbf{\theta }$, obtaining

$$\begin{array}{c}\hfill P\left(N\right|\widehat{G})\propto \pi \left(N\right)\widehat{\theta }(N-\widehat{N})\frac{1}{N^{\widehat{L}-1}}\frac{1}{2\pi }\int d\overline{\theta }d\omega e^{\mathrm{i}N\omega \overline{\theta }}{I}^{\left(\theta \right)}(\omega ,\overline{\theta })\end{array}$$

(58)

where $I^{\left(\theta \right)}(\omega ,\overline{\theta })$ is given by

$$\begin{array}{c}\hfill I^{\left(\theta \right)}=\prod _{i=1}^{\widehat{N}}\left(\sum _{\theta }p\left(\theta \right){\theta }^{{\kappa }_i}{e}^{-\theta (\overline{\theta }/2-\mathrm{i}\omega )}\right).\end{array}$$

(59)

The integrals in Equation (58) can be calculated at the saddle point getting

$$\begin{array}{c}\hfill P\left(N\right|\widehat{G})\propto \pi \left(N\right)\widehat{\theta }(N-\widehat{N})\frac{1}{N^{\widehat{L}-1}}{e}^{N\frac{{\left({\overline{\theta }}^{\star }\right)}^2}{2}}\left[\prod _{i=1}^{\widehat{N}}\left(\sum _{\theta }p\left(\theta \right){\theta }^{{\kappa }_i}{e}^{-\theta {\overline{\theta }}^{\star }}\right)\right]\end{array}$$

(60)

where

$$\begin{array}{c}\hfill {\overline{\theta }}^{\star }=\frac{1}{N}\sum _{i=1}^{\widehat{N}}\frac{{\sum }_{\theta }p\left(\theta \right){\theta }^{{\kappa }_i+1}{e}^{-\theta {\overline{\theta }}^{\star }}}{{\sum }_{\theta }p\left(\theta \right){\theta }^{{\kappa }_i}{e}^{-\theta {\overline{\theta }}^{\star }}}.\end{array}$$

(61)

In Figure 4 we display the marginal a posteriori distribution $P\left(N\right|\widehat{G})$ on the true number of nodes in the simplified assumption in which $\widehat{G}$ is regular and all degree $\kappa$ are the same demonstrating that the sampled network can modify significantly the prior assumptions on the total number of nodes in the network.

8. Conclusions

In this paper we have proposed grand canonical network ensembles formed by networks of varying number of nodes. The grand canonical network ensembles we have introduced are both sparse and exchangeable, i.e., have a finite average degree and are invariant under permutation of the node labels. The grand canonical ensembles are hierarchical network models in which first the network size is selected, then the degree sequence (or the sequence of latent variables) and finally the network adjacency matrix is selected. The model circumvents the difficulties imposed by the Aldous-Hoover theorem that states that exchangeable infinite sparse network ensembles vanish, as the network is a mixture of finite networks, although the networks can have an arbitrarily large network size. Here we show how the grand-canonical ensembles can be used to perform a Bayesian estimation of the network parameters when only partial information about the network structures is known. This a posteriori estimation of the network parameters can then be used for network reconstruction.

The grand canonical framework for sparse exchangeable network ensembles is here described for the case simple networks but has the potential to be extended to generalized network structures including directed, bipartite networks, multiplex networks and simplicial complexes following the lines outlined in ref. [32].

In conclusion we hope that this work, proposing hierarchical grand canonical network ensembles able to treat networks of different size and relating network theory to statistical mechanics will stimulate further results of mathematicians, physicists, and computer scientists working in network science and related machine learning problems.