A Mixture Model of Truncated Zeta Distributions with Applications to Scientific Collaboration Networks ^†

Abstract

The degree distribution has attracted considerable attention from network scientists in the last few decades to have knowledge of the topological structure of networks. It is widely acknowledged that many real networks have power-law degree distributions. However, the deviation from such a behavior often appears when the range of degrees is small. Even worse, the conventional employment of the continuous power-law distribution usually causes an inaccurate inference as the degree should be discrete-valued. To remedy these obstacles, we propose a finite mixture model of truncated zeta distributions for a broad range of degrees that disobeys a power-law behavior in the range of small degrees while maintaining the scale-free behavior. The maximum likelihood algorithm alongside the model selection method is presented to estimate model parameters and the number of mixture components. The validity of the suggested algorithm is evidenced by Monte Carlo simulations. We apply our method to five disciplines of scientific collaboration networks with remarkable interpretations. The proposed model outperforms the other alternatives in terms of the goodness-of-fit.

1. Introduction

Network science focuses on the study of complex networks such as telecommunication, computer, biological, cognitive, and social networks. A network consists of nodes and links. The topological structure, which explores how nodes are connected in the system, has been investigated with great interest [1,2,3]. Network researchers have examined foundational network topologies using various network-related quantities such as the degree distribution, clustering coefficient, and average path length. Most networks are dynamic, so accordingly, network-related quantities also change over time. Studying the evolution of these network quantities could provide insight into the behavior of individuals expressed by nodes and the change of topological properties of the network. The evolution of a network has been studied from various perspectives, e.g., the community [4,5], rich-get-richer [1], node heterogeneity [2,6], and link persistence [3].

The degree of a node is the number of links connected to a node. The degree distribution of a network, $P\left(k\right)$, tells us the probability of degree k that a randomly chosen node will have. One challenge in studying network science is to develop simplified measures that capture the network structure. The degree distribution is one of such measures that help to find influential nodes in the system. Many attempts have been made to study the degree distribution using Poisson, exponential, and power-law distributions. In particular, the analysis of the power-law degree distribution has been considered as one of the basic steps, and networks that have power-law degree distributions are often referred to as scale-free networks. We can frequently observe power-law degree distributions in collaboration, World Wide Web, protein–protein interactions, and semantic networks [7,8,9,10]. The emergence of hubs (highly connected nodes) is a consequence of a scale-free property of networks. A rich-get-richer mechanism, also called a popularity effect [11], has been known to produce power-law degree distributions and hubs [1,6].

Many dynamic network models have been developed to explain the power-law degree distribution in real networks. The most widely known dynamic model for the power-law degree distribution is the rich-get-richer generative model in Barabasi and Albert (1999) [1], called the BA model. They employ the preferential attachment mechanism in which nodes with more neighbors tend to receive more links from other nodes. Specifically, the algorithm of the BA model has the following parts with the parameter m that controls the number of links over time:

• Growth: At each time point, a node enters the network. Then the node tries to connect with m nodes in the network.
• Preferential attachment: The newly entered node connects with node i with probability proportional to the degree of node i.

The BA model yields a power-law degree distribution with exponent 3, i.e., $P\left(k\right)\propto k^{-3}$. The exact form, given by Bollobas et al. in [12], is:

$$\begin{array}{c}\hfill P\left(k\right|m)=\frac{2m(m+1)}{k(k+1)(k+2)},k=m,m+1,\cdots .\end{array}$$

(1)

There are many variant models to cover a broad range of power-law exponents [13,14,15], degree correlations [16], accelerating growth [17,18,19], and node heterogeneity [6,20,21]. Another model for generating the power-law degree distribution is a copying network model presented in Kumar et al. (2000) [22]. In this model, newly entered nodes randomly select some existing nodes and copy some of the links.

The power-law distribution has the form of $P\left(k\right)\propto k^{-\alpha }$, which can be expressed as $lnP\left(k\right)=-\alpha lnk+constant$. Therefore, a straight line of $lnP\left(k\right)$ plot on $lnk$ (log-log plot) can be an indication of a power-law relationship, and its slope is a power-law exponent. In real networks, however, the degree distribution does not have a shape of a straight line in the entire range. There are many empirical distributions where the power-law behavior is not observed in the range of small degrees. Many variants of the power-law distribution are developed to address this issue, such as generalized power-law distributions [23,24], composite distributions with threshold [25,26], and power-law distributions with an exponential cutoff [27,28]. However, these methods do not consider the essential foundation of the power-law. According to the BA model and its variants, the power-law nature is an inherent property exhibited from the preferential attachment rule. The model presented in this paper preserves the power-law nature to avoid manual modifications of the power-law distribution function, given by $P\left(k\right)\propto k^{-\alpha }$.

Note that the BA model has a parameter m. Jordan (2006) [29] relieved the constant m condition that, at each time, the number M of connections can change over time according to the distribution of M. The degree distribution turned out to be

$$\begin{array}{c}\hfill P\left(k\right)=\frac{2E\left[M\right(M+1\left)\mathbb{1}\right(M\le k\left)\right]}{k(k+1)(k+2)},k=1,2,\cdots .\end{array}$$

(2)

We note the following statistical property.

Theorem 1.

Suppose that M has a finite support, $M=1,\cdots ,M_{max}$. Then the degree distribution of Jordan’s model can be expressed as a mixture distribution, given by

$$\begin{array}{c}\hfill P\left(k\right)=\sum _{m=1}^{M_{max}}{w}_{m}P\left(k\right|m),k=1,2,\cdots ,\end{array}$$

(3)

where $w_m=P(M=m)$ is a mixture weight corresponding to the mth mixture component $P\left(k\right|m)$ in Equation (1), $m=1,\cdots ,M_{max}$.

Proof. 

Note that we can rewrite $P\left(k\right|m)$ and $P\left(k\right)$ as

$$\begin{array}{cc}\hfill P\left(k\right|m)& =\frac{2m(m+1)}{k(k+1)(k+2)}\mathbb{1}(k\ge m)\hfill \\ \hfill P\left(k\right)& =\sum _{m=1}^{M_{max}}{w}_m\frac{2m(m+1)}{k(k+1)(k+2)}\mathbb{1}(m\le k)\hfill \end{array}$$

from Equations (1) and (2). Then, we have

$$\begin{array}{cc}\hfill P\left(k\right)& =\sum _{m=1}^{M_{max}}{w}_m\frac{2m(m+1)}{k(k+1)(k+2)}\mathbb{1}(m\le k)\hfill \\ \hfill & =\sum _{m=1}^{M_{max}}{w}_{m}P\left(k\right|m)\hfill \end{array}$$

by $\mathrm{𝟙}(k\ge m)=\mathrm{𝟙}(m\le k)$. □

Theorem 1 suggests that the degree distribution might be expressed as a mixture distribution. For example, we consider a network where a new node connects to one or two existing nodes, with probabilities $P(M=1)=2/3$ and $P(M=2)=1/3$. Then we have the following degree distribution in a mixture form of the two BA model’s distributions with different m,

$$\begin{array}{c}\hfill P\left(k\right)=\frac{2}{3}P\left(k\right|m=1)+\frac{1}{3}P\left(k\right|m=2).\end{array}$$

Inspired by this property, we consider a mixture model as an explanation of the deviation from the power-law in the range of small degrees.

Moreover, many studies have considered the degree as a continuous variable using continuity assumptions [30,31]. This approach may mislead researchers since the degree is discrete-valued. Therefore, a discrete power-law distribution, called a truncated zeta distribution, is used in this paper.

In this study, we propose a mixture model of truncated zeta distributions for the analysis of degree distributions. The proposed model covers the entire range of the degree distribution through a mixture of truncated zeta distributions while maintaining the scale-free nature of a network. We can characterize the degree distribution more accurately through the discrete version of the power-law distribution. In addition, we present the maximum likelihood estimation algorithm along with a model selection method. A simulation study examines the validity of the proposed estimation procedure. In addition, real collaboration networks are investigated with the proposed model to describe the characteristics of the degree distribution.

We focus on analyzing actual scientific collaboration networks and have made significant advancements compared to the previous work in Jung and Phoa (2020) [32]. The major improvements are as follows:

• We detected some inconsistency in the scientific collaboration data. For example, “Smith, James,” “Smith, John,” and “Smith, Jacob” are all stated as “Smith, J” before 2007. Hence, we change the period of data to avoid the author name inconsistency for accurate inferences.
• Moreover, a more elaborate analysis of the real data is conducted with noteworthy interpretations.
• The validity of the presented algorithm is addressed with Monte Carlo simulations.
• Extensive comparison studies are performed to show the superiority of the proposed model. We compared the proposed model with generalized Pareto models as well as base models that lack discreteness or mixture nature.
• We provide more detailed explanations throughout the paper.

The rest of the paper is organized as follows. The continuous and discrete power-law distributions are defined in Section 2, and the proposed mixture model of truncated zeta distributions is defined in Section 3. Section 4 presents the estimation method of the mixture model, and the validity of the estimation procedure is demonstrated in Section 5. In Section 6, we analyze the scientific collaboration network by applying the proposed mixture model with interpretations. Section 7 concludes the paper.

2. The Power-Law Distribution

2.1. Continuous Power-Law Distribution

The probability density function (pdf) of the continuous power-law distribution parameterized by the power-law exponent $\alpha >1$ and the minimum value $l>0$, denoted by $PL(\alpha ,l)$, is given by [7]

$$\begin{array}{c}\hfill f\left(x\right|\alpha ,l)=(\alpha -1)l^{\alpha -1}{x}^{-\alpha },x\ge l.\end{array}$$

(4)

Note that the support is real values greater than or equal to l.

The complementary cumulative distribution function is useful to describe the tail of the power-law distribution, expressed as

$$\begin{array}{c}\hfill P(X\ge x)={\left(\frac{x}{l}\right)}^{-\alpha +1},x\ge l.\end{array}$$

Its moments are given by

$$\begin{array}{c}\hfill E\left[X^m\right]=\frac{\alpha -1}{\alpha -1-m}{l}^m,\end{array}$$

provided $\alpha >m+1$. We can deduce that $PL(\alpha ,l)$ has the mean and variance given by

$$\begin{array}{c}\hfill E\left[X\right]=\frac{\alpha -1}{\alpha -2}l,Var\left[X\right]=\frac{\alpha -1}{(\alpha -3){(\alpha -2)}^2}{l}^2,\end{array}$$

where the mean and variance are well-defined for $\alpha >2$ and $\alpha >3$, respectively.

The continuous power-law distribution has been widely used in the analysis of the degree distribution. To analyze the discrete-valued variable degree, we need an approximation method. Setting a constant c, $0\le c\le 1$, for the correction of continuity, the degree distribution can be approximated by

$$\begin{array}{c}\hfill f\left(k\right|\alpha ,l)\approx {\int }_{k-c}^{k+1-c}f\left(x\right|\alpha ,l)dx\end{array}$$

(5)

for an integer value k.

One of the most common approaches is to round values to the nearest integer, which corresponds to $c=0.5$ [7,33]. This rounding approach is acceptable when considering the tail part of the power-law distribution. However, if k is small, the constant c that satisfies the exact equation of (5) may be considerably less than $0.5$, and it should be avoided. According to Clauset et al. (2009) in [7], the rounding approach is reasonable for $k>6$.

Since the node degree is usually small, the approximation may lead to misunderstanding when performing statistical analysis on the degree distribution, such as generating node degrees and fitting the distribution to real data. We consider the exact version of the discrete power-law in the next subsection.

2.2. Truncated Zeta Distribution

The truncated zeta distribution, denoted by $TZ(\alpha ,l)$, is a discrete form of the power-law distribution. Parameters are the same as the continuous power-law distribution in which $\alpha >1$ is the power-law exponent and $l>0$ is the minimum value. The probability mass function (pmf) of $TZ(\alpha ,l)$ is given by [7]

$$\begin{array}{c}\hfill g\left(k\right|\alpha ,l)=\frac{1}{\zeta (\alpha ,l)}{k}^{-\alpha },k=l,l+1,\cdots ,\end{array}$$

(6)

where $\zeta (·,·)$ is the Hurwitz zeta function

$$\begin{array}{c}\hfill \zeta (\alpha ,l)=\sum _{k=l}^{\infty }{k}^{-\alpha },\end{array}$$

which can be regarded as the normalizing constant of the distribution. The Hurwitz zeta function in Equation (6) corresponds to the continuous counterpart $1/(\alpha -1)l^{\alpha -1}$ in Equation (4) via the upper Riemann sum approximation, expressed as

$$\begin{array}{c}\hfill \zeta (\alpha ,l)\gtrsim {\int }_l^{\infty }{x}^{-\alpha }dx=1/(\alpha -1)l^{\alpha -1}.\end{array}$$

The complementary cumulative distribution function is given by

$$\begin{array}{c}\hfill P(K\ge k)=\frac{\zeta (\alpha ,k)}{\zeta (\alpha ,l)},k\ge l.\end{array}$$

Next, the moments of the truncated zeta distribution are expressed as

$$\begin{array}{c}\hfill E\left[K^m\right]=\frac{\zeta (\alpha -m,l)}{\zeta (\alpha ,l)},\alpha >m+1.\end{array}$$

We can straightforwardly derive the mean and variance as

$$\begin{array}{c}\hfill E\left[X\right]=\frac{\zeta (\alpha -1,l)}{\zeta (\alpha ,l)},Var\left[X\right]=\frac{\zeta (\alpha ,l)\zeta (\alpha -2,l)-{\zeta }^2(\alpha -1,l)}{{\zeta }^2(\alpha ,l)},\end{array}$$

provided $\alpha >2$ for the mean and $\alpha >3$ for the variance.

Figure 1 shows some pmfs of $TZ(\alpha ,l)$ when $\alpha =2.50$. We can check that the straight line of a log-log plot can be strong evidence of a power-law behavior.

3. Truncated Zeta Mixture Model

We consider the finite mixture model of truncated zeta distributions by fixing the power-law exponent $\alpha$ for mixture components while varying minimum values to produce a mixture of truncated zeta distributions. The probability mass function is represented as

$$\begin{array}{c}\hfill p\left(k\right|\alpha ,L,w)=\sum _{l=1}^L{w}_{l}g\left(k\right|\alpha ,l),k=1,2,\cdots ,\end{array}$$

(7)

where $g\left(k\right|\alpha ,l)$ is the pmf of $TZ(\alpha ,l)$, and L is the number of mixture components. Mixture weights $w=(w_1,w_2,\cdots ,w_L)$

$$\begin{array}{c}\hfill w_l\ge 0,l=1,2,\cdots ,L-1,w_L>0,and\sum _{l=1}^L{w}_l=1.\end{array}$$

In this paper, we assume that the minimum value l is equal to 1, but it can be modified according to the data. The tail of most real networks follows the power-law distribution, and Equation (7) has the exact power-law behavior for sufficiently large degrees.

Theorem 2.

For k larger than or equal to L, the truncated zeta mixture distribution in Equation (7) has the exact power-law relationship, given by

$$\begin{array}{c}\hfill p\left(k\right|\alpha ,L,w)\propto k^{-\alpha }.\end{array}$$

Proof. 

By using the pmf of $TZ(\alpha ,l)$, we can write

$$\begin{array}{c}\hfill p\left(k\right|\alpha ,L,w)=\left(\sum _{l=1}^L\frac{w_l}{\zeta (\alpha ,l)}\right)k^{-\alpha }.\end{array}$$

Since the term inside the bracket is independent with k, the pmf of the mixture is proportional to $k^{-\alpha }$. □

Mixture models may suffer from the non-identifiability issue even for finite mixtures. The following theorem proves that the proposed truncated zeta mixture model is identifiable.

Theorem 3.

The mixture distribution Equation (7) is identifiable with respect to α, L, and w.

Proof. 

Let $p\left(k\right|{\alpha }_1,L_1,w_1)$ and $p\left(k\right|{\alpha }_2,L_2,w_2)$ be the mixture distributions of ${\alpha }_1$, $L_1$, $w_1=(w_{11},w_{12},\cdots ,w_{1L})$ and ${\alpha }_2$, $L_2$, $w_2=(w_{21},w_{22},\cdots ,w_{2L})$, respectively. Suppose that $p\left(k\right|{\alpha }_1,L_1,w_1)$ and $p\left(k\right|{\alpha }_2,L_2,w_2)$ are identical, i.e., $p\left(k\right|{\alpha }_1,L_1,w_1)=p\left(k\right|{\alpha }_2,L_2,w_2)$ for all $k=1,2,\cdots$. Further, we define the slope function $s\left(k\right|\alpha ,L,w)$ of the log-log degree distribution, given by

$$\begin{array}{c}\hfill s\left(k\right|\alpha ,L,w)=\frac{lnp(k+1|\alpha ,L,w)-lnp(k|\alpha ,L,w)}{ln(k+1)-ln\left(k\right)},k=1,2,\cdots .\end{array}$$

The equality of the two mixture distributions give the identical slope function, $s\left(k\right|{\alpha }_1,L_1,w_1)=s\left(k\right|{\alpha }_2,L_2,w_2)$ for all $k=1,2,\cdots$. We then have $\alpha =s\left(k\right|\alpha ,L,w)$ for the sufficiently large k$(\ge max\{L_1,L_2\})$. Therefore, we obtain ${\alpha }_1={\alpha }_2$. The number of mixture components L is the largest integer k such that $s(k-1|\alpha ,L,w)\ne \alpha$, and we also have $L_1=L_2$. Let L$(=L_1=L_2)$ and $\alpha$$(={\alpha }_1={\alpha }_2)$ be the common number of mixture components and the power-law exponent.

Using $p\left(k\right|{\alpha }_1,L_1,w_1)=p\left(k\right|{\alpha }_2,L_2,w_2)$ for $k=1,2,\cdots ,L$, we have the following L equations:

$$\begin{array}{cc}\hfill w_{11}g\left(1\right|\alpha ,1)& =w_{21}g\left(1\right|\alpha ,1),\hfill \\ \hfill w_{11}g\left(2\right|\alpha ,1)+w_{12}g\left(2\right|\alpha ,2)& =w_{21}g\left(2\right|\alpha ,1)+w_{22}g\left(2\right|\alpha ,2),\hfill \\ \hfill ⋮\\ \hfill w_{11}g\left(L\right|\alpha ,1)+\cdots +w_{1L}g\left(L\right|\alpha ,L)& =w_{21}g\left(L\right|\alpha ,1)+\cdots +w_{2L}g\left(L\right|\alpha ,L).\hfill \end{array}$$

By solving these equations, we get $w_{1l}=w_{2l}$ for $l=1,2,\cdots ,L$. Thus, the mixture of truncated zeta distributions is identifiable. □

In Figure 2, we depict some log-log plots of mixture distributions. We can see that this model can handle empirical degree distributions that do not follow the power-law distribution at small degrees.

4. Estimation Algorithm

We use the Expectation-Maximization (EM) algorithm to estimate the exponent parameter $\alpha$ and mixture weights w for a given number of mixture components L. Let $k=(k_1,k_2,\cdots ,k_N)$ be the observed data, and $z_n$ be the membership of $k_n$, where the membership $z_n$ is assigned as $z_n=l$ if $k_n$ is from the lth mixture component $TZ(\alpha ,l)$. We consider the membership variable $z=(z_1,z_2,\cdots ,z_N)$ as missing. Let $\theta =(\alpha ,w)=(\alpha ,w_1,\cdots ,w_L)$ be the parameters of the mixture model.

The complete-data likelihood function is given by

$$\begin{array}{cc}\hfill \mathcal{L}\left(\theta \right|k,z)& =\prod _{n=1}^{N}p\left(z_n\right|w)g\left(k_n\right|\alpha ,z_n)\hfill \\ \hfill & =\prod _{n=1}^N{w}_{z_n}\frac{1}{\zeta (\alpha ,z_n)}{k}_n^{-\alpha },k_n=z_n,z_n+1,\cdots .\hfill \end{array}$$

We proceed by taking the logarithms of both sides to have the complete-data log-likelihood function:

$$\begin{array}{c}\hfill ln\mathcal{L}\left(\theta \right|k,z)=\sum _{n=1}^N\left(ln{w}_{z_n}-ln\zeta (\alpha ,z_n)-\alpha ln{k}_n\right).\end{array}$$

We define $Q\left(\theta \right|{\theta }^{\ast })$ as the expected value of the log-likelihood given the observed data k and the current parameter estimate ${\theta }^{\ast }=({\alpha }^{\ast },w^{\ast })$, which can be expressed as

$$\begin{array}{cc}\hfill Q\left(\theta \right|{\theta }^{\ast })& =E\left[ln\mathcal{L}\left(\theta \right|k,Z)|k,{\theta }^{\ast }\right]\hfill \\ \hfill & =\sum _{n=1}^N\sum _{l=1}^L\gamma (l,n,{\theta }^{\ast })\left(ln{w}_l-ln\zeta (\alpha ,l)-\alpha ln{k}_n\right),\hfill \end{array}$$

where $\gamma (l,n,{\theta }^{\ast })$ is the membership responsibility of the nth observation $k_n$ corresponding to the lth mixture component $TZ(\alpha ,l)$. They are defined by the posterior probabilities of mixture component memberships for each observation,

$$\begin{array}{cc}\hfill \gamma (l,n,{\theta }^{\ast })& =P(z_n=l|k_n,{\theta }^{\ast })\hfill \\ \hfill & =\frac{w_l^{\ast }g\left(k_n\right|{\alpha }^{\ast },l)}{{\sum }_{l^{\prime }=1}^L{w}_{l^{\prime }}^{\ast }g\left(k_n\right|{\alpha }^{\ast },l^{\prime })},l=1,2,\cdots ,L,n=1,2,\cdots ,N.\hfill \end{array}$$

(8)

The E-step computes membership responsibilities.

In the M-step, a new parameter estimate $\check{\theta }={\mathrm{argmax}}_{\theta }Q\left(\theta \right|{\theta }^{\ast })$ is computed using the quantity previously computed in the E-step. To find $\check{w}$, we need to solve the following optimization problem:

$$\begin{array}{cc}\hfill Maximize& \sum _{n=1}^N\sum _{l=1}^L\gamma (l,n,{\theta }^{\ast })ln{w}_l,\hfill \\ \hfill Subjectto& \sum _{l=1}^L{w}_l=1,and{w}_l\ge 0,l=1,2,\cdots ,L.\hfill \end{array}$$

The Lagrange multiplier method yields

$$\begin{array}{c}\hfill {\check{w}}_l=\frac{1}{N}\sum _{n=1}^N\gamma (l,n,{\theta }^{\ast }),l=1,2,\cdots ,L.\end{array}$$

(9)

Next, $\check{\alpha }$ can be found in the partial derivative of Q with respect to $\alpha$, given by

$$\begin{array}{c}\hfill \frac{\partial Q\left(\theta \right|{\theta }^{\ast })}{\partial \alpha }=-\sum _{n=1}^N\sum _{l=1}^L\gamma (l,n,{\theta }^{\ast })\left(\frac{{\zeta }_{\alpha }(\alpha ,l)}{\zeta (\alpha ,l)}+ln{k}_n\right).\end{array}$$

Here, ${\zeta }_{\alpha }(\alpha ,l)$ is the partial derivative of the Hurwitz zeta function with respect to $\alpha$, given by

$$\begin{array}{c}\hfill {\zeta }_{\alpha }(\alpha ,l)=\frac{\partial \zeta (\alpha ,l)}{\partial \alpha }=-\sum _{k=l}^{\infty }(lnk)k^{-\alpha }.\end{array}$$

Then, the equation

$$\begin{array}{c}\hfill \frac{\partial Q\left(\theta \right|{\theta }^{\ast })}{\partial \alpha }=0\end{array}$$

(10)

gives the desired $\check{\alpha }$. Unfortunately, a closed-form solution does not exist in general. In this paper, we employ Brent’s method [34] to solve Equation (10) with respect to $\alpha$.

The two steps are necessarily repeated until the convergence obtains the final parameter estimate $\widehat{\theta }$. The process is summarized in Algorithm 1.

In order to select the number of mixture components L, we employ the Bayesian information criterion (BIC) considering the trade-off between the goodness-of-fit and the complexity of the model. BIC is given by the following formula:

$$\begin{array}{c}\hfill BIC=LlnN-2\sum _{n=1}^{N}lnp\left(k_n\right|\alpha ,L,w),\end{array}$$

where $\alpha$ and w are the obtained estimated parameters given the number of mixture components L. We choose L giving the smallest BIC, where the candidates of L are the integers from 1 to the minimum of the two values: 100 and the nearest integer to 0.90$\times \left(maximumdegree\right)$.

Algorithm 1: EM Algorithm

5. Monte Carlo Simulation

We study the validity of the presented estimation methods using the synthetic data. We consider three values of true power-law exponents $\alpha =2.50,3.00,3.50$ and also three values of the number of mixture components $L=2,4,6$. For each pair of $\alpha$ and L, we generate 1000 samples from the finite mixture of truncated zeta distributions in Equation (7). Each process is repeated 30 times. Mixture weights $w_l$, $l=1,2,\cdots ,L$ are set to $1/L$ for each dataset.

We assume that the true number of components is given. Algorithm 1 is applied to each dataset to obtain the parameter estimate $\widehat{\theta }=(\widehat{\alpha },\widehat{w})$.

Table 1. Estimates of $\theta$ of the EM algorithm applied to the synthetic data with the true number of mixture components. We present the average (standard deviation in parentheses) values of estimated parameters $\widehat{\theta }=(\widehat{\alpha },\widehat{w})$ of 30 datasets for each case.

$\mathit{\alpha }$	L	${\mathit{w}}_{\mathit{l}}$	$\widehat{\mathit{\alpha }}$	$\widehat{\mathit{w}}$
2.50	2	0.50	$2.{51}_{\left(0.05\right)}$	$[0.{49}_{\left(0.02\right)},0.{51}_{\left(0.02\right)}]$
	4	0.25	$2.{51}_{\left(0.06\right)}$	$[0.{25}_{\left(0.01\right)},0.{25}_{\left(0.02\right)},0.{25}_{\left(0.03\right)},0.{25}_{\left(0.02\right)}]$
	6	0.17	$2.{50}_{\left(0.07\right)}$	$[0.{17}_{\left(0.02\right)},0.{16}_{\left(0.02\right)},0.{17}_{\left(0.03\right)},0.{18}_{\left(0.04\right)},0.{16}_{\left(0.04\right)},0.{16}_{\left(0.04\right)}]$
3.00	2	0.50	$3.{01}_{\left(0.09\right)}$	$[0.{49}_{\left(0.02\right)},0.{51}_{\left(0.02\right)}]$
	4	0.25	$3.{03}_{\left(0.10\right)}$	$[0.{25}_{\left(0.01\right)},0.{24}_{\left(0.02\right)},0.{25}_{\left(0.03\right)},0.{26}_{\left(0.03\right)}]$
	6	0.17	$3.{03}_{\left(0.11\right)}$	$[0.{17}_{\left(0.01\right)},0.{17}_{\left(0.02\right)},0.{17}_{\left(0.03\right)},0.{17}_{\left(0.03\right)},0.{16}_{\left(0.03\right)},0.{17}_{\left(0.03\right)}]$
3.50	2	0.50	$3.{50}_{\left(0.14\right)}$	$[0.{50}_{\left(0.02\right)},0.{50}_{\left(0.02\right)}]$
	4	0.25	$3.{51}_{\left(0.13\right)}$	$[0.{25}_{\left(0.02\right)},0.{25}_{\left(0.02\right)},0.{26}_{\left(0.02\right)},0.{25}_{\left(0.02\right)}]$
	6	0.17	$3.{50}_{\left(0.13\right)}$	$[0.{17}_{\left(0.01\right)},0.{17}_{\left(0.01\right)},0.{17}_{\left(0.02\right)},0.{17}_{\left(0.02\right)},0.{15}_{\left(0.03\right)},0.{17}_{\left(0.03\right)}]$

presents the parameter estimation result. We find that the average values of estimated parameters are very close to the true parameters, implying that the EM algorithm works well in estimating the exponent parameter $\alpha$ and weights w.

Next, we assume that the number L of mixture components is not given. We will check whether the presented algorithm can find the appropriate number of mixture components. Note that BIC is used as a model selection criterion for the generated datasets.

Table 2. The estimation result of L applied to the maximum likelihood method. $\widehat{L}$ is selected with the smallest BIC for each dataset. The average, standard deviation, and count of $\widehat{L}$ over 30 datasets are presented.

$\mathit{\alpha }$	L	Average $\widehat{\mathit{L}}$	St.Dev. $\widehat{\mathit{L}}$	Count $\widehat{\mathit{L}}$
2.50	2	2.03	0.18	{2: 29, 3: 1}
	4	4.00	0.00	{4: 30}
	6	5.87	0.34	{6: 26, 5: 4}
3.00	2	2.00	0.00	{2: 30}
	4	4.00	0.00	{4: 30}
	6	6.00	0.00	{6: 30}
3.50	2	2.00	0.00	{2: 30}
	4	4.00	0.00	{4: 30}
	6	5.97	0.18	{6: 29, 5: 1}

shows the result of determining L. The result indicates that the maximum likelihood method yields a reasonable estimation result in finding L.

6. Application: Collaboration Networks

6.1. The Data

We study the scientific collaboration network obtained from the Web of Science, where a large-scale database collects the information of all published scientific articles in the world. Among all 275 disciplines, we randomly choose five, which are Biotechnology and Applied Microbiology, Computer Science, Environmental Science, Materials Science, and Physical Chemistry, for demonstration. We reorganize the Web of Science database into a network data structure such that the nodes of networks are authors, and two authors are connected by an undirected link if there is at least one paper co-authored by them. We employ the data from 2007 to 2016 as the inconsistency of author’s names is observed before the year 2007. The data from 2007 to 2009 are used to accumulate data so that the dependence of node degrees can be ignorable. We analyze the degree distribution of the data from 2010 to 2016.

Table 3. The summary statistics of collaboration networks in the year 2016. For each field, we present the number of authors (nodes) and links between them. The mean, median, standard deviation, and maximum of degrees are also presented.

Field	Number of		Degree Distribution
	Nodes	Links	Mean	Median	St.Dev.	Max.
Biotechnology & A. M.	729,478	3,977,919	10.91	7.00	19.52	2250
Computer Science	528,267	1,765,283	6.68	4.00	15.19	989
Environmental Science	680,924	3,291,780	9.67	5.00	17.42	1149
Materials Science	1,154,908	6,861,189	11.88	6.00	28.47	3801
Physical Chemistry	727,213	4,150,183	11.41	6.00	23.42	2260

shows the summary statistics in the year 2016.

6.2. Application of the Truncated Zeta Mixture Model

We apply the presented estimation method to degree distributions from 2010 to 2016.

In Figure 3, we plot degree distributions and the estimated fitting lines of the proposed model in the years 2001 and 2016. The plots suggest that the truncated zeta mixture shows a good performance in fitting degree distributions. The beginning points (see the vertical lines in Figure 3) of power-law behaviors are reasonably estimated via $\widehat{L}$. This result shows the usefulness of the proposed model for the degree distribution that deviates from the power-law distribution on a small degree part.

Figure 4 shows the temporal changes of $\widehat{L}$ and $\widehat{\alpha }$. The estimated number L of mixture components exhibits a rising tendency, and $\widehat{\alpha }$ shows an opposite trend. They constantly move towards large L and small $\alpha$ regions for all fields, indicating that the stabilization of the degree distribution has not yet been achieved. Many network scientists [1,13,14] have concentrated on the stationary or the converged degree distribution. The result implies that, however, non-stationary network models are of great importance, such as acceleration models [15]. Moreover, many works in real data analysis have focused on the power-law exponent of a snapshot of a network. However, the temporal variation of $\widehat{\alpha }$ tells us that the significance of temporal models can account for the temporal change of the power-law exponent.

It should be noted that $\widehat{L}$ tends to approach different values across fields. According to Equation (3) and the relevant interpretation in Section 1, the number of mixture components L and mixture weights w are closely related to the distribution of the number of links in the system. Therefore, $\widehat{L}$ could be a measure to the distribution of the number of links. On the other hand, $\widehat{\alpha }$ seems to converge a value near $2.80$ for all fields, suggesting that $\alpha$ could be a network-specific quantity instead of a field-specific quantity.

We now focus on the field of Computer Science.

Table 4. The results of $\widehat{L}$, $\widehat{\alpha }$, $\widehat{w}$, and BIC using the proposed model for the discipline of Computer Science. We also present the number of new links and nodes as well as the mean degree over time.

Year	Discrete Mixture								Number of New		Mean Deg.
	$\widehat{\mathit{L}}$	$\widehat{\mathit{\alpha }}$	${\widehat{\mathit{w}}}_1$	${\widehat{\mathit{w}}}_2$	${\widehat{\mathit{w}}}_3$	${\widehat{\mathit{w}}}_4$	${\widehat{\mathit{w}}}_5$	BIC	Nodes	Links
2010	11	3.07	0.18	0.32	0.25	0.12	0.06	1,010,549	-	-	4.70
2011	13	3.06	0.17	0.30	0.25	0.12	0.06	1,241,555	43,237	132,560	4.94
2012	14	3.06	0.16	0.30	0.25	0.13	0.07	1,486,634	45,004	144,305	5.16
2013	13	2.84	0.16	0.30	0.26	0.13	0.07	1,813,101	56,392	256,646	5.78
2014	16	2.86	0.15	0.29	0.26	0.14	0.07	2,137,666	56,091	215,236	6.03
2015	16	2.74	0.15	0.29	0.26	0.14	0.07	2,481,780	57,317	277,983	6.48
2016	16	2.72	0.14	0.28	0.26	0.14	0.07	2,810,563	55,800	234,714	6.68

presents the result of applying the model to the field of Computer Science in more detail. We can observe an interesting pattern in the estimated mixture weights, where ${\widehat{w}}_1$ and ${\widehat{w}}_2$ show decreasing trends whilst ${\widehat{w}}_3$, ${\widehat{w}}_4$, and ${\widehat{w}}_5$ show the opposite. According to Jordan’s model [29] and Equation (3), mixture weights have much to do with the number of newly made links. The decreasing trend of $w_1$ and $w_2$ and the increasing trend of $w_3$, $w_4$, ⋯ indicates the increasing number of links over time. As shown in Table 4, the number of created links tends to increase over time. Since there are many new links compared to new nodes (authors), the average degree gradually increased from 4.70 in 2010 to 6.68 in 2016. The increase in the average degree also explains that the state of this network is still evolving. With the rapid advancement in technology and science, many publications have been produced by researchers. In particular, Computer Science is making greater progress due to recently emerging areas: artificial intelligence and big data. The proposed model tries to explain the increasing mean degree in two ways: decreasing power-law exponents $\alpha$ and an appropriate change in the mixture weight, suggesting that the proposed model is helpful to describe the change of the network pattern.

6.3. Comparison to Other Models

Our model is developed to deal with two essential characteristics of the degree distribution: the non-power-law behavior at small degrees and the discreteness. We study the superiority of the proposed model to base models that lack one of these characteristics.

We first consider the standard discrete power-law distribution, $TZ(\alpha ,1)$. The estimates of the power-law exponent $\alpha$ are obtained by fitting the data into $TZ(\alpha ,1)$, and the result is presented in

Table 5. The results of $\widehat{L}$, $\widehat{\alpha }$, $\widehat{w}$, and BIC using the continuous mixture model for the discipline of Computer Science. Estimated $\alpha$ and BIC for the standard discrete power-law distribution $TZ(\alpha ,1)$ are also presented.

Year	Continuous Mixture						Zeta
	$\widehat{\mathit{L}}$	$\widehat{\mathit{\alpha }}$	${\widehat{\mathit{w}}}_1$	${\widehat{\mathit{w}}}_2$	${\widehat{\mathit{w}}}_3$	BIC	$\widehat{\mathit{\alpha }}$	BIC
2010	4	2.30	0.20	0.40	0.30	1,053,289	1.60	1,170,929
2011	4	2.29	0.19	0.39	0.30	1,290,015	1.59	1,436,579
2012	5	2.29	0.18	0.38	0.30	1,540,812	1.57	1,719,325
2013	5	2.26	0.17	0.37	0.30	1,871,783	1.56	2,090,369
2014	5	2.24	0.16	0.36	0.30	2,201,834	1.55	2,462,841
2015	5	2.22	0.16	0.35	0.31	2,550,130	1.54	2,852,831
2016	5	2.21	0.15	0.34	0.30	2,884,064	1.53	3,229,651

. The $\widehat{\alpha }$ estimates of the standard discrete power-law distribution are smaller than those of the mixture distribution. As we can observe in Figure 5, the small-degree data that deviates from the power-law makes the exponent small. The estimated fitting lines indicate that the standard discrete power-law distribution is inappropriate to describe the degree distribution.

Next, the degree distribution is applied to the mixture of the continuous power-law distribution using the rounding method with the constant $c=0.5$ of the continuity correction. The continuous version of the mixture of truncated zeta distributions can be constructed by the substitution of $TZ(\alpha ,l)$ to $PL(\alpha ,l-c)$. In addition, we imitate the method in Section 4 for the continuous mixture to compare the performance of the continuous mixture with the discrete mixture. The estimation procedure is identical to the case of the discrete mixture model. Table 5 presents the estimation result of $\alpha$ and mixture weights w. The estimates $\widehat{L}$ and $\widehat{\alpha }$ are much smaller in continuous mixture models due to the non-exactness of the continuous model. In addition, estimated mixture weights $\widehat{w}$ are considerably different from the discrete mixture. According to Equation (3), mixture weights are involved in the distribution of the number of links. Therefore, we should use the discrete version of the power-law to determine mixture weights correctly. Figure 5 shows that the fitting lines of the continuous mixture seem to deviate from empirical distributions.

As we can see in Table 4 and Table 5, the smallest BIC values are achieved in the proposed model as well. Thus, we can conclude that the proposed model outperforms the continuous model as well as the standard discrete power-law model.

Next, we compare the proposed model with generalized Pareto distributions in which all degree ranges are covered. We use both continuous and discrete types of generalized Pareto distributions. The complementary cumulative density function of the continuous generalized Pareto distribution $GPD(\mu =0.5,\sigma ,\xi )$ is given by

$$\begin{array}{c}\hfill P(X\ge x)={\left(1+\frac{\xi (x-\mu )}{\sigma }\right)}^{-1/\xi },x\ge \mu .\end{array}$$

There are few studies concerning the discrete version of the generalized Pareto distribution. We here use the model in Prieto et al. (2013) [35], expressed as $DGP(\alpha ,\lambda ,\mu =1)$. This distribution has advantages over the continuous distribution since the node degree is discrete.

Table 6. The results of discrete and continuous generalized Pareto distributions applied to the field of Computer Science.

Year	Continuous Generalized Pareto			Discrete Generalized Pareto
	$\widehat{\mathit{\sigma }}$	$\widehat{\mathit{\xi }}$	BIC	$\widehat{\mathit{\alpha }}\left({10}^4\right)$	$\widehat{\mathit{\lambda }}\left({10}^{-6}\right)$	BIC
2010	3.54	0.14	1,032,317	2.51	9.53	1,043,227
2011	3.68	0.16	1,267,627	5.42	4.17	1,282,398
2012	3.83	0.16	1,517,145	5.65	3.81	1,535,652
2013	3.93	0.21	1,852,799	5.69	3.34	1,911,678
2014	4.09	0.22	2,183,341	1.40	12.92	2,249,684
2015	4.20	0.25	2,535,774	5.63	2.98	2,633,601
2016	4.31	0.26	2,871,026	3.49	4.65	2,980,178

shows the results of discrete and continuous generalized Pareto distributions applied to the field of Computer Science. In Figure 6, we plot fitting results in the years 2010 and 2016.

Interestingly, Figure 6 suggests that the continuous version has better fitting results. The discrete version has difficulty in explaining a range of large degrees. We can see that our model performs better than the two generalized Pareto distributions, as well indicated by the BIC values (see Table 4 and Table 6).

7. Concluding Remark

Inspired by Jordan’s model [29], a novel mixture model for the degree distribution is proposed to describe the entire range of degrees while maintaining the power-law or the scale-free property of a network. The truncated zeta distribution enables us to analyze discrete distributions for accuracy purposes. The parameter estimation procedure is presented along with the model selection criterion for determining the number of mixture components. A simulation study shows the validity of the suggested estimation procedure. The practical performance of the model is studied through the comparison analysis with the other techniques.

We perform the real data analysis on five disciplines of the scientific collaboration data obtained from the Web of Science. We observe the increasing tendency in the number of mixture components and the decreasing tendency in the power-law exponent. In addition, mixture weights change over time. It can be suggested from these results that the analyzed networks are still in an evolving state, highlighting the practical importance of non-stationary temporal network models. The non-convergence of the degree distribution might be due to the short-term analysis performed. Determining whether the collaboration network will stabilize the equilibrium remains as future work.

We can observe power-law distributions not only in the degree distribution but also in sandpile avalanches, species extinctions, city sizes, and so on. The proposed model could be useful when (i) the distribution does not follow the power-law only in small values while the power-law is suitable for large values, (ii) the background knowledge does not support the manual modification of the power-law relationship, or (iii) a mixture distribution can be regarded as reasonable for describing data.