A Fusion Link Prediction Method Based on Limit Theorem

Featured Application

The proposed theory can guide the design of combination methods, and the proposed TLF method can fuse multiple similarity indices in link prediction.

Abstract

The theoretical limit of link prediction is a fundamental problem in this field. Taking the network structure as object to research this problem is the mainstream method. This paper proposes a new viewpoint that link prediction methods can be divided into single or combination methods, based on the way they derive the similarity matrix, and investigates whether there a theoretical limit exists for combination methods. We propose and prove necessary and sufficient conditions for the combination method to reach the theoretical limit. The limit theorem reveals the essence of combination method that is to estimate probability density functions of existing links and nonexistent links. Based on limit theorem, a new combination method, theoretical limit fusion (TLF) method, is proposed. Simulations and experiments on real networks demonstrated that TLF method can achieve higher prediction accuracy.

1. Introduction

Limit theory is a basic theoretical issue and has attracted wide interest across many fields. On the 100th anniversary of its foundation, Science raised 125 unresolved scientific questions, and many of these issues related to limit theory [1]. Link prediction predicts missing links in current networks and new or dissolution links in future networks [2]. With continuous improvement of link prediction methods and, the theoretical limit of link prediction has attracted considerable research interest [3].

Considering structure or attribute features, link prediction methods based on classification have been proposed by computer science community [4,5]. Subsequently, more insightful methods of network structure, such as similarity based methods [6], have become a focus, these methods pay more attention to the physical meaning. At the same time, similarity index fusion methods are springing up [7,8]. Recent years, with the development of deep learning, some deep features extraction methods have been proposed [9,10], the fusion of structure and attribute information has been attached importance again [11,12,13,14]. These methods have strong consistency. We divide link prediction method into single and combination methods, based on whether they use multidimension information, and whether they define the relation of multidimension information directly. For example, single methods, such as RA index [15], which defines the relation of common neighbors and degree of nodes directly; and classification based methods, index fusion methods, fusion of structure and attribute information methods belong to link prediction combination methods.

Most combination methods perform better than single methods that will be fused, and are robust to many network types. However, what is the reason for this improved accuracy and robustness, and is there a theoretical limit for combination methods? This paper proposes the mathematic description of combination methods, and obtains the necessary and sufficient conditions for theoretical limit. The limit theorem also has important practical application value. It reveals the ultimate goal of combination methods that is to estimate probability density functions of existing links and nonexistent links. Thus, an appropriate form of the transformation function could be selected from the complete set. Based on the limit theorem, a new combination method, theoretical limit fusion (TLF) method, is proposed. We use the Parzen kernel method [16] of destiny estimation in the TLF method. Simulations and empirical studies have shown that TLF method can achieve higher prediction accuracy.

Section 2 introduces a mathematical description for the theoretical limit of combination methods and evaluation metrics for link prediction. Section 3 proposes and proves necessary and sufficient conditions for the theoretical limit of combination methods. Section 4 proposes a fusion link prediction method based on limit theorem (TLF method). Section 5 provides simulation examples for limit theorem and proposed TLF method with other combination methods, and gives comparison experiments in real networks. Section 6 and Section 7 discuss some results and conclude the paper.

2. Problem Description and Evaluation Metrics

2.1. Problem Description

Given a network $G(V,E)$ at time t, where $V=\left\{v_1,v_2,\cdots ,v_N\right\}$ is the set of nodes and $E=\left\{e_1,e_2,\cdots ,e_M\right\}$ is the set of links. The observed links, E, are randomly divided into training, $E^T$, and probe, $E^P$, sets, where $E=E^T\cup E^P$ and $E^T\cap E^P=\varnothing$. Link prediction aims to predict missing links at current network or new links for a future time $t\prime (t\prime >t)$ [2]. Link prediction combination methods fuse several similarity indices and obtain a synthetic index and can be described in mathematic as follows. Let $X={(X_1,X_2,\cdots ,X_n)}^T$ be the scores of existing links as given by n structural similarity indices, and follow probability density function (pdf) $f(\mathit{x})=f(x_1,x_2,\cdots ,x_n)$. Let $Y={(Y_1,Y_2,\cdots ,Y_n)}^T$ be the scores of nonexistent links as n structural similarity indices, and follow $g(\mathit{x})=g(x_1,x_2,\cdots ,x_n)$. We need to find the transformation function, $l(\mathit{x})$, and obtain the synthetic score, $X=l(X)$, $Y=l(Y)$ that maximizes evaluation metrics. Figure 1 is the diagram of combination methods.

Figure 1. Combination methods.

2.2. Evaluation Metrics

Let the synthetic score $X=l(X)$ follow pdf $f_X(x)$, and $Y=l(Y)$ follow $g_Y(x)$. X and Y are independent. We have the following metrics.

2.2.1. Area under the Receiver Operation Characteristics Curve (AUC)

A receiver operating characteristics (ROC) curve is a two-dimensional depiction of classifier performance [17]. In the field of link prediction, the ROC curve abscissa represents the probability of nonexistent links i.e., the false positive rate (FPR), when the link prediction score is greater than some threshold, $\mu$, and $\mathrm{FPR}={\int }_{\mu }^{\infty }{g}_Y(x)dx$. The ordinate represents the probability of missing links, i.e., the true positive rate (TPR), when score >$\mu$, and $\mathrm{TPR}={\int }_{\mu }^{\infty }{f}_X(x)dx$, TPR is equivalent to Recall. According to [18], AUC can be derived as

$$\begin{array}{ll}P(X>Y)& ={\displaystyle \underset{X>Y}{\iint }{f}_X(x)g_Y(y)dxdy}\\ & =\frac{1}{2}{\displaystyle \underset{X>Y}{\iint }{f}_X(x)g_Y(y)dxdy}+\frac{1}{2}\left(1-{\displaystyle \underset{X\le Y}{\iint }{f}_X(x)g_Y(y)dxdy}\right)\\ & =\frac{1}{2}{\displaystyle \iint \mathrm{sgn}(x-y)f_X(x)g_Y(y)dxdy+}\frac{1}{2}\\ & =\frac{1}{2}\mathbb{E}\left[\mathrm{sgn}\left(X-Y\right)+1\right],\end{array}$$

(1)

where

$$\mathrm{sgn}(x)=\{\begin{array}{ll}1,& x>0\\ 0,& x=0\\ -1,& x<0\end{array}$$

(2)

In the real network, original data is randomly divided into training set and the probe set. Equation (1) means that for n independent comparisons, if there are n′ comparisons where the missing link returns a higher score and n″ comparisons where the missing and nonexistent links return the same score, we can obtain the algorithm expression of AUC:

$$\mathrm{AUC}=\frac{n\prime +0.5n″}{n}$$

(3)

2.2.2. Precision

Precision can be defined as the ratio of correct to (correct and error) prediction proportions when score >$\mu$, i.e.,

$$\begin{array}{ll}\mathrm{Precision}& =\frac{P({\omega }_1){\displaystyle {\int }_{\mu }^{+\infty }{f}_X(x)dx}}{P({\omega }_1){\displaystyle {\int }_{\mu }^{+\infty }{f}_X(x)dx}+P({\omega }_2){\displaystyle {\int }_{\mu }^{+\infty }{g}_Y(x)dx}}\\ & =\frac{P({\omega }_1)\mathrm{TPR}}{P({\omega }_1)\mathrm{TPR}+P({\omega }_2)\mathrm{FPR}}.\end{array}$$

(4)

In the real network, if the top L links are predicted ones, with m links being right (i.e., there are m links in E^P), then

$$\mathrm{Precision}=\frac{m}{L}$$

(5)

Owing to the imbalance of positive and negative samples, link prediction usually uses AUC metric. In application, high Precision means target links are accurate, and these links can be used directly. AUC and Precision are two important metrics in link prediction, we will study the theoretical limit using the two metrics.

3. Theoretical Limit Theorem

Theorem 1.

Let $X={(X_1,X_2,\cdots ,X_n)}^T$ and $Y={(Y_1,Y_2,\cdots ,Y_n)}^T$ be random vectors following the joint distributions $f(\mathit{x})$ and $g(\mathit{x})$, respectively, where $m\{\mathit{x}:f(\mathit{x})/g(\mathit{x})=C,g(\mathit{x})\ne 0,$ $\forall C\in ℝ\}=0$. (m represents the measure of a set.) Then the following conditions are equivalent.

(a)

A monotonically increasing function $r(x)$ exists, such that $l(\mathit{x})=r[f(\mathit{x})/g(\mathit{x})],g(\mathit{x})\ne 0$, a.e. $\mathit{x}\in {ℝ}^n$.

(b)

Transformation function $l(\mathit{x})$produces maximum AUC. If we add a condition in Theorem that prior probability of existing and nonexistent links be $P({\omega }_1)$ and $P({\omega }_2)$, respectively. Then the following conditions are equivalent to (a) and (b):

(c)

for any $\alpha$, there exists the corresponding threshold ${\mu }_l$ for transformation $l(\mathit{x})$, and satisfies $\alpha =P({\omega }_1){\int }_{{\mu }_l}^{+\infty }{f}_X(x)dx+P({\omega }_2){\int }_{{\mu }_l}^{+\infty }{g}_Y(x)dx$, such that transformation function $l(\mathit{x})$ produces maximum Precision.

Proof. 

$(\mathrm{a})\Rightarrow (\mathrm{b})$:

From the equivalent definition, AUC maximum is the maximum area under the ROC curve. For any FPR, if the TPRs corresponding to the ROC curve reach maximum, then the AUC reaches the maximum, i.e.,

$$\mathrm{FPR}={\displaystyle {\int }_{\mu }^{\infty }{g}_Y(x)dx}={\displaystyle {\int }_{E(l(\mathit{x})>\mu )}g(\mathit{x})}d\mathit{x},$$

(6)

$$\mathrm{TPR}={\displaystyle {\int }_{\mu }^{+\infty }{f}_X(x)dx=}{\displaystyle {\int }_{E(l(\mathit{x})>\mu )}f(\mathit{x})d\mathit{x}},$$

(7)

where $E\left(l(\mathit{x})>\mu \right)$ is a set $\left\{\mathit{x}\in {ℝ}^n:\mu \in ℝ,l(\mathit{x})>\mu \right\}$, and $m\left\{\mathit{x}:l(\mathit{x})=C,\forall C\in ℝ\right\}=0$.

We use Lagrange’s undetermined multipliers to solve this problem. For any specified FPR (denoted as FPR₀), the TPR corresponding to the ROC curve reaches maximum is equivalent as $\phi$ reaches maximum,

$$\begin{array}{ll}\phi & ={\displaystyle {\int }_{E(l(\mathit{x})>\mu )}f(\mathit{x})d\mathit{x}}+\lambda \left[{\mathrm{FPR}}_0-{\displaystyle {\int }_{E(l(\mathit{x})>\mu )}g(\mathit{x})d\mathit{x}}\right]\\ & =\lambda {\mathrm{FPR}}_0+{\displaystyle {\int }_{E(l(\mathit{x})>\mu )}\left[f(\mathit{x})-\lambda g(\mathit{x})\right]d\mathit{x}}.\end{array}$$

(8)

Function $\phi$ will be maximized if we choose set $E(l(\mathit{x})>\mu )$ such that the integrand is positive, i.e., if

$$f(\mathit{x})-\lambda g(\mathit{x})>0,$$

(9)

then $\mathit{x}\in E(l(\mathit{x})>\mu )$. Which means, no matter what is $\lambda$, if we select the set of $\mathit{x}$ which makes the integrand $f(\mathit{x})-\lambda g(\mathit{x})$ always be positive, the function $\phi$ will reach maximum; if the set contains $\mathit{x}$ that makes the integrand be negative, function $\phi$ will decrease. Let $l(\mathit{x})=f(\mathit{x})/g(\mathit{x})$ and $\mu =\lambda$, and the set, $E(l(\mathit{x})>\mu )$, equals to $E(f(\mathit{x})/g(\mathit{x})>\lambda )$, which satisfies (8), i.e.,

$$\phi =\lambda {\mathrm{FPR}}_0+{\displaystyle {\int }_{E(f(\mathit{x})/g(\mathit{x})>\lambda )}\left[f(\mathit{x})-\lambda g(\mathit{x})\right]d\mathit{x}}$$

(10)

Thus, for any FPR, the TPR corresponding to the ROC curve reaches the maximum, so the AUC reaches the maximum when X and Y are transformed by $l(\mathit{x})=f(\mathit{x})/g(\mathit{x})$.

Let $r(x)$ be a monotonically increasing function; and $h(x)$ be the inverse function of $r(x)$. If $h\prime (x)=1/r\prime (x)$, then $h(x)$ and $r(x)$ have the same monotonicity, and both are increasing functions. Thus, $|h\prime (x)|=h\prime (x)$. The pdf of $X_2=r(X_1)$ is $f_{X_2}(x)=f_{X_1}[h(x)]h\prime (x)$, and the pdf of $Y_2=r(Y_1)$ is $g_{Y_2}(x)=g_{Y_1}[h(x)]h\prime (x)$. Thus,

$$\begin{array}{ll}\mathrm{AUC}& =P(X_2>Y_2)={\displaystyle {\int }_{-\infty }^{+\infty }{f}_{X_2}(x){\displaystyle {\int }_{-\infty }^x{g}_{Y_2}(y)}}dydx\\ & ={\displaystyle {\int }_{-\infty }^{+\infty }{f}_{X_1}\left(h(x)\right)h\prime (x){\displaystyle {\int }_{-\infty }^x{g}_{Y_1}\left(h(y)\right)h\prime (y)}}dydx\\ & ={\displaystyle {\int }_{-\infty }^{+\infty }{f}_{X_1}(x){\displaystyle {\int }_{-\infty }^{x}g{}_{Y_1}(y)}}dydx\\ & =P(X_1>Y_1).\end{array}$$

(11)

We have proved $(\mathrm{a})\Rightarrow (\mathrm{b})$.

$(\mathrm{b})\Rightarrow (\mathrm{a})$: If $l_2(\mathit{x})\ne r[l(\mathit{x})]$, where $r(x)$ is increasing function, there exists $l_2(\mathit{x})$ such that $X,Y$ transforming from $l_2(\mathit{x})$ can also produce maximum AUC, and then the corresponding ROC curves are the same. Otherwise, if ROC curves are different, except the same part, for any FPR, there is at least a ROC curve which doesn’t reach maximum TPR, and contradict with maximum AUC. Since $m\{\mathit{x}:f(\mathit{x})/g(\mathit{x})=C,$ $g(\mathit{x})\ne 0,$ $\forall C\in ℝ\}=0$ and the ROC curve is the same for any point $(\mathrm{FPR},\mathrm{TPR})$ on the two ROC curves, thus,

• For any $\mathrm{FPR}\in [0,1]$, and any ${\mu }_{\mathrm{FPR}}$, there exist ${\mu }_{2\mathrm{FPR}}$, such that $E(l(\mathit{x})>{\mu }_{\mathrm{FPR}})=E(l_2(\mathit{x})>{\mu }_{2\mathrm{FPR}})$ for a.e. $\mathit{x}\in {ℝ}^n$;
• For any ${\mu }_{\mathrm{FPR}}^{\ast }>{\mu }_{\mathrm{FPR}}$, if $E(l(\mathit{x})>{\mu }_{\mathrm{FPR}}^{\ast })=E(l_2(\mathit{x})>{\mu }_{2\mathrm{FPR}}^{\ast })$ and $E(l(\mathit{x})>{\mu }_{\mathrm{FPR}})=E(l_2(\mathit{x})>{\mu }_{2\mathrm{FPR}})$, then ${\mu }_{2\mathrm{FPR}}^{\ast }>{\mu }_{2\mathrm{FPR}}$.

Let $y_1=l(\mathit{x})$, then a set of $y_1$ exist with nonzero measure, such that $l_2(\mathit{x})\ne r[l(\mathit{x})]$, i.e., $m\{y_1:l_2(\mathit{x})\ne r[l(\mathit{x})]\}\ne 0$. Let $\sigma =\{y_1:l_2(\mathit{x})\ne r[l(\mathit{x})]\}$. If $y_1\in \sigma$, $l_2(\mathit{x}),l_1(\mathit{x})$ satisfies function relation $l_2(\mathit{x})=s[l(\mathit{x})]$, but $s(x)$ is not increasing, then for any ${\mu }_1\in \sigma$, condition (ii) does not hold. If $y_1\in \sigma$, $l_2(\mathit{x})$ and $l(\mathit{x})$ are not functionally related, then neither condition (i) or (ii) hold. Thus $(\mathrm{b})\Rightarrow (\mathrm{a})$ is established.

$(\mathrm{c})\iff (\mathrm{b})$: Let $k=\mathrm{TPR}/\mathrm{FPR}$ be the slope of the secant for any point on the ROC curve to the origin, then $\mathrm{Precision}=k/(k+\lambda ),\lambda =P({\omega }_2)/P({\omega }_1)$. For any $\alpha$, that $l(\mathit{x})$ produces maximum Precision is equivalent that k reaches maximum. And equivalent that for any $\alpha$, $\alpha =P({\omega }_1){\int }_{{\mu }_l}^{+\infty }{f}_X(x)dx+P({\omega }_2){\int }_{{\mu }_l}^{+\infty }{g}_Y(x)dx$, $\mathrm{TPR}={\int }_{{\mu }_l}^{+\infty }{f}_X(x)dx$ is maximum. Since this condition is established for any $\alpha$, then it is equivalent that for any $\mathrm{FPR}\in [0,1]$, the corresponding TPR reaches maximum, and equivalent to $l(\mathit{x})$ produces maximum AUC. ☐

Note 1: the condition $m\{\mathit{x}:f(\mathit{x})/g(\mathit{x})=C,g(\mathit{x})\ne 0,\forall C\in ℝ\}=0$ is for exclusion that when $f(\mathit{x})/g(\mathit{x})=C$, (C is a constant), transformation function can be defined randomly on set $\sigma =\{\mathit{x}:f(\mathit{x})/g(\mathit{x})=C,g(\mathit{x})\ne 0,\forall C\in ℝ\}\cap {ℝ}^n$. For example, let us construct the pdf of some random vector $\tilde{X}$ as

$$\tilde{f}(\mathit{x})=\{\begin{array}{l}f(\mathit{x}),\mathit{x}\in {ℝ}^n\backslash \sigma \\ kg(\mathit{x}),\mathit{x}\in \sigma \end{array},k\in ℝ,k<\frac{f(\mathit{x})}{g(\mathit{x})},$$

(12)

${\int }_{{ℝ}^n}\tilde{f}(\mathit{x})d\mathit{x}=1$. Let the transformation function be

$$l(\mathit{x})=\{\begin{array}{l}\frac{f(\mathit{x})}{g(\mathit{x})},\mathit{x}\in {ℝ}^n\backslash \sigma \\ l^{\ast }(\mathit{x}),\mathit{x}\in \sigma \end{array}$$

(13)

then no matter how $l^{\ast }(\mathit{x})$ is defined, only when $l^{\ast }(\mathit{x})<\min [f(\mathit{x})/g(\mathit{x})]$ can the $l(\mathit{x})$ produce the maximum AUC of $(\tilde{X},Y)$. In particular, if $f(\mathit{x})=g(\mathit{x}),$ $\mathit{x}\in {ℝ}^n$, regardless of how $l(\mathit{x})$ is defined, AUC = 0.5. Thus, maximum AUC = 0.5.

Note 2: The arbitrariness of the ratio $\alpha$ must be emphasized in condition (c). If we omitted “any $\alpha$”, then (b) $\Rightarrow$ (c) can be established but (c) $\Rightarrow$ (b) cannot. The meaning of $\alpha$ in application is a ratio of the whole data, for any $l(\mathit{x})$, a ratio $\alpha$ corresponds a threshold $\mu$.

Theorem shows that no matter which evaluation criteria choose, transformation functions that provide maximum link prediction accuracy constitute a function cluster, $\mathsf{\Phi }=\{l(\mathit{x}):l(\mathit{x})=r[f(\mathit{x})/g(\mathit{x})]$, $g(\mathit{x})\ne 0\}$, where $r(x)$ is a monotonically increasing function. Therefore, the accuracy of the combination method must be greater than or equal to the accuracy of each single dimension.

4. A Fusion Link Prediction Method Based on Limit Theorem

4.1. The Algorithm

The limit theorem of combination method shows that when selecting transformation function as $l(\mathit{x})=f(\mathit{x})/g(\mathit{x})$ or its monotone increasing transformation, the AUC and Precision of synthetic score reaches the maximum. In the real network, because $f(\mathit{x})$ and $g(\mathit{x})$ are unknown, the pdfs need to be estimated from multidimensional data. Let the estimated pdfs be $\widehat{f}(\mathit{x})$ and $\widehat{g}(\mathit{x})$. On the basis of limit theorem, we define the transformation function as the ratio of estimated pdfs, i.e.,

$$\widehat{l}(\mathit{x})=\widehat{f}(\mathit{x})/\widehat{g}(\mathit{x})$$

(14)

then we obtained the synthetic score, $s=\widehat{l}(\mathit{x})$, and used for link prediction. This method is called theoretical limit fusion (TLF) method.

Before evaluating $f(\mathit{x})$ and $g(\mathit{x})$, the input link prediction scores need to be normalized,

$$s_k^{\ast }(i,j)=\frac{0.5N^2·s_k(i,j)}{{\displaystyle {\sum }_{i=1}^N{\displaystyle {\sum }_{j=1}^N{s}_k(i,j)}}},k=1,2,\cdots ,d$$

(15)

$s_k(i,j)$ represents the k-th similarity score for node pair $i,j$. N is the dimension of adjacent matrix, and d is the number of similarity indices.

The limit theorem of combination method transformed the link prediction indices fusion problem into the estimation of pdfs. Statistical methods for estimating density functions can be applied to this problem, directly. The Parzen kernel method [16] of destiny estimation is used in this paper. The multivariate kernel density estimate defined as:

$$\widehat{f}(\mathit{x})=\frac{1}{n_s{h}^d}{\displaystyle \sum _{i=1}^{n_s}K\left[\frac{1}{h}\left(\mathit{x}-{\mathit{x}}_i\right)\right]}$$

(16)

where h is the window width, $n_s$ is the sample size, and $K\left(\mathit{x}\right)$ is a multivariate kernel defined for d-dimensional $\mathit{x}$, such that

$${\displaystyle {\int }_{{ℝ}^d}K\left(\mathit{x}\right)d\mathit{x}}=1$$

(17)

A form of the pdf estimate commonly used is Gauss kernel,

$$K\left(\mathit{x}\right)=\frac{1}{{\left(2\pi \right)}^{d/2}}\exp \left(\frac{-{\mathit{x}}^T\mathit{x}}{2}\right)$$

(18)

In summary, the steps of TLF are listed as Table 1.

Table 1. The steps of theoretical limit fusion (TLF) method.

Step 1	Divide the network into training set, ET, and probe set, EP;
Step 2	According to Equation (15), normalize these similarity indices, then we distinguish existing links and nonexistent links in the training set;
Step 3	Based on Equation (16), estimate the pdfs of existing links and nonexistent links, and we obtain the estimated pdfs as $\widehat{f}(\mathit{x})$ and $\widehat{g}(\mathit{x})$;
Step 4	Obtain the synthetic score of n structure similarity indices according to Equation (14);
Step 5	Calculate the accuracy such as AUC metric or Precision metric on the probe set.

4.2. Complexity Analysis

For a given undirected, unweighted graph G(V, E), let $N=\left|V\right|$ be the number of nodes and let $m=\left|E\right|$ be the number of edges, and let $n_s$ be the sample size. During the estimation of pdfs in (16), the entire samples are scanned once. A scan of samples requires time $O(d·n_s)$ and it is less than $O(N^2)$. This is the step of model training or pdf estimation. Among all combination methods, there is an inevitable time complexity, that is to obtain the similarity matrix or final link prediction scores according to Equation (14). This step requires time $O(d·n_s·N^2)$. So, the TLF method will take time more than $O(N^3)$. The main space needs to storage estimator and adjacent matrix or final similarity matrix. The spatial complexity is $O(N^2)$.

5. Simulation and Experiment

We programmed the algorithm using Matlab (MathWorks, Beijing, China, 2014), and runs on a single machine equipped with RedHat 6.4. The host memory is 16 GB, with 3.4 GHz CPU, and the Matlab version is R2014b. In simulations from Section 5.1, 4-dimisional pdfs are supported to verify limit theorem and the effectiveness of TLF method. We also test the resulting method in real networks. We use TLF method to fuse 4 local similarity indices, CN [19], AA [20], RA and PA [21,22]. These indices are 4 simple indices with low computation complexity about $O(N·<{k>}^2)$, where <k> represents the average degree of nodes in a network. CN index only considers common neighbors of node pairs; PA index only considers the degree of two nodes; AA and RA consider both common neighbors and degree of nodes with different weights. And compare the method with fusion methods such as naïve Bayes and logistic regression and other global indices with computation complexity more than $O(N^3)$.

5.1. Simulation Examples

Four types of structural similarity indices were simulated to evaluate node pairs with and without links. The pdfs of the structural similarity indices are also provided. We construct 3 groups of known distributions for the similarity indices pdfs. One thousand samples extracted from 10,000 existing links and 100,000 samples of nonexistent links were generated following the appropriate pdfs. The 1000 samples serve as probe set; the 100,000 samples with 1000 probe links serve as unknown links for training; and the remaining 9000 samples serve as train set of existing links. Each sample had 4 dimensions to simulate 4 similarity scores. We first compute AUC and Precision for each dimension, then use proposed TLF method to obtain the synthetic score and calculate the AUC and Precision, compared with other combination methods such as Naïve Bayes and logistic regression. Finally, we calculate AUC and Precision using the theoretical limit theorem and compare with the above methods.

Let random vectors $X={(X_1,X_2,X_3,X_4)}^T$ and $Y={(Y_1,Y_2,Y_3,Y_4)}^T$ be the scores of existing and nonexistent links, which follow $f(\mathit{x})=f(x_1,x_2,\cdots ,x_n)$ and $g(\mathit{x})=g(x_1,x_2,\cdots ,x_n)$ pdfs, respectively.

Let $f(\mathit{x}),g(\mathit{x})$ are 4-dimensional normal distributions,

$$f(\mathit{x})=\frac{1}{{(2\pi )}^{p/2}{\left|\Sigma \right|}^{1/2}}\exp \left\{-\frac{1}{2}{\left(\mathit{x}-\mathit{\mu }\right)}^T{\Sigma }^{-1}\left(\mathit{x}-\mathit{\mu }\right)\right\},$$

(19)

where $\mathrm{diag}\left(\Sigma \right)1={\left({\sigma }_1^2,{\sigma }_2^2,{\sigma }_3^2,{\sigma }_4^2\right)}^T$, and ${\Sigma }_{ij}={\mathit{r}}_{ij}{\sigma }_i{\sigma }_j$.

The parameter sets for the 2 groups of simulation examples are as follows.

Group 1: ${\mathsf{\Theta }}_{1f}=\{{\mathit{\mu }}_{1f},{\Sigma }_{1f}\}$, and ${\mathsf{\Theta }}_{1g}=\{{\mathit{\mu }}_{1g},{\Sigma }_{1g}\}$;

Group 2: ${\mathsf{\Theta }}_{2f}=\{{\mathit{\mu }}_{2f},{\Sigma }_{2f}\}$ and ${\mathsf{\Theta }}_{2g}=\{{\mathit{\mu }}_{2g},{\Sigma }_{2g}\}$.

In each group, ${\mathit{\mu }}_{1f}={(1,2,1.7,2.1)}^T$, ${\mathit{\mu }}_{1g}={(1.3,2.5,2.1,2.8)}^T$, ${\mathit{\mu }}_{2f}={(1,2,1.7,2.1)}^T$, ${\mathit{\mu }}_{2g}={(1.5,3.5,2.8,3)}^T$, $\mathrm{diag}\left({\Sigma }_{1f}\right)1={\left({1.5}^2,{2.2}^2,3^2,{2.5}^2\right)}^T$, $\mathrm{diag}\left({\Sigma }_{1g}\right)1={\left(2^2,{2.2}^2,3^2,{2.5}^2\right)}^T$, $\mathrm{diag}\left({\Sigma }_{2f}\right)1={\left({1.5}^2,{2.2}^2,3^2,{2.5}^2\right)}^T$, $\mathrm{diag}\left({\Sigma }_{2g}\right)1={\left({2.5}^2,{3.5}^2,4^2,{2.5}^2\right)}^T$, ${\mathit{r}}_{1f}={\mathit{r}}_{1g}=\left[\begin{array}{cccc}1& 0.8& 0.76& 0.56\\ 0.8& 1& 0.85& 0.74\\ 0.76& 0.85& 1& 0.93\\ 0.56& 0.74& 0.93& 1\end{array}\right]$, and ${\mathit{r}}_{2f}={\mathit{r}}_{2g}=\left[\begin{array}{cccc}1& 0.62& 0.45& 0.34\\ 0.62& 1& 0.28& 0.47\\ 0.45& 0.28& 1& 0.65\\ 0.34& 0.47& 0.65& 1\end{array}\right]$.

The window width h of TLF method in the group 1 and 2 is h = 0.1.

Group 3: Let

$$\begin{array}{c}{f}_3(\mathit{x})=x_1{x}_2{x}_3{x}_4+x_1{x}_4+x_3\exp (x_1)\log (x_2)\\ (0\le x_1\le 3,1\le x_2\le 3,3\le x_3\le 5,2\le x_4\le 3.5)\end{array},$$

(20)

and

$$\begin{array}{c}{g}_3(\mathit{x})=x_1{x}_2{x}_3{x}_4+x_3\exp (x_1)\log (x_2)\\ (0\le x_1\le 4,1\le x_2\le 3,3\le x_3\le 5,2.5\le x_4\le 5)\end{array}$$

(21)

We ignore the constant that makes the integral of $f(\mathit{x}),g(\mathit{x})$ equal to 1. The simulation results of group 3 are shown as Table 2.

Table 2. Simulation results of group 1 and group 2. The bold figure indicates the best accuracy in each dimension and combination method.

Parameters	Accuracy	Dim1	Dim2	Dim3	Dim4	NB	LR	TLF	Theoretical Limit	Transform by Increasing Function
Group 1	AUC	0.554	0.566	0.547	0.585	0.610	0.668	0.691	0.738	0.738
	Precision	0.047	0.015	0.014	0.027	0.038	0.020	0.097	0.120	0.120
Group 2	AUC	0.569	0.660	0.604	0.622	0.765	0.676	0.786	0.792	0.792
	Precision	0.114	0.140	0.081	0.038	0.153	0.051	0.212	0.241	0.241

The window width h of TLF method in the group 3 is h = 0.1.

The simulation results in Table 2 and Table 3 show us that we can calculate the theoretical limit of combination method based on Theorem 1, and the limit AUC and Precision are highest among all listed methods, though we cannot list all possible conditions. Results also show that TLF method can fuse the information effectively, and obtain the optimum accuracy. We also verify that the transformation of monotonically increasing function does not change the theoretical limit. Theorem 1 provides a platform that can compare each combination method by constructing some distributions, and direct an effect combination method TLF.

Table 3. Simulation results of group 3. The bold figure indicates the best accuracy in each dimension and combination method.

Accuracy	Dim1	Dim2	Dim3	Dim4	NB	LR	TLF	Theoretical Limit	Transform by Increasing Function
AUC	0.770	0.505	0.488	0.878	0.938	0.923	0.950	0.956	0.956
Precision	0.567	0.007	0.007	0.654	0.711	0.100	0.815	0.858	0.858

5.2. Experiments in Real Networks

The significance of simulation is that the theoretical limit can be derived by theoretical calculation or numerical calculation, and all combination methods can be used to compare with it, finding shortcomings and gaps to design a more rational method. However, the simulation data is different from real network data. We use TLF method to fuse several similarity indices and test in real networks. The basic similarity indices we use are Common Neighbor index (CN) [19], Adamic-Adar index (AA) [15], Resource Allocation index (RA) and Preferential Attachment index (PA) [21,22]. These indices are local indices. Several global indices such as Katz index [23], Average Commute Time index (ACT) and Cosine Similarity Time index (Cos+) are served as comparisons [24,25]. The definitions of the above indices and their meanings are listed as Table 4.

Table 4. Definitions and descriptions of similarity indices.

Index	Equation	Description
CN	$s_{\mathrm{CN}}(i,j)=\left|\mathsf{\Gamma }(i)\cap \mathsf{\Gamma }(j)\right|$	$\mathsf{\Gamma }(i)$ is the set of neighbors of node i. $|·|$ represents cardinality of a set. CN index denotes the common neighbors between nodes i and j.
AA	$s_{\mathrm{AA}}(i,j)={\displaystyle \sum _{z\in \mathsf{\Gamma }(i)\cap \mathsf{\Gamma }(j)}\frac{1}{\log k_z}}$	AA index weights the common neighbors by the reciprocal of the logarithm of each node’s degree.
RA	$s_{\mathrm{RA}}(i,j)={\displaystyle \sum _{z\in \mathsf{\Gamma }(i)\cap \mathsf{\Gamma }(j)}\frac{1}{k_z}}$	RA index weights the common neighbors by the reciprocal of each node’s degree.
PA	$s_{\mathrm{PA}}(i,j)=k_i{k}_j$	PA index expresses preferential attachment by node’s degree.
Katz	$s_{\mathrm{Katz}}(i,j)=\left[\underset{n\to \infty }{\lim }{\displaystyle \sum _{m=1}^n{\left(\alpha A\right)}^m}\right]$	A is adjacent matrix of network. Katz index considers all path between two nodes and gives more weights, $\alpha$, to the shorter paths.
ACT	$s_{\mathrm{ACT}}(i,j)=\frac{1}{l_{ii}^{+}+l_{jj}^{+}-2l_{ij}^{+}}$	$l_{xy}^{+}$ is the corresponding element in $L^{+}$, and $L^{+}$ denotes the pseudo-inverse of laplacian matrix.
Cos+	$s_{\mathrm{Cos}+}(i,j)=\frac{v_i^T{v}_j}{\left|v_i\right|·\left|v_j\right|}=\frac{l_{ij}^{+}}{\sqrt{l_{ii}^{+}·l_{jj}^{+}}}$	According to $L^{+}$, Cos+ calculates cosine similarity of two vectors in matrix $L^{+}$.

We use TLF method to fuse 4 local similarity indices, and compare with fusion method such as naïve Bayes and logistic regression and other global indices. Our experiments are performed on 11 different real networks. (1) Food Web Everglades Web (FWEW) [26]; (2) Food Web Florida Bay(FWFB) [27]; (3) Protein-protein Interactions Cell (PPI Cell) [28]; (4) CKM-3 [29]; (5) Netscience (NS) [30]; (6) Yeast [31]; (7) Political Blogosphere(PB) [32]; (8) Email [33]; (9) CA-GrQc(CG) [34]; (10) Com-dblp(CD) [35]; (11) Email Enron (EE) [36,37]. The basic topological features of 11 real networks are listed in Table 5. Each original data is randomly divided into training set of 90% links, and the probe set of 10% links.

Table 5. Basic topological features of 6 example networks. |V| and |E| are the total numbers of nodes and links, respectively. <k> represents the average degree of nodes in a network. C and r are the clustering coefficient and assortative coefficient respectively. H is the degree heterogeneity, defined as $H=\frac{<k^2>}{{<k>}^2}$.

Data	|V|	|E|	<k>	r	C	H
FWEW	69	880	25.51	−0.298	0.560	1.275
FWFB	128	2075	32.42	−0.112	0.335	1.24
PPI_Cell	127	237	3.732	0.035	0.455	1.649
CKM-3	246	423	3.439	0.102	0.356	1.335
Yeast	2375	11,693	9.85	0.469	0.378	3.48
PB	1222	16,717	27.36	−0.221	0.361	2.97
NS	1589	2742	3.451	0.462	0.889	2.011
Email	1133	5451	9.622	0.078	0.297	1.942
CG	5242	14,496	1.11	0.659	0.720	3.05
CD	425,957	1,049,866	4.93	0.267	0.267	4.412
EE	36,692	183,831	10.02	−0.111	0.746	13.9

Table 6 and Table 7 show the comparisons between TLF method and other combination methods or global indices using AUC and Precision metrics. Each result is the average of 10 realizations. When calculating the Precision metric in Equation (5), we take L = 100 in datasets 1 to 8, and take L = 1000 in datasets 9 to 11. In the large networks, TLF method needs to sample to save the computing sources, and in datasets 10 to 11, the under-sampling rate is set as 1000.

Table 6. Comparisons of the AUC value between TLF and other combination methods or global indices. In each network, the selected window width h is along with the AUC value. The bold figure indicates the best AUC.

Data	CN	AA	RA	PA	ACT	Cos+	Katz	NB	LR	TLF
FWEW	0.687	0.694	0.714	0.819	0.793	0.511	0.727	0.825	0.832	0.876 (h = 0.1)
FWFB	0.624	0.624	0.624	0.742	0.727	0.649	0.680	0.749	0.762	0.781 (h = 0.1)
PPI_Cell	0.736	0.745	0.740	0.699	0.779	0.783	0.822	0.753	0.679	0.831 (h = 0.3)
CKM-3	0.661	0.665	0.661	0.585	0.560	0.535	0.928	0.683	0.675	0.713 (h = 0.15)
Yeast	0.918	0.918	0.915	0.869	0.903	0.958	0.962	0.925	0.934	0.968 (h = 0.2)
PB	0.922	0.928	0.928	0.906	0.890	0.932	0.934	0.931	0.936	0.949 (h = 0.3)
NS	0.994	0.994	0.995	0.709	0.558	0.507	0.996	0.998	0.999	0.999 (h = 0.2)
Email	0.849	0.852	0.851	0.817	0.801	0.889	0.908	0.865	0.870	0.912 (h = 0.15)
CG	0.966	0.965	0.967	0.992	0.549	0.679	0.996	0.984	0.991	0.994 (h = 0.1)
CD	0.962	0.968	0.971	0.943	0.912	0.971	0.915	0.975	0.973	0.982 (h = 0.15)
EE	0.981	0.984	0.984	0.927	0.903	0.980	0.514	0.985	0.987	0.992 (h = 0.15)

Table 7. Comparisons of the Precision value between TLF and other combination methods or global indices. In each network, the corresponding window width h is the same as Table 6. The bold figure indicates the best Precision.

Data	CN	AA	RA	PA	ACT	Cos+	Katz	NB	LR	TLF
FWEW	0.143	0.145	0.162	0.334	0.271	0.004	0.196	0.301	0.325	0.543
FWFB	0.071	0.072	0.083	0.240	0.184	0.029	0.148	0.249	0.283	0.382
PPI_Cell	0.052	0.048	0.073	0.012	0.045	0.061	0.058	0.072	0.068	0.085
CKM-3	0.051	0.059	0.062	0.011	0.001	0.003	0.061	0.060	0.062	0.064
Yeast	0.652	0.703	0.461	0.439	0.487	0.291	0.721	0.712	0.723	0.785
PB	0.381	0.320	0.212	0.100	0.129	0.298	0.381	0.411	0.395	0.452
NS	0.820	0.971	0.982	0.008	0.004	0.006	0.823	0.988	0.986	0.991
Email	0.202	0.253	0.214	0.039	0.031	0.086	0.231	0.263	0.289	0.347
CG	0.972	0.969	0.967	0.991	0.557	0.663	0.998	0.983	0.989	0.996
CD	0.901	0.924	0.931	0.892	0.867	0.937	0.912	0.939	0.942	0.951
EE	0.981	0.984	0.987	0.924	0.898	0.912	0.516	0.988	0.985	0.992

The results show us that TLF method performs better than other fusion methods such as naïve Bayes and logistic regression, no matter what evaluation metric use. Almost all combination methods are better than 4 basic indices. From the limit theorem, combination methods are dependent with each dimension. The promotion of fusion index is restrict to each similarity index. Experiment results also exposed this problem: if the single similarity indices perform not well, the fusion index cannot significantly improve the accuracy. For example, in the CKM-3 network, though we use TLF method to fuse 4 basic similarity indices can improve the AUC obviously, it cannot be better than Katz index (0.928).

6. Discussion

Many combination methods try to find the nonlinear relation of every dimensions, and want to obtain a more reasonable fusion function to promote the prediction accuracy. For example, link prediction method based on the choquet fuzzy integral [7] uses fuzzy measures to measure the importance of each similarity index in the fusion process and the interaction between them. Logistic regression based index adopts logistic function to learn the relation of multiple structural features and obtain an adaptive link prediction method [38]. In fact, according to the limit theorem, the nonlinear relation is the ratio of two joint probability destiny functions or its monotone increasing transformation. The best fusion function is a measurement of difference between existing and nonexistent links, and it reflects the relativity of existing and nonexistent links. The essence of combination methods is trying to approximate the pdfs from many aspects. Limit theorem provides a unified interpretation for all combination methods. On the basis of theoretical limit theorem, the proposed TLF method evaluates two pdfs directly, and it has a better fusion effect from results of simulation and experiment in real network.

7. Conclusions

This paper proposes mathematic description of link prediction combination methods and derives the limit theorem. Before the mathematic description we proposed, many combination methods have been put forward and widely used. However, all these methods are groping respectively without unified explanation. Limit theorem solved this problem and provided a guidance for link prediction method design. The TLF method based on limit theorem can achieve higher prediction accuracy.