Estimating Mixed Memberships in Directed Networks by Spectral Clustering

Abstract

Community detection is an important and powerful way to understand the latent structure of complex networks in social network analysis. This paper considers the problem of estimating community memberships of nodes in a directed network, where a node may belong to multiple communities. For such a directed network, existing models either assume that each node belongs solely to one community or ignore variation in node degree. Here, a directed degree corrected mixed membership (DiDCMM) model is proposed by considering degree heterogeneity. An efficient spectral clustering algorithm with a theoretical guarantee of consistent estimation is designed to fit DiDCMM. We apply our algorithm to a small scale of computer-generated directed networks and several real-world directed networks.

1. Introduction

Many real-world complex networks have community structure such that nodes within the same community (also known as cluster or module) have more links than across communities. For example, in social networks, communities can be groups of students in the same department; in co-authorship networks, a community can be formed by researchers in the same field. However, community structure for a real-world network is usually not directly observable. To process this problem, community detection, also known as graph clustering, is a popular tool for uncovering a latent community structure in a network [1,2]. For decades, many community detection methods have been proposed for non-overlapping undirected networks in which each node belongs to a single community, and the interactions between two nodes are symmetric or undirected. The stochastic block model (SBM) [3] is a popular generative model for non-overlapping undirected networks. In SBM, it is assumed that each node only belongs to one community and that nodes in the same community have the same expectation degrees. Ref. [4] proposes the classical degree corrected stochastic block model (DCSBM) which extends SBM by considering variation in node degree. In recent years, numerous algorithms have been developed to estimate node community for non-overlapping undirected networks generated from SBM and DCSBM, see [5,6,7,8,9,10,11,12,13,14,15]. For recent developments about SBM, see the wonderful review paper [16].

However, in most real-world networks, a node may belong to more than one community at a time. In recent years, the problem of estimating mixed memberships for the undirected network has received a lot of attention [17,18,19,20,21,22,23,24,25,26,27,28,29], and references therein. Ref. [17] extends the SBM model from non-overlapping undirected networks to mixed membership undirected networks and designs the mixed membership stochastic block (MMSB) model. Based on the MMSB model, ref. [24] designs a model called the degree corrected mixed membership (DCMM) model by considering degree heterogeneity, where DCMM can also be seen as an extension of the non-overlapping model DCSBM, and ref. [24] also develops an efficient and provably consistent spectral algorithm. Ref. [27] presents a spectral algorithm under MMSB and establishes per-node rates for mixed memberships by sharp row-wise eigenvector deviation. Ref. [29] proposes an overlapping continuous community assignment model (OCCAM), which is also an extension of MMSB, by considering degree heterogeneity. To fit OCCAM, ref. [29] develops a spectral algorithm requiring a relatively small fraction of mixed nodes when building theoretical frameworks. Ref. [26] finds the cone structure inherent in the normalization of the eigen-decomposition of the population adjacency matrix under DCMM and develops a spectral algorithm to hunt corners in the cone structure.

Though the above works are encouraging and appealing, they focus on undirected networks. In reality, there exist substantial directed networks, such as citation networks, protein–protein interaction networks, and the hyperlink network of websites. In recent years, a lot of works with encouraging results have been developed for directed networks. Ref. [30] proposes a stochastic co-block model (ScBM) and its extension DC-ScBM by considering degree heterogeneity to model non-overlapping directed networks, where ScBM and DC-ScBM can model directed networks whose row nodes may be different from column nodes, and the number of row communities may also be different from the number of column communities. Ref. [31] studies the theoretical guarantees for the algorithm DSCORE [32] and its variants designed under DC-ScBM. Ref. [33] studies the spectral clustering algorithms designed by a data-driven regularization of the adjacency matrix under ScBM. Ref. [34] studies higher-order spectral clustering of directed graphs by designing a nearly linear time algorithm. Based on the fact that the above works only consider non-overlapping directed networks, ref. [35] develops a directed mixed membership stochastic block model (DiMMSB), which is an extension of ScBM, and models directed networks with mixed memberships. DiMMSB can also be seen as a direct extension of MMSB from an undirected network to a directed network.

Recall that DCSBM, DCMM, and DCScBM are extensions of SBM, MMSB, and ScBM by considering node degree variation, respectively, this paper aims at proposing a model as an extension of DiMMSB by considering node degree heterogeneity and building an efficient spectral algorithm to fit the proposed model. In this paper, we focus on the directed network with mixed membership. Our contributions are as follows:

(i)

We propose a novel generative model for directed networks with a mixed membership, the directed degree corrected mixed membership (DiDCMM) model. DiDCMM models a directed network with mixed memberships when row nodes have degree heterogeneities, while column nodes do not. We present the identifiability of DiDCMM under popular conditions which are also required by models modeling mixed membership networks when considering degree heterogeneity. Meanwhile, our results also show that modeling a directed network with mixed membership when considering degree heterogeneity for both row and column nodes needs nontrivial conditions. DiDCMM can be seen as an extension of the DCScBM model from a non-overlapping directed network to an overlapping directed network. DiDCMM also extends the DCMM model from an undirected network to a directed network and extends the DiMMSB model by considering node degree heterogeneity. For a detailed comparison of our DiDCMM with previous models, see Remark 2.

(ii)

To fit DiDCMM, we present a spectral algorithm called DiMSC, which is designed based on the investigation that there exists an ideal cone structure inherent in the normalized version of the left singular vectors and an ideal simplex structure inherent in the right singular vectors of the population adjacency matrix. We prove that our DiMSC exactly recovers the membership matrices for both row and column nodes in the oracle case under DiDCMM, and this also supports the identifiability of DiDCMM. We obtain the upper bounds of error rates for each row (and column) node and show that our method produces asymptotically consistent parameter estimations under mild conditions. Our theoretical results are consistent with classical results when DiDCMM degenerates to SBM and MMSB under mild conditions. Numerical results of simulated directed networks support our theoretical results and show that our approach outperforms its competitors. We also apply our algorithm to several real-world directed networks to test the existence of highly mixed nodes and asymmetric structures between row and column communities.

Notations. 

We take the following general notations in this paper. For a vector x and fixed $q>0$, ${\parallel x\parallel }_q$ denotes its $l_q$-norm. For a matrix M, $M^{\prime }$ denotes the transpose of the matrix M, $\parallel M\parallel$ denotes the spectral norm, ${\parallel M\parallel }_F$ denotes the Frobenius norm, and ${\parallel M\parallel }_{2\to \infty }$ denotes the maximum $l_2$-norm of all the rows of M. Let $\mathrm{rank}\left(M\right)$ denote the rank of matrix M. Let ${\sigma }_i\left(M\right)$ be the i-th largest singular value of matrix M, and ${\lambda }_i\left(M\right)$ denote the i-th largest eigenvalue of the matrix M ordered by the magnitude. $M(i,:)$ and $M(:,j)$ denote the i-th row and the j-th column of matrix M, respectively. $M(S_r,:)$ and $M(:,S_c)$ denote the rows and columns in the index sets $S_r$ and $S_c$ of matrix M, respectively. For any matrix M, we simply use $Y=\max (0,M)$ to represent $Y_{ij}=\max (0,M_{ij})$ for any $i,j$. For any matrix $M\in {\mathbb{R}}^{m\times m}$, let $\mathrm{diag}\left(M\right)$ be the $m\times m$ diagonal matrix whose i-th diagonal entry is $M(i,i)$. Here, $\mathbf{1}$ and $\mathbf{0}$ are column vectors with all entries being ones and zeros, respectively; $e_i$ is a column vector whose i-th entry is one, while other entries are zero. C is a positive constant that may vary occasionally.

2. The Directed Degree Corrected Mixed Membership Model

Consider a directed network $\mathcal{N}=({\mathcal{V}}_r,{\mathcal{V}}_c,\mathcal{E})$, where ${\mathcal{V}}_r=\{1,2,\dots ,n_r\}$ is the set of row nodes, ${\mathcal{V}}_c=\{1,2,\dots ,n_c\}$ is the set of column nodes ($n_r$ and $n_c$ indicate the number of row nodes and the number of column nodes, respectively), and $\mathcal{E}$ is the set of edges. Note that when ${\mathcal{V}}_r={\mathcal{V}}_c$ such that row nodes are the same as column nodes, $\mathcal{N}$ is a traditional directed network [31,36,37,38,39,40,41,42]; when ${\mathcal{V}}_r\ne {\mathcal{V}}_c$, $\mathcal{N}$ is a bipartite network (also known as a bipartite graph) [30,33,35,43,44,45]; see Figure 1 for illustrations of the topological structures for a directed network and a bipartite network. Without confusion, we also call bipartite networks directed networks occasionally in this paper.

Figure 1. Illustration for directed network and bipartite network. Panel (a): directed network; Panel (b): bipartite network.

We assume that the row nodes of the directed network $\mathcal{N}$ belong to K perceivable communities (called row communities in this paper)

$$\begin{array}{c}\hfill {\mathcal{C}}_r^{\left(1\right)},{\mathcal{C}}_r^{\left(2\right)},\dots ,{\mathcal{C}}_r^{\left(K\right)},\end{array}$$

(1)

and the column nodes of the directed network $\mathcal{N}$ belong to K perceivable communities (called column communities in this paper)

$$\begin{array}{c}\hfill {\mathcal{C}}_c^{\left(1\right)},{\mathcal{C}}_c^{\left(2\right)},\dots ,{\mathcal{C}}_c^{\left(K\right)}.\end{array}$$

(2)

Define an $n_r\times K$ row nodes membership matrix ${\mathrm{\Pi }}_r$ and an $n_c\times K$ column nodes membership matrix ${\mathrm{\Pi }}_c$ such that ${\mathrm{\Pi }}_r(i,:)$ is a $1\times K$ probability mass function (PMF) for row node i, ${\mathrm{\Pi }}_c(j,:)$ is a $1\times K$ PMF for column node j, and

$$\begin{array}{cc}\hfill & {\mathrm{\Pi }}_r(i,k)\mathrm{is}\mathrm{the}\mathrm{weight}\mathrm{of}\mathrm{row}\mathrm{node}i\mathrm{on}{\mathcal{C}}_r^{\left(k\right)},1\le k\le K,\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & {\mathrm{\Pi }}_c(j,k)\mathrm{is}\mathrm{the}\mathrm{weight}\mathrm{of}\mathrm{column}\mathrm{node}j\mathrm{on}{\mathcal{C}}_c^{\left(k\right)},1\le k\le K.\hfill \end{array}$$

(4)

Call row node i ‘pure’ if ${\mathrm{\Pi }}_r(i,:)$ is degenerate (i.e., one entry is 1, all other $K-1$ entries are 0) and ‘mixed’ otherwise. The same definitions hold for column nodes. Note that mixed nodes considered in this article are not the boundary nodes introduced in [46] since boundary nodes are defined based on non-overlapping networks, while mixed nodes belong to multiple communities.

Let $A\in {\{0,1\}}^{n_r\times n_c}$ be the bi-adjacency matrix of $\mathcal{N}$ such that for each entry, $A(i,j)=1$ if there is a directional edge from row node i to column node j, and $A(i,j)=0$ otherwise. So, the i-th row of A records how row node i sends edges, and the j-th column of A records how column node j receives edges. Let P be a $K\times K$ matrix such that

$$\begin{array}{c}\hfill P(k,l)\ge 0\mathrm{for}1\le k,l\le K.\end{array}$$

(5)

Note that since we consider a directed network in this paper, P may be asymmetric.

Without loss of generality, suppose that row nodes have degree heterogeneities, while column nodes do not i.e., row nodes have variation in degree, while column nodes do not. Note that in a directed network, if column nodes have degree heterogeneities while row nodes do not, to detect memberships of both row nodes and column nodes, we set the transpose of the adjacency matrix as input when applying our algorithm DiMSC. Meanwhile, in a directed network, if both row and column nodes have degree heterogeneity, to model such a directed network with mixed memberships, we need nontrivial constraints on the degree heterogeneities between row nodes and column nodes for model identifiability, for detail, see Remark 1.

Let ${\theta }_r$ be an $n_r\times 1$ vector whose i-th entry is the positive degree heterogeneity of row node i. For all pairs of $(i,j)$ with $1\le i\le n_r,1\le j\le n_c$, DiDCMM models the entries of A such that $A(i,j)$ are independent Bernoulli random variables satisfying

$$\begin{array}{c}\hfill \mathbb{P}(A(i,j)=1)={\theta }_r\left(i\right)\sum _{k=1}^K\sum _{l=1}^K{\mathrm{\Pi }}_r(i,k){\mathrm{\Pi }}_c(j,l)P(k,l).\end{array}$$

(6)

Equation (6) means that $\mathbb{P}(A(i,j)=1)={\theta }_r\left(i\right){\mathrm{\Pi }}_r(i,:)P{\mathrm{\Pi }}_c^{\prime }(j,:)$, i.e., the probability of generating a directional edge from row node i to column node j is ${\theta }_r\left(i\right){\mathrm{\Pi }}_r(i,:)P{\mathrm{\Pi }}_c^{\prime }(j,:)$, and this probability is controlled by the degree heterogeneity parameter ${\theta }_r\left(i\right)$ of row node i, the connecting matrix P, and the memberships of nodes i and j. Equation (6) functions similarly to Equation (1.4) in [24], and both equations define the probability of generating an edge. For comparison, Equation (6) defines the probability of generating a directional edge under DiDCMM for a directed network, while Equation (1.4) in [24] defines the probability of generating an edge under DCMM for an undirected network, i.e., DiDCMM can be seen as an extension of DCMM from an undirected network to a directed network.

Introduce the degree heterogeneity diagonal matrix ${\mathrm{\Theta }}_r\in {\mathbb{R}}^{n_r\times n_r}$ for row nodes such that

$$\begin{array}{c}\hfill {\mathrm{\Theta }}_r(i,i)={\theta }_r\left(i\right)\mathrm{for}1\le i\le n_r.\end{array}$$

(7)

Equation (7) uses a diagonal matrix ${\mathrm{\Theta }}_r$ to contain all degree heterogeneities, and ${\mathrm{\Theta }}_r$ is useful for further theoretical analysis through Equation (8).

Definition 1. 

Call model (1)–(6) the directed degree corrected mixed membership (DiDCMM) model, and denote it by $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$.

The following conditions are sufficient for the identifiability of DiDCMM:

• (I1) $\mathrm{rank}\left(P\right)=K$, and P has unit diagonals.
• (I2) There is at least one pure node for each of the K row and K column communities.

When building statistical models for a network in which nodes can belong to multiple communities, the full rank requirement of connecting matrix P and pure nodes condition are always necessary for model identifiability, see models for an undirected network such as MMSB considered in [23,27], DCMM considered in [24,26], and OCCAM considered in [26,29]. Meanwhile, if models modeling networks with mixed memberships consider degree heterogeneity, the unit diagonals requirement on connecting matrix P is also necessary for model identifiability, see the identifiability requirement of DCMM and OCCAM considered in [24,26,29]. Furthermore, based on the fact that DiDCMM, DCMM, and OCCAM can include the well-known model SBM, letting P have unit diagonals is not a serious problem since many wonderful works study a special case of SBM when P has unit diagonals and a network has K equal size clusters (this special case of SBM is also known as a planted partition model), see [12,47,48,49,50,51,52].

Let $\mathrm{\Omega }=\mathbb{E}\left[A\right]$ be the expectation of the adjacency matrix A. Under DiDCMM, we have

$$\begin{array}{c}\hfill \mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }.\end{array}$$

(8)

We refer to $\mathrm{\Omega }$ as the population adjacency matrix. Since $\mathrm{rank}\left({\mathrm{\Theta }}_r\right)=K,\mathrm{rank}\left({\mathrm{\Pi }}_r\right)=K,\mathrm{rank}\left({\mathrm{\Pi }}_c\right)=K$ and $\mathrm{rank}\left(P\right)=K$ by Equation (7) and Conditions (I1) and (I2), the rank of $\mathrm{\Omega }$ is K. Recall that K is the number of communities, and it is much smaller than network size. We see that $\mathrm{\Omega }$ has a low dimensional structure. The form of $\mathrm{\Omega }$ given in Equation (8) is powerful to build the spectral algorithm developed in this paper to fit DiDCMM. Analyzing properties of the population adjacency matrix to build a spectral algorithm fitting statistical model is a common strategy in community detection, for example, references [24,26,27,35] also use this strategy to design their algorithms fitting DCMM, MMSB, and DiDCMM.

For $1\le k\le K$, let ${\mathcal{I}}_r^{\left(k\right)}=\{i\in \{1,2,\dots ,n_r\}:{\mathrm{\Pi }}_r(i,k)=1\}$ and ${\mathcal{I}}_c^{\left(k\right)}=\{j\in \{1,2,\dots ,n_c\}:{\mathrm{\Pi }}_c(j,k)=1\}$. By Condition (I2), $I_r^{\left(k\right)}$ and $I_c^{\left(k\right)}$ are nonempty for all $1\le k\le K$. For $1\le k\le K$, select one row node from ${\mathcal{I}}_r^{\left(k\right)}$ to construct the index set ${\mathcal{I}}_r$, i.e., ${\mathcal{I}}_r$ is the indices of row nodes corresponding to K pure row nodes, one from each community, and ${\mathcal{I}}_c$ is defined similarly. W.L.O.G., let ${\mathrm{\Pi }}_r({\mathcal{I}}_r,:)=I_K$ and ${\mathrm{\Pi }}_c({\mathcal{I}}_c,:)=I_K$ (Lemma 2.1 [27] has a similar setting to design their spectral algorithm under MMSB.), where $I_K$ is the $K\times K$ identity matrix. The proposition below shows that the DiDCMM model is identifiable.

Proposition 1. 

(Identifiability). When Conditions (I1) and (I2) hold, DiDCMM is identifiable: for eligible $(P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$ and $(\tilde{P},{\tilde{\mathrm{\Pi }}}_r,{\tilde{\mathrm{\Pi }}}_c,{\tilde{\mathrm{\Theta }}}_r)$, set $\mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }$ and $\tilde{\mathrm{\Omega }}={\tilde{\mathrm{\Theta }}}_r{\tilde{\mathrm{\Pi }}}_r\tilde{P}{\tilde{\mathrm{\Pi }}}_c^{\prime }$. If $\mathrm{\Omega }=\tilde{\mathrm{\Omega }}$, then ${\mathrm{\Theta }}_r={\tilde{\mathrm{\Theta }}}_r,{\mathrm{\Pi }}_r={\tilde{\mathrm{\Pi }}}_r,{\mathrm{\Pi }}_c={\tilde{\mathrm{\Pi }}}_c$ and $P=\tilde{P}$.

 Remark 1.

(The reason that we do not model a directed network with mixed memberships where both row and column nodes have degree heterogeneities). Suppose both row and column nodes have degree heterogeneities in a mixed membership directed network. To model such a directed network, the probability of generating an edge from row node i to column node j is

$$\begin{array}{c}\hfill \mathbb{P}(A(i,j)=1)={\theta }_r\left(i\right){\theta }_c\left(j\right)\sum _{k=1}^K\sum _{l=1}^K{\mathrm{\Pi }}_r(i,k){\mathrm{\Pi }}_c(j,l)P(k,l),\end{array}$$

where ${\theta }_c$ is an $n_r\times 1$ vector whose j-th entry is the degree heterogeneity of column node j. Set $\mathrm{\Omega }=\mathbb{E}\left[A\right]$, then $\mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }{\mathrm{\Theta }}_c$, where ${\mathrm{\Theta }}_c\in {\mathbb{R}}^{n_c\times n_c}$ is a diagonal matrix whose j-th diagonal entry ${\theta }_c\left(j\right)$. Set $\mathrm{\Omega }=U\mathrm{\Lambda }{V}^{\prime }$ as the compact SVD of Ω. Follow similar analysis as Lemma 1, we see that $U={\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r$ and $V={\mathrm{\Theta }}_c{\mathrm{\Pi }}_c{B}_c$ (without causing confusion, we still use $B_c$ here for convenience.). For model identifiability, follow similar analysis as the proof of Proposition 1, since $\mathrm{\Omega }({\mathcal{I}}_r,{\mathcal{I}}_c)={\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Pi }}_r({\mathcal{I}}_r,;)P{\mathrm{\Pi }}_c^{\prime }({\mathcal{I}}_c,:){\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)={\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)P{\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)=U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$, we see that ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)P{\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)=U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$. To obtain ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ and ${\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)$ from $U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$, when P has unit diagonals, we see that it is impossible to recover ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ and ${\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)$ unless we add a condition that ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)={\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)$. Now, suppose ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)={\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)$ holds and call it Condition (I3); we have ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)P{\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)=U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$; hence, ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)={\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)=\sqrt{\mathrm{diag}\left(U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right)}$ when P has unit diagonals. However, Condition (I3) is nontrivial since it requires ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)={\mathrm{\Theta }}_c({\mathcal{I}}_c,{\mathcal{I}}_c)$, and we always prefer a directed network in which there are no connections between row nodes degree heterogeneities and column nodes degree heterogeneities. For example, when all nodes are pure in a directed network, ref. [30] models such directed network using model DC-ScBM such that $\mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }{\mathrm{\Theta }}_c$ when all nodes are pure, and ${\mathrm{\Theta }}_r$ and ${\mathrm{\Theta }}_c$ are independent under DC-ScBM. Because Condition (I3) is nontrivial, we do not model a mixed membership directed network with all nodes having degree heterogeneities.

For DiDCMM’s identifiability, the number of row communities should equal that of column communities when both row and column nodes may belong to more than one community. However, when only row nodes have mixed memberships while column nodes do not, the number of row communities can be lesser than that of column communities, and this is also discussed in [53]. All proofs of our theoretical results are provided in the Appendix A.1.

Unless specified, we treat Conditions (I1) and (I2) as default from now on. Proposition 1 is important since it guarantees that our model DiDCMM is well-defined, and we can design efficient spectral algorithms to fit DiDCMM based on its identifiability. The reason that we do not consider degree heterogeneity for column nodes for our DiDCMM is mainly for its identifiability. As analyzed in Remark 1, considering degree heterogeneity for both row and column nodes make the model unidentifiable unless adding some nontrivial conditions on model parameters. Meanwhile, many previous statistical models in the community detection areas are identifiable, and spectral algorithms can be applied to fit them. For examples, SBM [3], DCSBM [4], MMSB [17], DCMM [24], OCCAM [29], ScBM (and DCScBM), [30], and DiMMSB [35] are identifiable. Especially, though different statistical models may have different requirements on model parameters for identifiability, the proof of identifiability enjoys a similar idea as that of Proposition 1, for instance, Proposition 1.1 [24] and Theorem 2.1 [27] build theoretical guarantees on identifiability for DCMM and MMSB, respectively.

Remark 2. 

We compare our DiDCMM with some previous models in this remark.

• When ${\mathrm{\Theta }}_r=\rho I$ for $\rho >0$, Equation (8) gives $\mathrm{\Omega }=\rho {\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }$ and DiDCMM degenerates to DiMMSB [35], where ρ is known as a sparsity parameter [9,27,35]. So, DiDCMM includes DiMMSB as a special case, and the relationship between DiDCMM and DiMMSB is similar to that between DCSBM [3,4]. Meanwhile, DiDCMM considers degree heterogeneity parameter ${\mathrm{\Theta }}_r$ at the cost that DiDCMM requires P to have unit diagonals for model identifiability, while there is no such requirement for P on DiMMSB’s identifiability. Note that both DiDCMM and DiMMSB are identifiable only when P is a full-rank square matrix.
• When ${\mathrm{\Theta }}_r=\rho I$ for $\rho >0$ and all nodes are pure, DiDCMM reduces to ScBM [30]. DiDCMM can model a directed network in which nodes enjoy overlapping memberships, while ScBM cannot. Meanwhile, DiDCMM enjoys this advantage at the cost of requiring $\mathrm{rank}\left(P\right)=K$ for model identifiability, while ScBM is identifiable even when P is not a square matrix, i.e., ScBM can model a directed network in which the number of row communities can be different from the number of column communities. A comparison between DiDCMM and DCScBM [30] is similar.
• When ${\mathrm{\Theta }}_r=\rho I$ and the network is undirected, DiDCMM reduces to MMSB [17]. However, DiDCMM models directed networks with mixed memberships, while MMSB only models undirected networks with mixed memberships. Again, DiDCMM enjoys its advantage at the cost of P having unit diagonals for its identifiability (not that DiDCMM allows P to be asymmetric since DiDCMM models directed networks), while MMSB is identifiable even when P has non-unit diagonals (note that P is symmetric under MMSB since it models undirected networks). Meanwhile, the identifiability of both DiDCMM and MMSB requires the square matrix P to have full rank.
• When ${\mathrm{\Theta }}_r=\rho I$, the network is undirected and all nodes are pure, DiDCMM reduces to SBM [3]. For comparison, DiDCMM models directed networks and allows nodes to belong to multiple communities, while SBM only models undirected networks in which a node only belongs to one community. Meanwhile, DiDCMM enjoys these advantages at the cost of requiring P to be full rank with unit diagonals for its identifiability, while SBM is identifiable even when P is not full rank and P has non-unit diagonals. Note that DiDCMM allows P to be asymmetric, while P must be symmetric for SBM since DiDCMM models directed networks, while SBM models undirected networks. Comparison between DiDCMM and DCSBM [4] is similar.
• Compared with DCMM introduced in [24] and OCCAM introduced in [29], DCMM, and OCCAM model undirected networks with mixed memberships, while DiDCMM models directed networks with mixed memberships. DiDCMM, DCMM, and OCCAM all consider degree heterogeneity for overlapping networks, and they are identifiable only when the full rank matrix P has unit diagonals. These three models are identifiable only when the square matrix P is full rank. Meanwhile, DiDCMM allows P to be asymmetric, while P must be symmetric for DCMM and OCCAM since DiDCMM models directed networks, while DCMM and OCCAM model undirected networks.

3. Algorithm

The primary goal of the proposed algorithm is to estimate the row membership matrix ${\mathrm{\Pi }}_r$ and column membership matrix ${\mathrm{\Pi }}_c$ from the observed adjacency matrix A with given K. We start by considering the ideal case when $\mathrm{\Omega }$ is known, and then we extend what we learn in the ideal case to the real case.

3.1. The Ideal Simplex (IS), the Ideal Cone (IC), and the Ideal DiMSC

Recall that $\mathrm{rank}\left(\mathrm{\Omega }\right)=K$ under Conditions (I1) and (I2), and K is much smaller than $\min \{n_r,n_c\}$. Let $\mathrm{\Omega }=U\mathrm{\Lambda }{V}^{\prime }$ be the compact singular value decomposition of $\mathrm{\Omega }$ such that $U\in {\mathbb{R}}^{n_r\times K},\mathrm{\Lambda }\in {\mathbb{R}}^{K\times K},V\in {\mathbb{R}}^{n_c\times K}$, $U^{\prime }U=I_K,V^{\prime }V=I_K$. The goal of the ideal case is to use $U,\mathrm{\Lambda }$, and V to exactly recover ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$. As stated in [8,24], ${\theta }_r$ is one of the major nuisances, and similar to [7], we remove the effect of ${\theta }_r$ by normalizing each row of U to have a unit $l_2$ norm. Set $U_{*}\in {\mathbb{R}}^{n_r\times K}$ by $U_{*}(i,:)=\frac{U(i,:)}{{\parallel U(i,:)\parallel }_F}$, and let $N_U$ be the $n_r\times n_r$ diagonal matrix such that $N_U(i,i)=\frac{1}{{\parallel U(i,:)\parallel }_F}$ for $1\le i\le n_r$. Then, $U_{*}$ can be rewritten as $U_{*}=N_{U}U$. The existences of the ideal cone (IC for short) structure inherent in $U_{*}$ and the ideal simplex (IS for short) structure inherent in V are guaranteed by the following lemma.

Lemma 1.  (Ideal Simplex and Ideal Cone). Under $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$, there exist a unique $K\times K$ matrix $B_r$ and a unique $K\times K$ matrix $B_c$ such that

• $U={\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r$, where $B_r={\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$, and $U_{*}=Y{U}_{*}({\mathcal{I}}_r,:)$ where
  $Y=N_M{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)$ with $N_M$ being an $n_r\times n_r$ diagonal matrix whose diagonal entries are positive. Meanwhile, $U_{*}(i,:)=U_{*}(\overline{i},:)$ if ${\mathrm{\Pi }}_r(i,:)={\mathrm{\Pi }}_r(\overline{i},:)$ for $1\le i,\overline{i}\le n_r$.
• $V={\mathrm{\Pi }}_c{B}_c$, where $B_c=V({\mathcal{I}}_c,:)$. Meanwhile, $V(j,:)=V(\overline{j},:)$ if ${\mathrm{\Pi }}_c(j,:)={\mathrm{\Pi }}_c(\overline{j},:)$ for $1\le j,\overline{j}\le n_c$.

Lemma 1 says that the rows of V form a K-simplex in ${\mathbb{R}}^K$ which we call the ideal simplex (IS), with the K rows of $B_c$ being the vertices. Such IS is also found in [24,27,35]. Lemma 1 also shows that the form of $U_{*}=Y{U}_{*}({\mathcal{I}}_r,:)$ is actually the ideal cone structure mentioned in [26]. Meanwhile, we remove the influence of ${\theta }_r$ by normalizing each row of U to have a unit norm in this paper. Using the idea of the entry-wise ratio in [8] also works, where ref. [24] develops their spectral algorithms to fit DCMM using the idea of entry-wise ratio. Designing algorithms based on the nonnegative matrix factorization [25] to fit DiDCMM by adding some constraints on $\mathrm{\Omega }$ may also work. We leave the study of using these ideas to fit DiDCMM or its submodels for our future work.

For column nodes (recall that column nodes have no degree heterogeneities), since $B_c$ is full rank if V and $B_c$ are known in advance, ideally we can exactly recover ${\mathrm{\Pi }}_c$ by setting ${\mathrm{\Pi }}_c=V{B}_c^{\prime }{\left(B_c{B}_c^{\prime }\right)}^{-1}\equiv V{B}_c^{-1}$. For convenience, to transfer the ideal case to the real case, set $Z_c=V{B}_c^{-1}$. Since $Z_c\equiv {\mathrm{\Pi }}_c$, we have

$$\begin{array}{c}\hfill {\mathrm{\Pi }}_c(j,:)=\frac{Z_c(j,:)}{\parallel Z_c{(j,:)\parallel }_1},1\le j\le n_c.\end{array}$$

With given V, since it enjoys IS structure $V={\mathrm{\Pi }}_c{B}_c\equiv {\mathrm{\Pi }}_{c}V({\mathcal{I}}_c,:)$, as long as we can obtain $V({\mathcal{I}}_c,:)$ (i.e., $B_c$), we can recover ${\mathrm{\Pi }}_c$ exactly. As mentioned in [24,27], for such IS, the successive projection (SP) algorithm [54] (i.e., Algorithm A2 in the Appendix E) can be applied to V with K column communities to find the column corner matrix $B_c$. The above analysis gives how to recover ${\mathrm{\Pi }}_c$ with given $\mathrm{\Omega }$ and K under DiDCMM ideally.

Next, we aim to recover ${\mathrm{\Pi }}_r$ from U with the given K. Since $\mathrm{rank}\left(U_{*}\right)=K$, $\mathrm{rank}\left(U_{*}({\mathcal{I}}_r,:)\right)=K$. As $U_{*}({\mathcal{I}}_r,:)\in {\mathbb{R}}^{K\times K}$, the inverse of $U_{*}({\mathcal{I}}_r,:)$ exists. Therefore, Lemma 1 also gives that

$$\begin{array}{c}\hfill Y=U_{*}{U}_{*}^{-1}({\mathcal{I}}_r,:).\end{array}$$

(9)

Equation (9) holds because $U_{*}=Y{U}_{*}({\mathcal{I}}_r,:)$ and $U_{*}({\mathcal{I}}_r,:)$ is a nonsingular matrix. By Lemma 1, we know that for row nodes, their membership matrix ${\mathrm{\Pi }}_r$ appears in the expression of Y. Therefore, we aim to use Equation (9) to find the exact expression of ${\mathrm{\Pi }}_r$ using $U,V$, and $\mathrm{\Lambda }$ by putting Y at the left-hand side of equality. For our next step, we aim at finding ${\mathrm{\Pi }}_r$ using Equation (9). Since $Y=N_M{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)$ by Lemma 1 and $U_{*}=N_{U}U$, using $N_M{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)$ and $N_{U}U$ to replace Y and $U_{*}$ in Equation (9), respectively, we have $N_U^{-1}{N}_M{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)=U{U}_{*}^{-1}({\mathcal{I}}_r,:)$, which gives

$$\begin{array}{c}\hfill N_U^{-1}{N}_M{\mathrm{\Pi }}_r=U{U}_{*}^{-1}({\mathcal{I}}_r,:)N_U({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r).\end{array}$$

(10)

From Equation (10), we have found the expression of ${\mathrm{\Pi }}_r$ as a function of $U,U_{*},{\mathrm{\Theta }}_r,N_U$, and ${\mathcal{I}}_r$, where we do not move $N_U^{-1}{N}_M$ to the right-hand side of Equation (10) because it is a diagonal matrix and does not influence the expression of ${\mathrm{\Pi }}_r$, see our next step for details. When designing a spectral algorithm in the ideal case with given $\mathrm{\Omega }$ and K, we aim at recovering ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ by taking advantage of the singular value decomposition of $\mathrm{\Omega }$. We find that though Equation (10) provides an expression for ${\mathrm{\Pi }}_r$ by $\mathrm{\Omega }$’s SVD, there is a term ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ which relates to degree heterogeneity, and we aim at expressing ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ through $\mathrm{\Omega }$’s SVD. By the proof of Lemma 1, we know that ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)=\mathrm{diag}\left(U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right)$ when Condition (I1) holds. Thus, substituting $\mathrm{diag}\left(U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right)$ for ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ in Equation (10), we obtain an expression of ${\mathrm{\Pi }}_r$ such that this expression is directly related to $\mathrm{\Omega }$’s SVD and two index set ${\mathcal{I}}_r$ and ${\mathcal{I}}_c$. For convenience, set $J_{*}=N_U({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)\equiv \mathrm{diag}\left(U_{*}({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right),Z_r=N_U^{-1}{N}_M{\mathrm{\Pi }}_r,Y_{*}=U{U}_{*}^{-1}({\mathcal{I}}_r,:)$. By Equation (10), we have

$$\begin{array}{c}\hfill Z_r=Y_{*}{J}_{*}\equiv U{U}_{*}^{-1}({\mathcal{I}}_r,:)\mathrm{diag}\left(U_{*}({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right).\end{array}$$

(11)

Equation (11) looks similar to Equation (7) of [55]. However, Equation (11) is related to two index sets ${\mathcal{I}}_r$ and ${\mathcal{I}}_c$, while Equation (7) of [55] is only related to one index set because Equation (11) aims at designing a spectral algorithm for directed network generated under DiDCMM and Equation (7) of [55] aims at reviewing the generation of the SVM-cone-DCMMSB algorithm proposed in [26] for undirected network generated under DCMM. Meanwhile, since $N_U^{-1}{N}_M$ is an $n_r\times n_r$ positive diagonal matrix, we have

$$\begin{array}{c}\hfill {\mathrm{\Pi }}_r(i,:)=\frac{Z_r(i,:)}{\parallel Z_r{(i,:)\parallel }_1},1\le i\le n_r.\end{array}$$

(12)

With given $\mathrm{\Omega }$ and K, we can obtain $U,V$; thus, the above analysis shows that once the two index sets ${\mathcal{I}}_r$ and ${\mathcal{I}}_c$ are known, we can exactly recover ${\mathrm{\Pi }}_r$ by Equations (11) and (12). Meanwhile, from Equation (10), we see that it is important to express ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ as a combination of $U,V,\mathrm{\Lambda }$, and the two index sets ${\mathcal{I}}_r$ and ${\mathcal{I}}_c$, where we successfully obtain an expression of ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ by Condition (I1), the unit diagonal constraint on P. Otherwise, if P has no unit diagonals, we cannot obtain an expression of ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ unless adding some nontrivial conditions on model parameters, just as analyzed in Remark 1. Similarly, references [24,26] also design their spectral algorithms to fit DCMM by using the unit diagonal constraint on P to obtain an expression of a sub-matrix of degree heterogeneity matrix, see Equations (6)–(8) of [55] as an example.

Given $\mathrm{\Omega }$ and K, to recover ${\mathrm{\Pi }}_r$ in the ideal case, we need to obtain $Z_r$ by Equation (11), which means that the only difficulty is in finding the index set ${\mathcal{I}}_r$ since $V({\mathcal{I}}_c,:)$ can be obtained by SP algorithm from the IS structure $V={\mathrm{\Pi }}_{c}V({\mathcal{I}}_c,:)$. From Lemma 1, we know that $U_{*}=Y{U}_{*}({\mathcal{I}}_r,:)$ forms the IC structure. In [26], their SVM-cone algorithm (i.e., Algorithm A3 in the Appendix F) can exactly obtain the row nodes corner matrix $U_{*}({\mathcal{I}}_r,:)$ from the ideal cone $U_{*}=Y{U}_{*}({\mathcal{I}}_r,:)$ as long as the Condition ${\left(U_{*}({\mathcal{I}}_r,:)U_{*}^{\prime }({\mathcal{I}}_r,:)\right)}^{-1}\mathbf{1}>0$ holds (see Lemma 2).

Lemma 2. 

Under $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$, ${\left(U_{*}({\mathcal{I}}_r,:)U_{*}^{\prime }({\mathcal{I}}_r,:)\right)}^{-1}\mathbf{1}>0$ holds.

Based on the above analysis, we are now ready to give the following four-stage algorithm which we call ideal DiMSC. Input $\mathrm{\Omega },K$. Output: ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$.

• Let $\mathrm{\Omega }=U\mathrm{\Lambda }{V}^{\prime }$ be the compact SVD of $\mathrm{\Omega }$ such that $U\in {\mathbb{R}}^{n_r\times K},V\in {\mathbb{R}}^{n_c\times K},\mathrm{\Lambda }\in {\mathbb{R}}^{K\times K},U^{\prime }U=I,V^{\prime }V=I$. Let $U_{*}=N_{U}U$, where $N_U$ is an $n_r\times n_r$ diagonal matrix whose i-th diagonal entry is $\frac{1}{{\parallel U(i,:)\parallel }_F}$ for $1\le i\le n_r$.
• Run the SP algorithm on V assuming that there are K column communities to obtain the column corner matrix $V({\mathcal{I}}_c,:)$ (i.e.,$B_c$). Run the SVM-cone algorithm on $U_{*}$ assuming that there are K row communities to obtain ${\mathcal{I}}_r$.
• Set $J_{*}=\mathrm{diag}\left(U_{*}({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right),Y_{*}=U{U}_{*}^{-1}({\mathcal{I}}_r,:),Z_r=Y_{*}{J}_{*}$ and $Z_c=V{V}^{-1}({\mathcal{I}}_c,:)$.
• Recover ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ by setting ${\mathrm{\Pi }}_r(i,:)=\frac{Z_r(i,:)}{\parallel Z_r{(i,:)\parallel }_1}$ for $1\le i\le n_r$, and ${\mathrm{\Pi }}_c(j,:)=\frac{Z_c(j,:)}{\parallel Z_c{(j,:)\parallel }_1}$ for $1\le j\le n_c$.

The following theorem guarantees that ideal DiMSC exactly recovers nodes memberships, and this verifies the identifiability of DiDCMM in turn. Meanwhile, it should be noted that many spectral algorithms designed to fit identifiable statistical models in the community detection area can exactly recover node memberships for the ideal case. For example, the spectral clustering for K many clusters algorithm addressed in [5] under SBM, the regularized spectral clustering designed in [7] under DCSBM, the SCORE algorithm designed in [8] under DCSBM, the two algorithms designed and studied in [9] under SBM and DCSBM, the RSC-$\tau$ algorithm studied in [11] under SBM, the mixed-SCORE algorithm designed in [24] under DCMM, the DI-SIM algorithm designed in [30] under DCScBM, the D-SCORE algorithm studied in [31,32] under DCScBM, the SVM-cone-DCMMSB algorithm designed in [26] under DCMM, and the SPACL algorithm designed in [27] under MMSB can exactly recover membership matrices under respective models for the ideal case by using the population adjacency matrix to replace the adjacency matrix in the input of these algorithms. The fact that ideal cases for the above spectral algorithms can return community information also supports the identifiability of the above models.

Theorem 1. 

Under $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$, the ideal DiMSC exactly recovers the row nodes membership matrix ${\mathrm{\Pi }}_r$ and the column nodes membership matrix ${\mathrm{\Pi }}_c$.

To demonstrate that $U_{*}$ has the ideal cone structure, we drew Panel (a) of Figure 2. The simulated data used for Panel (a) is generated from $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$ with $n_r=600,n_c=400,K=3$; each row (and column) community has 120 pure nodes. For the 240 mixed row nodes, we set ${\mathrm{\Pi }}_r(i,1)=\mathrm{rand}\left(1\right)/2,{\mathrm{\Pi }}_r(i,2)=\mathrm{rand}\left(1\right)/2,{\mathrm{\Pi }}_r(i,3)=1-{\mathrm{\Pi }}_r(j,1)-{\mathrm{\Pi }}_r(j,2)$, where $\mathrm{rand}\left(1\right)$ is any random number in $(0,1)$,

Figure 2. Panel (a): plot of $U_{*}$ and the hyperplane formed by $U_{*}({\mathcal{I}}_r,:)$. Blue points denote rows respective to mixed row nodes of $U_{*}$, and black points denote the K rows of the corner matrix $U_{*}({\mathcal{I}}_r,:)$. The plane in Panel (a) is the hyperplane formed by the triangle of the 3 rows of $U_{*}({\mathcal{I}}_r,:)$. Panel (b): plot of V and the ideal simplex formed by $V({\mathcal{I}}_c,:)$. Blue points denote rows respective to mixed column nodes of V, and black points denote the K rows of the corner matrix $V({\mathcal{I}}_c,:)$. Since $K=3$, for visualization, we have projected these points from ${\mathbb{R}}^3$ to ${\mathbb{R}}^2$.

and i is a mixed row node. For the 40 mixed column nodes, set ${\mathrm{\Pi }}_c(j,1)=\mathrm{rand}\left(1\right)/2$,

${\mathrm{\Pi }}_c(j,2)=\mathrm{rand}\left(1\right)/2,{\mathrm{\Pi }}_c(j,3)=1-{\mathrm{\Pi }}_c(j,1)-{\mathrm{\Pi }}_c(j,2)$. For the degree heterogeneity parameter, set ${\theta }_r\left(i\right)=\mathrm{rand}\left(1\right)$ for all row nodes i. The matrix P is set as

$$P=\left[\begin{array}{ccc}1& 0.4& 0.3\\ 0.2& 1& 0.1\\ 0.1& 0.4& 1\end{array}\right].$$

Under such a setting, after computing $\mathrm{\Omega }$ and obtaining $U_{*},V$ from $\mathrm{\Omega }$, we can plot Figure 2. Panel (a) shows that all rows respective to mixed row nodes of $U_{*}$ are located at one side of the hyperplane formed by the K rows of $U_{*}({\mathcal{I}}_r,:)$, and this phenomenon occurs since each row of $U_{*}$ is a scaled convex combination of the K rows of $U_{*}({\mathcal{I}}_r,:)$ guaranteed by the IC structure $U_{*}=Y{U}_{*}({\mathcal{I}}_r,;)$. Thus Panel (a) shows the existence of the ideal cone structure formed by $U_{*}$. Similarly, to demonstrate that V has the ideal simplex structure, we drew Panel (b) of Figure 2, where Panel (b) is obtained under the same setting as Panel (a). Panel (b) shows that rows respective to mixed column nodes of V are located inside of the simplex formed by the K rows of $V({\mathcal{I}}_c,:)$, and this phenomenon occurs since each row of V is a convex linear combination of the K rows of $V({\mathcal{I}}_c,:)$ guaranteed by the IS structure $V={\mathrm{\Pi }}_{c}V({\mathcal{I}}_c,;)$. Thus Panel (b) shows the existence of the ideal simplex structure formed by V.

3.2. Dimsc Algorithm

We now extend the ideal case to the real case. Set $\tilde{A}=\widehat{U}\widehat{\mathrm{\Lambda }}{\widehat{V}}^{\prime }$ to be the top-K-dimensional SVD of A such that $\widehat{U}\in {\mathbb{R}}^{n_r\times K},\widehat{V}\in {\mathbb{R}}^{n_c\times K},\widehat{\mathrm{\Lambda }}\in {\mathbb{R}}^{K\times K},{\widehat{U}}^{\prime }\widehat{U}=I_K,{\widehat{V}}^{\prime }\widehat{V}=I_K$, and $\widehat{\mathrm{\Lambda }}$ contains the top K singular values of A. Let ${\widehat{U}}_{*}$ be the row-wise normalization of $\widehat{U}$ such that $\widehat{U}=N_{\widehat{U}}\widehat{U}$, where $N_{\widehat{U}}\in {\mathbb{R}}^{n_r\times n_r}$ is a diagonal matrix whose i-th diagonal entry is $\frac{1}{\parallel \widehat{U}{(i,:)\parallel }_F}$. For the real case, we use ${\widehat{J}}_{*},{\widehat{Y}}_{*},{\widehat{Z}}_r,{\widehat{Z}}_c,{\widehat{\mathrm{\Pi }}}_r,{\widehat{\mathrm{\Pi }}}_c$ given in Algorithm 1 to estimate $J_{*},Y_{*},Z_r,Z_c,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c$, respectively. Algorithm 1 called directed mixed simplex and cone (DiMSC for short) algorithm is a natural extension of the ideal DiMSC to the real case.

Algorithm 1: Directed Mixed Simplex and Cone (DiMSC) algorithm

Require: The adjacency matrix $A\in {\mathbb{R}}^{n_r\times n_c}$ of a directed network, the number of row (column) communities K. Ensure: The estimated $n_r\times K$ row membership matrix ${\widehat{\mathrm{\Pi }}}_r$ and the estimated $n_c\times K$ column membership matrix ${\widehat{\mathrm{\Pi }}}_c$. 1: Obtain $\tilde{A}=\widehat{U}\widehat{\mathrm{\Lambda }}{\widehat{V}}^{\prime }$, the top-K-dimensional SVD of A. Compute ${\widehat{U}}_{*}$ from $\widehat{U}$. 2: Apply SP algorithm (i.e., Algorithm A2) on the rows of $\widehat{V}$ assuming there are K column communities to obtain ${\widehat{\mathcal{I}}}_c$, the index set returned by SP algorithm. 3: Similarly, apply SVM-cone algorithm (i.e., Algorithm 3) on the rows of ${\widehat{U}}_{*}$ with K row communities to obtain ${\widehat{\mathcal{I}}}_r$, the index set returned by SVM-cone algorithm. 4: Set ${\widehat{J}}_{*}=\mathrm{diag}\left({\widehat{U}}_{*}({\widehat{I}}_r,:)\widehat{\mathrm{\Lambda }}{\widehat{V}}^{\prime }({\widehat{\mathcal{I}}}_c,:)\right),{\widehat{Y}}_{*}=\widehat{U}{\widehat{U}}_{*}^{-1}({\widehat{\mathcal{I}}}_r,:),{\widehat{Z}}_r={\widehat{Y}}_{*}{\widehat{J}}_{*}$ and ${\widehat{Z}}_c=\widehat{V}{\widehat{V}}^{-1}({\widehat{\mathcal{I}}}_c,:)$. Then, set ${\widehat{Z}}_r=\max (0,{\widehat{Z}}_r)$ and ${\widehat{Z}}_c=\max (0,{\widehat{Z}}_c)$. 5: Estimate ${\mathrm{\Pi }}_r(i,:)$ by ${\widehat{\mathrm{\Pi }}}_r(i,:)={\widehat{Z}}_r(i,:)/{\parallel {\widehat{Z}}_r(i,:)\parallel }_1,1\le i\le n_r$ and estimate ${\mathrm{\Pi }}_c(j,:)$ by ${\widehat{\mathrm{\Pi }}}_c(j,:)={\widehat{Z}}_c(j,:)/{\parallel {\widehat{Z}}_c(j,:)\parallel }_1,1\le j\le n_c$.

In the third step, we set the negative entries of ${\widehat{Z}}_r$ as 0 by setting ${\widehat{Z}}_r=\max (0,{\widehat{Z}}_r)$ for the reason that weights for any row node should be nonnegative, while there may exist some negative entries of ${\widehat{Y}}_{*}{\widehat{J}}_{*}$. A similar argument holds for ${\widehat{Z}}_c$. The flowchart of DiMSC is displayed in Figure 3. Meanwhile, in community detection, researchers often use top-K-dimensional SVD of A or its variants such as Laplacian matrix or regularized Laplacian matrix to design their spectral clustering algorithms to fit identifiable statistical models such as spectral methods designed or studied in [5,7,8,9,11,24,26,27,29,31,33,35,56]. Furthermore, as discussed in [57], the ${\mathrm{SVS}}^{+}$ and ${\mathrm{SVS}}^{*}$ algorithms may be used as substitutions of the SP algorithm in our DiMSC for a better estimation of ${\mathrm{\Pi }}_r$. When applying the entry-wise normalization idea developed in [8] to deal with U, as analyzed in [24], we obtain a simplex structure, and we can use the SP algorithm (or the combinatorial vertex search and sketched vertex search approaches developed in [24]) to hunt for the corners. The above ideas suggest that we can design different spectral algorithms to fit our model DiDCMM. We leave them for our future work. In particular, in this paper, we apply the SVM-cone algorithm to hunt for the corners of the cone structure inherent in $U_{*}$ mainly for the theoretical convenience of the SVM-cone algorithm because ref. [26] has developed a nice theoretical framework on the performance for the SVM-cone algorithm.

Figure 3. Flowchart of Algorithm 1.

3.3. Computational Complexity

The computing cost of DiMSC mainly comes from SVD, SP, and SVM-cone. The computational complexity of SVD is $O\left(\max (n_r,n_c)\min (n_r^2,n_c^2)\right)$. Since the adjacency matrix A for real-world network data sets is usually sparse, using the power method discussed in [58], the computation complexity for obtaining the top-K-dimensional SVD of A is only slightly larger than $O\left(\max (n_r^2,n_c^2)K\right)$ [8,24]. The SP algorithm step in DiMSC has a complexity of $O\left(\max (n_r,n_c)K^2\right)$ [24]. The complexity of the one-class SVM step for SVM-cone algorithm is $O\left(\max (n_r,n_c)K^2\right)$ [26,59]. The complexity of the K-means step for SVM-cone algorithm is $O\left(\max (n_r,n_c)K^2\right)$ [60]. Since the number of communities K considered in this paper is much smaller than the network size, the total complexity of DiMSC is $O\left(\max (n_r^2,n_c^2)K\right)$. Results in Section 5 show that, for a computer-generated network with 15,000 nodes under SBM, DiMSC takes hundreds of seconds to process a standard personal computer (Thinkpad X1 Carbon Gen 8) using MATLAB R2021b. Meanwhile, many spectral methods developed under models SBM, DCSBM, MMSB, ScBM, DCScBM, OCCAM, DCMM, and DiMMSB for community detection also have complexity $O\left(\max (n_r^2,n_c^2)K\right)$, see spectral algorithms designed or studied in [5,7,8,9,11,24,26,27,29,30,31,33,35,61,62]. Researchers design spectral algorithms for community detection under various identifiable statistical models mainly for their convenience on building a theoretical guarantee of consistent estimation, and we also provide a theoretical guarantee on DiMSC’s estimation consistency in next section.

4. Consistency Results

In this section, we show the consistency of our algorithm for fitting the DiDCMM by proving that the sample-based estimates ${\widehat{\mathrm{\Pi }}}_r$ and ${\widehat{\mathrm{\Pi }}}_c$ concentrate around the true mixed membership matrices ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$. Throughout this paper, K is a known positive integer. Set ${\theta }_{r,\max }={\max }_{1\le i\le n_r}{\theta }_r\left(i\right)$ and ${\theta }_{r,\min }={\min }_{1\le i\le n_r}{\theta }_r\left(i\right)$. Assume that

Assumption 1. 

$P_{\max }\max (\parallel {\theta }_r{\parallel }_1,{\theta }_{r,\max }{n}_c)\ge \log (n_r+n_c)$.

Assumption 1 means that the network cannot be too sparse, and it also means that we allow ${\theta }_{r,\max }$ to go to zero with increasing numbers of row nodes and column nodes. When building theoretical guarantees on consistent estimation, controlling network sparsity is popular in the community detection area. For examples, Condition (2.9) of [8], Theorem 3.1 of [9], Condition (2.13) of [24], Assumption 3.1 of [27], and Assumption 2 of [31] all control network sparsity for their theoretical analysis. Especially, when DiDCMM reduces to SBM by letting ${\mathrm{\Theta }}_r=\rho I,n=n_r=n_c,{\mathrm{\Pi }}_r={\mathrm{\Pi }}_c$, and all nodes are pure for $\rho >0$, Assumption A1 requires that $\rho n\gg \log \left(n\right)$, which is consistent with the sparsity requirement in [8,9,24,31]. As analyzed in [55], we know that our requirement on network sparsity is optimal since it matches the sharp threshold of obtaining a connected Erdös–Rényi (ER) random graph [63] when SBM reduces to an ER random graph by letting $K=1$.

For notation convenience, set $\varpi =\max (\parallel \widehat{U}{\widehat{U}}^{\prime }-U{U}^{\prime }{\parallel }_{2\to \infty },\parallel \widehat{V}{\widehat{V}}^{\prime }-V{V}^{\prime }{\parallel }_{2\to \infty }),{\widehat{f}}_r$$={\max }_{1\le i\le n_r}\parallel e_i^{\prime }({\widehat{\mathrm{\Pi }}}_r-{\mathrm{\Pi }}_r{\mathcal{P}}_r){\parallel }_1,{\widehat{f}}_c={\max }_{1\le j\le n_c}{\parallel e_j^{\prime }({\widehat{\mathrm{\Pi }}}_c-{\mathrm{\Pi }}_c{\mathcal{P}}_c)\parallel }_1$, and ${\pi }_{r,\min }={\min }_{1\le k\le K}$${\mathbf{1}}^{\prime }{\mathrm{\Pi }}_r{e}_k$, where $\varpi$ is the row-wise singular vector deviation which can be bounded by Theorem 4.4 of [64], ${\widehat{f}}_r$ and ${\widehat{f}}_c$ measures per node clustering error of DiMSC, and ${\pi }_{r,\min }$ measures the minimum summation of row nodes belonging to a certain row community. Increasing ${\pi }_{r,\min }$ makes the network tend to be more balanced and vice versa. Meanwhile, row-wise singular vector deviation is important when building a theoretical guarantee of spectral methods fitting models for a network with mixed memberships, for example, refs. [24,26,27,35] also consider $\varpi$ when building consistent estimation for their spectral methods.

The next theorem gives theoretical bounds on estimations of memberships for both row and column nodes, which is the main theoretical result for our DiMSC method.

Theorem 2. 

Under $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$, let ${\widehat{\mathrm{\Pi }}}_r$ and ${\widehat{\mathrm{\Pi }}}_c$ be obtained from Algorithm 1, when Assumption 1 holds, suppose ${\sigma }_K\left(\mathrm{\Omega }\right)\ge C\sqrt{{\theta }_{r,\max }{P}_{\max }(n_r+n_c)\log (n_r+n_c)}$, with probability at least $1-o\left({(n_r+n_c)}^{-3}\right)$, we have

$$\begin{array}{cc}\hfill & {\widehat{f}}_r=O\left(\frac{K^{5.5}{\theta }_{r,\max }^{15}\varpi {\kappa }^{4.5}\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)\kappa \left({\mathrm{\Pi }}_c\right){\lambda }_1^{1.5}\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)}{{\theta }_{r,\min }^{15}{\pi }_{r,\min }}\right),{\widehat{f}}_c=O\left(\varpi K\kappa \left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)\sqrt{{\lambda }_1\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)}\right).\hfill \end{array}$$

In Theorem 2, the Condition ${\sigma }_K\left(\mathrm{\Omega }\right)\ge C\sqrt{{\theta }_{r,\max }{P}_{\max }(n_r+n_c)\log (n_r+n_c)}$ is necessary when applying Theorem 4.4 [64] to obtain a theoretical upper bound of $\varpi$. When building a theoretical guarantee on estimation consistency for spectral methods fitting models modeling network with mixed memberships, it is necessary to have a lower bound requirement on ${\sigma }_K\left(\mathrm{\Omega }\right)$, see [24,26,27,35]. Actually, this requirement matches with the consistent requirement on $\frac{{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}$ obtained from the theoretical upper bound of error rates for a balanced network, see Remark 4 for details. Meanwhile, similar to [7,11,30], we can design a spectral algorithm via an application of regularized Laplacian matrix to fit DiDCMM.

The following corollary is obtained by adding conditions on model parameters similar to Corollary 3.1 in [27], where these conditions give a directed network in which each community has the same order of size, and each node has the same order of degree, i.e., a balanced network.

Corollary 1. 

Under $DiDCM{M}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$, when conditions of Theorem 2 hold, suppose ${\lambda }_K\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)=O\left(\frac{n_r}{K}\right),{\lambda }_K\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)=O\left(\frac{n_c}{K}\right),{\pi }_{r,\min }=O\left(\frac{n_r}{K}\right)$ and $K=O\left(1\right)$, with probability at least $1-o\left({(n_r+n_c)}^{-3}\right)$, we have

$$\begin{array}{cc}\hfill & {\widehat{f}}_r=O\left({\left(\frac{{\theta }_{r,\max }}{{\theta }_{r,\min }}\right)}^{15.5}\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{P_{\max }\log (n_r+n_c)}{{\theta }_{r,\min }{n}_c}}\right),{\widehat{f}}_c=O\left({\left(\frac{{\theta }_{r,\max }}{{\theta }_{r,\min }}\right)}^{0.5}\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{P_{\max }\log (n_r+n_c)}{{\theta }_{r,\min }{n}_r}}\right).\hfill \end{array}$$

Meanwhile,

• when ${\theta }_{r,\max }=O\left(\rho \right),{\theta }_{r,\min }=O\left(\rho \right)$ (i.e., $\frac{{\theta }_{r,\min }}{{\theta }_{r,\max }}=O\left(1\right)$), we have

  $$\begin{array}{cc}\hfill & {\widehat{f}}_r=O\left(\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{P_{\max }\log (n_r+n_c)}{\rho n_c}}\right),{\widehat{f}}_c=O\left(\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{P_{\max }\log (n_r+n_c)}{\rho n_r}}\right).\hfill \end{array}$$
• when $n_r=O\left(n\right),n_c=O\left(n\right)$ and ${\theta }_{r,\max }=O\left(\rho \right),{\theta }_{r,\min }=O\left(\rho \right)$, we have

  $$\begin{array}{cc}\hfill & {\widehat{f}}_r=O\left(\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{P_{\max }\log \left(n\right)}{\rho n}}\right),{\widehat{f}}_c=O\left(\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{P_{\max }\log \left(n\right)}{\rho n}}\right).\hfill \end{array}$$

Consider a directed mixed membership network under the settings of Corollary 1 when ${\theta }_{r,\max }=O\left(\rho \right),{\theta }_{r,\min }=O\left(\rho \right)$ for $\rho >0$, to obtain consistent estimations for both row nodes and column nodes, by Corollary 1, $\frac{{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}$ should shrink slower than $\sqrt{\frac{\log (n_r+n_c)}{\rho \min (n_r,n_c)}}$, where consistent estimation means that the theoretical upper bound of error rate goes to zero when increasing network size. Especially, when $n_r=O\left(n\right)$ and $n_c=O\left(n\right)$, $\frac{{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}$ should shrink slower than $\sqrt{\frac{\log \left(n\right)}{n}}$. We further assume that $P=(2-\beta )I_K+(\beta -1)\mathbf{1}{\mathbf{1}}^{\prime }$ for $\beta \in [1,2)\cup (2,\infty )$ and let $\tilde{P}=\rho P$ (note that for this P, we have ${\sigma }_K\left(P\right)=|\beta -2|$ and $P_{\max }=\max (1,\beta -1)$). So the diagonal elements for $\tilde{P}$ are $\rho$ and non-diagonal elements are $\rho (\beta -1)$. Set $p_{\mathrm{in}}$ as the diagonal entries of $\tilde{P}$, and $p_{\mathrm{out}}$ as the non-diagonal entries of $\tilde{P}$, we have $p_{\mathrm{in}}=\rho$, $p_{\mathrm{out}}=\rho (\beta -1)$, and $\frac{|p_{\mathrm{in}}-p_{\mathrm{out}}|}{\sqrt{\max (p_{\mathrm{in}},p_{\mathrm{out}})}}=\frac{\sqrt{\rho }|\beta -2|}{\sqrt{\max (1,\beta -1)}}=\frac{\sqrt{\rho }{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}$. Hence, for consistent estimation, we see that $\frac{|p_{\mathrm{in}}-p_{\mathrm{out}}|}{\sqrt{\max (p_{\mathrm{in}},p_{\mathrm{out}})}}$ should shrink slower than $\sqrt{\frac{\log (n_r+n_c)}{\min (n_r,n_c)}}$ by Corollary 1 and should shrink slower than $\sqrt{\frac{\log \left(n\right)}{n}}$ when $n_r=O\left(n\right)$ and $n_c=O\left(n\right)$, where this result is consistent with classical separation condition for a standard network with two equal-sized clusters by applying the separation condition and sharp threshold criterion developed in [55].

Remark 3. 

When the network is undirected (i.e., $n_r=n_c=n,{\mathrm{\Pi }}_r={\mathrm{\Pi }}_c$) with $K=O\left(1\right)$ by setting ${\theta }_r\left(i\right)=\rho$ for $1\le i\le n_r$, DiDCMM degenerates to MMSB considered in [27], the upper bound of error rate for DiMSC is $O\left(\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{\log \left(n\right)}{\rho n}}\right)$ when $P_{\max }=1$. Replacing the Θ in [24] by $\mathrm{\Theta }=\sqrt{\rho }I$, their DCMM model degenerates to MMSB. Then, their conditions in Theorem 2.2 are our Assumption 1 and ${\lambda }_K\left({\mathrm{\Pi }}^{\prime }\mathrm{\Pi }\right)=O\left(\frac{n}{K}\right)$, where $\mathrm{\Pi }={\mathrm{\Pi }}_r={\mathrm{\Pi }}_c$ for MMSB. When $K=O\left(1\right)$, the error bound in Theorem 2.2 in [24] is $O\left(\frac{1}{{\sigma }_K\left(P\right)}\sqrt{\frac{\log \left(n\right)}{\rho n}}\right)$, which is consistent with ours.

Remark 4. 

By Lemma A5 in the Appendix D, we know ${\sigma }_K\left(\mathrm{\Omega }\right)\ge {\theta }_{r,\min }{\sigma }_K\left(P\right){\sigma }_K\left({\mathrm{\Pi }}_r\right){\sigma }_K\left({\mathrm{\Pi }}_c\right)$. To ensure the Condition ${\sigma }_K\left(\mathrm{\Omega }\right)\ge C{\left({\theta }_{r,\max }{P}_{\max }(n_r+n_c)\log (n_r+n_c)\right)}^{1/2}$ in Theorem 2 holds, we need

$$\begin{array}{c}\hfill \frac{{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}\ge C{\left(\frac{{\theta }_{r,\max }(n_r+n_c)\log (n_r+n_c)}{{\theta }_{r,\min }^2{\lambda }_K\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right){\lambda }_K\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)}\right)}^{1/2}.\end{array}$$

(13)

When $K=O\left(1\right),n_r=O\left(n\right),n_c=O\left(n\right),{\lambda }_K\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)=O\left(\frac{n_r}{K}\right),{\lambda }_K\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)=O\left(\frac{n_c}{K}\right)$ and ${\theta }_{r,\max }=O\left(\rho \right),{\theta }_{r,\min }=O\left(\rho \right)$, Equation (13) gives that $\frac{{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}$ should shrink slower than $\sqrt{\frac{\log \left(n\right)}{\rho n}}$, which matches with the consistency requirement on $\frac{{\sigma }_K\left(P\right)}{\sqrt{P_{\max }}}$ of Corollary 1.

For convenience, we need the following definition.

Definition 2. 

Let $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$ be a special case of $DiMMD{F}_{n_r,n_c}(K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$ when ${\mathrm{\Theta }}_r=\rho I,n_r=n_c=n,{\lambda }_K\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)=O(n/K),{\lambda }_K\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)=O(n/K),{\pi }_{r,\min }=O(n/K),K=O\left(1\right)$, and $\tilde{P}=\rho P$ has diagonal entries $p_{\mathrm{in}}={\alpha }_{\mathrm{in}}\frac{\log \left(n\right)}{n}$ and non-diagonal entries $p_{\mathrm{out}}={\alpha }_{\mathrm{out}}\frac{\log \left(n\right)}{n}$.

$DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$ denotes a special directed network such that row communities have nearly equal sizes since ${\lambda }_K\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)=O(n/K)$, and column communities also have nearly equal sizes. By Corollary 1, for consistent estimation, we need $\frac{|p_{\mathrm{in}}-p_{\mathrm{out}}|}{\sqrt{\max (p_{\mathrm{in}},p_{\mathrm{out}})}}\gg \sqrt{\frac{\log \left(n\right)}{n}}$ under $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$. Since $\frac{|p_{\mathrm{in}}-p_{\mathrm{out}}|}{\sqrt{\max (p_{\mathrm{in}},p_{\mathrm{out}})}}=\frac{|{\alpha }_{\mathrm{in}}-{\alpha }_{\mathrm{out}}|\sqrt{\frac{\log \left(n\right)}{n}}}{\sqrt{\max ({\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})}}$, for consistent estimation, we need

$$\begin{array}{c}\hfill \frac{|{\alpha }_{\mathrm{in}}-{\alpha }_{\mathrm{out}}|}{\sqrt{\max ({\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})}}\gg 1\end{array}$$

(14)

Our numerical results in Section 5 support that DiMSC can estimate memberships for both row and column nodes when the threshold $\frac{|{\alpha }_{\mathrm{in}}-{\alpha }_{\mathrm{out}}|}{\sqrt{\max ({\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})}}\gg 1$ holds under $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$.

Remark 5. 

When $K=2$, the network is undirected (i.e., ${\mathrm{\Pi }}_r={\mathrm{\Pi }}_c$), all nodes are pure, and each community has an equal size, $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$ reduces to the SBM case such that nodes connect with probability $p_{\mathrm{in}}$ within clusters and $p_{\mathrm{out}}$ across clusters. This case has been well studied in recent years, see [50] and references therein. Especially, for this case, ref. [50] finds that exact recovery is possible if $|\sqrt{{\alpha }_{\mathrm{in}}}-\sqrt{{\alpha }_{\mathrm{out}}}|>\sqrt{2}$ and impossible if $|\sqrt{{\alpha }_{\mathrm{in}}}-\sqrt{{\alpha }_{\mathrm{out}}}|<\sqrt{2}$. For convenience, we use $SBM(n,p_{\mathrm{in}},p_{\mathrm{out}})$ to denote this case. Our numerical results in Section 5 show that DiMSC return consistent estimation under $SBM(n,p_{\mathrm{in}},p_{\mathrm{out}})$ when ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ are set in the impossible region of exact recovery but satisfy Equation (14).

Remark 6. 

In information theory, Shannon entropy [65] quantifies the amount of information in a variable, and it is a measure of uncertainty information of a probability distribution. We use a node membership entropy (NME) derived from Shannon theory to measure the node’s uncertainty about the node and all communities [66,67]. For row node i with membership ${\mathrm{\Pi }}_r(i,:)$, since ${\sum }_{k=1}^K{\mathrm{\Pi }}_r(i,k)=1$ and ${\mathrm{\Pi }}_r(i,k)$ can be seen as the probability that row node i belongs to row cluster k for $1\le k\le K$, NME of row node i is the Shannon entropy related to ${\mathrm{\Pi }}_r(i,:)$:

$$\begin{array}{c}\hfill NME\left(i\right)=-\sum _{k=1}^K{\mathrm{\Pi }}_r(i,k)\log \left({\mathrm{\Pi }}_r(i,k)\right).\end{array}$$

(15)

For column node j with membership ${\mathrm{\Pi }}_c(j,:)$, we can also obtain its NME by Equation (15). In particular, if a node belongs to each cluster with equal probability $\frac{1}{K}$, its NME is $\log \left(K\right)$ which is the maximum among all NME; if a node belongs to two clusters with equal probability $\frac{1}{2}$, its NME is $\log \left(2\right)$ which is less than $\log \left(K\right)$ when $K\ge 3$. Generally, we see that recovering memberships for mixed nodes is harder than for pure nodes since NME is 0 for pure nodes, while NME is larger than 0 for mixed nodes by the definition of NME.

5. Simulations

In this section, several experiments are conducted to investigate the performance of our DiMSC under DiDCMM. We compare our DiMSC with three model-based methods that can be thought of as special cases of our model DiDCMM. Model-based methods we compare include the DISIM algorithm proposed in [30], the DSCORE algorithm studied in [31], and the DiPCA algorithm which is obtained by using the adjacency matrix A to replace the regularized graph Laplacian matrix in the DISIM algorithm. Similar to [24,27], for simulations, we measure the errors for the inferred community membership matrices instead of simply each node. We measure the performance of DiMSC and its competitors by the mixed Hamming error rate (MHamm for short) defined below

$$\begin{array}{cc}\hfill & \mathrm{MHamm}=\max (\frac{{\min }_{\mathcal{P}\in S_{\mathcal{P}}}{\parallel {\widehat{\mathrm{\Pi }}}_r\mathcal{P}-{\mathrm{\Pi }}_r\parallel }_1}{n_r},\frac{{\min }_{\mathcal{P}\in S_{\mathcal{P}}}{\parallel {\widehat{\mathrm{\Pi }}}_c\mathcal{P}-{\mathrm{\Pi }}_c\parallel }_1}{n_c}),\hfill \end{array}$$

(16)

where $S_{\mathcal{P}}$ is the set of $K\times K$ permutation matrices.

For all simulations in this section, unless specified, we set the parameters $(n_r,n_c,K,P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\mathrm{\Theta }}_r)$ under DiDCMM as follows: let each row community and each column community have $n_0$ pure nodes; let all mixed row nodes (and mixed column nodes) have membership $(1/K,1/K,\dots ,1/K)$; for $z\ge 1$, we generate the degree parameters for row nodes as below: let ${\overline{\theta }}_r\in {\mathbb{R}}^{n_r\times 1}$ such that $1/{\overline{\theta }}_r\left(i\right)\stackrel{iid}{\sim }U(1,z)$ for $1\le i\le n_r$, where $U(1,z)$ denotes the uniform distribution on $[1,z]$, and set ${\theta }_r=\rho {\overline{\theta }}_r$, where we use $\rho$ to control the sparsity of the network; when $K=2$, P is set as

$$\begin{array}{cc}\hfill P_1& =\left[\begin{array}{cc}1& 0.1\\ 0.2& 1\end{array}\right]\mathrm{or}{P}_2=\left[\begin{array}{cc}0.8& 0.1\\ 0.2& 0.9\end{array}\right];\hfill \end{array}$$

when $K=3$,

$$\begin{array}{cc}\hfill P_3& =\left[\begin{array}{ccc}1& 0.1& 0.3\\ 0.2& 1& 0.4\\ 0.5& 0.2& 1\end{array}\right]\mathrm{or}{P}_4=\left[\begin{array}{ccc}0.8& 0.1& 0.3\\ 0.2& 0.9& 0.4\\ 0.5& 0.2& 1\end{array}\right];\hfill \end{array}$$

where $P_2$ and $P_4$ have non-unit diagonals, and we consider the two cases because we want to investigate DiMSC’s sensitivity when P has non-unit diagonals such that P disobeys Condition (I1).

After obtaining $P,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\theta }_r$, similar to the five simulation steps in [8], each simulation experiment contains the following steps:

(a) Let ${\mathrm{\Theta }}_r$ be the $n_r\times n_r$ diagonal matrix such that ${\mathrm{\Theta }}_r(i,i)={\theta }_r\left(i\right),1\le i\le n_r$. Set $\mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }$.

(b) Let W be an $n_r\times n_c$ matrix such that $W(i,j)$ are independent centered-Bernoulli with parameters $\mathrm{\Omega }(i,j)$. Let $\tilde{A}=\mathrm{\Omega }+W$.

(c) Set ${\tilde{S}}_r=\{i:{\sum }_{j=1}^{n_c}\tilde{A}(i,j)=0\}$ and ${\tilde{S}}_c=\{j:{\sum }_{i=1}^{n_r}\tilde{A}(i,j)=0\}$, i.e., ${\tilde{S}}_r$ (${\tilde{S}}_c$) is the set of row (column) nodes with 0 edges. Let A be the adjacency matrix obtained by removing rows respective to nodes in ${\tilde{S}}_r$ and removing columns respective to nodes in ${\tilde{S}}_c$ from $\tilde{A}$. Similarly, update ${\mathrm{\Pi }}_r$ by removing nodes in ${\tilde{S}}_r$ and update ${\mathrm{\Pi }}_c$ by removing nodes in ${\tilde{S}}_c$.

(d) Apply the DiMSC algorithm (and its competitors) to A. Record MHamm under investigations.

(e) Repeat (b)–(d) 50 times, and report the averaged MHamm over the 50 repetitions.

Let $n_{r,A}$ be the number of rows of A and $n_{c,A}$ be the number of columns of A. In our experiments, $n_{r,A}$ and $n_{c,A}$ are usually very close to $n_r$ and $n_c$; therefore we do not report the exact values of $n_{r,A}$ and $n_{c,A}$. After providing the above steps about how to generate A numerically under DiDCMM and how to record the error rates, now we describe our experiments in detail. We consider six experiments here. In experiments 1–6, we study the influence of the fraction of pure nodes, degree heterogeneity, connectivity across communities, sparsity, phase transition, and network size on performances of these methods, respectively.

Experiment 1 (a): Fraction of pure nodes. Set $n_r=200,n_c=300,z=5,\rho =1$ and P as $P_1$. Let $n_0$ range in $\{10,20,30,\dots ,100\}$. The numerical results are shown in Panel (a) of Figure 4. The results show that as the fraction of pure nodes increases for both row and column communities, all approaches perform better. Meanwhile, DiMSC performs best among all methods in Experiment 1 (a).

Figure 4. Errors against increasing $n_0$. y-axis: MHamm. Panel (a): Experiment 1 (a); Panel (b): Experiment 1 (b); Panel (c): Experiment 1 (c); Panel (d): Experiment 1 (d).

Experiment 1 (b): Fraction of pure nodes. All parameters are set the same as Experiment 1 (a) except that we set P as $P_2$ here. The numerical results are shown in Panel (b) of Figure 4. The results show that all methods perform better as $n_0$ increases, DiMSC outperforms its competitors, and DiMSC enjoys satisfactory performance even when P has non-unit diagonals.

Experiment 1 (c): Fraction of pure nodes. Set $n_r=600,n_c=900,z=5$, $\rho =1$, and P as $P_3$. Let $n_0$ range in $\{20,40,60,\dots ,200\}$. The numerical results are shown in Panel (c) of Figure 4, and we see that all methods perform better when there are more pure nodes and our DiMSC performs best.

Experiment 1 (d): Fraction of pure nodes. All parameters are set the same as Experiment 1 (c) except that we set P as $P_4$ here. The numerical results are shown in Panel (d) of Figure 4, and the analysis is similar to that of Experiment 1 (b).

Experiment 2 (a): Degree heterogeneity. Set $n_r=200,n_c=300,n_0=80,\rho =1$, and P as $P_1$. Let z range in $\{2,3,4,\dots ,12\}$. A lager z generates lesser edges. The results are displayed in Panel (a) of Figure 5. The results suggest that the error rates of DiMSC for both row and column nodes tend to increase as z increases. This phenomenon happens because decreasing degree heterogeneities for row nodes lowers the number of edges in the directed network; thus the network becomes harder to be detected for both row and column nodes. Meanwhile, DiMSC outperforms its competitors in this experiment, and it is interesting to see that the error rates of DI-SIM, DiPCA, and DSCORE are almost the same for this experiment.

Figure 5. Errors against increasing z. y-axis: MHamm. Panel (a): Experiment 2 (a); Panel (b): Experiment 2 (b); Panel (c): Experiment 2 (c); Panel (d): Experiment 2 (d); Panel (e): Experiment 2 (e); Panel (f): Experiment 2 (f); Panel (g): Experiment 2 (g); Panel (h): Experiment 2 (h).

Experiment 2 (b): Degree heterogeneity. All parameters are set the same as Experiment 2 (a) except that we set P as $P_2$ here. The results are displayed in Panel (b) of Figure 5, and we see that DiMSC performs satisfactorily when the directed network is not too sparse (i.e., a small z case) even when P has non-unit diagonals. Meanwhile, DiMSC significantly outperforms its competitors in this experiment.

Experiment 2 (c): Degree heterogeneity. Set $n_r=600,n_c=900,n_0=150$, $\rho =1$, and P as $P_3$. Let z range in $\{2,3,4,\dots ,12\}$. The results are shown in Panel (c) of Figure 5 and can be analyzed similarly to Experiment 2 (a).

Experiment 2 (d): Degree heterogeneity. All parameters are set the same as Experiment 2 (c) except that we set P as $P_4$ here. The results are displayed in Panel (d) of Figure 5 and are similar to that of Experiment 2 (b).

Experiment 2 (e): Degree heterogeneity. All parameters are set the same as Experiment 2(a) except that we set $n_0=0$ (so there are no pure nodes in both row and column communities), and all mixed row nodes have two different memberships (0.9, 0.1) and (0.1, 0.9), each with $\frac{n_r}{K}=100$ number of row nodes, and all mixed column nodes also have the above two memberships, each with $\frac{n_c}{K}=150$ number of column nodes. Panel (e) of Figure 5 shows the results, and we see that DiMSC performs satisfactorily for a small z even for the case when there are no pure nodes for both row and column communities. Meanwhile, DiMSC performs better than its competitors when $z<7$, and it perform poorer than its competitors when $z\ge 8$ for this experiment. Furthermore, compared with numerical results of Experiment 2 (a), we see that DI-SIM, DiPCA, and DSCORE have better performances in Experiment 2 (e). The possible reason is the memberships $(0.9,0.1)$ and $(0.1,0.9)$ are close to $(1,0)$ and $(0,1)$ somewhat.

Experiment 2 (f): Degree heterogeneity. All parameters are set the same as Experiment 2 (b) except that we set ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ the same as Experiment 2 (e). The results are shown in Panel (f) of Figure 5 and are similar to that of Experiment 2 (e).

Experiment 2 (g): Degree heterogeneity. All parameters are set the same as Experiment 2 (c) except that we set $n_0=0$, all mixed row nodes have three different memberships (0.8, 0.1, 0.1), (0.1, 0.8, 0.1), and $(0.1,0.1,0.8)$, each with $\frac{n_r}{K}=200$ number of row nodes, and all mixed column nodes also have the above four memberships, each with $\frac{n_c}{K}=300$ number of column nodes. The results are displayed in Panel (g) of Figure 5 and are similar to that of Experiment 2 (e).

Experiment 2 (h): Degree heterogeneity. All parameters are set the same as Experiment 2 (d) except that we set ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ the same as Experiment 2 (g). The results are shown in Panel (h) of Figure 5 and are similar to that of Experiment 2 (e).

Experiment 3 (a): Connectivity across communities. Set $n_r=200,n_c=300$, $n_0=80$, $z=5,\rho =1$. Set

$$P=\left[\begin{array}{cc}1& \beta -1\\ \beta -1& 1\end{array}\right].$$

and let $\beta$ range in $\{1,1.2,1.4,\dots ,4\}$. Decreasing $|\beta -2|$ increases the hardness of detecting such directed networks. Note that $\mathbb{P}(A(i,j)=1)=\mathrm{\Omega }(i,j)={\theta }_r\left(i\right){\mathrm{\Pi }}_r(i,:)P{\mathrm{\Pi }}_c^{\prime }(j,:)$ gives ${\max }_{i,j}\mathrm{\Omega }(i,j)={\theta }_{r,\max }{P}_{\max }$ should be no larger than 1. Since $P_{\max }$ may be larger than one in this experiment, after obtaining ${\theta }_r$, we need to update ${\theta }_r$ as ${\theta }_r/P_{\max }$. The results are displayed in Panel (a) of Figure 6, and they support the arguments given after Corollary 1 such that DiMSC performs better when $|\beta -2|$ increases and vice versa. Meanwhile, our DiMSC outperforms its competitors in this experiment.

Figure 6. Errors against increasing $\beta$. y-axis: MHamm. Panel (a): Experiment 3 (a); Panel (b): Experiment 3 (b); Panel (c): Experiment 3 (c); Panel (d): Experiment 3 (d); Panel (e): Experiment 3 (e); Panel (f): Experiment 3 (f); Panel (g): Experiment 3 (g); Panel (h): Experiment 3 (h).

Experiment 3 (b): Connectivity across communities. All parameters are set the same as Experiment 3 (a) except that we set

$$P=\left[\begin{array}{cc}0.8& \beta -1\\ \beta -1& 0.9\end{array}\right].$$

The results are displayed in Panel (b) of Figure 6, and we see that DiMSC performs better when $|\beta -2|$ increases even for the case that P has non-unit diagonals.Meanwhile, our DiMSC performs better than its competitors here.

Experiment 3 (c): Connectivity across communities. Set $n_r=600,n_c=900,n_0=150,z=5,\rho =1$. Set

$$P=\left[\begin{array}{ccc}1& \beta -1& \beta -1\\ \beta -1& 1& \beta -1\\ \beta -1& \beta -1& 1\end{array}\right].$$

and let $\beta$ range in $\{1,1.2,1.4,\dots ,4\}$. The results are displayed in Panel (c) of Figure 6 and can be analyzed similarly to Experiment 3 (a).

Experiment 3 (d): Connectivity across communities. All parameters are set the same as Experiment 3(c) except that we set

$$P=\left[\begin{array}{ccc}0.8& \beta -1& \beta -1\\ \beta -1& 0.9& \beta -1\\ \beta -1& \beta -1& 1\end{array}\right].$$

The results are displayed in Panel (d) of Figure 6 and can be analyzed similarly to Experiment 3 (b).

Experiment 3 (e): Connectivity across communities. All parameters are set the same as Experiment 3(a) except that we let ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ be the same as that of Experiment 2 (e) (so there are no pure nodes in both row and column communities.). Panel (e) of Figure 6 shows the results, and we see that DiMSC enjoys better performance when $|\beta -2|$ increases even in the case that there are no pure nodes for both row and column communities. Meanwhile, all methods have competitive performances for this experiment, and the possible reason that DiMSC’s competitors enjoy better performances here than in Experiment 3 (a) is analyzed in Experiment 2 (e).

Experiment 3 (f): Connectivity across communities. All parameters are set the same as Experiment 3 (b) except that we set ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ the same as Experiment 2 (e). The results are displayed in Panel (f) of Figure 6 and can be analyzed similarly to Experiment 3 (e).

Experiment 3 (g): Connectivity across communities. All parameters are set the same as Experiment 3 (c) except that we let ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ be the same as that of Experiment 2 (g) (so there are no pure nodes). Panel (g) of Figure 6 shows the results, and the analysis is similar to that of Experiment 3 (b).

Experiment 3 (h): Connectivity across communities. All parameters are set the same as Experiment 3 (d) except that we set ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ the same as Experiment 2 (g). Panel (h) of Figure 6 shows the results, and the analysis is similar to that of Experiment 3 (b).

Experiment 4 (a): Sparsity. Set $n_r=200,n_c=300,n_0=80,z=5,$ and P as $P_1$. Let $\rho$ range in $\{0.2,0.3,\dots ,1\}$. A larger $\rho$ indicates a denser network. Panel (a) in Figure 7 displays the simulation results of this experiment. We see that DiMSC performs better as the simulated directed network becomes denser, and DiMSC significantly outperforms its competitors in this experiment.

Figure 7. Errors against increasing $\rho$. y-axis: MHamm: Panel (a): Experiment 4 (a); Panel (b): Experiment 4 (b); Panel (c): Experiment 4 (c); Panel (d): Experiment 4 (d); Panel (e): Experiment 4 (e); Panel (f): Experiment 4 (f); Panel(g): Experiment 4 (g); Panel (h): Experiment 4 (h).

Experiment 4 (b): Sparsity. All parameters are set the same as Experiment 4 (a) except that P is set as $P_2$. Panel (b) of Figure 7 shows the results, and the analysis is similar to that of Experiment 2 (b).

Experiment 4 (c): Sparsity. Set $n_r=600,n_c=900,n_0=150,z=5,$ and P as $P_3$. Let $\rho$ range in $\{0.2,0.3,\dots ,1\}$. Panel (c) of Figure 7 shows the results, and the analysis is similar to that of Experiment 4 (a).

Experiment 4 (d): Sparsity. All parameters are set the same as Experiment 4 (c) except that P is set as $P_4$. Panel (d) of Figure 7 displays the results, and the analysis is similar to that of Experiment 4 (b).

Experiment 4 (e): Sparsity. All parameters are set the same as Experiment 4 (a) except that we let ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ be the same as that of Experiment 2 (e). Panel (e) of Figure 7 shows the results, and we see that DiMSC’s error rates decrease for a denser directed network even when all nodes are mixed. Meanwhile, all methods enjoy similar performances in this experiment.

Experiment 4 (f): Sparsity. All parameters are set the same as Experiment 4 (b) except that we set ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ the same as Experiment 2(e). Panel (f) of Figure 7 shows the results, and the analysis is similar to that of Experiment 4 (e).

Experiment 4 (g): Sparsity. All parameters are set the same as Experiment 4 (c) except that we let ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ be the same as that of Experiment 2 (g). Panel (g) of Figure 7 shows the results, and the analysis is similar to that of Experiment 4 (e).

Experiment 4 (h): Sparsity. All parameters are set the same as Experiment 4 (d) except that we set ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ the same as Experiment 2(g). Panel (h) of Figure 7 shows the results, and the analysis is similar to that of Experiment 4 (e).

Experiment 5 (a): Phase transition. Under $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$, set $K=2,n=n_r=n_c=300$. Let each row community have 100 pure nodes, each column community have 120 pure nodes, and all mixed nodes have membership $(1/2,1/2)$. Since $\max (p_{\mathrm{in}},p_{\mathrm{out}})=\max ({\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})\frac{\log \left(n\right)}{n}\le 1$, ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ should be set in $(0,\frac{n}{\log \left(n\right)}]$. We let ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ be in the range of $\{2.5,5,7.5,\dots ,50\}$. Panel (a) of Figure 8 displays the results. We see that DiMSC performs satisfactorily when ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ satisfy Equation (14), and this means that DiMSC achieves the threshold provided in Equation (14) under $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$.

Figure 8. Phase transition for DiMSC: darker pixels represent lower error rates. The red lines represent $\frac{|{\alpha }_{\mathrm{in}}-{\alpha }_{\mathrm{out}}|}{\sqrt{\max ({\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})}}=1$. Panel (a): Experiment 5 (a); Panel (b): Experiment 5 (b).

Experiment 5 (b): Phase transition. Under $DiDCMM(n,K,{\mathrm{\Pi }}_r,{\mathrm{\Pi }}_c,{\alpha }_{\mathrm{in}},{\alpha }_{\mathrm{out}})$, set $K=3,n=n_r=n_c=300$. Let each row community have 60 pure nodes, each column community have 80 pure nodes, and all mixed nodes have membership $(1/3,1/3,1/3)$. We also let ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ be in the range of $\{2.5,5,7.5,\dots ,50\}$. Panel (b) of Figure 8 displays the results, and the analysis is similar to that of Experiment 5 (a).

For Experiments 1–5, we can conclude that DiMSC outperforms its competitors, and this supports our analysis in Remark 6 because DiMSC is designed to estimate mixed memberships, while its competitors are designed for community partition of pure nodes.

Experiment 6: Network size. Under $SBM(n,p_{\mathrm{in}},p_{\mathrm{out}})$, let ${\alpha }_{\mathrm{in}}=2$ and ${\alpha }_{\mathrm{out}}=0.0001$. On the one hand, we have $\sqrt{{\alpha }_{\mathrm{in}}}-\sqrt{{\alpha }_{\mathrm{out}}}=\sqrt{2}-0.01<\sqrt{2}$, i.e., ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ locates in the impossible region of exact recovery introduced in [50]. On the other hand, we have $\frac{{\alpha }_{\mathrm{in}}-{\alpha }_{\mathrm{out}}}{\sqrt{{\alpha }_{\mathrm{in}}}}>1$, i.e., ${\alpha }_{\mathrm{in}}$ and ${\alpha }_{\mathrm{out}}$ satisfy Equation (14) for DiMSC’s consistent estimation. Let n range in $\{1000,2000,3000,\dots ,15000\}$. For each n in this experiment, we report the averaged error rate and running time of DiMSC over 10 independent repetitions. The results are shown in Figure 9. From Panel (a) of Figure 9, we see that DiMSC enjoys satisfactory performance with a small error rate for this experiment. Panels (b) of Figure 9 says that DiMSC processes computer-generated networks of up to 15,000 nodes within hundreds of seconds.

Figure 9. Numerical results for Experiment 6. Panel (a): MHamm; Panel (b): running time.

Remark 7. 

For visuality, we provide some examples of different types of directed networks generated under DiDCMM in this remark. Let ${\theta }_r\left(i\right)=0.9+\frac{i^2}{9n_r^2}$ for $1\le i\le n_r$. Let each row community has $n_{r,0}$ pure nodes, and each column community has $n_{c,0}$ pure nodes. Let all mixed nodes have membership $(1/K,\dots ,1/K)$. For the setting of P, we set it as

$$\begin{array}{cc}\hfill P_a& =\left[\begin{array}{cc}0.9& 0.05\\ 0.1& 0.95\end{array}\right]\mathrm{or}{P}_b=\left[\begin{array}{cc}0.1& 0.95\\ 0.9& 0.05\end{array}\right]\mathrm{or}{P}_c=\left[\begin{array}{cc}12& 1\\ 0& 12\end{array}\right]\frac{\log \left(n_r\right)}{n_r}\mathrm{or}{P}_d=\left[\begin{array}{cc}0& 12\\ 12& 1\end{array}\right]\frac{\log \left(n_r\right)}{n_r}\mathrm{or}\hfill \\ \hfill P_e& =\left[\begin{array}{ccc}12& 1& 0\\ 0& 12& 0\\ 1& 0& 12\end{array}\right]\frac{\log \left(n_r\right)}{n_r}\mathrm{or}{P}_f=\left[\begin{array}{ccc}1& 0& 12\\ 12& 0& 0\\ 0& 12& 1\end{array}\right]\frac{\log \left(n_r\right)}{n_r},\hfill \end{array}$$

where $K=2$ when P is $P_a,P_b,P_c$ or $P_d$, and $K=3$ when P is $P_e$ or $P_f$. Meanwhile, we can generate different types of directed networks under DiDCMM by considering the above six different settings of P, where these different types are also considered in Experiments 1–6, and we mainly provide the visuality for these directed networks with different structures provided in different P for this remark. Note that we allow P to have non-unit diagonals here because Condition (I1) is mainly for our theoretical buildings, and results for previous experiments show that DiMSC performs stable even when P has non-unit diagonals. We consider below eight settings.

Model Setup 1: Set $n_r=16,n_{r,0}=6,n_c=16,n_{c,0}=7$, and P as $P_a$. For this setup, a directed network with 16 row nodes and 16 column nodes is generated from DiDCMM. Figure 10 shows a directed network $\mathcal{N}$ generated under Model Setup 1, where we also report DiMSC’s error rate. Figure 10 says that there are more directed edges sent from row nodes 1–6 to column nodes 1–7 than from row nodes 7–12 to column nodes 1–7 for $P_a$. With given adjacency matrix A and known memberships ${\mathrm{\Pi }}_r$ and ${\mathrm{\Pi }}_c$ for this setup, readers can apply our DiMSC directly to A given in Panel (a) of Figure 10 to check the effectiveness of DiMSC.

Figure 10. Illustration for a directed network under Model Setup 1. Panel (a): Adjacency matrix of $\mathcal{N}$, where black square denotes 1; Panel (b): directed network $\mathcal{N}$, where red (blue) points indicate row (column) nodes. The error rate MHamm defined in Equation (16) of our DiSMC algorithm for this directed network $\mathcal{N}$ is 0.0377.

Model Setup 2: All settings are the same as Model Setup 1 except that we let P be $P_b$. The directed network $\mathcal{N}$ and its adjacency matrix are shown in Figure 11. We see that there are more directed edges sent from row nodes 1–6 to column nodes 10–16 than from row nodes 7–12 to column nodes 10–16 for $P_b$, which means that directed network generated using $P_b$ and directed network from $P_a$ has different structures.

Figure 11. Illustration for a directed network under Model Setup 2. Panel (a): Adjacency matrix A; Panel (b): directed network $\mathcal{N}$. MHamm of DiMSC for this directed network $\mathcal{N}$ is 0.0424.

Model Setup 3: Set $n_r=32,n_{r,0}=14,n_c=28,n_{c,0}=12$, and P as $P_a$. For this setup, a bipartite network with 32 row nodes and 28 column nodes are generated from DiDCMM. Figure 12 shows this bipartite network and its adjacency matrix.

Figure 12. Illustration for a bipartite network under Model Setup 3. Panel (a): Adjacency matrix A; Panel (b): bipartite network $\mathcal{N}$. MHamm of DiMSC for this bipartite network $\mathcal{N}$ is 0.0313.

Model Setup 4: All settings are the same as Model Setup 3 except that we let P be $P_b$. Figure 13 displays the results, and we see that the bipartite network from $P_b$ also has a different structure compared with the one generated from using $P_a$ under DiDCMM.

Figure 13. Illustration for a bipartite network under Model Setup 4. Panel (a): Adjacency matrix A; Panel (b): bipartite network $\mathcal{N}$. MHamm of DiMSC for this bipartite network $\mathcal{N}$ is 0.0320.

Model Setup 5: Set $n_r=100,n_{r,0}=48,n_c=100,n_{c,0}=45$, and P as $P_c$. Figure 14 shows the row and column communities for a directed network generated from Setup 5 under DiDCMM, where we plot the directed network directly.

Figure 14. Illustration for a directed network under Model Setup 5. Panels (a,b) show the row and column communities, respectively. In these two panels, dots in the same color are pure nodes in the same communities, and a square indicates mixed nodes. MHamm of DiMSC for this directed network $\mathcal{N}$ is 0.0181.

Model Setup 6: All settings are the same as Model Setup 5 except that we let P be $P_d$. Figure 15 shows a directed network obtained from this setup, and we see that the structure of the directed network from $P_d$ in Figure 15 differs a lot from that of the directed network from $P_c$ shown in Figure 14.

Figure 15. Illustration for a directed network under Model Setup 6. Panels (a,b) show the row and column communities, respectively. MHamm of DiMSC for this directed network $\mathcal{N}$ is 0.0185.

Model Setup 7: Set $n_r=100,n_{r,0}=30,n_c=100,n_{c,0}=32$, and P as $P_e$. Figure 16 shows a directed network generated from this setup.

Figure 16. Illustration for a directed network under Model Setup 7. Panels (a,b) show the row and column communities, respectively. MHamm of DiMSC for this directed network $\mathcal{N}$ is 0.0266.

Model Setup 8: All settings are the same as Model Setup 7 except that we let P be $P_f$. Figure 17 displays a directed network generated from this setup, and we see that directed networks from $P_f$ and $P_e$ have different structures by comparing Figure 16 and Figure 17.

Figure 17. Illustration for a directed network under Model Setup 8. Panels (a,b) show the row and column communities, respectively. MHamm of DiMSC for this directed network $\mathcal{N}$ is 0.0279.

6. Application to Real-World Directed Networks

For the empirical directed networks considered here, row nodes are always the same as column nodes, so we let $n_r=n_c=n$. For ${\widehat{\mathrm{\Pi }}}_r$, we call node i highly mixed node if $0.8\ge {\max }_{1\le k\le K}{\widehat{\mathrm{\Pi }}}_r(i,k)$, similar for ${\widehat{\mathrm{\Pi }}}_c$. A highly mixed node tells us whether a node has mixed memberships and belongs to multiple communities. Let ${\tau }_r=\frac{|i:0.8\ge {\max }_{1\le k\le K}{\widehat{\mathrm{\Pi }}}_r(i,k)|}{n}$ be the proportion of highly mixed nodes among all nodes to measure the mixability of all row communities. Define ${\tau }_c$ similar to ${\tau }_r$. Let ${\widehat{\ell }}_r$ be a vector such that ${\widehat{\ell }}_r\left(i\right)={\mathrm{argmax}}_{1\le k\le K}{\widehat{\mathrm{\Pi }}}_r(i,k)$ for $1\le i\le n$, where we use ${\widehat{\ell }}_r\left(i\right)$ to denote the home base row community of node i. Define ${\widehat{\ell }}_c$ similar to ${\widehat{\ell }}_r$. To measure the asymmetric structure of a directed network, we use

$$\begin{array}{c}\hfill {\mathrm{Hamm}}_{rc}=\frac{{\min }_{\mathcal{P}\in S_{\mathcal{P}}}{\parallel {\widehat{\mathrm{\Pi }}}_c\mathcal{P}-{\widehat{\mathrm{\Pi }}}_r\parallel }_1}{n},\end{array}$$

where a large ${\mathrm{Hamm}}_{rc}$ means that the structure of row clusters differs a lot from that of column clusters. For $1\le i\le n$, let $d_r\left(i\right)={\sum }_{j=1}^{n}A(i,j)$ be the number of edges sent by node i, $d_c\left(i\right)={\sum }_{j=1}^{n}A(j,i)$ be the number of edges received by node i, where $d_r\left(i\right)$ (and $d_c\left(i\right)$) is the out degree (in degree) of node i. Since there are many nodes with zero in degree or out degree for real-world directed network, we need the below pre-processing: for any directed network $\mathcal{N}$, we let $A_m$ be its adjacency matrix for any positive integer m such that $A_m$ is connected, and every node has at least m in degree and m out degree in $A_m$.

We apply DiMSC to the following real-world directed networks to discover their mixability, asymmetries, and directional communities.

Political blogs: This is a directed network of hyperlinks between weblogs on US politics [68]. In this data, node means a blog, and edge means a hyperlink. This data can be downloaded from http://www-personal.umich.edu/~mejn/netdata/ (accessed on 28 August 2022). It is well-known that there are two parties, “liberal” and “conservative”, so $K=2$ for this data. The are 1490 nodes in the original data. After pre-processing, $A_1\in {\{0,1\}}^{813\times 813},A_3\in {\{0,1\}}^{495\times 495},A_6\in {\{0,1\}}^{285\times 285},A_9\in {\{0,1\}}^{158\times 158}$, where we focus on the cases when $m=1,3,6,9$ for this data here. Meanwhile, we use political blogs $A_m$ to denote this network when its adjacency matrix is $A_m$, where every node has a degree at least m. Similar notations hold for other real-world directed networks used in this paper.

Wikipedia links (gan): This directed network consists of the Wikilinks of Wikipedia in the Gan Chinese language (gan). In this data, node means an article, and the directed edge is a Wikilink [69]. This data can be downloaded from http://konect.cc/networks/wikipedia_link_gan (accessed on 28 August 2022). There are 9189 nodes in the original data. After pre-processing, $A_1\in {\{0,1\}}^{6012\times 6012},A_{30}\in {\{0,1\}}^{820\times 820},A_{60}\in {\{0,1\}}^{559\times 569},A_{90}\in {\{0,1\}}^{240\times 240}$, where we study the cases $m=1,30,60,90$ for this data. The leading 20 singular values of $A_1,A_{30},A_{60},A_{90}$ shown in Panels (e)–(h) of Figure 18 suggest $K=2$ for these four adjacency matrices, where [30] also uses eigengap to estimate K.

Figure 18. Leading 20 singular values of real-world directed networks used in this paper. Panel (a): political blogs $A_1$; Panel (b): political blogs $A_3$; Panel (c): political blogs $A_6$; Panel (d): political blogs $A_9$; Panel (e): Wikipedia links (gan) $A_1$; Panel (f): Wikipedia links (gan) $A_{30}$; Panel (g): Wikipedia links (gan) $A_{60}$; Panel (h): Wikipedia links (gan) $A_{90}$; Panel (i): Wikipedia links (nah) $A_1$; Panel (j): Wikipedia links (nah) $A_{20}$; Panel (k): Wikipedia links (nah) $A_{30}$; Panel (l): Wikipedia links (nah) $A_{40}$.

Wikipedia links (nah): This network consists of the Wikilinks of the N $\overline{\mathrm{a}}$huatl language (nah) [69] and can be downloaded from http://konect.cc/networks/wikipedia_link_nah/ (accessed on 28 August 2022). The original data has 10285 nodes. After pre-processing, $A_1\in {\{0,1\}}^{6924\times 6924},A_{20}\in {\{0,1\}}^{1057\times 1057},A_{30}\in {\{0,1\}}^{486\times 486},A_{40}\in {\{0,1\}}^{136\times 136}$. Panel (i) of Figure 18 suggests $K=4$ for $A_1$, and Panels (j)–(l) of Figure 18 suggest $K=2$ for $A_{20},A_{30}$, and $A_{40}$. Note that it only takes around 4 seconds for DiMSC to estimate memberships of Wikipedia links (nah) $A_1$.

The proportions of highly mixed nodes and ${\mathrm{Hamm}}_{rc}$ when applying DiMSC on the above real-world directed networks are reported in Table 1. For the political blogs network, small ${\tau }_r,{\tau }_c$, and ${\mathrm{Hamm}}_{rc}$ indicate that there are only a few highly mixed nodes, and the structure of row communities is similar to that of column communities, i.e., there is a slight asymmetry for this data. For Wikipedia links (gan) $A_1$ and Wikipedia links (nah) $A_1$, they have a large proposition of highly mixed nodes in both row and column communities, and the row communities differ a lot from column communities, suggesting heavy asymmetric structure between row and column communities for these two data. For Wikipedia links (gan) $A_{30},A_{60}$, and Wikipedia links (nah) $A_{20}$, we see that the proportion of highly mixed nodes for row (column) communities is small (large), and there is a slight asymmetric for these data. For Wikipedia links (gan) $A_{90}$ and Wikipedia links (nah) $A_{30},A_{40}$, there is no highly mixed node, and the structure of row clusters is similar to that of column clusters. For visualization, we plot the row and column communities as well as highly mixed nodes by applying DiMSC to some of these directed networks in Figure 19 and Figure 20.

Table 1. ${\tau }_r,{\tau }_c$, and ${\mathrm{Hamm}}_{rc}$ obtained from DiMSC for real-world directed networks used in this paper.

Data	$\mathit{\tau }$${}_{\mathit{r}}$	$\mathit{\tau }$${}_{\mathit{c}}$	Hamm${}_{\mathbf{rc}}$
Political blogs $A_1$	0.0455	0.1353	0.0893
Political blogs $A_3$	0.0481	0.1570	0.0705
Political blogs $A_6$	0.0386	0.1368	0.0662
Political blogs $A_9$	0.0443	0.1772	0.0771
Wikipedia links (gan) $A_1$	0.1505	0.6051	0.3528
Wikipedia links (gan) $A_{30}$	0.0817	0.1902	0.0547
Wikipedia links (gan) $A_{60}$	0.0054	0.1145	0.0664
Wikipedia links (gan) $A_{90}$	0	0	0.0203
Wikipedia links (nah) $A_1$	0.2718	0.3521	0.2065
Wikipedia links (nah) $A_{20}$	0.0937	0.1722	0.0488
Wikipedia links (nah) $A_{30}$	0	0	0.0046
Wikipedia links (nah) $A_{40}$	0	0	0

Figure 19. Row and column communities detected by DiMSC for political blogs. Colors indicate clusters, and a green square indicates highly mixed nodes, where the row and column communities are obtained from ${\widehat{\ell }}_r$ and ${\widehat{\ell }}_c$, respectively. Panel (a): political blogs $A_1$; Panel (b): political blogs $A_1$; Panel (c): political blogs $A_3$; Panel (d): political blogs $A_3$; Panel (e): political blogs $A_6$; Panel (f): political blogs $A_6$; Panel (g): political blogs $A_9$; Panel (h): political blogs $A_9$.

Figure 20. Row and column communities detected by DiMSC for Wikipedia links (gan) $A_{90}$ and Wikipedia links (nah) $A_{40}$. Colors indicate clusters, where the row and column communities are obtained from ${\widehat{\ell }}_r$ and ${\widehat{\ell }}_c$, respectively. Panel (a): Wikipedia links (gan) $A_{90}$; Panel (b): Wikipedia links (gan) $A_{90}$; Panel (c): Wikipedia links (nah) $A_{40}$; Panel (d): Wikipedia links (nah) $A_{40}$.

7. Discussion and Conclusions

In this paper, we propose a novel directed degree corrected mixed membership (DiDCMM) model. DiDCMM models a directed network with mixed memberships for row nodes with degree heterogeneities and column nodes without degree heterogeneities. DiDCMM is identifiable when the two well-used Conditions (I1) and (I2) hold. It should be mentioned that a model modeling a directed network with mixed memberships for both row and column nodes with degree heterogeneities is unidentifiable unless considering some nontrivial conditions. To fit the model, we propose a provably consistent spectral algorithm called DiMSC to infer community memberships for both row and column nodes in a directed network generated by DiDCMM. DiMSC is designed based on the SVD of the adjacency matrix, where we apply the SP algorithm to hunt for the corners in the simplex structure and the SVM-cone algorithm to hunt for the corners in the cone structure. The theoretical results of DiMSC show that it consistently recovers memberships of both row nodes and column nodes under mild conditions. Meanwhile, when DiDCMM degenerates to MMSB, our theoretical results match that of Theorem 2.2 [24] when their DCMM degenerates to MMSB under mild conditions. Experiments conducted on synthetic directed networks generated from DiDCMM verify the effectiveness and the stability of Conditions (I1) and (I2) of DiMSC. Results for real-world directed networks show that DiMSC reveals highly mixed nodes and asymmetries in the structure of row and column communities. The model DiDCMM and the algorithm DiMSC developed in this paper are useful to discover asymmetry for a directed network with mixed memberships. DiDCMM can also generate an artificially directed network with mixed memberships as a benchmark directed network for research purposes. We wish that DiDCMM and DiMSC can be widely applied in social network analysis.

The proposed model DiDCMM and the algorithm DiMSC can be extended in many ways. Similar to [24,57], we may obtain an ideal simplex from U using the idea of the entry-wise ratio proposed in [8]. Meanwhile, DiMSC is designed based on the SVD of the adjacency matrix, and similar to [5,7,11,30], we may design spectral algorithms based on the regularized Laplacian matrix under DiDCMM. Extending DiDCMM from an un-weighted directed network to a weighted directed network with an application of the distribution-free idea introduced in [62] is one of our future research directions. The SVD step of DiMSC can be accelerated by the random projection and random sampling ideas introduced in [70] to process large-scale directed networks. Instead of simply using eigengap to find K, in our future work, it is worth focusing on estimating the number of communities in a directed network generated under ScBM (and DCScBM) [30] and DiDCMM. Ref. [46] proposes an algorithm to uncover boundary nodes that spread information between communities in undirected social networks. It is an interesting topic to extend works in [46] to directed networks generated from ScBM, DCScBM, and DiDCMM. We leave them for our future work.

Appendix A.1. Proof of Proposition 1

Proof. 

Let $\mathrm{\Omega }=U\mathrm{\Lambda }{V}^{\prime }$ be the compact singular value decomposition of $\mathrm{\Omega }$. Lemma 1 gives $V={\mathrm{\Pi }}_c{B}_c\equiv {\mathrm{\Pi }}_{c}V({\mathcal{I}}_c,:)$. Since $\mathrm{\Omega }=\tilde{\mathrm{\Omega }}$, V also equals to ${\tilde{\mathrm{\Pi }}}_{c}V({\mathcal{I}}_c,:)$, which gives that ${\mathrm{\Pi }}_c={\tilde{\mathrm{\Pi }}}_c$.

Since $\mathrm{\Omega }({\mathcal{I}}_r,{\mathcal{I}}_c)={\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Pi }}_r({\mathcal{I}}_r,:)P{\mathrm{\Pi }}_c^{\prime }({\mathcal{I}}_c,:)={\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)P=U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$ by Condition (I2), we have ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)P=U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$, which gives that ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)=\mathrm{diag}\left(U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right)$. From this step, we see that if P’s diagonal entries are not ones, we cannot obtain ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)=\mathrm{diag}\left(U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right)$ which leads to a consequence that ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$ does not equal to ${\tilde{\mathrm{\Theta }}}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$; hence Condition (I1) is necessary by Condition (I1). Since $\mathrm{\Omega }=\tilde{\mathrm{\Omega }}$, we also have ${\tilde{\mathrm{\Theta }}}_r({\mathcal{I}}_r,{\mathcal{I}}_r)=\mathrm{diag}\left(U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)\right)$, which gives that ${\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)={\tilde{\mathrm{\Theta }}}_r({\mathcal{I}}_r,{\mathcal{I}}_r)$. Since ${\tilde{\mathrm{\Theta }}}_r({\mathcal{I}}_r,{\mathcal{I}}_r)\tilde{P}$ also equals to $U({\mathcal{I}}_r,:)\mathrm{\Lambda }{V}^{\prime }({\mathcal{I}}_c,:)$, we have $P=\tilde{P}$.

Lemma 1 gives that $U={\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r$, where $B_r={\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$. Since $\mathrm{\Omega }=\tilde{\mathrm{\Omega }}$, we also have $U={\tilde{\mathrm{\Theta }}}_r{\tilde{\mathrm{\Pi }}}_r{\tilde{B}}_r$. Since ${\tilde{B}}_r={\tilde{\mathrm{\Theta }}}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)={\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$, we have ${\tilde{B}}_r=B_r$. Since $U={\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r={\tilde{\mathrm{\Theta }}}_r{\tilde{\mathrm{\Pi }}}_r{\tilde{B}}_r={\tilde{\mathrm{\Theta }}}_r{\tilde{\mathrm{\Pi }}}_r{B}_r$, we have ${\mathrm{\Theta }}_r{\mathrm{\Pi }}_r={\tilde{\mathrm{\Theta }}}_r{\tilde{\mathrm{\Pi }}}_r$. Since each row of ${\mathrm{\Pi }}_r$ or ${\tilde{\mathrm{\Pi }}}_r$ is a PMF, ${\mathrm{\Theta }}_r={\tilde{\mathrm{\Theta }}}_r,{\mathrm{\Pi }}_r={\tilde{\mathrm{\Pi }}}_r$, and the claim follows.    □

Appendix B.1. Proof of Lemma 1

Proof. 

First, we consider U and V. Since $\mathrm{\Omega }=U\mathrm{\Lambda }{V}^{\prime }$, we have $U=\mathrm{\Omega }V{\mathrm{\Lambda }}^{-1}$ since $V^{\prime }V=I_K$. Recall that $\mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }$, we have $U={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }V{\mathrm{\Lambda }}^{-1}={\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r$, where we set $B_r=P{\mathrm{\Pi }}_c^{\prime }V{\mathrm{\Lambda }}^{-1}$ and sure it is unique. Since $U({\mathcal{I}}_r,:)={\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Pi }}_r({\mathcal{I}}_r,:)B_r={\mathrm{\Theta }}_r({\mathcal{I}}_r,{\mathcal{I}}_r)B_r$, we have $B_r={\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$.

Similarly, since $\mathrm{\Omega }=U\mathrm{\Lambda }{V}^{\prime }$, we have $V^{\prime }={\mathrm{\Lambda }}^{-1}{U}^{\prime }\mathrm{\Omega }$ since $U^{\prime }U=I_K$, hence $V={\mathrm{\Omega }}^{\prime }U{\mathrm{\Lambda }}^{-1}$. Recall that $\mathrm{\Omega }={\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }$, we have $V={\left({\mathrm{\Theta }}_r{\mathrm{\Pi }}_{r}P{\mathrm{\Pi }}_c^{\prime }\right)}^{\prime }U{\mathrm{\Lambda }}^{-1}={\mathrm{\Pi }}_c{P}^{\prime }{\mathrm{\Pi }}_r^{\prime }{\mathrm{\Theta }}_{r}U{\mathrm{\Lambda }}^{-1}={\mathrm{\Pi }}_c{B}_c$, where we set $B_c=P^{\prime }{\mathrm{\Pi }}_r^{\prime }{\mathrm{\Theta }}_{r}U{\mathrm{\Lambda }}^{-1}$ and sure it is unique. Since $V({\mathcal{I}}_c,:)={\mathrm{\Pi }}_c({\mathcal{I}}_c,:)B_c=B_c$, we have $B_c=V({\mathcal{I}}_c,:)$. Meanwhile, for $1\le j\le n_c$, we have $V(j,:)=e_j^{\prime }{\mathrm{\Pi }}_c{B}_c={\mathrm{\Pi }}_c(j,:)B_c$. Hence, we have $V(j,:)=V(\overline{j},:)$ as long as ${\mathrm{\Pi }}_c(j,:)={\mathrm{\Pi }}_c(\overline{j},:)$.

Now, we show the ideal cone structure that appears in $U_{*}$. For convenience, set $M={\mathrm{\Pi }}_r{B}_r$, hence $U={\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r$ gives $U={\mathrm{\Theta }}_{r}M$. Hence, we have $U(i,:)=e_i^{\prime }U={\mathrm{\Theta }}_r(i,i)M(i,:)$. Therefore, $U_{*}(i,:)=\frac{U(i,:)}{{\parallel U(i,:)\parallel }_F}=\frac{M(i,:)}{{\parallel M(i,:)\parallel }_F}$, combine it with the fact that $B_r={\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$, we have

$$\begin{array}{cc}\hfill U_{*}& =\left[\begin{array}{cccc}\frac{1}{{\parallel M(1,:)\parallel }_F}& & & \\ & \frac{1}{{\parallel M(2,:)\parallel }_F}& & \\ & & \ddots & \\ & & & \frac{1}{\parallel M(n_r,:){\parallel }_F}\end{array}\right]{\mathrm{\Pi }}_r{B}_r\hfill \\ & =\left[\begin{array}{c}{\mathrm{\Pi }}_r(1,:)/{\parallel M(1,:)\parallel }_F\\ {\mathrm{\Pi }}_r(2,:)/{\parallel M(2,:)\parallel }_F\\ ⋮\\ {\mathrm{\Pi }}_r(n_r,:)/{\parallel M(n_r,:)\parallel }_F\end{array}\right]{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U_{*}({\mathcal{I}}_r,:).\hfill \end{array}$$

Therefore, we have

$$\begin{array}{c}\hfill Y=\left[\begin{array}{c}{\mathrm{\Pi }}_r(1,:)/{\parallel M(1,:)\parallel }_F\\ {\mathrm{\Pi }}_r(2,:)/{\parallel M(2,:)\parallel }_F\\ ⋮\\ {\mathrm{\Pi }}_r(n_r,:)/{\parallel M(n_r,:)\parallel }_F\end{array}\right]{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)=N_M{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r),\end{array}$$

where $N_M$ is a diagonal matrix with $N_M(i,i)=\frac{1}{{\parallel M(i,:)\parallel }_F}$ for $1\le i\le n_r$. All entries of Y are nonnegative, and since we assume that each community has at least one pure node, no row of Y is 0.

Then, we prove that $U_{*}(i,:)=U_{*}(\overline{i},:)$ when ${\mathrm{\Pi }}_r(i,:)={\mathrm{\Pi }}_r(\overline{i},:)$. For $1\le i\le n_r$, we have

$$\begin{array}{cc}\hfill U_{*}(i,:)& =e_i^{\prime }{U}_{*}=\frac{1}{{\parallel M(i,:)\parallel }_F}{e}_i^{\prime }M=\frac{1}{\parallel {\mathrm{\Pi }}_r(i,:)B_r{\parallel }_F}{\mathrm{\Pi }}_r(i,:)B_r,\hfill \end{array}$$

and the claim follows immediately.    □

Appendix B.2. Proof of Lemma 2

Proof. 

Since $I=U^{\prime }U=B_r^{\prime }{\mathrm{\Pi }}_r^{\prime }{\mathrm{\Theta }}_r^2{\mathrm{\Pi }}_r{B}_r=U^{\prime }({\mathcal{I}}_r,:){\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Pi }}_r^{\prime }{\mathrm{\Theta }}_r^2{\mathrm{\Pi }}_r{\mathrm{\Theta }}^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$ and $\mathrm{rank}\left(U({\mathcal{I}}_r,:)\right)=K$ (i.e., the inverse of $U({\mathcal{I}}_r,:)$ exists), we have ${\left(U({\mathcal{I}}_r,:)U^{\prime }({\mathcal{I}}_r,:)\right)}^{-1}={\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Pi }}_r^{\prime }{\mathrm{\Theta }}_r^2{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)$. Since $U_{*}({\mathcal{I}}_r,:)=N_U({\mathcal{I}}_r,{\mathcal{I}}_r)U({\mathcal{I}}_r,:)$, we have

$$\begin{array}{c}\hfill {\left(U_{*}({\mathcal{I}}_r,:)U_{*}^{\prime }({\mathcal{I}}_r,:)\right)}^{-1}=N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Theta }}^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r){\mathrm{\Pi }}_r^{\prime }{\mathrm{\Theta }}_r^2{\mathrm{\Pi }}_r{\mathrm{\Theta }}_r^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r)N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r).\end{array}$$

Since all entries of $N_U^{-1}({\mathcal{I}}_r,{\mathcal{I}}_r),{\mathrm{\Pi }}_r,{\mathrm{\Theta }}_r$ and nonnegative and $N,{\mathrm{\Theta }}_r$ are diagonal matrices, we see that all entries of ${\left(U_{*}({\mathcal{I}}_r,:)U_{*}^{\prime }({\mathcal{I}}_r,:)\right)}^{-1}$ are nonnegative, and its diagonal entries are strictly positive, hence we have ${\left(U_{*}({\mathcal{I}}_r,:)U_{*}^{\prime }({\mathcal{I}}_r,:)\right)}^{-1}\mathbf{1}>0$.    □

Appendix B.3. Proof of Theorem 1

Proof. 

For column nodes, Remark A1 guarantees that SP algorithm returns ${\mathcal{I}}_c$ when the input is V with K column communities, hence ideal DiMSC recovers ${\mathrm{\Pi }}_c$ exactly. For row nodes, Remark A2 guarantees that SVM-cone algorithm returns ${\mathcal{I}}_r$ when the input is $U_{*}$ with K row communities, hence ideal DiMSC recovers ${\mathrm{\Pi }}_r$ exactly, and this theorem follows.    □

Appendix E.1. Proof of Theorem 2

Proof. 

We bound $\parallel e_j^{\prime }({\widehat{\mathrm{\Pi }}}_c-{\mathrm{\Pi }}_c{\mathcal{P}}_c){\parallel }_1$ first. Recall that $Z_c={\mathrm{\Pi }}_c,{\mathrm{\Pi }}_c(j,:)=\frac{Z_c(j,:)}{\parallel Z_c{(j,:)\parallel }_1},{\widehat{\mathrm{\Pi }}}_c(i,:)=\frac{{\widehat{Z}}_c(j,:)}{\parallel {\widehat{Z}}_c{(j,:)\parallel }_1}$, for $1\le j\le n_c$, since

$$\begin{array}{cc}\hfill \parallel e_j^{\prime }({\widehat{\mathrm{\Pi }}}_c-{\mathrm{\Pi }}_c{\mathcal{P}}_c){\parallel }_1& =\parallel \frac{e_j^{\prime }{\widehat{Z}}_c}{\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1}-\frac{e_j^{\prime }{Z}_c{\mathcal{P}}_c}{\parallel e_j^{\prime }{Z}_c{\mathcal{P}}_c{\parallel }_1}{\parallel }_1={\parallel \frac{e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{Z}_c{\parallel }_1-e_j^{\prime }{Z}_c{\mathcal{P}}_c{\parallel e_j^{\prime }{\widehat{Z}}_c\parallel }_1}{\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1{\parallel e_j^{\prime }{Z}_c\parallel }_1}\parallel }_1\hfill \\ & =\parallel \frac{e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{Z}_c{\parallel }_1-e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1+e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1-e_j^{\prime }{Z}_c\mathcal{P}{\parallel e_j^{\prime }{\widehat{Z}}_c\parallel }_1}{\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1{\parallel e_j^{\prime }{Z}_c\parallel }_1}{\parallel }_1\hfill \\ \hfill & \le \frac{\parallel e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{Z}_c{\parallel }_1-e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1{\parallel }_1+\parallel e_j^{\prime }{\widehat{Z}}_c\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1-e_j^{\prime }{Z}_c{\mathcal{P}}_c\parallel e_j^{\prime }{\widehat{Z}}_c{{\parallel }_1\parallel }_1}{\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1{\parallel e_j^{\prime }{Z}_c\parallel }_1}\hfill \\ \hfill & =\frac{|\parallel e_j^{\prime }{Z}_c{\parallel }_1-\parallel e_j^{\prime }{\widehat{Z}}_c{\parallel }_1|+\parallel e_j^{\prime }{\widehat{Z}}_c-e_j^{\prime }{Z}_c{\mathcal{P}}_c{\parallel }_1}{\parallel e_j^{\prime }{Z}_c{\parallel }_1}\le \frac{2\parallel e_j^{\prime }({\widehat{Z}}_c-Z_c{\mathcal{P}}_c){\parallel }_1}{\parallel e_j^{\prime }{Z}_c{\parallel }_1}\hfill \\ \hfill & =\frac{2\parallel e_j^{\prime }({\widehat{Z}}_c-Z_c{\mathcal{P}}_c){\parallel }_1}{\parallel e_j^{\prime }{\mathrm{\Pi }}_c{\parallel }_1}=2\parallel e_j^{\prime }({\widehat{Z}}_c-Z_c{\mathcal{P}}_c){\parallel }_1\le 2\sqrt{K}{\parallel e_j^{\prime }({\widehat{Z}}_c-Z_c{\mathcal{P}}_c)\parallel }_F,\hfill \end{array}$$

by Lemma A9, we have

$$\begin{array}{c}\hfill \parallel e_j^{\prime }({\widehat{\mathrm{\Pi }}}_c-{\mathrm{\Pi }}_c{\mathcal{P}}_c){\parallel }_1=O(\sqrt{K}\parallel e_j^{\prime }({\widehat{Z}}_c-Z_c{\mathcal{P}}_c){\parallel }_F)=O\left(\varpi K\kappa \left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)\sqrt{{\lambda }_1\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)}\right).\end{array}$$

For row nodes $1\le i\le n_r$, recall that $Z_r=Y_{*}{J}_{*}\equiv N_U^{-1}{N}_M{\mathrm{\Pi }}_r,{\widehat{Z}}_r={\widehat{Y}}_{*}{\widehat{J}}_{*},{\mathrm{\Pi }}_r(i,:)=\frac{Z_r(i,:)}{\parallel Z_r{(i,:)\parallel }_1}$ and ${\widehat{\mathrm{\Pi }}}_r(i,:)=\frac{{\widehat{Z}}_r(i,:)}{\parallel {\widehat{Z}}_r{(i,:)\parallel }_1}$, where $N_M$ and M are defined in the proof of Lemma 1 such that $U={\mathrm{\Theta }}_{r}M\equiv {\mathrm{\Theta }}_r{\mathrm{\Pi }}_r{B}_r$ and $N_M(i,i)=\frac{1}{{\parallel M(i,:)\parallel }_F}$, similar as the proof for column nodes, we have

$$\begin{array}{c}\hfill \parallel e_i^{\prime }({\widehat{\mathrm{\Pi }}}_r-{\mathrm{\Pi }}_r{\mathcal{P}}_r){\parallel }_1\le \frac{2\parallel e_i^{\prime }({\widehat{Z}}_r-Z_r{\mathcal{P}}_r){\parallel }_1}{\parallel e_i^{\prime }{Z}_r{\parallel }_1}\le \frac{2\sqrt{K}{\parallel e_i^{\prime }({\widehat{Z}}_r-Z_r{\mathcal{P}}_r)\parallel }_F}{\parallel e_i^{\prime }{Z}_r{\parallel }_1}.\end{array}$$

Now, we provide a lower bound of $\parallel e_i^{\prime }{Z}_r{\parallel }_1$ as below

$$\begin{array}{cc}\hfill \parallel e_i^{\prime }{Z}_r{\parallel }_1& =\parallel e_i^{\prime }{N}_U^{-1}{N}_M{\mathrm{\Pi }}_r{\parallel }_1=\parallel N_U^{-1}(i,i)e_i^{\prime }{N}_M{\mathrm{\Pi }}_r{\parallel }_1=N_U^{-1}(i,i){\parallel N_M(i,i)e_i^{\prime }{\mathrm{\Pi }}_r\parallel }_1=\frac{N_M(i,i)}{N_U(i,i)}\hfill \\ \hfill & ={\parallel U(i,:)\parallel }_F{N}_M(i,i)={\parallel U(i,:)\parallel }_F\frac{1}{{\parallel M(i,:)\parallel }_F}={\parallel U(i,:)\parallel }_F\frac{1}{\parallel e_i^{\prime }{M\parallel }_F}\hfill \\ \hfill & ={\parallel U(i,:)\parallel }_F\frac{1}{\parallel e_i^{\prime }{\mathrm{\Theta }}_r^{-1}{U\parallel }_F}={\parallel U(i,:)\parallel }_F\frac{1}{\parallel {\mathrm{\Theta }}_r^{-1}(i,i)e_i^{\prime }{U\parallel }_F}={\theta }_r\left(i\right)\ge {\theta }_{r,\min }.\hfill \end{array}$$

Therefore, by Lemma A9, we have

$$\begin{array}{cc}\hfill \parallel e_i^{\prime }({\widehat{\mathrm{\Pi }}}_r-{\mathrm{\Pi }}_r{\mathcal{P}}_r){\parallel }_1& \le \frac{2\sqrt{K}{\parallel e_i^{\prime }({\widehat{Z}}_r-Z_r{\mathcal{P}}_r)\parallel }_F}{\parallel e_i^{\prime }{Z}_r{\parallel }_1}\le \frac{2\sqrt{K}{\parallel e_i^{\prime }({\widehat{Z}}_r-Z_r{\mathcal{P}}_r)\parallel }_F}{{\theta }_{r,\min }}\hfill \\ \hfill & =O\left(\frac{K^{5.5}{\theta }_{r,\max }^{15}\varpi {\kappa }^{4.5}\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)\kappa \left({\mathrm{\Pi }}_c\right){\lambda }_1^{1.5}\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)}{{\theta }_{r,\min }^{15}{\pi }_{r,\min }}\right).\hfill \end{array}$$

   □

Appendix E.2. Proof of Corollary 1

Proof. 

Under conditions of Corollary 1, we have

$$\begin{array}{cc}\hfill & \parallel e_i^{\prime }({\widehat{\mathrm{\Pi }}}_r-{\mathrm{\Pi }}_r{\mathcal{P}}_r){\parallel }_1=O\left(\frac{K^{5.5}{\theta }_{r,\max }^{15}\varpi {\kappa }^{4.5}\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)\kappa \left({\mathrm{\Pi }}_c\right){\lambda }_1^{1.5}\left({\mathrm{\Pi }}_r^{\prime }{\mathrm{\Pi }}_r\right)}{{\theta }_{r,\min }^{15}{\pi }_{r,\min }}\right)=O\left(\frac{{\theta }_{r,\max }^{15}\varpi \sqrt{n_r}}{{\theta }_{r,\min }^{15}}\right),\hfill \\ \hfill & \parallel e_j^{\prime }({\widehat{\mathrm{\Pi }}}_c-{\mathrm{\Pi }}_c{\mathcal{P}}_c){\parallel }_1=O\left(\varpi K\kappa \left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)\sqrt{{\lambda }_1\left({\mathrm{\Pi }}_c^{\prime }{\mathrm{\Pi }}_c\right)}\right)=O\left(\varpi \sqrt{n_c}\right).\hfill \end{array}$$

Under conditions of Corollary 1, Lemma A7 gives $\varpi =O\left(\frac{\sqrt{P_{\max }{\theta }_{r,\max }\log (n_r+n_c)}}{{\theta }_{r,\min }{\sigma }_K\left(P\right)\sqrt{n_r{n}_c}}\right)$, which gives that

$$\begin{array}{cc}\hfill & \parallel e_i^{\prime }({\widehat{\mathrm{\Pi }}}_r-{\mathrm{\Pi }}_r{\mathcal{P}}_r){\parallel }_1=O\left(\frac{{\theta }_{r,\max }^{15}\varpi \sqrt{n_r}}{{\theta }_{r,\min }^{15}}\right)=O\left(\frac{{\theta }_{r,\max }^{15.5}\sqrt{P_{\max }\log (n_r+n_c)}}{{\theta }_{r,\min }^{16}{\sigma }_K\left(P\right)\sqrt{n_c}}\right),\hfill \\ \hfill & \parallel e_j^{\prime }({\widehat{\mathrm{\Pi }}}_c-{\mathrm{\Pi }}_c{\mathcal{P}}_c){\parallel }_1=O\left(\varpi \sqrt{n_c}\right)=O\left(\frac{\sqrt{P_{\max }{\theta }_{r,\max }\log (n_r+n_c)}}{{\theta }_{r,\min }{\sigma }_K\left(P\right)\sqrt{n_r}}\right).\hfill \end{array}$$

By basic algebra, this corollary follows.    □