Latent Class Analysis with Arbitrary-Distribution Responses

Abstract

The latent class model has been proposed as a powerful tool in understanding human behavior for various fields such as social, psychological, behavioral, and biological sciences. However, one important limitation of the latent class model is that it is primarily applied to data with binary responses or categorical responses, making it fail to model real-world data with continuous or negative responses. In many applications, ignoring the weights throws out a lot of potentially valuable information contained in the weights. To address this limitation, we propose a novel generative model, the arbitrary-distribution latent class model (adLCM). Our model enables the generation of data’s response matrix from an arbitrary distribution with a latent class structure. When compared to the latent class model, our adLCM is both more realistic and general. To our knowledge, our adLCM is the first model for latent class analysis with any real-valued responses, including continuous, negative, and signed values, thereby extending the classical latent class model beyond its traditional limitation to binary or categorical outcomes. We investigate the identifiability of the model and propose an efficient algorithm for estimating the latent classes and other model parameters. We show that the proposed algorithm enjoys consistent estimation. The performance of our algorithm is evaluated using both computer-generated data and real-world personality test data.

1. Introduction

The latent class model (LCM) [1,2,3] is a powerful tool for categorical data, with many applications across various areas such as social, psychological, behavioral, and biological sciences. These applications include movie rating [4,5], psychiatric evaluation [6,7,8,9], educational assessments [10], political surveys [11,12,13,14], transport economics personal interviews [15], and disease etiology detection [16,17,18]. In categorical data, subjects (individuals) typically respond to several items (questions). LCM is a theoretical model that categorizes subjects into disjoint groups, known as latent classes, according to their response pattern to a collection of categorical items. Latent classes assist researchers in better understanding human behaviors. For example, in movie rating, latent classes may represent different groups of users with an affinity for certain movie themes; in psychological tests, latent classes may represent different types of personalities. In educational assessments, latent classes may indicate different levels of abilities. In political surveys, latent classes may represent distinct types of political ideologies. In transport economics personal interviews, each latent class stands for a partition of the population. In disease etiology detection, latent classes may represent different disease categories. To infer latent classes for categorical data generated from LCM, various approaches have been developed in recent years, including maximum likelihood estimation techniques [19,20,21,22,23], nonnegative matrix factorization (NMF) [24], tensor-based methods [25,26], and spectral clustering approaches [27,28,29].

To mathematically describe categorical data, let R be the N-by-J observed response matrix such that $R(i,j)$ represents subject i’s response to item j, where N denotes the number of subjects and J denotes the number of items. For LCM, many researchers focus on binary choice data where elements of the observed response matrix R only take 0 or 1 [10,16,26,30,31,32,33,34,35,36,37]. LCM models the binary (or categorical) response matrix by generating its elements from a Bernoulli (or Binomial) distribution. Binary responses can be agree/disagree responses in psychiatric evaluation, correct/wrong responses in educational assessments, and presence/absence of symptoms in disease etiology detection. For real-world categorical data from various online personality tests in the link https://openpsychometrics.org/_rawdata/ (accessed on 11 August 2025), the ranges of most categorical responses are $\{0,1,2,\dots ,m\}$, where m is an integer like 2, 5, and 10. However, categorical data is more than binary or categorical responses. Categorical data with negative or continuous responses is also commonly encountered in the real world, and ignoring such weighted data may lose potentially meaningful information [38]. For example, in the buyer–seller rating e-commerce data [39], elements of the observed response matrix take values of $\{-1,0,1\}$ (for convenience, we call such R as a signed response matrix in this paper) since sellers are rated by users by applying three levels of rating, “Positive”, “Neutral”, and “Negative”. In the users–jokes rating categorical data Jester 100 [40], with the data source link: https://eigentaste.berkeley.edu/dataset/archive/ (accessed on 11 August 2025), all responses (i.e., ratings) are continuous numbers in the range $[-10,10]$. The aforementioned data cannot be generated from a Bernoulli or Binomial distribution. On the other hand, a substantial body of work has been developed to address polytomous responses or continuous responses, such as profile analysis (LPA) and factor mixture models (FMMs) [41,42,43,44,45,46,47], while their restrictive Gaussian or factorial assumptions limit their applicability to more general response types with irregular scales, such as real-valued ratings or signed categorical scores as aforementioned. Therefore, it is desirable to develop a more flexible model for data with arbitrary-distribution responses. With this motivation, our key contributions to the literature of latent class analysis are summarized as follows.

• Model. We propose a novel, identifiable, and generative statistical model, the arbitrary-distribution latent class model (adLCM), for data with arbitrary-distribution responses, where the responses can be continuous or negative values. Our adLCM allows the elements of an observed response matrix R to be generated from any distribution provided that the population version of R under adLCM enjoys a latent class structure. For example, our adLCM allows R to be generated from Bernoulli, Normal, Poisson, Binomial, Uniform, and Exponential distributions, etc. By considering a specifically designed discrete distribution, our adLCM can also model signed response matrices. For details, please refer to Examples 1–7.
• Algorithm. We develop an easy-to-implement algorithm, spectral clustering with K-means (SCK), to infer latent classes for arbitrary-distribution response matrices generated from arbitrary distribution under the proposed model. Our algorithm is designed based on a combination of two popular techniques: the singular value decomposition (SVD) and the K-means algorithm.
• Theoretical property. We build a theoretical framework to show that SCK enjoys consistent estimation under adLCM. We also provide Examples 1–7 to show that the theoretical performance of the proposed algorithm can be different when the observed response matrices R are generated from different distributions under the proposed model.
• Empirical validation. We conduct extensive simulations to validate our theoretical insights. Additionally, we apply our SCK approach to two real-world personality test datasets with meaningful interpretations.

The remainder of this paper is organized as follows. Section 2 offers a comprehensive review of related works. Section 3 describes the model. Section 4 details the algorithm. Section 5 establishes the consistency results and provides examples for further analysis. Section 6 contains numerical studies that verify our theoretical findings and examine the performance of the proposed method. Section 7 demonstrates the proposed method using two real-world datasets. Section 8 concludes the paper with a brief discussion of contributions and future work.

The following notations will be used throughout the paper. For any positive integer m, let $\left[m\right]$ and $I_{m\times m}$ be $\left[m\right]:=\{1,2,\dots ,m\}$ and the $m\times m$ identity matrix, respectively. For any vector x and any $q>0$, ${\parallel x\parallel }_q$ denotes x’s $l_q$-norm. For any matrix M, $M^{\prime }$ denotes its transpose, $\parallel M\parallel$ denotes its spectral norm, ${\parallel M\parallel }_F$ denotes its Frobenius norm, $\mathrm{rank}(M)$ denotes its rank, ${\sigma }_i(M)$ denotes its i-th largest singular value, ${\lambda }_i(M)$ denotes its i-th largest eigenvalue ordered by magnitude, $M(i,:)$ denotes its i-th row, and $M(:,j)$ denotes its j-th column. Let $\mathbb{R}$ and $\mathbb{N}$ be the set of real numbers and nonnegative integers, respectively. For any random variable X, $\mathbb{E}(X)$ and $\mathbb{P}(X=a)$ are the expectation and the probability that X equals to a, respectively. Let ${\mathbb{M}}_{m,K}$ be the collection of all $m\times K$ matrices where each row has only one 1 and all others 0.

2. Related Literature

Categorical data have been widely collected in various fields ranging from social and psychological research to political and transportation sciences [4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]. The latent class model (LCM) has long been a cornerstone for modeling unobserved heterogeneity in categorical data that are not directly measurable [48,49]. Since [48] provided one of the earliest formal frameworks for latent structure models in sociology, LCM has been widely applied in psychology and medicine for identifying subgroups of individuals with similar symptom or trait profiles. For example, Meyer et al. [6] applied latent classes to represent different personality or clinical profiles in psychometric assessments. In political science, LCM is used to analyze voting patterns and public opinion. For instance, Poole [11] applied an unfolding approach to cluster legislators based on their yes/no voting patterns. Wu et al. [18] developed a nested latent class model that distinguished pathogen combinations causing childhood pneumonia in epidemiology. LCM is also employed in health outcomes research [16] or to cluster clinical profiles [17].

For the classical LCM, the responses of categorical data are often assumed to be binary or categorical with nonnegative integers, such as the binary responses of agree/disagree (or yes/no) or ordinal categories in certain surveys [26,30,31,32,33,34,35,36,37]. Compared to traditional clustering methods like K-means and K-modes, LCM provides a model-based clustering approach with the selection of the clusters based on rigorous statistical tests [28]. Various studies in the literature have extended and refined latent class techniques in diverse directions. For example, some studies focus on the fundamental concern of identifiability in LCM to evaluate the feasibility of recovering model parameters and latent classes [37,50]. Gyllenberg et al. [51] stated that the LCM with binary responses is not identifiable; this problem has received considerable attention in [34,37] for extended LCMs. Another extension is the grade-of-membership model (GoM), which allows individuals to have partial membership in multiple classes rather than a single discrete class [52]. Qing [28] detected mixed memberships in categorical data with polytomous responses based on the GoM model and derived theoretical guarantees as well; see also [53,54,55]. Meanwhile, the estimation of LCM has also spurred extensive methodological innovation. For example, the maximum likelihood estimates are typically used via the expectation–maximization algorithm to iteratively refine class membership and parameter estimates [22,23,56]. Clinton et al. [12] considered Bayesian inference using MCMC techniques to analyze roll-call voting data, treating the class memberships as parameters to be sampled, see also [57,58,59] for more Bayesian reference. Recently, the tensor-based [25,26] and spectral-based algorithms [27,28] have also been popular to infer the latent memberships in the LCM framework.

As mentioned earlier, while classical latent class models are powerful for categorical data, they have a significant limitation as they typically assume binary or ordinal categories for the response. The literature has stated that the data could have arbitrary distribution with general responses such as negative or continuous values; ignoring this information may lead to a misunderstanding of the latent class structures [38]. Various methods have been proposed to address polytomous or continuous response data, such as latent profile analysis (LPA) and factor mixture models (FMM) [41,42,43,44,45,46,47]. However, for more general response types—such as the Jester joke data with ratings continuously varying within $[-10,10]$ [40], e-commerce data with scores in the range $\{-1,0,1\}$ [39], or the Advogato trust data with relationship values of $\{0.6,0.8,1\}$[60]—the classical LCM, LPA, and FMM are not applicable, due to LPA’s Gaussian assumption and FMM’s factorial assumption. To address this limitation, this paper proposes an arbitrary-distribution latent class model to enable the generation of data’s response matrix from an arbitrary distribution with a latent class structure. Hence, it allows any real-valued responses, including continuous, negative, and signed values, not merely limited to sampling weights, and thereby extends the classical LCM beyond its traditional response limitation.

In recent years, a noteworthy trend in latent structure learning is the use of spectral and other matrix/tensor-based methods to learn latent class structures as alternatives to traditional likelihood-based inference. Spectral algorithms offer an alternative by leveraging linear algebra (eigen-decomposition, singular value decomposition) or tensors of the data with theoretical soundness and ease of implementation [25]. In the context of network community detection—a problem analogous to finding latent classes of nodes—spectral clustering on graph Laplacians became a dominant approach for diverse types of networks [27,61,62,63]. Such algorithms have been extended to mixed-membership stochastic block models to allow overlapping communities [64,65]. In the latent class analysis realm, spectral algorithms have recently been developed to consistently estimate LCM parameters. For example, Chen and Gu [27] applied a spectral method for identifying GoM parameters from binary response data. Similarly, Anandkumar et al. [25] described tensor decomposition techniques to solve latent variable models via the factorization of third-order moments. These spectral methods, surveyed comprehensively by [66], achieve sound estimation and asymptotic consistency. Though recent advances extend spectral learning to polytomous categorical data, most existing applications still focus on binary settings. This paper focuses on the spectral method for arbitrary-distribution latent class modeling, and develops an easy-to-implement spectral clustering algorithm based on singular value decomposition (SVD) and the K-means algorithm.

3. Arbitrary-Distribution Latent Class Model

Unlike most researchers that focus on binary responses, in our arbitrary-distribution response setting in this paper, all elements of the observed response matrix R are allowed to be any real value, i.e., $R\in {\mathbb{R}}^{N\times J}$.

Consider categorical data with N subjects and J items, where the N subjects belong to K disjoint extreme latent profiles (also known as latent classes). Throughout this paper, the number of classes K is assumed to be a known integer. To describe the membership of each subject, we let Z be a $N\times K$ matrix such that $Z(i,k)$ is 1 if subject i belongs to the k-th extreme latent profile and $Z(i,k)$ is 0 otherwise. Call Z the classification matrix in this paper. For each subject $i\in \left[N\right]$, it is assumed to belong to a single extreme latent profile. For convenience, define ℓ as an N-by-1 vector whose i-th entry $\mathsf{\ell }(i)$ is k if the i-th subject belongs to the k-th extreme latent profile for $i\in \left[N\right]$. Thus, for subject $i\in \left[N\right]$, we have $Z(i,\mathsf{\ell }(i))=1$ and the other $(K-1)$ entries of the $K\times 1$ classification vector $Z(i,:)$ are 0.

Introduce the $J\times K$ item parameter matrix $\mathrm{\Theta }\in {\mathbb{R}}^{J\times K}$. For $k\in \left[K\right]$, our arbitrary-distribution latent class model (adLCM) assumes that $\mathrm{\Theta }(j,k)$ collects the conditional-response expectation for the response of the i-th subject to the j-th item under arbitrary distribution $\mathcal{F}$ provided that subject i belongs to the k-th extreme latent profile. Specifically, for $i\in \left[N\right],j\in \left[J\right]$, given the classification vector $Z(i,:)$ of subject i and the item parameter matrix $\mathrm{\Theta }$, our adLCM assumes that for arbitrary distribution $\mathcal{F}$, the conditional response expectation of the i-th subject to the j-th item is

$$\begin{array}{c}\hfill \mathbb{E}(R(i,j)|Z(i,:),\mathrm{\Theta })=\sum _{k=1}^{K}Z(i,k)\mathrm{\Theta }(j,k)\equiv \mathrm{\Theta }(j,\mathsf{\ell }(i)).\end{array}$$

(1)

Based on Equation (1), our adLCM can be simplified as follows.

Definition 1.

Let $R\in {\mathbb{R}}^{N\times J}$ denote the observed response matrix. Let $Z\in {\mathbb{M}}_{N,K}$ be the classification matrix and $\mathrm{\Theta }\in {\mathbb{R}}^{J\times K}$ be the item parameter matrix. For $i\in \left[N\right], j\in \left[J\right]$, our arbitrary-distribution latent class model (adLCM) assumes that for an arbitrary distribution $\mathcal{F}$, $R(i,j)$ are independent random variables generated from the distribution $\mathcal{F}$ and the expectation of $R(i,j)$ under the distribution $\mathcal{F}$ should satisfy the following formula:

$$\begin{array}{c}\hfill \mathbb{E}(R(i,j))=R_0(i,j),\mathrm{where}{R}_0:=Z{\mathrm{\Theta }}^{\prime }.\end{array}$$

(2)

Definition 1 says that adLCM is determined by the classification matrix Z, the item parameter matrix $\mathrm{\Theta }$, and the distribution $\mathcal{F}$. For brevity, we denote adLCM by $adLCM(Z,\mathrm{\Theta },\mathcal{F})$. Under adLCM, $\mathcal{F}$ is allowed to be any distribution as long as Equation (2) is satisfied under $\mathcal{F}$, i.e., adLCM only requires the expectation (i.e., population) response matrix $R_0$ of the observed response matrix R to be $Z{\mathrm{\Theta }}^{\prime }$ under any distribution $\mathcal{F}$.

Remark 1.

For the case that $\mathcal{F}$ is a Bernoulli distribution, all elements of Θ have a range in $[0,1]$, R only contains binary responses (i.e., $R(i,j)\in \{0,1\}$ for $i\in \left[N\right], j\in \left[J\right]$ when $\mathcal{F}$ is a Bernoulli distribution), and Equation (1) becomes $\mathbb{P}(R(i,j)=1|Z(i,:),\mathrm{\Theta })=\mathrm{\Theta }(j,\mathsf{\ell }(i))$. For this case, adLCM reduces to the LCM model for data with binary responses.

Remark 2.

It should be noted that Equation (2) does not hold for all distributions. For instance, we cannot set $\mathcal{F}$ as a t-distribution because the expectation of a t-distribution is always 0, which cannot capture the latent structure required by adLCM; $\mathcal{F}$ cannot be a Cauchy distribution whose expectation does not even exist; $\mathcal{F}$ cannot be a Chi-square distribution because the expectation of a Chi-square distribution is its degrees of freedom, which is a fixed positive integer and cannot capture the latent structure required by adLCM. We will provide some examples to demonstrate that Equation (2) can be satisfied for different distributions of $\mathcal{F}$. For details, please refer to Examples 1–7.

Remark 3.

It should also be noted that the ranges of the observed response matrix R and the item parameter matrix Θ depend on distribution $\mathcal{F}$. For example, when $\mathcal{F}$ is a Bernoulli distribution, $R\in {\{0,1\}}^{N\times J}$ and $\mathrm{\Theta }\in {[0,1]}^{J\times K}$; when $\mathcal{F}$ is a Poisson distribution, $R\in {\mathbb{N}}^{N\times J}$ and $\mathrm{\Theta }\in {[0,+\infty )}^{J\times K}$. If we let $\mathcal{F}$ be a Normal distribution, $R\in {\mathbb{R}}^{N\times J}$ and $\mathrm{\Theta }\in {(-\infty ,+\infty )}^{J\times K}$. For details, please refer to Examples 1–7.

The following proposition shows that adLCM is identifiable as long as there exists at least one subject for every extreme latent profile.

Proposition 1

(Identifiability). Consider the adLCM as in Equation (2): when each extreme latent profile has at least one subject, the model is identifiable: for any other valid parameter set $(\tilde{Z},\tilde{\mathrm{\Theta }})$, if $\tilde{Z}{\tilde{\mathrm{\Theta }}}^{\prime }=Z{\mathrm{\Theta }}^{\prime }$, then $(Z,\mathrm{\Theta })$ and $(\tilde{Z},\tilde{\mathrm{\Theta }})$ are identical up to a permutation of the K extreme latent profiles.

All proofs of theoretical results developed in this paper are given in the Appendix A. The condition that each extreme latent profile must contain at least one subject means that each extreme latent profile cannot be an empty set and we have $\mathbf{rank}(Z)=K$.

Remark 4.

Note that Z and $\tilde{Z}$ are the same up to a permutation of the K latent classes in Proposition 1. A permutation is acceptable since the equivalence of Z and $\tilde{Z}$ should not rely on how we label each of the K extreme latent profiles. A similar argument holds for the identity of Θ; and $\tilde{\mathrm{\Theta }}$.

The observed response matrix R along with the ground-truth classification matrix Z and the item parameter matrix $\mathrm{\Theta }$ can be generated using our adLCM as follows: let $R(i,j)$ be a random variable generated by distribution $\mathcal{F}$ with expected value $R_0(i,j)$ for $i\in \left[N\right], j\in \left[J\right]$, where $R_0=Z{\mathrm{\Theta }}^{\prime }$ satisfies the latent structure required by adLCM. In latent class analysis, given the observed response matrix R generated from $adLCM(Z,\mathrm{\Theta },\mathcal{F})$, our goal is to infer the classification matrix Z and the item parameter matrix $\mathrm{\Theta }$. Proposition 1 ensures that the model parameters Z and $\mathrm{\Theta }$ can be reliably inferred from the observed response matrix R. In the following two sections, we will develop a spectral algorithm to fit adLCM and show that this algorithm yields consistent estimation.

4. A Spectral Method for Parameter Estimation

In addition to providing a more general model for latent class analysis, we are also interested in estimating the model parameters. In this section, we focus on the parameter estimation problem within the adLCM framework by developing an efficient and easy-to-implement spectral method.

To provide insight into developing an algorithm for the adLCM, we first consider an oracle case where we observe the expectation response matrix $R_0$ given in Equation (2). We would like to estimate Z and $\mathrm{\Theta }$ from $R_0$. Recall that the item parameter matrix $\mathrm{\Theta }$ is a J-by-K matrix; here we let $\mathrm{rank}(\mathrm{\Theta })=K_0$, where $K_0$ is a positive integer and it is no larger than K. As $R_0=Z{\mathrm{\Theta }}^{\prime },\mathrm{rank}(Z)=K$, and $\mathrm{rank}(\mathrm{\Theta })=K_0\le K$, we see that $R_0$ is a rank-$K_0$ matrix. As the number of extreme latent profiles K is usually far smaller than the number of subjects N and the number of items J, the N-by-J population response matrix $R_0$ enjoys a low-dimensional structure. Next, we will demonstrate that we can greatly benefit from the low-dimensional structure of $R_0$ when we aim to develop a method to infer model parameters under adLCM.

Let $R_0=U\Sigma V^{\prime }$ be the compact singular value decomposition (SVD) of $R_0$ such that $\Sigma$ is a $K_0\times K_0$ diagonal matrix collecting the $K_0$ nonzero singular values of $R_0$. Write $\Sigma =\mathrm{diag}({\sigma }_1(R_0),{\sigma }_2(R_0),\dots ,{\sigma }_{K_0}(R_0))$. The $N\times K_0$ matrix U collects the corresponding left singular vectors and it satisfies $U^{\prime }U=I_{K_0\times K_0}$. Similarly, the $J\times K_0$ matrix V collects the corresponding right singular vectors and it satisfies $V^{\prime }V=I_{K_0\times K_0}$. For $k\in \left[K\right]$, let $N_k$ be the number of subjects that belong to the k-th extreme latent profile, i.e., $N_k={\sum }_{i=1}^{N}Z(i,k)$. The ensuing lemma constitutes the foundation of our estimation method.

Lemma 1.

Under $adLCM(Z,\mathrm{\Theta },\mathcal{F})$, let $R_0=U\Sigma V^{\prime }$ be the compact SVD of $R_0$. The following statements are true.

• The left singular vectors matrix U can be written as

  $$\begin{array}{c}\hfill U=ZX,\end{array}$$

  (3)
  where X is a $K\times K_0$ matrix.
• U has K distinct rows such that for any two distinct subjects i and $\overline{i}$ that belong to the same extreme latent profile (i.e., $\mathsf{\ell }(i)=\mathsf{\ell }(\overline{i})$), we have $U(i,:)=U(\overline{i},:)$.
• Θ can be written as

  $$\begin{array}{c}\hfill \mathrm{\Theta }=V\Sigma U^{\prime }Z{(Z^{\prime }Z)}^{-1}.\end{array}$$

  (4)
• Furthermore, when $K_0=K$, for all $k\in \left[K\right],l\in \left[K\right]$, and $k\ne l$, we have

  $$\begin{array}{c}\hfill {\parallel X(k,:)-X(l,:)\parallel }_F={(N_k^{-1}+N_l^{-1})}^{1/2}.\end{array}$$

  (5)

From now on, for the simplicity of our further analysis, we let $K_0\equiv K$. Hence, the last statement of Lemma 1 always holds.

The second statement of Lemma 1 indicates that the rows of U corresponding to subjects assigned to the same extreme latent profile are identical. This circumstance implies that the application of a clustering algorithm to the rows of U can yield an exact reconstruction of the classification matrix Z after a permutation of the K extreme latent profiles.

In this paper, we adopt the K-means clustering algorithm, an unsupervised learning technique that groups similar data points into K clusters. This clustering technique is detailed as follows,

$$\begin{array}{c}\hfill (\overline{\overline{Z}},\overline{\overline{X}})=\arg {\min }_{\overline{Z}\in {\mathbb{M}}_{N,K},\overline{X}\in {\mathbb{R}}^{K\times K}}{\parallel \overline{Z}\overline{X}-\overline{U}\parallel }_F^2,\end{array}$$

(6)

where $\overline{U}$ is any $N\times K$ matrix. For convenience, call Equation (6) a “Run K-means algorithm on all rows of $\overline{U}$ with K clusters to obtain $\overline{\overline{Z}}$” because we are interested in the classification matrix $\overline{\overline{Z}}$. Let $\overline{U}$ in Equation (6) be U; the second statement of Lemma 1 guarantees that $\overline{\overline{Z}}=Z\mathcal{P},\overline{\overline{X}}={\mathcal{P}}^{\prime }X$, where $\mathcal{P}$ is a $K\times K$ permutation matrix, i.e., the running K-means algorithm on all rows of U exactly recovers Z up to a permutation of the K extreme latent profiles.

After obtaining Z from U, $\mathrm{\Theta }$ can be recovered subsequently by Equation (4). The above analysis suggests the following Algorithm 1, Ideal SCK, where SCK stands for Spectral Clustering with K-means. Ideal SCK returns a permutation of $(Z,\mathrm{\Theta })$, which also supports the identifiability of the proposed model as stated in Proposition 1.

Algorithm 1 Ideal SCK

Require:  The expectation response matrix $R_0$ and the number of extreme latent profiles K. Ensure:  A permutation of Z and $\mathrm{\Theta }$.   1: Obtain $U\Sigma V^{\prime }$, the top K SVD of $R_0$.   2: Run K-means algorithm on all rows of U with K clusters to obtain $Z\mathcal{P}$, a permutation of Z.   3: Equation (4) gives $V\Sigma U^{\prime }Z\mathcal{P}{({(Z\mathcal{P})}^{\prime }Z\mathcal{P})}^{-1}=\mathrm{\Theta }\mathcal{P}$, a permutation of $\mathrm{\Theta }$.

For the real case, the response matrix R is observed rather than the expectation response matrix $R_0$. We now move from the ideal scenario to the real scenario, intending to estimate Z and $\mathrm{\Theta }$ when the observed response matrix R is a random matrix generated from an unknown distribution $\mathcal{F}$ satisfying Equation (2) with K extreme latent profiles under adLCM. The expectation of R is $R_0$ according to Equation (2) under adLCM, so intuitively, the singular values and singular vectors of R will be close to those of $R_0$. Set $\widehat{R}=\widehat{U}\widehat{\Sigma }{\widehat{V}}^{\prime }$ as the top K SVD of R, where $\widehat{\Sigma }$ is a $K\times K$ diagonal matrix collecting the top K singular values of R. Write $\widehat{\Sigma }=\mathrm{diag}({\sigma }_1(R),{\sigma }_2(R),\dots ,{\sigma }_K(R))$. As $\mathbb{E}(R)=R_0$ and the $N\times J$ matrix $R_0$ has K nonzero singular values, while the other $(\min (N,J)-K)$ singular values are zeros; we see that $\widehat{R}$ should be a good approximation of $R_0$. Matrices $\widehat{U}\in {\mathbb{R}}^{N\times K},\widehat{V}\in {\mathbb{R}}^{J\times K}$ collect the corresponding left and right singular vectors and satisfy ${\widehat{U}}^{\prime }\widehat{U}={\widehat{V}}^{\prime }\widehat{V}=I_{K\times K}$. The above analysis implies that $\widehat{U}$ should have roughly K distinct rows because $\widehat{U}$ is a slightly perturbed version of U. Therefore, to obtain a good estimation of the classification matrix Z, we should apply the K-means algorithm on all rows of $\widehat{U}$ with K clusters. Let $\widehat{Z}$ be the estimated classification matrix returned by applying the K-means method on all rows of $\widehat{U}$ with K clusters. Then we are able to obtain a good estimation of $\mathrm{\Theta }$ according to Equation (4) by setting $\widehat{\mathrm{\Theta }}=\widehat{V}\widehat{\Sigma }{\widehat{U}}^{\prime }\widehat{Z}{({\widehat{Z}}^{\prime }\widehat{Z})}^{-1}$. Algorithm 2, referred to as SCK, is a natural extension of the Ideal SCK from the oracle case to the real case. Note that in our SCK algorithm there are only two inputs: the observed response matrix R and the number of latent classes K, i.e., SCK does not require any tuning parameters.

Algorithm 2 Spectral Clustering with K-means (SCK for short)

Require:  The observed response matrix $R\in {\mathbb{R}}^{N\times J}$ and the number of extreme latent profiles K. Ensure:  $\widehat{Z}$ and $\widehat{\mathrm{\Theta }}$.   1: Obtain $\widehat{R}=\widehat{U}\widehat{\Sigma }{\widehat{V}}^{\prime }$, the top K SVD of R.   2: Run K-means algorithm on all rows of $\widehat{U}$ with K clusters to obtain $\widehat{Z}$.   3: Obtain an estimate of $\mathrm{\Theta }$ by setting $\widehat{\mathrm{\Theta }}={\widehat{R}}^{\prime }\widehat{Z}{({\widehat{Z}}^{\prime }\widehat{Z})}^{-1}$.

Here, we evaluate the computational cost of our SCK algorithm. The computational cost of the SVD step involved in the SCK approach is $O(max(N^2,J^2)K)$. For the K-means algorithm, its complexity is $O(Nl{K}^2)$ with l being the number of K-means iterations. In all experimental studies considered in this paper, l is set as 100 for the K-means algorithm. The complexity of the last step in SCK is $O(JNK)$. Since $K\ll min(N,J)$ in this paper, as a consequence, the total time complexity of our SCK algorithm is $O(max(N^2,J^2)K)$.

5. Theoretical Properties

In this section, we present comprehensive theoretical properties of the SCK algorithm when the observed response matrix R is generated from the proposed model. Our objective is to demonstrate that the estimated classification matrix $\widehat{Z}$ and the estimated item parameter matrix $\widehat{\mathrm{\Theta }}$ both concentrate around the true classification matrix Z and the true item parameter matrix $\mathrm{\Theta }$, respectively.

Let $\mathcal{T}=\{{\mathcal{T}}_1,{\mathcal{T}}_2,\dots ,{\mathcal{T}}_K\}$ be the collection of true partitions for all subjects, where ${\mathcal{T}}_k=\{i:Z(i,k)=1\mathrm{for}i\in \left[N\right]\}$ for $k\in \left[K\right]$, i.e., ${\mathcal{T}}_k$ is the set of the true partition of subjects into the k-th extreme latent profile. Similarly, let $\widehat{\mathcal{T}}=\{{\widehat{\mathcal{T}}}_1,{\widehat{\mathcal{T}}}_2,\dots ,{\widehat{\mathcal{T}}}_K\}$ represent the collection of estimated partitions for all subjects, where ${\widehat{\mathcal{T}}}_k=\{i:\widehat{Z}(i,k)=1\mathrm{for}i\in \left[N\right]\}$ for $k\in \left[K\right]$. We use the measure defined in [61] to quantify the closeness of the estimated partition $\widehat{\mathcal{T}}$ and the ground-truth partition $\mathcal{T}$. Denote the Clustering error associated with $\mathcal{T}$ and $\widehat{\mathcal{T}}$ as

$$\begin{array}{c}\hfill \widehat{f}={\min }_{\pi \in S_K}{\max }_{k\in \left[K\right]}{\displaystyle \frac{|{\mathcal{T}}_k\cap {\widehat{\mathcal{T}}}_{\pi (k)}^c|+|{\mathcal{T}}_k^c\cap {\widehat{\mathcal{T}}}_{\pi (k)}|}{N_K}},\end{array}$$

(7)

where $S_K$ represents the set of all permutations of $\{1,2,\dots ,K\}$, and ${\widehat{\mathcal{T}}}_{\pi (k)}^c$ and ${\mathcal{T}}_k^c$ denote the complementary sets. As stated in Reference [61], $\widehat{f}$ evaluates the maximum proportion of subjects in the symmetric difference of ${\mathcal{T}}_k$ and ${\widehat{\mathcal{T}}}_{\pi (k)}$. Since the observed response matrix R is generated from adLCM with expectation $R_0$, and $\widehat{f}$ measures the performance of the SCK algorithm, it is expected that SCK estimates Z with a small Clustering error $\widehat{f}$.

For convenience, let $\rho ={\max }_{j\in \left[J\right],k\in \left[K\right]}\left|\mathrm{\Theta }(j,k)\right|$ and call it the scaling parameter. Let $B={\displaystyle \frac{\mathrm{\Theta }}{\rho }}$, and then we have ${\max }_{j\in \left[J\right],k\in \left[K\right]}\left|B(j,k)\right|=1$ and $R_0=\rho Z{B}^{\prime }$. Let $\tau ={\max }_{i\in \left[N\right], j\in \left[J\right]}|R(i,j)-R_0(i,j)|$ and $\gamma ={\max }_{i\in \left[N\right], j\in \left[J\right]}\mathrm{Var}(R(i,j))$ where $\mathrm{Var}(R(i,j))$ means the variance of $R(i,j)$. We require the following assumption to establish theoretical guarantees of consistency for our SCK method.

Assumption 1.

Assume $\gamma \ge {\displaystyle \frac{{\tau }^2\log (N+J)}{\max (N,J)}}$.

The following theorem presents our main result, which provides upper bounds for the error rates of our SCK algorithm under our adLCM.

Theorem 1.

Under $adLCM(Z,\mathrm{\Theta },\mathcal{F})$, if Assumption 1 is satisfied, with a probability of at least $1-o({(N+J)}^{-3})$,

$$\begin{array}{cc}\hfill & \widehat{f}=O({\displaystyle \frac{\gamma K^2{N}_{\max }\max (N,J)\log (N+J)}{{\rho }^2{N}_{\min }^{2}J}})\mathrm{and}{\displaystyle \frac{{\parallel \widehat{\mathrm{\Theta }}-\mathrm{\Theta }\mathcal{P}\parallel }_F}{{\parallel \mathrm{\Theta }\parallel }_F}}=O({\displaystyle \frac{\sqrt{\gamma K\max (N,J)\log (N+J)}}{\rho \sqrt{N_{\min }J}}}),\hfill \end{array}$$

where $N_{\max }={\max }_{k\in \left[K\right]}\left\{N_k\right\},N_{\min }={\min }_{k\in \left[K\right]}\left\{N_k\right\}$, and $\mathcal{P}$ is a permutation matrix.

Because our adLCM is distribution-free, Theorem 1 provides a general theoretical guarantee of the SCK algorithm when R is generated from adLCM for any distribution $\mathcal{F}$ as long as Equation (2) is satisfied. We can simplify Theorem 1 by considering additional conditions:

Corollary 1.

Under $adLCM(Z,\mathrm{\Theta },\mathcal{F})$, when Assumption 1 holds, if we make the additional assumption that ${\displaystyle \frac{N_{\max }}{N_{\min }}}=O(1)$ and $K=O(1)$, with a probability of at least $1-o({(N+J)}^{-3})$,

$$\begin{array}{c}\hfill \widehat{f}=O({\displaystyle \frac{\gamma \max (N,J)\log (N+J)}{{\rho }^{2}NJ}})\mathrm{and}{\displaystyle \frac{{\parallel \widehat{\mathrm{\Theta }}-\mathrm{\Theta }\mathcal{P}\parallel }_F}{{\parallel \mathrm{\Theta }\parallel }_F}}=O({\displaystyle \frac{\sqrt{\gamma \max (N,J)\log (N+J)}}{\rho \sqrt{NJ}}}).\end{array}$$

For the case $J=\beta N$ for any positive constant $\beta$, Corollary 1 implies that the SCK algorithm yields consistent estimation under adLCM since the error bounds in Corollary 1 decrease to zero as $N\to +\infty$ when $\rho$ and distribution $\mathcal{F}$ are fixed.

Recall that R is an observed response matrix generated from a distribution $\mathcal{F}$ with expectation $R_0=Z{\mathrm{\Theta }}^{\prime }=\rho Z{B}^{\prime }$ under adLCM and $\gamma$ is the maximum variance of $R(i,j)$ and it is closely related to the distribution $\mathcal{F}$, the ranges of $R,\rho ,B$, and $\gamma$ can vary depending on the specific distribution $\mathcal{F}$. The following examples provide the ranges of $R,\rho ,B$, the upper bound of $\gamma$, and the explicit forms of error bounds in Theorem 1 for different distribution $\mathcal{F}$ under our adLCM. Meanwhile, based on the explicitly derived error bounds for different distribution $\mathcal{F}$, we also investigate how the scaling parameter $\rho$ influences the performance of the SCK algorithm in these examples. For all pairs $(i,j)$ with $i\in \left[N\right],j\in \left[J\right]$, we consider the following distributions when $\mathbb{E}(R)=R_0$ in Equation (2) holds.

Example 1.

Let $\mathcal{F}$ be a Bernoulli distributionsuch that $R(i,j)\sim \mathrm{Bernoulli}(R_0(i,j))$, where $R_0(i,j)$ is the Bernoulli probability, i.e., $\mathbb{E}(R(i,j))=R_0(i,j)$. For this case, our adLCM degenerates to the LCM for data with binary responses. According to the properties of the Bernoulli distribution, we have the following conclusions.

• $R(i,j)\in \{0,1\}$, i.e., $R(i,j)$ only takes two values 0 and 1.
• $B(i,j)\in [0,1]$ and $\rho \in (0,1]$ because $R_0(i,j)$ is a probability located in $[0,1]$ and ${\max }_{i\in \left[N\right],j\in \left[J\right]}\left|B(i,j)\right|$ is assumed to be 1.
• $\tau \le 1$ because $\tau ={\max }_{i\in \left[N\right],j\in \left[J\right]}|R(i,j)-R_0(i,j)|\le 1$.
• $\gamma \le \rho$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))={\max }_{i\in \left[N\right],j\in \left[J\right]}{R}_0(i,j)(1-R_0(i,j))\le {\max }_{i\in \left[N\right],j\in \left[J\right]}{R}_0(i,j)={\max }_{i\in \left[N\right],j\in \left[J\right]}\rho (ZB)(i,j)\le \rho$.
• Let τ be its upper bound 1 and γ be its upper bound ρ, Assumption 1 becomes $\rho \ge {\displaystyle \frac{\log (N+J)}{\max (N,J)}}$, which means a sparsity requirement on R because ρ controls the probability of the numbers of ones in R for this case.
• Let γ be its upper bound ρ in Theorem 1, then we have

  $$\begin{array}{c}\hfill \widehat{f}=O({\displaystyle \frac{K^2{N}_{\max }\max (N,J)\log (N+J)}{\rho N_{\min }^{2}J}})\mathrm{and}{\displaystyle \frac{{\parallel \widehat{\mathrm{\Theta }}-\mathrm{\Theta }\mathcal{P}\parallel }_F}{{\parallel \mathrm{\Theta }\parallel }_F}}=O({\displaystyle \frac{\sqrt{K\max (N,J)\log (N+J)}}{\sqrt{\rho N_{\min }J}}}).\end{array}$$
  We observe that increasing ρ leads to a decrease in SCK’s error rates when $\mathcal{F}$ is a Bernoulli distribution.

Example 2.

Let $\mathcal{F}$ be a Binomial distribution such that $R(i,j)\sim \mathrm{Binomial}(m,{\displaystyle \frac{R_0(i,j)}{m}})$ for any positive integer m, where $R(i,j)$ is a random variable that reflects the number of successes in a fixed number of independent trials m with the same probability of success ${\displaystyle \frac{R_0(i,j)}{m}}$, i.e., $\mathbb{E}(R(i,j))=R_0(i,j)$. For this case, our adLCM reduces to the LCM for data with categorical responses. For a Binomial distribution, we have $\mathbb{P}(R(i,j)=r)=\left({\displaystyle \genfrac{}{}{0pt}{}{m}{r}}\right){({\displaystyle \frac{R_0(i,j)}{m}})}^r{(1-{\displaystyle \frac{R_0(i,j)}{m}})}^{m-r}$ for $r=0,1,2,\dots ,m$, where $\left({\displaystyle \genfrac{}{}{0pt}{}{•}{•}}\right)$ is a binomial coefficient. By the property of the Binomial distribution, we have the following conclusions.

• $R(i,j)\in \{0,1,2,\dots ,m\}$.
• $B(i,j)\in [0,1]$ and $\rho \in (0,m]$ because ${\displaystyle \frac{R_0(i,j)}{m}}$ is a probability that has a range in $[0,1]$.
• $\tau \le m$ because $\tau ={\max }_{i\in \left[N\right],j\in \left[J\right]}|R(i,j)-R_0(i,j)|\le m$.
• $\gamma \le \rho$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))=m{\displaystyle \frac{R_0(i,j)}{m}}(1-{\displaystyle \frac{R_0(i,j)}{m}})=R_0(i,j)(1-{\displaystyle \frac{R_0(i,j)}{m}})\le \rho$.
• Let τ be its upper bound m and γ be its upper bound ρ, then Assumption 1 becomes $\rho \ge {\displaystyle \frac{m^2\log (N+J)}{\max (N,J)}}$ which provides a lower bound requirement of the scaling parameter ρ.
• Let γ be its upper bound ρ in Theorem 1, then we obtain the exact forms of error bounds for SCK when $\mathcal{F}$ is a Binomial distribution, and we observe that increasing ρ reduces SCK’s error rates.

Remark 5.

When $\mathcal{F}$ is the Binomial distribution and Condition 1 of [28] holds ($K=O(1)$, $J=O(N)$, $N_{\max }/N_{\min }=O(1)$), the theoretical upper bounds for $\widehat{f}$ and ${\displaystyle \frac{|\widehat{\mathrm{\Theta }}-\mathrm{\Theta }\mathcal{P}|F}{\left|\mathrm{\Theta }\right|F}}$ correspond to the squared error bounds established in Theorem 1 of [28]. This optimality guarantee is consistent with Assumption 1, which aligns with Assumption 1 in [28]. The relationship arises because the theoretical results in [28] apply to the overlapping case (each subject can belong to multiple latent classes). Under mild conditions, spectral clustering methods in the non-overlapping case (each subject belongs to only one latent class) typically achieve an error bound that is the square of the bound for spectral methods in the overlapping case, while requiring similar sparsity levels. This phenomenon is generally observed in the area of community detection: for instance, spectral methods for non-overlapping networks [67,68,69,70] exhibit squared error bounds relative to those for overlapping networks [64,65,71,72], despite comparable sparsity requirements. We refer readers to [73] for further discussion of this phenomenon.

Example 3.

Let $\mathcal{F}$ be a Poisson distribution such that $R(i,j)\sim \mathrm{Poisson}(R_0(i,j))$, where $R_0(i,j)$ is the Poisson parameter, i.e., $\mathbb{E}(R(i,j))=R_0(i,j)$. By the properties of the Poisson distribution, the following conclusions can be obtained.

• $R(i,j)\in \mathbb{N}$, i.e., $R(i,j)$ is an nonnegative integer.
• $B(i,j)]\in [0,1]$ and $\rho \in (0,+\infty )$ because Poisson distribution can take any positive value for its mean.
• τ is an unknown positive value because we cannot know the exact upper bound of $R(i,j)$ when R is obtained from the Poisson distribution under the adLCM.
• $\gamma \le \rho$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))={\max }_{i\in \left[N\right],j\in \left[J\right]}{R}_0(i,j)\le \rho$.
• Let γ be its upper bound ρ, then Assumption 1 becomes $\rho \ge {\displaystyle \frac{{\tau }^2\log (N+j)}{\max (N,J)}}$ which is a lower bound requirement of ρ.
• Let γ be its upper bound ρ in Theorem 1 which obtains the exact forms of error bounds for the SCK algorithm when $\mathcal{F}$ is a Poisson distribution. It is easy to observe that increasing ρ leads to a decrease in SCK’s error rates.

Example 4.

Let $\mathcal{F}$ be a Normal distribution such that $R(i,j)\sim \mathrm{Normal}(R_0(i,j),{\sigma }^2)$, where $R_0(i,j)$ is the mean ( i.e., $\mathbb{E}(R(i,j))=R_0(i,j)$) and ${\sigma }^2$ is the variance parameter for Normal distribution. For this case, we have

• $R(i,j)\in \mathbb{R}$, i.e., $R(i,j)$ is a real value.
• $B(i,j)\in [-1,1]$ and $\rho \in (0,+\infty )$ because the mean of Normal distribution can take any value. Note that, unlike the cases when $\mathcal{F}$ is Bernoulli or Poisson, B can have negative elements for the Normal distribution case.
• Similar to Example 3, τ is an unknown positive value.
• $\gamma ={\sigma }^2$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))={\max }_{i\in \left[N\right],j\in \left[J\right]}{\sigma }^2={\sigma }^2$ for Normal distribution.
• Let γ be its exact value ${\sigma }^2$, then Assumption 1 becomes ${\sigma }^2\max (N,J)\ge {\tau }^2\log (N+J)$ which means that $\max (N,J)$ should be set larger than ${\displaystyle \frac{{\tau }^2\log (N+J)}{{\sigma }^2}}$ for our theoretical analysis.
• Let γ be its exact value ${\sigma }^2$ in Theorem 1 which provides the exact forms of error bounds for SCK. We observe that increasing the scaling parameter ρ (or decreasing the variance ${\sigma }^2$) reduces SCK’s error rates.

Example 5.

Let $\mathcal{F}$ be an Exponential distribution such that $R(i,j)\sim \mathrm{Exponential}({\displaystyle \frac{1}{R_0(i,j)}})$, where ${\displaystyle \frac{1}{R_0(i,j)}}$ is the Exponential parameter, i.e., $\mathbb{E}(R(i,j))=R_0(i,j)$. For this case, we have

• $R(i,j)\in {\mathbb{R}}_{+}$, i.e., $R(i,j)$ is a positive value.
• $B(i,j)\in (0,1]$ and $\rho \in (0,+\infty )$ because the mean of Exponential distribution can be any positive value.
• Similar to Example 3, τ is an unknown positive value.
• $\gamma \le {\rho }^2$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))={\max }_{i\in \left[N\right],j\in \left[J\right]}{R}_0^2(i,j)\le {\rho }^2$ for Exponential distribution.
• Let γ be its upper bound ${\rho }^2$, then Assumption 1 becomes ${\rho }^2\ge {\tau }^2\log (N+J)/\max (N,J)$, a lower bound requirement of ρ.
• Let γ be its upper bound ${\rho }^2$ in Theorem 1; the theoretical bounds demonstrate that ρ vanishes, which indicates that increasing ρ has no significant impact on the error rates of SCK.

Example 6.

Let $\mathcal{F}$ be a Uniform distribution such that $R(i,j)\sim \mathrm{Uniform}(0,2R_0(i,j))$, where $\mathbb{E}(R(i,j))={\displaystyle \frac{0+2R_0(i,j)}{2}}=R_0(i,j)$ holds immediately. For this case, we have

• $R(i,j)\in (0,2\rho )$ because $2R_0(i,j)\le 2\rho$.
• $B(i,j)\in (0,1]$ and $\rho \in (0,+\infty )$ because $\mathrm{Uniform}(0,2R_0(i,j))$ allows $2R_0(i,j)$ to be any positive value.
• τ is an unknown positive value with an upper bound $2\rho$.
• $\gamma \le {\displaystyle \frac{{\rho }^2}{3}}$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))={\max }_{i\in \left[N\right],j\in \left[J\right]}{\displaystyle \frac{{(2R_0(i,j)-0)}^2}{12}}=$${\max }_{i\in \left[N\right],j\in \left[J\right]}{\displaystyle \frac{R_0^2(i,j)}{3}}\le {\displaystyle \frac{{\rho }^2}{3}}$ for Uniform distribution.
• Let γ be its upper bound ${\displaystyle \frac{{\rho }^2}{3}}$, then Assumption 1 becomes ${\rho }^2\ge 3{\tau }^2\log (N+J)/\max (N,J)$, a lower bound requirement of ρ.
• Since ρ disappears in the error bounds when we let $\gamma ={\displaystyle \frac{{\rho }^2}{3}}$ in Theorem 1, increasing ρ does not significantly influence SCK’s error rates, a conclusion similar to Example 5.

Example 7.

Our adLCM can also model a signed response matrix by setting $\mathbb{P}(R(i,j)=1)={\displaystyle \frac{1+R_0(i,j)}{2}}$ and $\mathbb{P}(R(i,j)=-1)={\displaystyle \frac{1-R_0(i,j)}{2}}$, where $\mathbb{E}(R(i,j))={\displaystyle \frac{1+R_0(i,j)}{2}}-{\displaystyle \frac{1-R_0(i,j)}{2}}=R_0(i,j)$ and Equation (2) holds surely. For the signed response matrix, we have

• $R(i,j)\in \{-1,1\}$, i.e., $R(i,j)$ only takes two values −1 and 1.
• $B(i,j)\in [-1,1]$ and $\rho \in (0,1]$ because ${\displaystyle \frac{1+R_0(i,j)}{2}}$ and ${\displaystyle \frac{1-R_0(i,j)}{2}}$ are two probabilities which should be in the range $[0,1]$. Note that, similar to Example 4, $B(i,j)$ can be negative for the signed response matrix.
• $\tau \le 2$ because $R(i,j)\in \{-1,1\}$ and $R_0(i,j)\in [-1,1]$.
• $\gamma \le 1$ because $\gamma ={\max }_{i\in \left[N\right],j\in \left[J\right]}\mathrm{Var}(R(i,j))={\max }_{i\in \left[N\right],j\in \left[J\right]}(1-R_0^2(i,j))\le 1$.
• When setting $\tau =2$ and $\gamma =1$, Assumption 1 turns to be $\max (N,J)\ge 4\log (N+J)$.
• Setting γ as its upper bound 1 in Theorem 1 gives that increasing ρ reduces SCK’s error rates.

6. Simulation Studies

In this section, we conduct extensive simulation experiments to evaluate the effectiveness of the proposed method and validate our theoretical results in Examples 1–7.

6.1. Baseline Method

More than the SCK algorithm, here we briefly provide an alternative spectral method that can also be applied to fit our adLCM. Recall that $R_0=Z{\mathrm{\Theta }}^{\prime }$ under adLCM; it is easy to see that $R_0(i,:)=R_0(\overline{i},:)$ when two distinct subjects i and $\overline{i}$ belong to the same extreme latent profile for $i,\overline{i}\in \left[N\right]$. Therefore, the population response matrix $R_0$ features K disparate rows, and running the K-means approach on all rows of $R_0$ with K clusters can faithfully recover the classification matrix Z in terms of a permutation of the K extreme latent profiles. $R_0=Z{\mathrm{\Theta }}^{\prime }$ also gives that $\mathrm{\Theta }=R_0^{\prime }Z{(Z^{\prime }Z)}^{-1}$, which suggests the following ideal algorithm called Ideal RMK Algorithm 3).

Algorithm 3 Ideal RMK

Require:  $R_0,K$. Ensure:  A permutation of Z and $\mathrm{\Theta }$.   1: Run K-means algorithm on all rows of $R_0$ with K clusters to obtain $Z\mathcal{P}$, a permutation of Z.   2: Compute $R_0^{\prime }Z\mathcal{P}{({(Z\mathcal{P})}^{\prime }Z\mathcal{P})}^{-1}=\mathrm{\Theta }\mathcal{P}$, a permutation of $\mathrm{\Theta }$.

Algorithm 4, called RMK, is a natural generalization of the Ideal RMK from the oracle case to the real case because $\mathbb{E}(R)=R_0$ under adLCM. Unlike the SCK method, the RMK method does not need to obtain the SVD of the observed response matrix R.

Algorithm 4 Response Matrix with K-means (RMK for short)

Require:  $R,K$. Ensure:  $\widehat{Z},\widehat{\mathrm{\Theta }}$.   1: Run K-means algorithm on all rows of R with K clusters to obtain $\widehat{Z}$.   2: Obtain an estimate of $\mathrm{\Theta }$ by setting $\widehat{\mathrm{\Theta }}=R^{\prime }\widehat{Z}{({\widehat{Z}}^{\prime }\widehat{Z})}^{-1}$.

The computational cost of the first step in RMK is $O(lNJK)$, where l denotes the number of iterations for the K-means algorithm. The complexity of the second step in RMK is $O(JNK)$. Therefore, the overall computational cost of RMK is $O(lNJK)$. When $J=\beta N$ for a constant value $\beta \in (0,1]$, the complexity of RMK is $O(\beta lK{N}^2)$, and it is larger than the SCK’s complexity $O(K{N}^2)$ when $\beta l>1$. Therefore, SCK runs faster than RMK when $\beta l>1$, as confirmed by our numerical results in this section.

We also compare our SCK and RMK approaches with several existing methods to highlight their superior performance. The comparative methods include the probabilistic latent component analysis (PLCA) algorithm [20] (an expectation maximization method), the nonnegative matrix factorization (NMF) technique [24], and the HeteroClustering (HC) algorithm [29] (a spectral clustering approach).

6.2. Evaluation Metric

For the classification of subjects, when the true classification matrix Z is known, to evaluate how good the quality of the partition of the subjects into extreme latent profiles is, four metrics are considered including the Clustering error $\widehat{f}$ computed by Equation (7). The other three popular evaluation criteria are Hamming error [74], normalized mutual information (NMI) [75,76,77,78], and adjusted rand index (ARI) [78,79,80].

• Hamming error is defined as

  $$\begin{array}{c}\hfill \mathrm{Hamming}\mathrm{error}=N^{-1}{\min }_{\mathcal{P}\in {\mathcal{P}}_K}{\parallel \widehat{Z}-Z\mathcal{P}\parallel }_0,\end{array}$$

  where ${\mathcal{P}}_K$ denotes the collection of all K-by-K permutation matrices. Hamming error falls within the range $[0,1]$, and a smaller Hamming error indicates better classification performance.
• Let C be a $K\times K$ confusion matrix such that $C(k,l)$ is the number of common subjects between ${\mathcal{T}}_k$ and ${\widehat{\mathcal{T}}}_l$ for $k,l\in \left[K\right]$. NMI is defined as

  $$\begin{array}{c}\hfill \mathrm{NMI}={\displaystyle \frac{-2{\sum }_{k,l}C(k,l)\log (\frac{C(k,l)N}{C_{k.}{C}_{.l}})}{{\sum }_k{C}_{k.}\log (\frac{C_{k.}}{N})+{\sum }_l{C}_{.l}\log (\frac{C_{.l}}{N})}},\end{array}$$

  where $C_{k.}={\sum }_{m=1}^{K}C(k,m)$ and $C_{.l}={\sum }_{m=1}^{K}C(m,l)$. NMI is in the range $[0,1]$ and the larger it is, the better it is.
• ARI is defined as

  $$\begin{array}{c}\hfill \mathrm{ARI}={\displaystyle \frac{{\sum }_{k,l}\left(\genfrac{}{}{0pt}{}{C(k,l)}{2}\right)-\frac{{\sum }_k\left(\genfrac{}{}{0pt}{}{C_{k.}}{2}\right){\sum }_l\left(\genfrac{}{}{0pt}{}{C_{.l}}{2}\right)}{\left(\genfrac{}{}{0pt}{}{N}{2}\right)}}{\frac{1}{2}[{\sum }_k\left(\genfrac{}{}{0pt}{}{C_{k.}}{2}\right)+{\sum }_l\left(\genfrac{}{}{0pt}{}{C_{.l}}{2}\right)]-\frac{{\sum }_k\left(\genfrac{}{}{0pt}{}{C_{k.}}{2}\right){\sum }_l\left(\genfrac{}{}{0pt}{}{C_{.l}}{2}\right)}{\left(\genfrac{}{}{0pt}{}{N}{2}\right)}}},\end{array}$$

  where $\left({\displaystyle \genfrac{}{}{0pt}{}{.}{.}}\right)$ is a binomial coefficient. ARI falls within the range [−1,1] and the larger it is, the better it is.

For the estimation of $\mathrm{\Theta }$, we use the Relative $l_1$ error and the Relative $l_2$ error to evaluate the performance. The two criteria are defined as

$$\begin{array}{c}\hfill \mathrm{Relative}{l}_1\mathrm{error}={\min }_{\mathcal{P}\in {\mathcal{P}}_K}{\displaystyle \frac{{\parallel \widehat{\mathrm{\Theta }}-\mathrm{\Theta }\mathcal{P}\parallel }_1}{{\parallel \mathrm{\Theta }\parallel }_1}}\mathrm{and}\mathrm{Relative}{l}_2\mathrm{error}={\min }_{\mathcal{P}\in {\mathcal{P}}_K}{\displaystyle \frac{{\parallel \widehat{\mathrm{\Theta }}-\mathrm{\Theta }\mathcal{P}\parallel }_F}{{\parallel \mathrm{\Theta }\parallel }_F}}.\end{array}$$

The smaller both values are the better.

6.3. Synthetic Data

We conduct numerical studies to examine the accuracy and the efficiency of the aforementioned approaches by changing the scaling parameter $\rho$ and the number of subjects N. Unless specified, in all computer-generated response matrices, we set $K=3,J={\displaystyle \frac{N}{5}}$, and the $N\times K$ classification matrix Z is generated such that each subject belongs to one of the K extreme latent profiles with equal probability. For distributions that require B’s entries to be nonnegative, we let $B(j,k)=\mathrm{rand}(1)$ for $j\in \left[J\right],k\in \left[K\right]$, where $\mathrm{rand}(1)$ is a random value simulated from the uniform distribution on $[0,1]$. For a Normal distribution and signed response matrix that allow B to have negative entries, we let $B(j,k)=2\mathrm{rand}(1)-1$ for $j\in \left[J\right],k\in \left[K\right]$, i.e., $B(j,k)$ is in the range $[-1,1]$. Set $B_{\max }={\max }_{j\in \left[J\right],k\in \left[K\right]}\left|B(j,k)\right|$ because the generation process of B makes $|B(j,k)|\in [0,1]$, but it cannot guarantee that $B_{\max }=1$, which is required in the definition of B. Therefore, we update B by ${\displaystyle \frac{B}{B_{\max }}}$. For the scaling parameter $\rho$ and the number of subjects N, they are set independently for each distribution. After setting all model parameters $(K,N,J,Z,B,\rho )$, we can generate the observed response matrix R from distribution $\mathcal{F}$ with expectation $R_0=Z{\mathrm{\Theta }}^{\prime }=\rho Z{B}^{\prime }$ under our adLCM. By applying each method to R with K extreme latent profiles, we can compute the evaluation metrics of each method. In every simulation scenario, we generate 50 independent replicates and report the mean of Clustering error (as well as Hamming error, NMI, ARI, Relative $l_1$ error, Relative $l_2$ error, and running time) computed from the 50 repetitions for each method. All numerical results in this paper are reported using MATLAB R2024b on a standard personal computer (Thinkpad X1 Carbon Gen 8).

6.3.1. Bernoulli Distribution

When $R(i,j)\sim \mathrm{Bernoulli}(R_0(i,j))$ for $i\in \left[N\right], j\in \left[J\right]$, we consider the following two simulations.

Simulation 1(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{500}$. For the Bernoulli distribution, the scaling parameter $\mathit{\rho }$ should be set within the range $(\mathbf{0},\mathbf{1}]$ according to Example 1. Here, for simulation studies, we let $\mathit{\rho }$ be in the range $\{\mathbf{0}.\mathbf{1},\mathbf{0}.\mathbf{2},\mathbf{0}.\mathbf{3},\dots ,\mathbf{1}\}$.

Simulation 1(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{0}.\mathbf{1}$ and $\mathit{N}$ be in the range $\{\mathbf{1000},\mathbf{2000},\dots ,\mathbf{5000}\}$.

The results are presented in Figure 1. We observe that SCK and HC outperform the other three methods, while NMF and PLCA perform the poorest in estimating $(\mathit{Z},\mathrm{\Theta })$. As for running time, SCK and NMF run faster than their competitors across all settings. All methods achieve better performances as $\mathit{\rho }$ increases, which conforms to our analysis in Example 1. Additionally, all algorithms enjoy better performances when the number of subjects $\mathit{N}$ increases, as predicted by our analysis following Corollary 1.

6.3.2. Binomial Distribution

When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathrm{Binomial}(\mathit{m},{\displaystyle \frac{{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})}{\mathit{m}}})$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$, we consider the following two simulations.

Simulation 2(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{500}$ and $\mathit{m}=\mathbf{5}$. Recall that $\mathit{\rho }$’s range is $(\mathbf{0},\mathit{m}]$ when $\mathcal{F}$ is a Binomial distribution according to Example 2; here, we let $\mathit{\rho }$ be in the range $\{\mathbf{0}.\mathbf{2},\mathbf{0}.\mathbf{4},\mathbf{0}.\mathbf{6},\dots ,\mathbf{2}\}$.

Simulation 2(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{0}.\mathbf{1},\mathit{m}=\mathbf{5}$, and $\mathit{N}$ be in the range $\{\mathbf{1000},\mathbf{2000},\dots ,\mathbf{5000}\}$.

Figure 2 presents the corresponding results. We note that SCK, RMK, and HC enjoy similar error rates and they outperform NMF and PLCA in estimating $(\mathit{Z},\mathrm{\Theta })$. NMF runs slightly faster than SCK, while SCK runs faster than the other three approaches for this simulation. Meanwhile, increasing $\mathit{\rho }$ (and $\mathit{N}$) decreases error rates for all methods, which confirms our findings in Example 2 and Corollary 1.

6.3.3. Poisson Distribution

When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathrm{Poisson}({\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j}))$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$, we consider the following two simulations.

Simulation 3(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{500}$. Example 3 says that the theoretical range of $\mathit{\rho }$ is $(\mathbf{0},+\infty )$ when $\mathcal{F}$ is a Poisson distribution. Here, we let $\mathit{\rho }$ be in the range $\{\mathbf{0}.\mathbf{2},\mathbf{0}.\mathbf{4},\mathbf{0}.\mathbf{6},\dots ,\mathbf{2}\}$.

Simulation 3(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{0}.\mathbf{1}$ and $\mathit{N}$ be in the range $\{\mathbf{1000},\mathbf{2000},\dots ,\mathbf{5000}\}$.

Figure 3 displays the numerical results of Simulation 3(a) and Simulation 3(b). The results are similar to those of the Binomial distribution case: SCK, RMK, and HC perform similarly, while NMF and PLCA perform poorer in estimating $(\mathit{Z},\mathrm{\Theta })$. SCK runs slightly slower than NMF and both methods run faster than their competitors. All methods perform better as $\mathit{\rho }$ and $\mathit{N}$ increase, which supports our analysis in Example 3 and Corollary 1.

6.3.4. Normal Distribution

When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathrm{Normal}({\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j}),{\mathit{\sigma }}^{\mathbf{2}})$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$, we consider the following two simulations.

Simulation 4(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{500}$ and ${\mathit{\sigma }}^{\mathbf{2}}=\mathbf{2}$. According to Example 4, the scaling parameter $\mathit{\rho }$ can be set as any positive value when $\mathcal{F}$ is a Normal distribution. Here, we let $\mathit{\rho }$ be in the range $\{\mathbf{0}.\mathbf{2},\mathbf{0}.\mathbf{4},\mathbf{0}.\mathbf{6},\dots ,\mathbf{2}\}$.

Simulation 4(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{0}.\mathbf{5},{\mathit{\sigma }}^{\mathbf{2}}=\mathbf{2}$, and $\mathit{N}$ be in the range $\{\mathbf{1000},\mathbf{2000},\dots ,\mathbf{5000}\}$.

Figure 4 shows the results. We see that SCK, RMK, and HC have similar performances in estimating model parameters $(\mathit{Z},\mathrm{\Theta })$, while NMF and PLCA fail to estimate parameters in this simulation. For running time, SCK runs faster than RMK and HC. Additionally, the error rates of SCK, RMK, and HC decrease when the scaling parameter $\mathit{\rho }$ and the number of subjects $\mathit{N}$ increase, supporting our findings in Example 4 and Corollary 1.

6.3.5. Exponential Distribution

When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathrm{Exponential}({\displaystyle \frac{\mathbf{1}}{{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})}})$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$, we consider the following two simulations.

Simulation 5(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{300}$. According to Example 5, the range of the scaling parameter $\mathit{\rho }$ is $(\mathbf{0},+\infty )$ when $\mathcal{F}$ is an Exponential distribution. Here, we let $\mathit{\rho }$ be in the range $\{\mathbf{1},\mathbf{2},\dots ,\mathbf{20}\}$ for our numerical studies.

Simulation 5(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{1}$ and $\mathit{N}$ be in the range $\{\mathbf{300},\mathbf{600},\dots ,\mathbf{3000}\}$.

Figure 5 displays the results. We see that SCK, RMK, and HC provide satisfactory estimations for $\mathit{Z}$ and $\mathrm{\Theta }$ for their small error rates, large NMI, and large ARI, while NMF and PLCA perform poorer. For running time, NMF and SCK run faster than their competitors for large $\mathit{N}$. Meanwhile, we find that increasing $\mathit{\rho }$ does not significantly influence the performances of these methods and this verifies our theoretical analysis in Example 5 that $\mathit{\rho }$ disappears in the theoretical upper bounds of error rates by setting $\mathit{\gamma }={\mathit{\rho }}^{\mathbf{2}}$ in Theorem 1 for Exponential distribution. Furthermore, when we increase $\mathit{N}$, all methods perform better and this supports our analysis after Corollary 1.

6.3.6. Uniform Distribution

When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathrm{Uniform}(\mathbf{0},\mathbf{2}{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j}))$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$, we consider the following two simulations.

Simulation 6(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{120}$. According to Example 6, the scaling parameter $\mathit{\rho }$ can be set as any positive value when $\mathcal{F}$ is a Uniform distribution. Here, we let $\mathit{\rho }$ be in the range $\{\mathbf{1},\mathbf{2},\dots ,\mathbf{20}\}$.

Simulation 6(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{1}$ and $\mathit{N}$ be in the range $\{\mathbf{300},\mathbf{600},\dots ,\mathbf{3000}\}$.

Figure 6 displays the numerical results. We see that increasing $\mathit{\rho }$ does not significantly decrease or increase the estimation accuracies of these methods, which verifies our theoretical analysis in Example 6. For all settings, SCK runs faster than RMK and HC. When increasing $\mathit{N}$, the Clustering error and Hamming error (NMI and ARI) for SCK, RMK, and HC are 0 (1), and this suggests that they return the exact estimation of the classification matrix $\mathit{Z}$. This phenomenon occurs because $\mathit{N}$ is set quite large for Uniform distribution in Simulation 6(b). For the estimation of $\mathrm{\Theta }$, the error rates for SCK, RMK, and HC decrease when we increase $\mathit{N}$ and this is consistent with our findings following Corollary 1. The numerical results for running time are similar to previous simulations, and we omit the detailed analysis here for brevity.

6.3.7. Signed Response Matrix

For signed response matrices when $\mathbb{P}(\mathit{R}(\mathit{i},\mathit{j})=\mathbf{1})={\displaystyle \frac{\mathbf{1}+{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})}{\mathbf{2}}}$ and $\mathbb{P}(\mathit{R}(\mathit{i},\mathit{j})=-\mathbf{1})={\displaystyle \frac{\mathbf{1}-{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})}{\mathbf{2}}}$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$, we consider the following two simulations.

Simulation 7(a): changing $\mathit{\rho }$. Set $\mathit{N}=\mathbf{500}$. Recall that the theoretical range of the scaling parameter $\mathit{\rho }$ is $(\mathbf{0},\mathbf{1}]$ for signed response matrices according to our analysis in Example 7; here, we let $\mathit{\rho }$ be in the range $\{\mathbf{0}.\mathbf{1},\mathbf{0}.\mathbf{2},\dots ,\mathbf{1}\}$.

Simulation 7(b): changing $\mathit{N}$. Let $\mathit{\rho }=\mathbf{0}.\mathbf{2}$ and $\mathit{N}$ be in the range $\{\mathbf{1000},\mathbf{2000},\dots ,\mathbf{5000}\}$.

Figure 7 shows the results. We see that increasing $\mathit{\rho }$ and $\mathit{N}$ improves the estimation accuracies of SCK, HC, and RMK, which confirms our analysis in Example 7 and Corollary 1. Meanwhile, PLCA and NMF almost fail to estimate $\mathit{Z}$ in this simulation. Additionally, it is easy to see that SCK, RMK, and HC enjoy similar performances in estimating $\mathit{Z}$ and $\mathrm{\Theta }$, and SCK requires less computation time compared to RMK and HC.

6.3.8. Simulated Arbitrary-Distribution Response Matrices

For visuality, we plot two response matrices $\mathit{R}$ generated from the Normal distribution and the Poisson distribution under adLCM. Let $\mathit{K}=\mathbf{2},\mathit{N}=\mathbf{16},\mathit{J}=\mathbf{10},{\mathit{\sigma }}^{\mathbf{2}}=\mathbf{1},\mathsf{\ell }(\mathit{i})=\mathbf{1},$$\mathsf{\ell }(\mathit{i}+\mathbf{8})=\mathbf{2}$ for $\mathit{i}\in \left[\mathbf{8}\right]$, and $\mathrm{\Theta }(\mathit{j},\mathbf{1})=\mathbf{100},\mathrm{\Theta }(\mathit{j},\mathbf{2})=\mathbf{110}-\mathbf{10}\mathit{j}$ for $\mathit{j}\in \left[\mathbf{10}\right]$. Because ${\mathit{R}}_{\mathbf{0}}=\mathit{Z}{\mathrm{\Theta }}^{\prime }$ has been set, we can generate $\mathit{R}$ under different distributions with expectation ${\mathit{R}}_{\mathbf{0}}$ under the proposed adLCM. Here, we consider the following two settings.

Simulation 8 (a): When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathbf{Normal}({\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j}),{\mathit{\sigma }}^{\mathbf{2}})$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$.

Simulation 8 (b): When $\mathit{R}(\mathit{i},\mathit{j})\sim \mathbf{Poisson}({\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j}))$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$.

Figure 8 displays the response matrices $\mathit{R}$ generated for Simulation 8 (a) and 8 (b), respectively. Error rates of these methods for the corresponding observed response matrices are displayed in

Table 1. Error rates of different methods for $\mathit{R}$ in Figure 8. Values outside the parentheses correspond to the first panel of Figure 8, and those inside correspond to the second panel of Figure 8.

Method	Clustering	Hamming	NMI	ARI	Rel. ${\mathsf{\ell }}_1$	Rel. ${\mathsf{\ell }}_2$
	Error	Error			Error	Error
SCK	0 (0)	0 (0)	1.000 (1.000)	1.000 (1.000)	0.0024 (0.0254)	0.0032 (0.0295)
RMK	0 (0)	0 (0)	1.000 (1.000)	1.000 (1.000)	0.0024 (0.0245)	0.0032 (0.0295)
NMF	0 (0)	0 (0)	1.000 (1.000)	1.000 (1.000)	0.0024 (0.0245)	0.0032 (0.0295)
HC	0 (0)	0 (0)	1.000 (1.000)	1.000 (1.000)	0.0024 (0.0254)	0.0032 (0.0307)
PLCA	0 (0)	0 (0)	1.000 (1.000)	1.000 (1.000)	0.0024 (0.0245)	0.0032 (0.0295)

. We also plot the estimated item matrix $\widehat{\mathrm{\Theta }}$ for our SCK and RMK in Figure 9. We see that all approaches exactly recover $\mathit{Z}$ from $\mathit{R}$, while they estimate $\mathrm{\Theta }$ with slight perturbations. Meanwhile, since $\mathit{Z},\mathrm{\Theta },$ and $\mathit{K}$ are known for this simulation, the $\mathit{R}$ provided in Figure 8 can be regarded as benchmark response matrices, and readers can apply SCK and RMK (and other methods) to $\mathit{R}$ to check their effectiveness in estimating $\mathit{Z}$ and $\mathrm{\Theta }$.

In summary, we conduct extensive numerical experiments across a wide spectrum of data-generating distributions to evaluate the SCK and RMK methods compared against various benchmarks. The proposed SCK algorithm consistently demonstrates superior performance and efficiency for latent class analysis of various types of response. Across all simulation settings, SCK achieves highly accurate estimates of both the latent class memberships $\mathit{Z}$ and item parameters $\mathrm{\Theta }$, often matching or exceeding the performance of alternative methods like RMK and HC while significantly outperforming NMF and PLCA. Notably, SCK demonstrates exceptional computational efficiency, consistently running faster than RMK and HC, and substantially faster than the EM-based PLCA method, which is the slowest across all scenarios. Furthermore, the experiments reveal a critical limitation of NMF and PLCA: the two methods consistently fail to provide reliable parameter estimates when the response matrix contains negative values, as evidenced by their poor performance in simulations involving Normal distributions and signed response matrices. In contrast, SCK robustly handles all distributions, including those with negative or continuous responses, confirming its versatility as a powerful and efficient tool for the analysis of data under the proposed adLCM framework.

7. Real Data Applications

As the main goal of this paper is to introduce the proposed adLCM and the SCK algorithm for arbitrary-distribution response matrices, this section reports empirical results on two datasets. Because the true classification matrix and the true item parameter matrix are unknown for real data, and SCK runs much faster than RMK and HC though they may perform similarly in simulations, we only report the outcomes of the SCK approach. For real-world datasets, the number of extreme latent profiles $\mathit{K}$ is often unknown. Here, we infer $\mathit{K}$ for real-world data using the following strategy:

$$\begin{array}{c}\hfill \mathit{K}=\mathbf{arg}{\mathbf{min}}_{\mathit{k}\in [\mathbf{rank}(\mathit{R})]}\parallel \mathit{R}-\widehat{\mathit{Z}}{\widehat{\mathrm{\Theta }}}^{\prime }\parallel ,\end{array}$$

(8)

where $\widehat{\mathit{Z}}$ and $\widehat{\mathrm{\Theta }}$ are outputs in Algorithm 2 with inputs $\mathit{R}$ and $\mathit{k}$. The method specified in Equation (8) selects $\mathit{K}$ by picking the one that minimizes the spectral norm difference between $\mathit{R}$ and $\widehat{\mathit{Z}}{\widehat{\mathrm{\Theta }}}^{\prime }$. The determination of the number of extreme latent profiles K in our adLCM in a rigorous manner with theoretical guarantees remains a future direction.

7.1. International Personality Item Pool (IPIP) Personality Test Data

Background. We apply SCK to an experiment personality test dataset called the International Personality Item Pool (IPIP) personality test, which is obtainable for download at https://openpsychometrics.org/_rawdata/ (accessed on 11 August 2025). This data consists of 1005 subjects and 40 items. The IPIP data also records the age and gender of each subject. After dropping subjects with missing entries in their responses, age, or gender, and dropping two subjects that are neither male nor female, there are 896 subjects left, i.e., $\mathit{N}=\mathbf{896},\mathit{J}=\mathbf{40}$. All items are rated on a 5-point scale, where 1 = Strongly disagree, 2 = Disagree, 3 = Neither agree not disagree, 4 = Agree, 5 = Strongly agree, i.e., $\mathit{R}\in {\{\mathbf{1},\mathbf{2},\mathbf{3},\mathbf{4},\mathbf{5}\}}^{\mathbf{896}\times \mathbf{40}}$, a response matrix. Items 1–10 measure the personality factor Assertiveness (short as “AS”); Items 11–20 measure the personality factor Social confidence (short as “SC”); Items 21–30 measure the personality factor Adventurousness (short as “AD”); and Items 31–40 measure the personality factor Dominance (short as “DO”). The details of each item are depicted in Figure 10.

Analysis. We apply Equation (8) to infer $\mathit{K}$ for the IPIP dataset and find that the estimated value of $\mathit{K}$ is 3. We then apply the SCK algorithm to the response matrix $\mathit{R}$ with $\mathit{K}=\mathbf{3}$ to obtain the $\mathbf{896}\times \mathbf{3}$ matrix $\widehat{\mathit{Z}}$ and the $\mathbf{40}\times \mathbf{3}$ matrix $\widehat{\mathrm{\Theta }}$. The running time for SCK on this dataset is around 0.2 s.

Results. For convenience, we denote the estimated three extreme latent profiles as profile 1, profile 2, and profile 3. Based on $\widehat{\mathit{Z}}$ and the information of age and gender, we can obtain some basic information (shown in

Table 2. Descriptive statistics for each estimated extreme latent profile from $\widehat{\mathit{Z}}$ on the IPIP dataset.

Statistic	Profile 1	Profile 2	Profile 3
Size	276	226	394
# Male	123	129	241
# Female	153	97	153
Average age (male)	35.98	32.82	35.90
Average age (female)	35.54	31.38	38.71

) such as the size of each profile, number of males (females) in each profile, and the average age of males (and females) in each profile. From Table 2, we see that the number of females is larger than that of males for profile 1, while profiles 2 and 3 have more males. The average age of males (and females) in profile 2 is smaller than that of profiles 1 and 3, while the average age of females in profile 3 is the largest. We can also obtain the average point on each item for males (and females) in each estimated extreme latent profile and the results are shown in Figure 10. We observe that males in profile 3 tend to be more confident, more creative, more social, and more open to changes than males in profiles 1 and 2; males in profile 3 are more (less) dominant than males in profile 1 (profile 2). Males in profile 2 are more confident, creative, social, open to changes, and dominant than males in profile 1. Meanwhile, in the three estimated extreme latent profiles, females enjoy similar personalities to males. We also find that males in profile 3 (profile 2) are more (less) confident, creative, social, open to changes, and dominant than females in profile 3 (profile 2). Furthermore, it is interesting to see that, though males in profile 1 are less confident, creative, social, and open to changes than females in profile 1, they are more dominant than females in profile 1. We also plot the average point on each item in each estimated extreme latent profile regardless of gender in Figure 10 where we can draw similar conclusions as those for males. In Figure 10, we also plot the heatmap of the estimated item parameter matrix $\widehat{\mathrm{\Theta }}$. By comparing these results shown in Figure, we see that the $(\mathit{j},\mathit{k})$-th element in the matrix shown in the third panel of Figure 10 is close to $\widehat{\mathrm{\Theta }}(\mathit{j},\mathit{k})$ for $\mathit{j}\in \left[\mathbf{40}\right],\mathit{k}\in \left[\mathbf{3}\right]$. Such a result implies that the behavior differences on each item for every extreme latent profile are governed by the item parameter matrix $\mathrm{\Theta }$.

Remark 6.

Recall that $\mathbb{E}(\mathit{R})={\mathit{R}}_{\mathbf{0}}=\mathit{Z}{\mathrm{\Theta }}^{\prime }$ under the adLCM; then we have ${\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})=\mathrm{\Theta }(\mathit{j},\mathsf{\ell }(\mathit{i}))$ for $\mathit{i}\in \left[\mathit{N}\right],\mathit{j}\in \left[\mathit{J}\right]$. Then we have ${\sum }_{\mathsf{\ell }(\mathit{i})\equiv \mathit{k}}{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})={\sum }_{\mathsf{\ell }(\mathit{i})\equiv \mathit{k}}\mathrm{\Theta }(\mathit{j},\mathsf{\ell }(\mathit{i}))={\sum }_{\mathsf{\ell }(\mathit{i})\equiv \mathit{k}}\mathrm{\Theta }(\mathit{j},\mathit{k})={\mathit{N}}_{\mathit{k}}\mathrm{\Theta }(\mathit{j},\mathit{k})$, which gives that $\mathrm{\Theta }(\mathit{j},\mathit{k})={\displaystyle \frac{{\sum }_{\mathsf{\ell }(\mathit{i})\equiv \mathit{k}}{\mathit{R}}_{\mathbf{0}}(\mathit{i},\mathit{j})}{{\mathit{N}}_{\mathit{k}}}}$ for $\mathit{k}\in \left[\mathit{K}\right]$. This explains why the average value on the $\mathit{j}$-th item in the $\mathit{k}$-th estimated extreme latent profile approximates $\widehat{\mathrm{\Theta }}(\mathit{j},\mathit{k})$ for $\mathit{j}\in \left[\mathit{J}\right],\mathit{k}\in \left[\mathit{K}\right]$.

7.2. Big Five Personality Test with Random Number (BFPTRN) Data

Background.Our SCK method is also applied to personality test data: the Big Five Personality Test with Random Number (BFPTRN) data. This dataset can be downloaded from the same URL as the IPIP data. This data asks respondents to generate random numbers in certain ranges attached to 50 personality items. The Big Five personality traits are extraversion (items E1–E10), neuroticism (items N1–N10), agreeableness (items A1–A10), conscientiousness (items C1–C10), and openness (items O1–O10). The original BFPTRN data contains 1369 subjects. After excluding subjects with missing responses or missing random numbers and removing those with random numbers exceeding the specified range, there remain 1155 subjects, i.e., $\mathit{N}=\mathbf{1155},\mathit{J}=\mathbf{50}$. All items are rated using the same 5-point scale as the IPIP data, which results in $\mathit{R}\in {\{\mathbf{1},\mathbf{2},\mathbf{3},\mathbf{4},\mathbf{5}\}}^{\mathbf{1155}\times \mathbf{50}}$. The detail of each item and each range for random numbers can be found in Figure 11.

Analysis. The estimated number of extreme latent profiles for the BFPTRN dataset is 3. Applying the SCK approach to $\mathit{R}$ with $\mathit{K}=\mathbf{3}$ produces the $\mathbf{1155}\times \mathbf{3}$ matrix $\widehat{\mathit{Z}}$ and the $\mathbf{50}\times \mathbf{3}$ matrix $\widehat{\mathrm{\Theta }}$. SCK takes around 1.6 s to process this data.

Results. Without confusion, we also let profile 1, profile 2, and profile 3 represent the three estimated extreme latent profiles. Profiles 1,2, and 3 have 409, 320, and 426 subjects, respectively. Similar to the IPIP data, based on $\widehat{\mathit{Z}}$ and $\widehat{\mathrm{\Theta }}$, we can also obtain the heatmap of the average point on each subject for every profile, the heatmap of the average random number on each range for every profile, and the heatmap of $\widehat{\mathrm{\Theta }}$ as shown in Figure. We observe that there is no significant connection between the average point and the average random number on each item in each estimated extreme latent profile. From the first panel of Figure, we find that: for extroversion, subjects in profile 1 are the most extroverted, while subjects in profile 2 are the most introverted; for neuroticism, subjects in profile 3 are emotionally stable, while subjects in profiles 1 and 2 are emotionally unstable; for agreeableness, subjects in profiles 1 and 3 are easier to get along with than subjects in profile 2; for conscientiousness, subjects in profile 3 are more responsible than those in profiles 1 and 2; and for openness, subjects in profiles 1 and 3 are more open than those in profile 2. Meanwhile, the matrix shown in the first panel of Figure approximates $\widehat{\mathrm{\Theta }}$ well, which has been explained in Remark 6.

8. Conclusions and Future Work

In this paper, we introduced the arbitrary-distribution latent class model (adLCM), a novel class of latent class analysis models for data with arbitrary-distribution responses. We studied its model identifiability, developed an efficient inference method SCK to fit adLCM, and built a theoretical guarantee of estimation consistency for the proposed method under adLCM. On the methodology side, the new model adLCM provides exploratory and useful tools for latent class analysis in applications where the data may have arbitrary-distribution responses. adLCM allows the observed response matrix to be generated from any distribution as long as its expectation follows a latent class structure modeled by adLCM. In particular, the popular latent class model is a sub-model of our adLCM, and data with signed responses can also be modeled by adLCM. Ground-truth latent classes of data with responses generated from adLCM serve as benchmarks for evaluating latent class analysis approaches. On the algorithmic side, the SVD-based spectral method SCK is efficient and easy to implement. SCK requires no tuning parameters and it is applicable for data with arbitrary-distribution responses. This means that researchers in fields such as social, psychological, behavioral, and biological sciences, and beyond can design their tests/evaluations/surveys/interviews without worrying that the response should be binary or positive, as our method SCK is applicable for any kind of response matrices in latent class analysis. On the theoretic side, we established the rate of convergence for our method SCK under the proposed adLCM. We found that SCK exhibits different behaviors when the response matrices are generated from different distributions, and we conducted extensive experiments to verify our theoretical findings. Empirically, we applied our method to two real personality test datasets with meaningful results. We expect that our adLCM and SCK method will have broad applications for latent class analysis in understanding human behaviors for diverse fields, similar to the widespread use of latent class models in recent years.

There are several future directions worth exploring. First, methods with theoretical guarantees should be designed to determine the number of extreme latent profiles $\mathit{K}$ for observed response matrices generated from any distribution $\mathcal{F}$ under adLCM. Following [81], a possible approach to estimate $\mathit{K}$ is to count the number of significant singular values (i.e., those above a noise threshold) of $\mathit{R}$. Second, the grade of membership (GoM) model [52,82] provides a richer modeling capacity than the latent class model since GoM allows a subject to belong to multiple extreme latent profiles. Therefore, following the distribution-free idea developed in this work, it is meaningful to extend the model GoM to data with arbitrary-distribution responses. Third, like the LCM being equipped with individual covariates [50,83,84,85,86,87], it is worth considering adding individual covariates into the adLCM analysis. Fourth, our adLCM only considers static latent class analysis and it is meaningful to extend adLCM to the dynamic case [88]. Fifth, our SCK is a spectral clustering method, and it is possible to speed it up by application of the random-projection techniques [63] or the distributed spectral clustering idea [89] to deal with large-scale data for latent class analysis. Finally, while our proposed methods can estimate latent class memberships and item parameters, a meaningful extension would be to develop methodology and theoretical guarantees for explicitly estimating the latent distribution parameters in future work.