Variational Bayesian Inference for a Q-Matrix-Free Hidden Markov Log-Linear Additive Cognitive Diagnostic Model

Abstract

Cognitive diagnostic models (CDMs) are commonly used in educational assessment to uncover the specific cognitive skills that contribute to student performance, allowing for precise identification of individual strengths and weaknesses and the design of targeted interventions. Traditional CDMs, however, depend heavily on a predefined Q-matrix that specifies the relationship between test items and underlying attributes. In this study, we introduce a hidden Markov log-linear additive cognitive diagnostic model (HM-LACDM) that does not require a Q-matrix, making it suitable for analyzing longitudinal assessment data without prior structural assumptions. To support scalable applications, we develop a variational Bayesian inference (VI) algorithm that enables efficient estimation in large datasets. Additionally, we propose a method to reconstruct the Q-matrix from estimated item-effect parameters. The effectiveness of the proposed approach is demonstrated through simulation studies.

1. Introduction

Cognitive diagnostic models (CDMs), also known as diagnostic classification models (DCMs), have emerged as powerful psychometric tools that provide fine-grained information about examinees’ knowledge states and cognitive processes, delivering substantially richer feedback than traditional single test scores [1,2,3]. Unlike traditional item response theory (IRT) models [4,5,6] that place examinees on a continuous ability scale, CDMs classify examinees into discrete attribute mastery patterns, offering an attribute-based interpretation of test performance. Due to these advantages, CDMs have gained significant interest across various domains, including educational assessment [7,8], language testing [9], and psychological assessment [10].

While most CDM applications have focused on cross-sectional data, there is growing interest in longitudinal applications to measure examinees’ growth or change in attribute mastery over time. Longitudinal CDMs are particularly valuable for monitoring students’ learning progress, evaluating instructional effectiveness, and understanding the development of cognitive skills. Several approaches have been developed to model longitudinal assessment data with CDMs. The most straightforward approach involves fitting separate CDMs at each time point and comparing the resulting classifications [11,12]. However, this approach ignores dependencies between attribute mastery states across time points, which may lead to inconsistent classifications. To address these limitations, more sophisticated longitudinal DCMs have been proposed. Li et al. [13] proposed the LTA-DINA model, which combines the latent transition analysis (LTA) [14,15] framework with the deterministic input, noisy “and” gate (DINA) [16,17] model. Kaya and Leite [18] developed longitudinal cognitive diagnosis models that combine latent transition analysis with the DINA model and deterministic input, noisy “or” gate (DINO) model [10], demonstrating through simulation studies that these approaches yield satisfactory convergence and classification accuracy when tracking attribute mastery over time. Wang et al. [19] introduced a higher-order hidden Markov model for tracking skill acquisition, which models the development of latent attributes through a continuous higher-order proficiency variable. Madison and Bradshaw [20] proposed the transition diagnostic classification model (TDCM), which integrates LTA with the more general log-linear cognitive diagnosis model, allowing for flexible modeling of attribute relationships and transitions over time.

However, there are challenges in the application of these longitudinal CDM approaches due to their reliance on a pre-specified Q-matrix. Q-matrix [21] is essential to standard CDM approaches, mapping each test item to the specific attributes required to answer it correctly. Specifically, for a test with J items and K attributes, the Q-matrix is a $J\times K$ binary matrix, where $q_{jk}=1$ indicates that item j requires attribute k to answer correctly, and $q_{jk}=0$ otherwise. The accurate specification of the Q-matrix is crucial for valid diagnostic inferences from the CDMs. Rupp and Templin [22] demonstrated that Q-matrix misspecification can lead to biased parameter estimates and reduced classification accuracy. Kunina-Habenicht et al. [23] found that the impact of Q-matrix misspecification depends on the specific DCM and the pattern of misspecification. Madison and Bradshaw [24] showed that even small changes in the Q-matrix design can significantly affect classification accuracy in the LCDM framework. Despite the importance of the Q-matrix, specifying a Q-matrix is a challenging and time-consuming task [25], as it requires a deep understanding of the cognitive processes involved in answering each item. In practice, researchers often rely on expert judgment to construct the Q-matrix, which is subjective and may lead to inconsistencies across different assessments. This challenge can be magnified in longitudinal settings, where misclassifications at one time point can propagate and affect the interpretation of transitions over time.

Given the challenges associated with Q-matrix specification, researchers have explored various approaches to estimate or validate Q-matrices empirically, or to develop models that do not require a pre-specified Q-matrix. Data-driven Q-matrix validation methods have been proposed to refine expert-specified Q-matrices based on empirical data. De La Torre [26] developed a method for validating and refining Q-matrices for the DINA model using a discrimination index. This method has been extended to the generalized DINA (G-DINA) model by De La Torre and Chiu [27]. These methods typically start with an expert-specified Q-matrix and suggest modifications based on statistical criteria. More ambitious approaches aim to estimate the Q-matrix directly from response data with minimal prior specification. Liu et al. [28,29] developed self-learning algorithms for Q-matrix estimation that iteratively update both the Q-matrix and model parameters and established conditions under which the Q-matrix can be uniquely identified from the data. Chen et al. [30] formulated the Q-matrix estimation as a latent variable selection problem and employed the regularized maximum likelihood to estimate the Q-matrix. Chen et al. [31] introduced a Bayesian method for estimating the DINA Q-matrix, which incorporates prior knowledge while allowing for data-driven refinement. More recently, Balamuta and Culpepper [32] introduced exploratory restricted latent class models with monotonicity requirements under Pòlya–Gamma data augmentation, providing a powerful Bayesian framework for Q-matrix estimation. Similarly, Chen et al. [33] introduced a sparse latent class model (SLCM) for cognitive diagnosis that simultaneously estimates the Q-matrix and classifies examinees without requiring prior specification of the number of attributes. Yamaguchi [34] uses a partially known Q-matrix to simultaneously estimate the effects of active and nonactive attributes in a Bayesian framework, which can also be employed to estimate the unknown Q-matrix part.

Despite these advances, longitudinal CDMs face significant computational challenges, particularly when the number of attributes or time points increases. Most current estimation methods rely on Markov Chain Monte Carlo (MCMC) techniques, which can be computationally intensive and time-consuming. Instead of MCMC, variational inference (VI) has emerged as a powerful alternative for estimating complex models, including CDMs, due to its computational efficiency and scalability [35,36,37,38]. Over the past several years, a growing body of work has advanced the use of variational inference for psychometric models. Yamaguchi and Okada [39,40] introduced variational inference approaches for the DINA model and the saturated diagnostic classification model, demonstrating significant computational advantages over traditional MCMC methods while maintaining comparable accuracy. Building on this foundation, Yamaguchi and Martinez [41] extended the variational framework to hidden Markov diagnostic classification models, making longitudinal cognitive diagnostic analyses more tractable for large-scale applications. Further advancing the field, Wang et al. [42] developed an efficient VBEM-M algorithm for the log-linear cognitive diagnostic model that aligns variational posteriors with prior distributions, achieving both faster convergence and improved parameter recovery compared to conventional estimation methods. In this paper, we propose a novel approach that leverages variational inference to efficiently estimate parameters of our hidden Markov log-linear additive cognitive diagnostic model (HM-LACDM), which extends the sparse latent class model (SLCM) framework. We implement post hoc Q-matrix recovery based on the estimated posterior distributions of item-effect parameters, enabling us to determine the Q-matrix without prior specification. Our approach provides substantial computational advantages over traditional MCMC methods, making it particularly well-suited for analyzing complex longitudinal diagnostic data with numerous attributes or time points. We demonstrate that our proposed method can effectively recover the Q-matrix and accurately estimate the model parameter through simulation studies.

2. Notation and Model Formulation

Before proceeding with our methodology, we first establish the notation that will be used throughout this paper. For any integer n, we denote the set $\{1,2,\dots ,n\}$ as $\left[n\right]$. We denote the number of respondents as I, the number of items as J, the number of attributes as K, and the number of assessment time points as T. As discussed in the introduction, we denote the Q-matrix as $\mathrm{Q}\in {\mathbb{R}}^{J\times K}$, where $q_{jk}=1$ indicates that item j requires attribute k to answer correctly, and $q_{jk}=0$ otherwise. Formally, the Q-matrix is denoted as $\mathrm{Q}\in {\{0,1\}}^{J\times K}$. Let ${\mathbf{q}}_j$ represent the $j\mathrm{th}$ row vector of $\mathrm{Q}$, such that $\mathrm{Q}={[{\mathbf{q}}_1,\dots ,{\mathbf{q}}_J]}^{\prime }$. In our work, we assume that the Q-matrix is unknown and we will recover it post hoc based on inference from the variational distributions of the item-effect parameters.

We introduce the latent discrete random variable $\mathbf{Z}\in {\{0,1\}}^{I\times T\times K}$, which represents the attribute mastery profiles of all respondents across time points and attributes. For each respondent $i\in \left[I\right]$, let ${\mathbf{Z}}_i\in {\{0,1\}}^{T\times K}$ denote their complete attribute profile, where each entry $z_{i,t,k}=1$ indicates that respondent i has mastered attribute k at assessment point t, and $z_{i,t,k}=0$ otherwise. We denote by ${\mathbf{z}}_{i,t}\in {\{0,1\}}^K$ the attribute mastery pattern of respondent i at assessment point t. At each time point t, there are $L:=2^K$ possible attribute mastery patterns. There exists a natural bijective mapping $\mathrm{v}:{\{0,1\}}^K\to \left[L\right]$ that maps each attribute profile ${\mathbf{z}}_{i,t}$ to a unique latent class $c_{i,t}\in \left[L\right]$, such that $\mathrm{v}({\mathbf{z}}_{i,t})=c_{i,t}$ and ${\mathrm{v}}^{-1}(c_{i,t})={\mathbf{z}}_{i,t}$. With slight abuse of notation, we can omit the mapping $\mathrm{v}$ and simply denote the latent class as ${\mathbf{z}}_{i,t}=c_{i,t}\in \left[L\right]$. Let $\Delta ={[{\mathit{\delta }}_1,\dots ,{\mathit{\delta }}_L]}^{\prime }\in {\mathbb{R}}^{L\times (K+1)}$ be the design matrix for the item effect parameters, where ${\mathit{\delta }}_l\in {\mathbb{R}}^{K+1}$ is the $l\mathrm{th}$ row of $\Delta$. Each row can be constructed as ${\mathit{\delta }}_l=\left[\begin{array}{c}1\\ {\mathrm{v}}^{-1}(l)\end{array}\right]\in {\mathbb{R}}^{K+1}$, where the first element is the intercept term and the remaining elements correspond to the attribute pattern. For example, when $K=2$, we have

$$\begin{array}{c}\hfill \Delta =\left[\begin{array}{ccc}1& 0& 0\\ 1& 0& 1\\ 1& 1& 0\\ 1& 1& 1\end{array}\right].\end{array}$$

We denote the response matrix as $Y\in {\{0,1\}}^{I\times J\times T}$, where $y_{i,j,t}=1$ indicates that respondent i answered item j correctly at time point t, and $y_{i,j,t}=0$ otherwise. In our model, for respondent i at time point t, the response to item j is modeled as a Bernoulli random variable based on the mixture of generalized linear model (GLM) framework:

$$p(y_{i,j,t}=1\mathit{\mid }{\mathit{z}}_{i,t}=l,{\mathit{\beta }}_j)=\mathsf{\Psi }({\mathit{\beta }}_j^{\prime }{\mathit{\delta }}_l),$$

(1)

where $\mathsf{\Psi }(·)$ is the logistic link function, i.e., $\mathsf{\Psi }(x)={\displaystyle \frac{1}{1+\exp (-x)}}$, ${\mathit{\beta }}_j\in {\mathbb{R}}^{K+1}$ is the item-effect parameter vector for item j. For each item j, we set the prior distribution of the item-effect parameter ${\mathit{\beta }}_j$ as a multivariate normal distribution ${\mathit{\beta }}_j\sim \mathcal{N}({\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)},{\Sigma }_{{\mathit{\beta }}_j}^{(0)}),$ where ${\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)}\in {\mathbb{R}}^{K+1}$ is the prior mean and ${\Sigma }_{{\mathit{\beta }}_j}^{(0)}\in {\mathbb{R}}^{(K+1)\times (K+1)}$ is the prior covariance matrix. We denote the collection of item-effect parameters as $\mathit{\beta }={[{\mathit{\beta }}_1,\dots ,{\mathit{\beta }}_J]}^{\prime }\in {\mathbb{R}}^{J\times (K+1)}$, and the prior distribution of $\mathit{\beta }$ can be written as

$$p(\mathit{\beta })=\prod _{j=1}^{J}p({\mathit{\beta }}_j)\propto \prod _{j=1}^J\exp \left(-{\displaystyle \frac{1}{2}}{({\mathit{\beta }}_j-{\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)})}^{\prime }{{\Sigma }_{{\mathit{\beta }}_j}^{(0)}}^{-1}({\mathit{\beta }}_j-{\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)})\right).$$

(2)

Assuming conditional independence of responses $y_{i,j,t}$ given the attribute mastery patterns ${\mathbf{z}}_{i,t}$, and item parameters ${\mathit{\beta }}_j$, the joint likelihood of the observed data is

$$\begin{array}{cc}\hfill p(Y\mathit{\mid }\mathbf{Z},\mathit{\beta })& =\prod _{i=1}^I\prod _{t=1}^T\prod _{j=1}^{J}p(y_{i,j,t}\mathit{\mid }{\mathbf{z}}_{i,t},{\mathit{\beta }}_j)\hfill \\ & =\prod _{i=1}^I\prod _{t=1}^T\prod _{j=1}^J\prod _{l=1}^L\mathsf{\Psi }{\left((2y_{i,j,t}-1){\mathit{\beta }}_j^{\prime }{\mathit{\delta }}_l\right)}^{\mathbf{1}({\mathbf{z}}_{i,t}=l)}.\hfill \end{array}$$

(3)

To model the longitudinal transitions of latent attribute mastery, we employ a hidden Markov model (HMM) structure. In our model, we assume that the transition probabilities between latent classes are time-homogeneous, meaning that the transition probabilities remain constant across time points. This is a common assumption in HMMs and allows us to simplify the modeling process while still capturing the essential dynamics of the latent attribute mastery states. We denote the time-homogeneous transition matrix as $\mathbf{T}={\left[{\mathit{\tau }}_{l,l^{\prime }}\right]}_{l,l^{\prime }\in \left[L\right]}\in {[0,1]}^{L\times L}$, where each entry ${\mathit{\tau }}_{l,l^{\prime }}$ represents the probability of transitioning from latent class l at time point t to latent class $l^{\prime }$ at time point $t+1$. The transition matrix satisfies the constraint ${\sum }_{l^{\prime }\in \left[L\right]}{\mathit{\tau }}_{l,l^{\prime }}=1$ for all $l\in \left[L\right]$. For $t\ge 2$, the transition from the latent class at time point $t-1$ to the latent class at time point t is modeled as a categorical distribution

$$\begin{array}{cc}\hfill p({\mathbf{z}}_{i,t}\mathit{\mid }{\mathbf{z}}_{i,t-1},\mathbf{T})& =\prod _{l=1}^L\prod _{l^{\prime }=1}^L{\mathit{\tau }}_{l,l^{\prime }}^{\mathbf{1}({\mathbf{z}}_{i,t-1}=l,{\mathbf{z}}_{i,t}=l^{\prime })}.\hfill \end{array}$$

(4)

For $t=1$, the initial latent class distribution is modeled as a categorical distribution with prior probabilities $\mathit{\pi }=[{\mathit{\pi }}_1,\dots ,{\mathit{\pi }}_L]$, where ${\sum }_{l=1}^L{\mathit{\pi }}_l=1$. We have

$$\begin{array}{c}\hfill p({\mathbf{z}}_{i,1}\mathit{\mid }\mathit{\pi })=\prod _{l=1}^L{\mathit{\pi }}_l^{\mathbf{1}({\mathbf{z}}_{i,1}=l)}.\end{array}$$

(5)

We set the prior distribution of each row vector of the transition matrix ${\mathit{\tau }}_l=[{\mathit{\tau }}_{l,1},\dots ,{\mathit{\tau }}_{l,L}]$ as a Dirichlet distribution with concentration parameters ${\mathit{\omega }}_l=[{\mathit{\omega }}_{l,1}^{(0)},\dots ,{\mathit{\omega }}_{l,L}^{(0)}]$, i.e., ${\mathit{\tau }}_l\sim \mathrm{Dir}({\mathit{\omega }}_l)$. The prior distribution of the transition matrix $\mathbf{T}$ can be expressed as

$$\begin{array}{cc}\hfill p(\mathbf{T})& =\prod _{l=1}^{L}p({\mathit{\tau }}_l)\propto \prod _{l=1}^L\prod _{l^{\prime }=1}^L{\mathit{\tau }}_{l,l^{\prime }}^{{\mathit{\omega }}_{l,l^{\prime }}^{(0)}-1}.\hfill \end{array}$$

(6)

Similarly, we set the prior distribution of the initial latent class distribution $\mathit{\pi }$ as a Dirichlet distribution with concentration parameters $\mathit{\alpha }=[{\alpha }_1^{(0)},\dots ,{\alpha }_L^{(0)}]$, i.e., $\mathit{\pi }\sim \mathrm{Dir}(\mathit{\alpha })$. We have

$$\begin{array}{cc}\hfill p(\mathit{\pi })& \propto \prod _{l=1}^L{\mathit{\pi }}_l^{{\alpha }_l^{(0)}-1}.\hfill \end{array}$$

(7)

We can now summarize our HM-LACDM model as follows:

$$\begin{array}{cc}\hfill p(Y,\mathbf{Z},\mathit{\beta },\mathbf{T},\mathit{\pi })& =p(Y\mathit{\mid }\mathbf{Z},\mathit{\beta })p(\mathbf{Z}\mathit{\mid }\mathbf{T},\mathit{\pi })p(\mathit{\beta })p(\mathbf{T})p(\mathit{\pi })\hfill \\ \hfill & =p(Y\mathit{\mid }\mathbf{Z},\mathit{\beta })\left(\prod _{i=1}^{I}p({\mathbf{z}}_{i,1}\mathit{\mid }\mathit{\pi })\right)\left(\prod _{i=1}^I\prod _{t=2}^{T}p({\mathbf{z}}_{i,t}\mathit{\mid }{\mathbf{z}}_{i,t-1},\mathbf{T})\right)p(\mathit{\beta })p(\mathbf{T})p(\mathit{\pi }),\hfill \end{array}$$

(8)

where $p(Y\mathit{\mid }\mathbf{Z},\mathit{\beta })$ is the likelihood of the observed data given the latent attribute mastery profiles and item parameters (Equation (3)); $p(\mathbf{Z}\mathit{\mid }\mathbf{T},\mathit{\pi })$ represents the likelihood of the latent attribute mastery profiles given the transition matrix and initial distribution (Equations (5) and (4)); $p(\mathit{\beta })$ denotes the prior distribution of the item-effect parameters (Equation (2)); $p(\mathbf{T})$ specifies the prior distribution of the transition matrix (Equation (6)); and $p(\mathit{\pi })$ is the prior distribution of the initial latent class distribution (Equation (7)). A more detailed description of the data likelihood can be found in Appendix A.

3. Variational Inference for HM-LACDM

Variational inference (VI) [35,36,37,38] is a powerful computational technique that transforms Bayesian posterior inference into an optimization problem. Given a probabilistic model with latent variables $\mathbf{z}$ and observed data x, Bayesian inference seeks to compute the posterior distribution $p(\mathbf{z}\mathit{\mid }x)={\displaystyle \frac{p(x\mathit{\mid }\mathbf{z})p(\mathbf{z})}{p(x)}}$, where $p(x)$ is the marginal likelihood of the data. However, $p(x)$ is often intractable to compute, especially for complex models with high-dimensional latent variables. Instead of computing the intractable posterior distribution $p(\mathbf{z}\mathit{\mid }x)$ directly, VI approximates it with a simpler distribution $q(\mathbf{z})$ by minimizing the Kullback–Leibler (KL) divergence between $q(\mathbf{z})$ and the true posterior $p(\mathbf{z}\mathit{\mid }x)$. VI aims to maximize the evidence lower bound (ELBO), defined as:

$$\mathrm{ELBO}(q(\mathbf{z}))={\mathbb{E}}_q[\log p(x,\mathbf{z})]-{\mathbb{E}}_q[\log q(\mathbf{z})],$$

where $p(x,\mathbf{z})$ is the joint distribution of the observed data and latent variables, and $q(\mathbf{z})$ is the variational distribution. Since maximizing the ELBO is equivalent to minimizing the KL divergence to the true posterior, this approach provides a principled framework for approximate Bayesian inference that is both computationally efficient and scalable to large datasets. For our HM-LACDM, we employ coordinate ascent variational inference (CAVI) with mean-field factorization, which assumes the variational posterior factorizes as $q(\mathbf{z})={\prod }_{i=1}^{n}q(z_i)$. This enables iterative optimization of each component while keeping others fixed. Additionally, we handle the logistic link function using the Jaakkola–Jordan lower bound approximation Jaakkola and Jordan [43], which transforms the non-conjugate logistic likelihood into a tractable quadratic form suitable for Gaussian variational approximations. A more detailed introduction to variational inference can be found in Appendix B.1.

Leveraging the mean-field variational inference framework, we aim to approximate the posterior distribution of the model parameters via a factorized distribution:

$$\begin{array}{cc}\hfill p(\mathbf{Z},\mathit{\beta },\mathbf{T},\mathit{\pi }\mathit{\mid }Y)& \approx q(\mathbf{Z})q(\mathit{\beta })q(\mathbf{T})q(\mathit{\pi })\hfill \\ \hfill & =\left(\prod _{i=1}^{I}q({\mathbf{Z}}_i)\right)\left(\prod _{j=1}^{J}q({\mathit{\beta }}_j)\right)\left(\prod _{l=1}^{L}q({\mathit{\tau }}_l)\right)q(\mathit{\pi }).\hfill \end{array}$$

(9)

In this section, we present the variational posterior distributions and their iterative updates based on coordinate ascent variational inference (CAVI). Detailed derivations of the variational updates can be found in the Appendix B.

First, the variational posterior distribution of the item-effect parameters ${\mathit{\beta }}_j$ is assumed to follow a multivariate normal distribution $q({\mathit{\beta }}_j)\sim \mathcal{N}({\mathit{\beta }}_j\mathit{\mid }{\mathit{\mu }}_{{\mathit{\beta }}_j}^{(★)},{\Sigma }_{{\mathit{\beta }}_j}^{(★)})$, where ${\mathit{\mu }}_{{\mathit{\beta }}_j}^{(★)}\in {\mathbb{R}}^{K+1}$ is the variational mean and ${\Sigma }_{{\mathit{\beta }}_j}^{(★)}\in {\mathbb{R}}^{(K+1)\times (K+1)}$ is the variational covariance matrix. The iterative updates can be updated as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}{\Sigma }_{{\mathit{\beta }}_j}^{(★)}\hfill & ={\left({{\Sigma }_{{\mathit{\beta }}_j}^{(0)}}^{-1}+2\sum _{i=1}^I\sum _{l=1}^L\sum _{t=1}^T{\mathbb{E}}_q\left[\mathbf{1}\left({\mathbf{z}}_{i,t}=l\right)\right]h({\xi }_{j,l}){\mathit{\delta }}_l{\mathit{\delta }}_l^{\prime }\right)}^{-1},\hfill \\ {\mathit{\mu }}_{{\mathit{\beta }}_j}^{(★)}\hfill & ={\Sigma }_{{\mathit{\beta }}_j}^{(★)}\left({{\Sigma }_{{\mathit{\beta }}_j}^{(0)}}^{-1}{\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)}+\sum _{i=1}^I\sum _{l=1}^L\sum _{t=1}^T{\mathbb{E}}_q\left[\mathbf{1}\left({\mathbf{z}}_{i,t}=l\right)\right](y_{i,j,t}-{\displaystyle \frac{1}{2}}){\mathit{\delta }}_l\right),\hfill \end{array}\right.\end{array}$$

(10)

where ${\xi }_{j,l}$ is the variational parameter associated with item j and latent class l, and  $h({\xi }_{j,l})={\displaystyle \frac{\tanh ({\xi }_{j,l}/2)}{4{\xi }_{j,l}}}$. We use $\Xi$ to denote the collection of all variational parameters ${\left\{{\xi }_{j,l}\right\}}_{j\in \left[J\right],l\in \left[L\right]}$.

The variational posterior distribution of each row vector of the transition matrix ${\mathit{\tau }}_l$ is assumed to follow a Dirichlet distribution, i.e.,  $q({\mathit{\tau }}_l)\sim \mathrm{Dir}({\mathit{\tau }}_l\mathit{\mid }{\mathit{\omega }}_l^{(★)})$, where ${\mathit{\omega }}_l^{(★)}\in {\mathbb{R}}^L$ is the variational concentration parameter vector for the $l\mathrm{th}$ row of the transition matrix. The iterative updates can be updated as follows:

$$\begin{array}{cc}\hfill {\mathit{\omega }}_{l{l}^{\prime }}^{(★)}& ={\mathit{\omega }}_{l{l}^{\prime }}^{(0)}+\sum _{i=1}^I\sum _{t=2}^T{\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t-1}=l,{\mathbf{z}}_{i,t}=l^{\prime })\right].\hfill \end{array}$$

(11)

Similarly, the variational posterior distribution of the initial latent class distribution $\mathit{\pi }$ is assumed to follow a Dirichlet distribution, i.e., $q(\mathit{\pi })\sim \mathrm{Dir}(\mathit{\pi }\mathit{\mid }{\alpha }^{(★)})$, where ${\alpha }^{(★)}\in {\mathbb{R}}^L$ is the variational concentration parameter vector for the initial latent class distribution. The iterative updates can be updated as follows:

$$\begin{array}{cc}\hfill {\alpha }_l^{(★)}& ={\alpha }_l^{(0)}+\sum _{i=1}^I{\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,1}=l)\right].\hfill \end{array}$$

(12)

To update these variational posteriors, expectations of latent attribute mastery pattern, ${\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t}=l)\right]$ and ${\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t-1}=l,{\mathbf{z}}_{i,t}=l^{\prime })\right]$, are required. For all $i\in \left[I\right]$ and $l,l^{\prime }\in \left[L\right]$, the iterative updates of these expectations can be computed using the following equations:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc} \hfill & {\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t}=l)\right]={\displaystyle \frac{f_{i,t}(l)b_{i,t}(l)}{{\sum }_{l^{\prime }}{f}_{i,t}(l^{\prime })b_{i,t}(l^{\prime })}}, \forall t\in \left[T\right]\hfill \\ \hfill & {\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t-1}=l,{\mathbf{z}}_{i,t}=l^{\prime })\right]={\displaystyle \frac{f_{i,t-1}(l)b_{i,t}(l^{\prime }){\eta }_{l{l}^{\prime }}{\varphi }_{i,t}(l^{\prime })}{{\sum }_{\ell ,{\ell }^{\prime }}{f}_{i,t-1}(\ell )b_{i,t}({\ell }^{\prime }){\eta }_{\ell {\ell }^{\prime }}{\varphi }_{i,t}({\ell }^{\prime })}}, \forall t\ge 2.\hfill \end{array}\right.\end{array}$$

(13)

In the above equations, $f_{i,t}(l)$ and $b_{i,t}(l)$ are the forward and backward probabilities, respectively, which can be computed recursively using the forward–backward algorithm, also known as the Baum–Welch algorithm in the context of HMMs [44,45,46]. For all $i\in \left[I\right]$, $t\in \left[T\right]$, and $l\in \left[L\right]$, $f_{i,t}(l)$ and $b_{i,t}(l)$ are defined as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}{f}_{i,1}(l)\hfill & ={\kappa }_l{\varphi }_{i,1,l},\hfill \\ f_{i,t}(l)\hfill & ={\varphi }_{i,t,l}\sum _{l^{\prime }}{f}_{i,t-1}(l^{\prime }){\eta }_{l^{\prime }l} \forall t\ge 2,\hfill \\ b_{i,T}(l)\hfill & =1,\hfill \\ b_{i,t}(l)\hfill & =\sum _{l^{\prime }}{b}_{i,t+1}(l^{\prime }){\eta }_{l{l}^{\prime }}{\varphi }_{i,t+1,l^{\prime }} \forall t\le T-1,\hfill \end{array}\right.\end{array}$$

(14)

where ${\eta }_{l{l}^{\prime }}=\exp \left({\mathbb{E}}_q\left[\log {\mathit{\tau }}_{l{l}^{\prime }}\right]\right)$, ${\kappa }_l=\exp \left({\mathbb{E}}_q\left[\log {\mathit{\pi }}_l\right]\right)$, and 

$$\begin{array}{c}\hfill {\varphi }_{i,t,l}=\exp \left(\sum _{j=1}^J{\mathbb{E}}_q\left[\log \mathsf{\Psi }({\xi }_{j,l})+(y_{i,j,t}-{\displaystyle \frac{1}{2}}){\mathit{\beta }}_j^{\prime }{\mathit{\delta }}_l-{\displaystyle \frac{{\xi }_{j,l}}{2}}-h({\xi }_{j,l})\left({\left({\mathit{\beta }}_j^{\prime }{\mathit{\delta }}_l\right)}^2-{\xi }_{j,l}^2\right)\right]\right).\end{array}$$

During each iteration of the variational inference algorithm, we update the variational parameters $\Xi ={\left\{{\xi }_{j,l}\right\}}_{j\in \left[J\right],l\in \left[L\right]}$ using

$$\begin{array}{c}\hfill {\xi }_{j,l}={{\mathbb{E}}_q\left[{({\mathit{\beta }}_j^{\prime }{\mathit{\delta }}_l)}^2\right]}^{{\displaystyle \frac{1}{2}}}={\left({\mathit{\delta }}_l^{\prime }{\mathbb{E}}_q\left[{\mathit{\beta }}_j{\mathit{\beta }}_j^{\prime }\right]{\mathit{\delta }}_l\right)}^{{\displaystyle \frac{1}{2}}}.\end{array}$$

(15)

We summarize our variational inference algorithm for the HM-LACDM as follows (Algorithm 1):

Algorithm 1 Variational Inference for HM-LACDM

1: Input: Response data $Y\in {\{0,1\}}^{I\times J\times T}$, prior parameters ${\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)}$, ${\Sigma }_{{\mathit{\beta }}_j}^{(0)}$, ${\mathit{\omega }}_l^{(0)}$, ${\alpha }^{(0)}$ for all $j\in \left[J\right]$, $l\in \left[L\right]$. 2: Initialize: Set ${\mathit{\mu }}_{{\mathit{\beta }}_j}^{(★)}={\mathit{\mu }}_{{\mathit{\beta }}_j}^{(0)}$, ${\Sigma }_{{\mathit{\beta }}_j}^{(★)}={\Sigma }_{{\mathit{\beta }}_j}^{(0)}$, ${\mathit{\omega }}_l^{(★)}={\mathit{\omega }}_l^{(0)}$, ${\alpha }^{(★)}={\alpha }^{(0)}$ for all $j\in \left[J\right]$, $l\in \left[L\right]$. 3: Repeat until convergence: 1. Update ${\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t}=l)\right]$ and ${\mathbb{E}}_q\left[\mathbf{1}({\mathbf{z}}_{i,t-1}=l,{\mathbf{z}}_{i,t}=l^{\prime })\right]$ (Equation (13)). 2. Update ${\mathit{\mu }}_{{\mathit{\beta }}_j}^{(★)}$, ${\Sigma }_{{\mathit{\beta }}_j}^{(★)}$, ${\mathit{\omega }}_l^{(★)}$, and ${\alpha }^{(★)}$ (Equation (10)–(12)). 3. Update the variational parameters ${\xi }_{j,l}$ (Equation (15)). 4: Output: Variational posterior distributions $q(\mathbf{Z})$, $q(\mathit{\beta })$, $q(\mathbf{T})$, and $q(\mathit{\pi })$.

We include the details of the derivation of the variational updates in Appendix B.2. We also provide the explicit form of the ELBO in Appendix B.3.

4. Post Hoc Q-Matrix Recovery

Although our HM-LACDM does not require a pre-specified Q-matrix, recovering the Q-matrix is still valuable for interpretability and diagnostic purposes. We propose a post hoc Q-matrix recovery approach that leverages the estimated variational posterior distributions of the item-effect parameters. For item j, its item-effect parameter ${\mathit{\beta }}_j$ can be expressed as ${\mathit{\beta }}_j=[{\mathit{\beta }}_{j0},{\mathit{\beta }}_{j1},\dots ,{\mathit{\beta }}_{jK}]$, where ${\mathit{\beta }}_{j0}$ is the intercept term and ${\mathit{\beta }}_{jk}$ is the effect of attribute k on item j. If attribute k is not required for item j, then its corresponding effect parameter ${\mathit{\beta }}_{jk}$ should be zero. Based on this fact, we can perform hypothesis testing to determine whether each attribute k is required for item j based on the estimated posterior distribution of ${\mathit{\beta }}_{jk}$. We formulate the hypothesis testing problem as follows:

$$\begin{array}{c}\hfill {\mathrm{H}}_0:{\mathit{\beta }}_{jk}=0 \mathrm{vs}. {\mathrm{H}}_a:{\mathit{\beta }}_{jk}>0.\end{array}$$

We then estimate the Q-matrix entry ${\widehat{q}}_{jk}$ using a decision rule:

$${\widehat{q}}_{jk}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\hspace{1em}{z}_{jk}>z_{1-\alpha }\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

where $z_{jk}={\displaystyle \frac{{\mathit{\mu }}_{jk}^{★}}{{\sigma }_{jk}^{★}}}$ is the test statistic for the hypothesis testing of ${\mathit{\beta }}_{jk}$, ${\mathit{\mu }}_{jk}^{★}$ and ${\sigma }_{jk}^{★}$ are the posterior mean and standard deviation of ${\mathit{\beta }}_{jk}$, respectively, and $z_{1-\alpha }$ is the critical value from the standard normal distribution corresponding to the significance level $\alpha$.

However, due to the fact that we are testing multiple hypotheses simultaneously (one for each combination of item j and attribute k), we need to control the false discovery rate (FDR). We employ the Benjamini–Hochberg (BH) procedure [47] to control the FDR at level $\alpha$. The BH procedure begins by computing the test statistic $z_{jk}={\displaystyle \frac{{\mathit{\mu }}_{jk}^{★}}{{\sigma }_{jk}^{★}}}$ for all $(j,k)$ pairs where $j\in \left[J\right]$ and $k\in \left[K\right]$, followed by calculating the corresponding p-values $p_{jk}=1-\Phi (z_{jk})$, where $\Phi (·)$ is the cumulative distribution function of the standard normal distribution. The p-values are then sorted in ascending order as $p_{(1)}\le p_{(2)}\le \dots \le p_{(JK)}$. We find the largest index ℓ such that $p_{(\ell )}\le {\displaystyle \frac{\ell \alpha }{JK}}$ and reject all hypotheses corresponding to $p_{(1)},\dots ,p_{(\ell )}$, setting the corresponding Q-matrix entries to 1.

Although our approach is post hoc, it is designed to recover the Q-matrix if the ground truth Q-matrix satisfies certain identifiability conditions. A Q-matrix is said to be complete if it allows for the identification of all the possible attribute profiles of the $L=2^K$ different proficiency classes. Köhn and Chiu [48,49] have proved that the necessary condition for a $J\times K$ Q-matrix of a main-effects-only model to be complete is that the matrix must have full column rank ($\mathrm{rank}(\mathrm{Q})=K$), i.e., the columns of the Q-matrix must be linearly independent. Therefore, if the ground truth Q-matrix is complete, our post hoc recovery procedure should recover the Q-matrix with high probability as long as the sample size is sufficiently large. Our approach provides a principled method for Q-matrix recovery that accounts for multiple testing while maintaining strong statistical properties. The recovered Q-matrix can then be used for diagnostic interpretation and validation of the cognitive structure underlying the assessment.

5. Simulation Study

5.1. Single Run

In this section, we present simulation results from a single run of the HM-LACDM with 100 CAVI iterations. For this simulation, we generated synthetic datasets with $I=500$ individuals across $T=2$ time points, using a ground-truth Q-matrix with $K=3$ attributes and $J=21$ items (see Appendix C). The data generation parameters were configured as follows: we set the intercept parameter ${\mathit{\beta }}_{j0}=-3$ for all items $j\in \left[J\right]$. For items requiring n attributes, we set all main effect parameters to ${\mathit{\beta }}_{jk}={\displaystyle \frac{6}{n}}$ for all attributes k required by item j, ensuring that the total effect remains constant across items with different complexity levels. The initial state distribution of the latent attribute mastery profiles was generated as $\mathit{\pi }=\mathrm{softmax}(\mathbf{x})$, where $\mathbf{x}$ was sampled from an L-dimensional isotropic Gaussian distribution with $L=2^K=8$ possible latent classes, and $\mathrm{softmax}{(\mathbf{x})}_i={\displaystyle \frac{\exp (x_i)}{{\sum }_{j=1}^L\exp (x_j)}}$ for $i\in \left[L\right]$. For the transition dynamics, we modeled attribute-specific transition probabilities, where each attribute could independently transition between mastery states. Specifically, for each attribute $k\in \left[K\right]$, we set $p(z_{i,t,k}=1|z_{i,t-1,k}=1)=0.8$ (mastery retention probability), $p(z_{i,t,k}=0|z_{i,t-1,k}=1)=0.2$ (forgetting probability), $p(z_{i,t,k}=1|z_{i,t-1,k}=0)=0.3$ (learning probability), and $p(z_{i,t,k}=0|z_{i,t-1,k}=0)=0.7$ (non-mastery retention probability) for all time points $t\in \left[T\right]$ and individuals $i\in \left[I\right]$. The transition matrix $\mathbf{T}={\left[{\mathit{\tau }}_{l,l^{\prime }}\right]}_{l,l^{\prime }\in \left[L\right]}$ was constructed by assuming independence across attributes:

$${\mathit{\tau }}_{l,l^{\prime }}=\prod _{k=1}^{K}p(z_k=l_k^{\prime }|z_k=l_k),$$

where $l_k$ and $l_k^{\prime }$ are the $k\mathrm{th}$ bits of the binary representations of latent classes l and $l^{\prime }$.

We display trace plots of the evidence lower bound (ELBO) and model parameters to demonstrate convergence behavior during the iterative optimization process. Additionally, we provide parameter recovery plots that compare the estimated parameters against the true values used in data generation, illustrating the accuracy of our proposed method.

Figure 1a shows the trace of the ELBO during the CAVI iterations. We only show the first 30 iterations since the ELBO converges quickly and stabilizes after that. We can see that the ELBO increases monotonically, corresponding to the optimization process of CAVI.

Figure 1b displays the trace plots of item-effect parameters ${\mathit{\beta }}_j$ for all items $j\in \left[J\right]$. Different colors represent different types of parameters, including intercepts (red) and main effects of different magnitudes (orange, purple, and green), allowing us to visualize the convergence behavior of each parameter type. The blue lines represent the parameters that are not required by the items, which should converge to zero. We can see that all parameters converge after around 10 iterations, indicating that the CAVI algorithm has efficiently optimized the model parameters.

Figure 2 displays the parameter recovery results for the item-effect parameters ${\mathit{\beta }}_j$, the initial latent class distribution $\mathit{\pi }$, and the transition matrix $\mathbf{T}$. The x-axis represents the true values used in data generation, while the y-axis shows the estimated values obtained through CAVI optimization. The dashed diagonal line $y=x$ represents perfect recovery, where estimated values exactly match true values. As evident from the plots, the estimated item-effect parameters ${\mathit{\beta }}_j$, $\mathit{\pi }$, and transition matrix $\mathbf{T}$ closely align with the diagonal line, demonstrating that our HM-LACDM effectively recovers the true parameters. At the first time point ($t=1$), the profile prediction accuracy was 99.2%, while at the second time point ($t=2$) it was 99.6%. When aggregated across all the time points, the overall profile prediction accuracy was 98.8%.

We also conducted post hoc Q-matrix recovery using the estimated variational posterior distributions of the item-effect parameters. With a significance level of $\alpha =0.05$ for hypothesis testing, we achieved 100% accuracy in Q-matrix recovery, successfully identifying all entries in the ground-truth Q-matrix. It is important to note that our model faces an inherent identifiability issue. We can only identify the Q-matrix up to a permutation of the attributes, which is a common limitation in latent variable models with discrete latent classes. We would also like to highlight the computational efficiency of our approach. Leveraging the torch back-end in R, our algorithm completed all 100 CAVI iterations in just $4.51$ s on an Apple M1 Max MacBook Pro with 32 GB of RAM. This demonstrates the practical feasibility of our method for large-scale applications. While we set the maximum iterations to 100, the trace plots show that convergence typically occurs within 30 iterations, suggesting potential for even greater computational efficiency in practice.

5.2. Multiple Runs

In this section, we present results from multiple runs of the HM-LACDM to evaluate its robustness and performance across varying sample sizes and attribute settings. For each configuration, we generated $n=100$ independent datasets and applied our algorithm to each dataset, reporting the average performance metrics with their standard errors across all runs. We varied the sample size I (200 and 1000), the number of attributes K (3 and 4), and the number of time points T (2, 3, and 5). We additionally varied the item parameters to evaluate the performance of our algorithm under different levels of signal strength. Specifically, we considered three conditions: strong, moderate, and weak, corresponding to $p=0.95,0.90,0.80$, respectively, where p denotes the probability that a respondent answers an item correctly given mastery of all required attributes. Our definitions of strong–weak signals align with the “high-” and “low-quality items” used by Yamaguchi and Martinez [41], who similarly manipulated item response probabilities to reflect varying item quality while preserving comparable cognitive diagnostic characteristics. The parameters $\mathit{\pi }$ and $\mathbf{T}$ are set to the same values as in the single-run simulation.

Table 1. Root mean square error (RMSE) of estimated parameters under a strong signal level across varying sample sizes and attribute settings.

I	K	T	$\mathit{\beta }$ RMSE	$\mathit{\tau }$ RMSE	$\mathit{\pi }$ RMSE
200	3	2	0.66 (4.49 × 10−1)	0.071 (4.5 × 10−3)	0.018 (2.3 × 10−2)
200	3	3	0.45 (2.49 × 10−1)	0.049 (4.4 × 10−3)	0.013 (1.0 × 10−2)
200	3	5	0.31 (2.7 × 10−2)	0.034 (7.9 × 10−4)	0.011 (1.1 × 10−3)
200	4	2	0.42 (7.9 × 10−2)	0.057 (9.0 × 10−4)	0.016 (1.1 × 10−3)
200	4	3	0.34 (8.4 × 10−2)	0.047 (9.1 × 10−4)	0.016 (1.6 × 10−3)
200	4	5	0.27 (1.06 × 10−1)	0.030 (1.2 × 10−3)	0.016 (1.8 × 10−3)
1000	3	2	0.21 (2.2 × 10−2)	0.026 (7.6 × 10−4)	0.012 (3.8 × 10−4)
1000	3	3	0.20 (3.20 × 10−1)	0.017 (4.8 × 10−3)	0.013 (1.1 × 10−2)
1000	3	5	0.16 (3.09 × 10−1)	0.014 (6.7 × 10−3)	0.013 (7.8 × 10−3)
1000	4	2	0.18 (1.25 × 10−1)	0.032 (1.9 × 10−3)	0.0074 (1.7 × 10−3)
1000	4	3	0.16 (1.86 × 10−1)	0.021 (2.3 × 10−3)	0.0074 (1.4 × 10−3)
1000	4	5	0.15 (2.38 × 10−1)	0.015 (4.5 × 10−3)	0.0077 (3.3 × 10−3)

summarizes the root mean square error (RMSE) of the estimated parameters across different configurations. The RMSE values are reported as the mean (standard error) over $n=100$ independent runs. We observe that as the sample size increases from 200 to 1000, the RMSE of the estimated parameters decreases significantly, indicating consistency of our CAVI algorithm.

Figure 3 shows the distributions of RMSE for item-effect parameters ${\mathit{\beta }}_j$, initial latent class distribution $\mathit{\pi }$, and transition matrix $\mathbf{T}$ under the configuration $I=200$, $K=3$, and $T=5$. We can see that the distributions of RMSE are unimodal and centered around the mean RMSE values reported in Table 1.

Table 2. Q-matrix recovery performance under a strong signal level across varying sample sizes and attribute settings under different significance levels ($\alpha =0.05$/$\alpha =0.01$). FPR: false positive rate; FNR: false negative rate.

I	K	T	FPR	FNR	Accuracy
200	3	2	0.98%/0.73%	1.13%/1.21%	97.89%/98.06%
200	3	3	1.27%/0.65%	0.16%/0.19%	98.57%/99.16%
200	3	5	1.35%/0.41%	0%/0%	98.65%/99.59%
200	4	2	1.11%/0.42%	0.07%/0.11%	98.82%/99.48%
200	4	3	1.35%/0.46%	0.07%/0.08%	98.58%/99.46%
200	4	5	1.70%/0.62%	0.03%/0.04%	98.27%/99.34%
1000	3	2	1.97%/0.79%	0%/0%	98.03%/99.21%
1000	3	3	2.41%/1.29%	0.27%/0.29%	97.32%/98.43%
1000	3	5	3.11%/1.68%	0.11%/0.16%	96.78%/98.16%
1000	4	2	3.33%/1.48%	0.03%/0.03%	96.64%/98.49%
1000	4	3	3.25%/1.52%	0.06%/0.07%	96.69%/98.40%
1000	4	5	3.80%/2.05%	0.07%/0.09%	96.12%/97.86%

summarizes the Q-matrix recovery performance across different configurations. We report the false positive rate (FPR), false negative rate (FNR), and accuracy of the recovered Q-matrix entries. For each run, we set the significance level $\alpha$ to $0.05$ and $0.01$, and report the results for both levels. We observe that the FPR remains low across all configurations, indicating that our method effectively controls the false discovery rate. The FNR is 0 for all configurations, demonstrating that our method successfully identifies all required attributes for each item. The overall accuracy of the recovered Q-matrix is high, ranging from $96.12\%$ to $99.59\%$, indicating that our method is reliable for Q-matrix recovery.

The tables above present the results for the strong signal condition. Results for the moderate and weak signal conditions are provided in Appendix D.2; see

Table A3. Root mean square error (RMSE) of estimated parameters under a weak signal level across varying sample sizes and attribute settings.

I	K	T	$\mathit{\beta }$ RMSE	$\mathit{\tau }$ RMSE	$\mathit{\pi }$ RMSE
200	3	2	0.36 (2.26 × 10−1)	0.078 (6.4 × 10−3)	0.035 (1.8 × 10−2)
200	3	3	0.26 (7.4 × 10−2)	0.056 (6.0 × 10−3)	0.029 (6.1 × 10−3)
200	3	5	0.20 (2.5 × 10−2)	0.039 (3.8 × 10−3)	0.025 (5.1 × 10−3)
200	4	2	0.31 (1.12 × 10−1)	0.064 (1.9 × 10−3)	0.023 (8.3 × 10−3)
200	4	3	0.24 (6.8 × 10−2)	0.054 (2.7 × 10−3)	0.020 (5.9 × 10−3)
200	4	5	0.20 (1.19 × 10−1)	0.038 (2.1 × 10−3)	0.019 (7.5 × 10−3)
1000	3	2	0.16 (1.61 × 10−1)	0.037 (4.5 × 10−3)	0.023 (1.3 × 10−2)
1000	3	3	0.11 (1.5 × 10−2)	0.026 (2.3 × 10−3)	0.020 (2.6 × 10−3)
1000	3	5	0.089 (1.3 × 10−2)	0.019 (1.8 × 10−3)	0.017 (2.3 × 10−3)
1000	4	2	0.13 (1.1 × 10−2)	0.042 (2.8 × 10−3)	0.010 (1.6 × 10−3)
1000	4	3	0.12 (1.18 × 10−1)	0.029 (1.9 × 10−3)	0.011 (6.9 × 10−3)
1000	4	5	0.087 (7.0 × 10−2)	0.021 (1.2 × 10−3)	0.0093 (5.0 × 10−3)

,

Table A4. Root mean square error (RMSE) of estimated parameters under a moderate signal level across varying sample sizes and attribute settings.

I	K	T	$\mathit{\beta }$ RMSE	$\mathit{\tau }$ RMSE	$\mathit{\pi }$ RMSE
200	3	2	0.38 (1.97 × 10−1)	0.073 (4.6 × 10−3)	0.021 (1.4 × 10−2)
200	3	3	0.30 (1.89 × 10−1)	0.051 (6.3 × 10−3)	0.018 (8.6 × 10−3)
200	3	5	0.21 (2.3 × 10−2)	0.035 (2.2 × 10−3)	0.015 (2.9 × 10−3)
200	4	2	0.30 (5.4 × 10−2)	0.060 (1.6 × 10−3)	0.018 (3.2 × 10−3)
200	4	3	0.25 (8.7 × 10−2)	0.050 (1.6 × 10−3)	0.018 (5.1 × 10−3)
200	4	5	0.19 (7.9 × 10−2)	0.033 (1.9 × 10−3)	0.017 (4.0 × 10−3)
1000	3	2	0.14 (1.9 × 10−2)	0.028 (2.3 × 10−3)	0.014 (1.4 × 10−3)
1000	3	3	0.13 (1.40 × 10−1)	0.020 (4.8 × 10−3)	0.014 (6.4 × 10−3)
1000	3	5	0.087 (1.4 × 10−2)	0.015 (1.0 × 10−3)	0.013 (1.2 × 10−3)
1000	4	2	0.13 (1.1 × 10−2)	0.035 (1.6 × 10−3)	0.0076 (9.6 × 10−4)
1000	4	3	0.10 (8.9 × 10−3)	0.024 (1.1 × 10−3)	0.0076 (9.8 × 10−4)
1000	4	5	0.080 (6.1 × 10−3)	0.016 (7.9 × 10−4)	0.0075 (9.1 × 10−4)

,

Table A5. Q-matrix recovery performance under a weak signal level across varying sample sizes and attribute settings under different significance levels ($\alpha =0.05$/$\alpha =0.01$). FPR: false positive rate; FNR: false negative rate.

I	K	T	FPR	FNR	Accuracy
200	3	2	1.00%/0.22%	1.37%/2.21%	97.63%/97.57%
200	3	3	1.41%/0.62%	0.14%/0.19%	98.44%/99.19%
200	3	5	1.76%/0.54%	0.02%/0.02%	98.22%/99.44%
200	4	2	2.30%/0.84%	2.02%/4.10%	95.67%/95.06%
200	4	3	2.77%/1.12%	0.39%/0.92%	96.83%/97.96%
200	4	5	2.78%/0.92%	0.40%/0.42%	96.82%/98.67%
1000	3	2	3.06%/1.32%	0.38%/0.38%	96.56%/98.30%
1000	3	3	2.73%/1.22%	0%/0%	97.27%/98.78%
1000	3	5	1.76%/0.59%	0%/0%	98.24%/99.41%
1000	4	2	4.83%/2.15%	0%/0%	95.17%/97.85%
1000	4	3	4.87%/2.22%	0.40%/0.40%	94.73%/97.38%
1000	4	5	5.20%/2.23%	0.13%/0.13%	94.67%/97.64%

,

Table A6. Q-matrix recovery performance under a moderate signal level across varying sample sizes and attribute settings under different significance levels ($\alpha =0.05$/$\alpha =0.01$). FPR: false positive rate; FNR: false negative rate.

I	K	T	FPR	FNR	Accuracy
200	3	2	0.81%/0.37%	0.51%/0.57%	98.68%/99.06%
200	3	3	1.56%/1.05%	0.29%/0.33%	98.16%/98.62%
200	3	5	1.08%/0.32%	0%/0%	98.92%/99.68%
200	4	2	1.63%/0.61%	0.20%/0.61%	98.17%/98.78%
200	4	3	1.62%/0.41%	0.14%/0.15%	98.24%/99.44%
200	4	5	2.19%/0.79%	0.06%/0.07%	97.75%/99.14%
1000	3	2	2.02%/0.83%	0%/0%	97.98%/99.17%
1000	3	3	2.51%/0.98%	0.06%/0.08%	97.43%/98.94%
1000	3	5	2.25%/0.73%	0%/0%	97.75%/99.27%
1000	4	2	3.33%/1.23%	0%/0%	96.67%/98.77%
1000	4	3	3.39%/1.47%	0%/0%	96.61%/98.53%
1000	4	5	3.69%/1.38%	0%/0%	96.31%/98.62%

. Notably, the findings under the weak signal condition are qualitatively similar, further demonstrating the robustness and consistency of our method.

Finally, we highlight the computational efficiency of our approach. For a large configuration with $I=1000$, $K=4$, and $T=5$, the average runtime per run was $5.30$ s. For a small configuration with $I=200$, $K=3$, and $T=2$, the average runtime per run was $3.08$ s. These results demonstrate the scalability of our variational inference approach even with increasing model complexity and sample size. For a large configuration with $I=1000$, $K=4$, and $T=5$, the average runtime per replication was $5.30$ s. In contrast, for a smaller configuration with $I=200$, $K=3$, and $T=2$, the average runtime per replication was $3.08$ s. These results underscore the scalability of our variational inference approach, demonstrating its ability to handle increasingly complex models and larger sample sizes with only modest additional computational cost. All simulations were conducted on an Apple M1 Max MacBook Pro with 32 GB of RAM.

6. Discussion

In this paper, we introduced a novel hidden Markov log-linear additive Cognitive Diagnostic Model (HM-LACDM) that circumvents the necessity of specifying a predefined Q-matrix, thus addressing a critical limitation in traditional cognitive diagnostic models (CDMs). By integrating a variational Bayesian inference framework, our method significantly reduces computational complexity, making it particularly suitable for large-scale longitudinal cognitive assessments involving numerous attributes and multiple time points.

The simulation studies validate the robustness and efficiency of our proposed approach. Our variational inference algorithm consistently demonstrates convergence, parameter recovery accuracy, and computational efficiency across varying configurations of sample size, attribute complexity, and temporal length. Notably, the post hoc Q-matrix recovery method exhibited remarkable accuracy, effectively controlling the false discovery rate and reliably identifying the cognitive attributes underlying each test item.

Our approach’s scalability and computational efficiency make it highly suitable for practical applications in educational assessments, where swift and accurate diagnostics are crucial. It provides educators and psychometricians with an effective tool for longitudinally tracking student learning progression without the intensive and subjective Q-matrix specification process.

In summary, the proposed HM-LACDM and the accompanying variational Bayesian inference method represent substantial advances in the field of cognitive diagnostic modeling. They not only facilitate more robust and scalable inference but also significantly enhance interpretability through effective Q-matrix recovery, thereby opening new avenues for targeted instructional intervention and deeper insights into cognitive skill development.

Despite the advantages of our proposed framework, there are important limitations to acknowledge. First, the recovered Q-matrix is identifiable only up to permutation of the attributes, a common issue in latent-class models without additional constraints. In our simulation studies, we address this by aligning the estimated attributes with the ground truth through enumeration of all possible label permutations. This ensures that the reported recovery accuracy and simulation results are not confounded by label switching. For empirical data, when the Q-matrix is not fully available, partial domain knowledge can be used to anchor certain attributes, and ordering constraints can be imposed to break label symmetry. These strategies mitigate the relabeling issue and represent promising directions for extending our framework to empirical applications. Second, unlike a saturated CDM, our specification includes only intercept and main-effect parameters, excluding higher-order interaction terms. We do not impose explicit constraints on these parameters during estimation; instead, we assume that the monotonicity condition holds for the true model, ensuring that the data themselves satisfy this property. In practice, we guide the algorithm toward monotone solutions by initializing intercepts with negative values and main effects with positive values, and by running the algorithm multiple times with different random initializations to select the best-fitting solution. Under this setting, the one-sided test with positive effects used in our post hoc Q-matrix recovery is appropriate, as the parameters of substantive interest are non-negative by construction.

Future research could extend the framework in several directions: incorporating interaction terms to approximate saturated CDMs, relaxing the monotonicity assumption, or integrating informative priors to further improve identifiability. Importantly, while our simulation studies demonstrate robustness and scalability, applying the method to large-scale empirical assessment data remains a critical next step. Such real-data validation would allow us to evaluate the practical utility of the HM-LACDM in capturing nuanced learning dynamics and further establish its value in operational testing contexts.