Improved Doubly Robust Inference with Nonprobability Survey Samples Using Finite Mixture Models: Application to Health Monitoring SMS Survey Data

Abstract

Nonprobability sampling has been increasingly used in epidemiologic research, yet direct inference based on such samples is subject to selection bias. Current adjustment methods commonly rely on a reference probability-based survey sample that shares a set of covariates with the nonprobability sample. However, these common covariates are often limited and may bias estimates in the presence of population heterogeneity. Existing methods generally assume population homogeneity in models and fail to address such heterogeneity adequately. To overcome this limitation, we propose the Nonprobability Heterogeneity-adjusted Doubly Robust (NHDR) method, a novel inference framework that explicitly accounts for population heterogeneity during selection bias adjustment. NHDR proceeds in three stages: (1) identifying latent subpopulations via finite mixture modeling; (2) incorporating the resulting latent-class structure as a grouping variable into mixed-effects models for both the propensity score and outcome projection; and (3) constructing a doubly robust estimator that integrates these adjusted models. The key methodological contribution of NHDR is its formal integration of latent-class-based population structure into a doubly robust estimation framework, which enables more reliable inference under heterogeneous population settings. Simulation studies demonstrate that the proposed method control the coverage probabilities well in most scenarios. Under heterogeneous conditions, NHDR consistently outperforms existing methods achieving an average reduction in relative bias of approximately 1.8–4.5% and a corresponding decrease in mean squared error of about 5.1–15.5 compared to the benchmark method. We illustrate the practical utility of NHDR by applying it to estimate nine health indicators using data from the Health Monitoring SMS Survey in Guangzhou, China, with the seventh Guangdong Health Service Survey serving as the reference sample.

1. Introduction

Probability sampling has long been considered the gold standard for obtaining representative samples due to its well-established theoretical basis [1]. However, the implementation of probability sampling methods is increasingly challenged by declining response rates and rising costs [2,3,4]. In the last decade, nonprobability sampling, especially online surveys, has gained momentum as a cost-effective alternative. Reports indicate that online surveys comprise 73% of survey research, with 90% employing nonprobability sampling methods [5]. However, direct inference from nonprobability samples is vulnerable to selection and coverage bias. This is because participants are not randomly selected, and the underlying selection mechanism is generally unknown [6,7,8,9]. Furthermore, online nonprobability surveys may face self-selection biases, non-response and attrition, which heighten the risk of selection bias [1,10]. Therefore, adjusting for selection bias is essential when making inference from nonprobability samples [11,12,13]. A common solution is to integrate a probability sample (termed a reference sample) that shares a set of common covariates. The design weights from this reference sample are then used to anchor population-level inference.

Current methods for adjusting selection bias in inference with nonprobability samples broadly fall into three categories: (i) global adjustment, including methods such as inverse probability weighting (IPW) [14], propensity score matching [15], and kernel smoothing methods [16,17]; (ii) outcome-specific adjustment, including methods such as the superpopulation (SP) model [1,18], model-based calibration [1,18], and multilevel poststratification regression [19]; (iii) the combined approach, including the doubly robust (DR) method [20] and the multiply robust method [21]. These methods typically rely on a set of common covariates $\mathit{Z}$, available in both the nonprobability sample and the reference sample, to model the selection mechanism $R$ and the outcome $Y$. This common set $\mathit{Z}$ is often limited to basic socio-demographics, as the reference survey was designed for different purposes. A critical, yet often unstated, assumption underlying these methods is the homogeneity of the $\mathit{Z}-R$ and $\mathit{Z}-Y$ relationships across the population. However, this assumption is frequently violated by latent population heterogeneity, where the population comprises unobserved subgroups with distinct behavioral or risk profiles. When key drivers of inclusion or the outcome are unmeasured, this latent heterogeneity causes the $\mathit{Z}-R$ and $\mathit{Z}-Y$ relationships to vary substantially across subpopulations, leading to model misspecification. Consequently, inverse probability weighted, outcome-projected, and even doubly robust estimators can yield biased estimates; increasing the size of the nonprobability sample does not, by itself, resolve this problem. This failure mode is common in health behavior studies, where unmeasured attitudes or contextual factors segment the population into groups with distinct $\mathit{Z}-R$ and $\mathit{Z}-Y$ patterns.

In our context, latent population heterogeneity is defined by the existence of unobserved subpopulations (latent class, $LC$) that differ both in distribution of their characteristics $\mathit{Z}$ and in their underlying data-generating mechanisms [22,23]. Formally, for $k\ne l$, not only does the joint distribution $P\left(\mathit{Z}\right|LC=k)$ differ from $P\left(\mathit{Z}\right|LC=l)$, but the functional relationships also differ: $f\left(Y\right|\mathit{Z},LC=k)\ne f(Y|\mathit{Z},LC=l)$ and $P\left(R\right|\mathit{Z},LC=k)\ne P(R|\mathit{Z},LC=l)$. For instance, Herschbach et al. reported that distributions of smoking, exercise, and other factors differed between sexes, and their effects on stroke risk also varied by sex [24]. Delineating such heterogeneity often requires multiple indicators, with the number of latent groups being an empirical question. For example, to explore health status variations among COVID-19 survivors in Vietnam, Lan et al. utilized six indicators and identified two subpopulations [25], whereas Lv et al. used seven indicators and identified three subpopulations among residents aged 18–64 in Hangzhou, China [26]. Therefore, accurately identifying and accounting for this heterogeneity is crucial for constructing valid propensity score and outcome projection models.

Existing methods for addressing potential heterogeneity can be categorized into four types: (i) latent class models; (ii) the two-step approach; (iii) random parameter models; and (iv) latent classes models with random parameters within classes [27,28]. Among these, the two-step approach is the most widely used due to its computational simplicity and modelling flexibility, while the others may face challenges such as unstable parameter estimates, incorrect specification of parameter distributions, and excessive computational burden [27,28]. In the two-step approach, analysts first use unsupervised clustering to identify distinct subpopulations and then fit the analysis models within, or conditional on, the identified groups. Common clustering techniques include finite mixture modeling (FMM), latent class analysis, latent profile analysis, K-means, and Ward clustering. We adopt FMM for its distinctive advantages in identifying latent subpopulations in survey settings. Unlike methods such as K-means, which require pre-specified cluster numbers and are limited to continuous data, FMM provides a principled statistical framework that objectively determines the number of latent classes using information criteria. Crucially, FMM inherently accommodates mixed-type variables (continuous, binomial, and multinomial) [6,29]—exactly the form of typical survey covariates such as age, sex, and education level. This capability allows us to utilize the full set of available covariates without transformation or approximate distance measures, yielding a more faithful representation of the underlying heterogeneous population structure.

We propose a Nonprobability Heterogeneity-adjusted Doubly Robust (NHDR) inference method for nonprobability samples that explicitly accounts for population heterogeneity when adjusting for selection bias, thereby improving the reliability and accuracy of resulting inferences. The proposed method employs a two-step strategy with three stages. In stage 1, we use the FMM to identify heterogeneous subpopulations (referred as heterogeneous groups). In stage 2, we construct the propensity score model and the outcome projection model using mixed-effects models, with the heterogeneous groups defined in the first stage serving as a level-2 variable. In stage 3, we construct a doubly robust estimator by combining the results from these two models, following the method described by Chen et al. [20], and stabilize weights via kernel smoothing. The NHDR method makes a key methodological contribution by being the first, to our knowledge, to formally integrate latent classes—identified via FMM—directly into both the propensity score and outcome models within a doubly robust framework. Unlike standard methods that assume population homogeneity, NHDR explicitly models the group-specific $\mathit{Z}-R$ and $\mathit{Z}-Y$ relationships by incorporating the latent class membership as a grouping variable, thereby addressing the fundamental problem of model misspecification under heterogeneity. We then perform extensive simulation studies to examine the coverage probabilities, relative bias, mean squared error, and confidence interval width of the proposed methods. Finally, we apply NHDR method to the Health Monitoring SMS Survey data collected in Guangzhou China, using the data from the seventh Guangdong Health Service Survey as the reference sample.

2. Methods

2.1. Notations and Basic Settings

Let $\mathcal{U}=\{\mathrm{1,2},\dots ,N\}$ represent the finite population of size $N$. Let $S_A$ denote a sample of size $n_A$ obtained from $\mathcal{U}$ through nonprobability sampling, and $S_B$ denote a reference sample of size $n_B$ obtained through probability sampling. The sampling weight for individual $i$ in $S_B$ is denoted as $d_i$. Let $Y_i$ represent the target outcome for individual $i$, and ${\mathit{Z}}_i={\left(z_{1i},z_{2i},\dots ,z_{mi}\right)}^T$ be a vector of $m$ covariates. Note that $Y_i$ is collected only in $S_A$, while ${\mathit{Z}}_i$ is collected in both $S_A$ and $S_B$. In this study, the inference parameters of interest are the means for continuous outcome, and the prevalence for binary outcomes. We define $R_i$ as an indicator variable for inclusion in $S_A$, where a value of 1 indicates that individual $i$ belongs to $S_A$ and a value of 0 indicates that individual $i$ belongs to $S_B$. Key notations in Section 2 are summarized in

Table 1. Summary of key notations in Section 2.

Notation	Description
$\mathcal{U}$	Finite population of size $N$
$S_A$	Nonprobability sample of size $n_A$
$S_B$	Reference probability sample of size $n_B$
$d_i$	Sampling weight for individual $i$ in $S_B$
$Y_i$	Target outcome for individual $i$ in $S_A$
${\mathit{Z}}_i$	Vector of $m$ covariates $z_{1i},z_{2i},\dots ,z_{mi}$
$R_i$	Inclusion indicator for the nonprobability sample ($R_i$= 1 if $i\in S_A$)
$L{C}_M$	Pre-specified upper limit for the number of latent classes
$L{C}^{\ast }$	Optimal number of latent classes selected via FMMs
$L{C}_i$	Latent class membership ($L{C}^{\ast }\in [1,L{C}^{\ast }]$)
$d_i^{\ast }$	Scaled sampling weights for individual $i$ in $S_B$
$\pi ({\mathit{Z}}_i,L{C}_i,d_i;\mathit{\alpha })$	Logistic mixed effects model for the propensity score
$m\left(Y_i\right|{\mathit{Z}}_{\mathit{i}},L{C}_i;\mathit{\beta })$	Generalized linear mixed effects for outcome projection
$q_i$	Linear predictors from $\pi ({\mathit{Z}}_i,L{C}_i,d_i;\mathit{\alpha })$
$K(·)$	Kernel function (standard normal)
$h$	Bandwidth parameter for kernel smoothing
$w_i$	The kernel smoothed weights for individual $i$ in $S_A$

. The propensity score model for estimating inclusion probability of individual $i$ in $\mathcal{U}$ can be expressed as:

$${\pi }_i=P\left(R_i=1|{\mathit{Z}}_i,Y_i\right), i\in \mathcal{U}$$

The following assumptions are made for the above model [20,30,31]:

Assumption 1.

Positivity: All individuals in $\mathcal{U}$ have a positive probability of inclusion in $S_A$, that is, ${\pi }_i>0$ for $i\in \mathcal{U}$.

Assumption 2.

Independence: The probabilities of inclusion in $S_A$ for different individual are independent given the covariates ${\mathit{Z}}_i$, that is, $cov\left(R_i,R_j|{\mathit{Z}}_i,{\mathit{Z}}_j\right)=0$ for $i\ne j$.

Assumption 3.

Non-informativity: The nonprobability sample selection is uncorrelated with the target outcome given the covariates, that is, $P\left(R_i=1|{\mathit{Z}}_i,Y_i\right)=P\left(R_i=1|{\mathit{Z}}_i\right)$ for $i\in \mathcal{U}$.

2.2. Overview of the Proposed NHDR Method

The NHDR framework adjusts for selection bias in nonprobability samples when latent population heterogeneity is present. As illustrated in Figure 1, the procedure consists of three sequential stages:

Stage 1 (Identification of latent subpopulations): A finite mixture model (FMM) is fitted to the pooled covariate data from both samples to uncover the underlying latent class structure that captures population heterogeneity.

Stage 2 (Heterogeneity-adjusted modeling): The estimated latent class memberships are incorporated as a grouping variable into mixed-effects models, enabling the estimation of class-specific propensity scores and the projection of outcomes. This explicitly accounts for differing covariate-outcome and covariate-selection relationships across subpopulations.

Stage 3 (Doubly robust estimation): A doubly robust estimator is formed by combining the kernel-smoothed inverse propensity weights with the model-projected outcomes.

The following subsections detail the technical implementation of each stage.

2.3. Stage 1: Identification of Heterogeneous Populations Using Finite Mixture Modelling

Finite mixture modelling treats the population as a weighted combination of K latent classes, with covariates distributions varying across classes. It classifies individuals into mutually exclusive latent classes by computing the posterior probabilities of class membership [32]. In this study, to identify heterogeneous populations, we fit a finite mixture model using the combined samples from $S_A$ and $S_B$, based on the common covariates ${\mathit{Z}}_i$:

$$f\left({\mathit{Z}}_i\right)=\sum _{k=1}^{L{C}^{\ast }}P\left(L{C}_i=k\right)f_k\left({\mathit{Z}}_i|L{C}_i=k\right)=\sum _{k=1}^{L{C}^{\ast }}P\left(L{C}_i=k\right)\prod _{j=1}^m{f}_k\left(z_{ji}|L{C}_i=k\right)$$

Here $L{C}_i=k$ indicates that individual $i$ belongs to latent class $k$, with $k=1,\dots ,L{C}^{\ast }$, where $L{C}^{\ast }$ is the optimal number of latent classes. We determine $L{C}^{\ast }$ through three steps:

(1)

Set an upper limit for the number of latent classes $L{C}_M$ based on the research question; if prior information is unavailable, $L{C}_M$ can be set to 1–10 [33];

(2)

For each candidate number of classes from 1 to $L{C}_M$, fit the corresponding FMMs.

(3)

The optimal number of latent classes, $L{C}^{\ast }$, is then selected by minimizing the Bayesian Information Criterion (BIC), as its penalty term for model complexity is $\mathrm{l}\mathrm{o}\mathrm{g}\left(n\right)$, which allows for asymptotically consistent selection of the true number of classes as the sample size increases [34]. This has been shown to provide a stable and reliable solution in mixture modeling [35,36].

$P\left(L{C}_i=k\right)$ is the probability that individual $i$ belongs to latent class $k$ and $f_k\left({\mathit{Z}}_i|L{C}_i=k\right)$ is the distribution of ${\mathit{Z}}_i$ given $L{C}_i=k$. Assuming independence among covariates, $f_k\left({\mathit{Z}}_i|L{C}_i=k\right)={\prod }_{j=1}^m{f}_k\left(z_{ji}|L{C}_i=k\right)$. The Expectation-Maximization (EM) algorithm is used to estimate the posterior probabilities of class membership for each individual, which can be obtained directly by the depmixS4 package in the R statistical software [37]. The monotonicity and convergence properties of the EM algorithm are justified in Supplementary Section S2. Each individual is then classified into the latent class with the highest posterior probability.

2.4. Stage 2: Construction of Propensity Score Model and Outcome Projection Model Accounting for Population Heterogeneity

(1)

Propensity score model accounting for population heterogeneity: A logistic mixed effects model is used to estimate inclusion probabilities, treating the heterogeneous groups identified in stage 1 as a level-2 variable. The model is fitted with common covariates ${\mathit{Z}}_i$ using the combined sample from $S_A$ and $S_B$, with $S_B$ weighted by $d_i^{\ast }=a·d_i$, where $a={\displaystyle \frac{n_B}{{\sum }_{i\in S_B}{d}_i}}$. This scaling adjustment is applied to stabilize the estimation process by mitigating the disproportionate influence that can arise from the disparity in effective sample size between the weighted $S_B$ and $S_A$. Specifically, it prevents the composite likelihood from being dominated by a small number of units with extremely large design weights. As a result, this adjustment is crucial for achieving stable and efficient estimation of the model parameters [30]. We denote the model as $\pi ({\mathit{Z}}_i,L{C}_i,d_i;\mathit{\alpha })$, specified as:

Level-1:

$$\mathit{ln}\left({\displaystyle \frac{P\left(R_{ik}=1\right)}{1-P\left(R_{ik}=1\right)}}\right)={\alpha }_{0k}+\sum _{j=1}^m{\alpha }_{jk}{z}_{jik}, k=1,\dots ,L{C}^{\ast }$$

Level-2:

$${\alpha }_{0k}={\alpha }_{00}+{\epsilon }_0$$

$${\alpha }_{jk}={\alpha }_{j0}+{\epsilon }_j, j=1,\dots ,m$$

where $R_{ik}$ is the inclusion indicator for individual $i$ in latent class $k$ identified in stage 1, and $z_{jik}$ is the $j$th covariate for that individual. ${\alpha }_{0k}$ and ${\alpha }_{jk}$ are level-1 intercept and slopes, and ${\alpha }_{00}$ and ${\alpha }_{j0}$ are corresponding parameters for level-2. ${\epsilon }_0$ and ${\epsilon }_j$ are normally distributed residual terms.

(2)

Outcome projection model accounting for population heterogeneity: A generalized linear mixed effects model is constructed between common covariates ${\mathit{Z}}_i$ and $Y_i$ based on $S_A$. This model is used to project outcomes in $S_B$. Denote the model as $m\left(Y_i\right|{\mathit{Z}}_{\mathit{i}},L{C}_i;\mathit{\beta })$ with the following specification:

Level-1:

$$g\left(Y_{ik}\right)={\beta }_{0k}+\sum _{j=1}^m{\beta }_{jk}{z}_{jik}+e_{ik}, k=1,\dots ,L{C}^{\ast }$$

Level-2:

$${\beta }_{0k}={\beta }_{00}+e_0$$

$${\beta }_{jk}={\beta }_{j0}+e_j, j=1,\dots ,m$$

where $g(·)$ is the link function. For continuous outcomes, the identity link is used; for binary outcomes, the logit link is employed. $Y_{ik}$ is the outcome for individual $i$ in latent class $k$. ${\beta }_{0k}$ and ${\beta }_{jk}$ are the level-1 intercept and slopes, and ${\beta }_{00}$ and ${\beta }_{j0}$ are level-2 parameters. $e_{ik}$, $e_0$ and $e_j$ are normally distributed residual terms.

For both the propensity score model and the outcome projection model, we fitted linear-mixed effects using the lme4 package [38]. The model specification included random intercepts and random slopes for all predictors across the $L{C}^{\ast }$ groups, utilizing an unstructured covariance matrix for the random effects. This comprehensive structure posits that the class-specific coefficients follow a multivariate normal distribution, which enables ‘borrowing strength’ across classes for more efficient estimation, particularly in cases where some classes have limited data. By adopting this approach, the model directly estimates the variance components between classes for both baseline levels and covariate effects. Consequently, it effectively captures the nuanced differences in the relationships between $\mathit{Z}$ and $Y$, as well as $\mathit{Z}$ and $R$, across subpopulations, which is essential for addressing the core challenge of latent population heterogeneity. Parameter estimation proceeded in two stages: the conditional modes of the random effects were predicted via the penalized iteratively reweighted least squares (PIRLS) algorithm, while the covariance parameters (i.e., the variance components) were estimated by maximizing the restricted maximum likelihood (REML) criterion.

2.5. Stage 3: Construction of a Doubly Robust Estimator

Using the improved propensity score model $\pi ({\mathit{Z}}_i,L{C}_i,d_i;\widehat{\mathit{\alpha }})$ and the outcome projection model $m\left(Y_i\right|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }})$, we construct a doubly robust estimator for inference following Chen et al. [20].

First, compute the projected outcomes for the reference sample via $m\left(Y_i\right|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }})$, i.e., ${\widehat{Y}}_i, i\in S_B$.

Next, for individuals in $S_A$, compute the linear predictors $q_i=\mathit{ln}\left(\frac{P_i}{1-P_i}\right)$, with $P_i=P\left(R_i=1\right)$. Then apply kernel smoothing to $q_i$ as in Wang et al. [30], to obtained a more stable weight $w_i$:

$$w_i=\sum _{m\in S_B}{\displaystyle \frac{K\left\{\frac{q_i^A-q_m^B}{h}\right\}}{{\sum }_{n\in S_A}K\left\{\frac{q_n^A-q_m^B}{h}\right\}}}·d_m$$

where $q_n^A$ is the linear predictor for individual $n$ in $S_A$, and $q_m^B$ for individual $m$ in $S_B$. $K(·)$ is a kernel function. In this study, standard normal function is selected. The bandwidth $h$ is determined by applying Silverman’s rule of thumb to the linear predictors in $S_A$, i.e., $h=0.9·\mathrm{m}\mathrm{i}\mathrm{n}({\widehat{\sigma }}_{q^A},{\displaystyle \frac{IQ{R}_{q^A}}{1.349}})$. This rule provides a data-driven plug-in bandwidth selector that is derived to asymptotically minimize the mean integrated squared error (MISE) for kernel density estimation under a Gaussian reference [39]. The weights $w_i$ can be obtained by KWML package. The asymptotic efficiency improvement from kernel smoothing is sated in the following lemma:

Lemma 1.

Let ${\widehat{\mu }}_{IPW}$ be an estimator constructed using the unsmoothed inverse-probability weights ${\omega }_i^{IPW}=1/{\pi }_i$, and ${\widehat{\mu }}_{smooth}$ be the corresponding estimator using kernel-smoothed weights ${\omega }_i^{smooth}$. Under regularity conditions C1–C5 (stated in Supplementary Section S1), the asymptotic variance of ${\widehat{\mu }}_{smooth}$ is no longer than that of ${\widehat{\mu }}_{IPW}$, i.e., $\underset{n_A\to \infty }{\lim }Var\left({\widehat{\mu }}_{smooth}\right)\le \underset{n_A\to \infty }{\lim }Var\left({\widehat{\mu }}_{IPW}\right)$ (see proof of Lemma 1 in Supplementary Section S3).

Last, combine the projected outcome in $S_B$ and stabilized weights $w_i$. For continuous outcome, the mean estimator is:

$${\widehat{\mu }}_{NHDR}={\displaystyle \frac{1}{{\sum }_{i\in S_A}{w}_i}}\sum _{i\in S_A}{w}_i(Y_i-{\widehat{Y}}_i)+{\displaystyle \frac{1}{{\sum }_{i\in S_B}{d}_i}}\sum _{i\in S_B}{d}_i{\widehat{Y}}_i$$

We denote this method as Nonprobability Heterogeneity-adjusted Doubly Robust Inference (NHDR). The resulting estimator retains the double robustness property under the latent class structure. Specifically, ${\widehat{\mu }}_{NHDR}$ is consistent if, within each latent stratum defined by $L{C}_i$, either the propensity model $\pi ({\mathit{Z}}_i,L{C}_i,d_i;\widehat{\mathit{\alpha }})$ or the outcome model $m\left(Y_i\right|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }})$ is correctly specified. This follows from the fact that consistency of either model within a stratum ensures unbiased estimation of a stratum-specific mean; the population estimator, as a weighted average across strata, thus remains consistent if this condition holds stratum-wise. The consistency and asymptotic normality of ${\widehat{\mu }}_{NHDR}$ is sated in the following theorem:

Theorem 1

(Consistency and Asymptotic Normality of ${\widehat{\mathsf{\mu }}}_{\mathrm{N}\mathrm{H}\mathrm{D}\mathrm{R}}$). Under conditions C1–C8 in Supplementary Section S1, and assuming that the latent class membership ${LC}_i$ obtained from the Stage 1 finite mixture model is consistent, i.e., the classification probability is $O_p\left(n_A^{-1/2}\right)$, we have, as $N\to \infty$

$${\widehat{\mu }}_{NHDR}\stackrel{p}{\to }\mu$$

and

$$\sqrt{n_A}\left({\widehat{\mu }}_{NHDR}-\mu \right)\stackrel{d}{\to }N\left(0, V_{NHDR}\right)$$

with the asymptotic variance $V_{NHDR}$ defined below. A detailed proof is provided in Supplementary Section S4.

Referring to Chen et al. [20] and Wang et al. [30], the finite population variance of ${\widehat{\mu }}_{NHDR}$ is $Var\left({\widehat{\mu }}_{NHDR}\right)=V_{NHDR}+o\left(n_A^{-1}\right)$, where

$$V_{NHDR}={\displaystyle \frac{1}{N^2}}\sum _{i=1}^N\left(1-{\pi }_i\right){\pi }_i{\left[w_i\left\{Y_i-m\left(Y_i|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }}\right)-{\mu }_e\right\}-{\mathit{b}}^T{\mathit{Z}}_i\right]}^2+{\mathit{b}}^T{D}_1\mathit{b}+D_2$$

with ${\mathbf{b}}^T=\left[{\sum }_{i=1}^N{\pi }_i\left(1-{\pi }_i\right)\left\{Y_i-m\left(Y_i|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }}\right)-{\mu }_e\right\}\partial w_i/\partial {\mathit{\alpha }}^T\right]{\left\{{\sum }_{i=1}^N{\pi }_i\left(1-{\pi }_i\right){\mathit{Z}}_i{\mathit{Z}}_i^T\right\}}^{-1}$, ${\mu }_e={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N\left\{Y_i-m\left(Y_i|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }}\right)\right\}$, $D_1=N^{-2}{V}_p\left({\sum }_{i=1}^{N}I\left(R_i=0\right)d_i{\pi }_i{\mathit{Z}}_i\right)$, $D_2=N^{-2}{V}_p\left[{\sum }_{i=1}^{N}I\left(R_i=0\right)d_i\left\{m\left(Y_i|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }}\right)-N^{-1}{\sum }_{i=1}^{N}m\left(Y_i|{\mathit{Z}}_{\mathit{i}},L{C}_i;\widehat{\mathit{\beta }}\right)\right\}\right]$, where $V_p$ denotes the design-based variance under the probability sampling design for $S_B$, $I\left(R_i=0\right)$ indicates that individual $i$ belongs to $S_B$. A consistent sample estimator of $V_{NHDR}$ can be obtained by substituting the finite population quantities by consistent sample estimators. Detailed derivation can be found in Supplementary Section S5. Proof of consistency of NHDR variance estimator is shown in Supplementary Section S6.

3. Simulation Studies

We evaluate the reliability and accuracy of the proposed NHDR method through simulation studies under scenarios with and without population heterogeneity. For comparison, we include the unadjusted estimates (denoted as Naïve) and three existing methods: the doubly robust (DR) method, the inverse propensity weighting (IPW), and the superpopulation (SP) model.

3.1. Data-Generating Models

We consider a finite population of size 100,000, a size that better approximates real-world population scales, to enhance the practical relevance of our simulation study. The data-generating models for the outcome $Y$ and the inclusion probabilities $\pi$ differ between scenarios with and without population heterogeneity.

(1)

Under scenarios without population heterogeneity, continuous outcomes are generated as follows [20]:

$$Y_i=t_1{z}_{1i}+t_2{z}_{2i}+t_3{z}_{3i}+t_4{z}_{4i}+t_5{z}_{5i}+{ϵ}_i$$

where ${\mathit{Z}}_{\mathit{i}}={\left(z_{1i},z_{2i},z_{3i},z_{4i},z_{5i}\right)}^T$ comprises variables following Bernoulli, Uniform, Poisson, Chi-squared, and Normal distributions, respectively. This mixture reflects realistic data structures with diverse variable types. The covariance matrix is set to be an identity matrix, reflecting the conditional independence assumption common in FMMs. $\mathit{T}={\left(t_1,t_2,t_3,t_4,t_5\right)}^T$ encodes the signs of coefficients. ${ϵ}_i~N(0,{\sigma }_{ϵ}^2)$. We set ${\sigma }_{ϵ}^2$ to satisfy $\frac{{\sigma }_{ϵ}^2}{{\sigma }_{\mathit{Z}}^2+{\sigma }_{ϵ}^2}=\rho$, where ${\sigma }_{\mathit{Z}}^2={\sum }_{j=1}^{5}Var\left(z_j\right)$ is the total variance of the covariates. The parameter $\rho$ reflects both $\mathit{Z}-Y$ and $\mathit{Z}-R$ correlations. Binary outcomes are generated as: $\log \left(\frac{p_i}{1-p_i}\right)={\omega }_0+Y_i$, where ${\omega }_0$ is chosen so that the overall prevalence is 50%.

The inclusion probabilities for nonprobability sample $S_A$ depend on the covariates ${\mathit{Z}}_{\mathit{i}}$ through the following logistic model:

$$\log \left({\displaystyle \frac{{\pi }_i}{1-{\pi }_i}}\right)={\theta }_0+t_1{z}_{1i}+t_2{z}_{2i}+t_3{z}_{3i}+t_4{z}_{4i}+t_5{z}_{5i}+{ϵ}_i$$

where ${\theta }_0$ is chosen such that ${\sum }_{i=1}^N{\pi }_i=n_A$. The sample $S_A$ was then generated via Poisson sampling based on these probabilities.

(2)

Under scenarios with population heterogeneity, the population consists of at least two subpopulations. We define $N_{LC}$ as the number of heterogeneous populations. Here we illustrate the data-generating models for $N_{LC}=2$. For individuals in subpopulation 1 ($L{C}_i=1$) and subpopulation 2 ($L{C}_i=2$), continuous outcomes are generated as:

$$Y_{1i}=5+t_{10}+t_{11}{z}_{11i}+t_{12}{z}_{12i}+t_{13}{z}_{13i}+t_{14}{z}_{14i}+t_{15}{z}_{15i}+{ϵ}_{1i}$$

$$Y_{2i}=5+t_{20}+t_{21}{z}_{21i}+t_{22}{z}_{22i}+t_{23}{z}_{23i}+t_{24}{z}_{24i}+t_{25}{z}_{25i}+{ϵ}_{2i}$$

The distributions of $z_{kji}$ are the same as in the $N_{LC}=1$ case but with parameters set differently. ${ϵ}_{1i}~N(0,{\sigma }_{ϵ1}^2)$ and ${ϵ}_{2i}~N(0,{\sigma }_{ϵ2}^2)$. For $k=1, 2$, ${\sigma }_{ϵk}^2$ is set to satisfy $\frac{{\sigma }_{ϵk}^2}{{\sigma }_{{\mathit{Z}}_k}^2+{\sigma }_{ϵk}^2}=\rho$. The value of $t_{10}$ and $t_{20}$ are chosen so that the intraclass correlation coefficient (ICC) equals to a prespecified value, where ICC controls the degree of heterogeneity. A detailed determination is provided in Supplementary Section S7.

Binary outcomes for $L{C}_i=1$ and $L{C}_i=2$ are generated as:

$$\log \left({\displaystyle \frac{p_{1i}}{1-p_{1i}}}\right)={\omega }_{10}+Y_{1i}$$

$$\log \left({\displaystyle \frac{p_{2i}}{1-p_{2i}}}\right)={\omega }_{20}+Y_{2i}$$

with ${\omega }_{10}$ and ${\omega }_{20}$ chosen so that the prevalences in both subpopulations are 50%.

The nonprobability inclusion probabilities for $L{C}_i=1$ and $L{C}_i=2$ are given by:

$$\log \left({\displaystyle \frac{{\pi }_{1i}}{1-{\pi }_{1i}}}\right)={\theta }_{10}+t_{10}+t_{11}{z}_{11i}+t_{12}{z}_{12i}+t_{13}{z}_{13i}+t_{14}{z}_{14i}+t_{15}{z}_{15i}+{ϵ}_{1i}$$

$$\log \left({\displaystyle \frac{{\pi }_{2i}}{1-{\pi }_{2i}}}\right)={\theta }_{20}+t_{20}+t_{21}{z}_{21i}+t_{22}{z}_{22i}+t_{23}{z}_{23i}+t_{24}{z}_{24i}+t_{25}{z}_{25i}+{ϵ}_{2i}$$

where ${\theta }_{10}$ and ${\theta }_{20}$ are chosen such that ${\sum }_{i=1}^{N_1}{\pi }_{1i}=n_{A1}$ and ${\sum }_{i=1}^{N_2}{\pi }_{2i}=n_{A2}$. Poisson sampling is used within each subpopulation to obtain $S_{A1}$ and $S_{A2}$, and $S_A$ is formed by combining them. Under both scenarios with and without heterogeneity, $S_B$ is obtained via random sampling with sampling weights $d_i=N/n_B$.

3.2. Simulated Scenarios

Following existing studies [25,40,41], we set the number of latent classes ($N_{LC}$) to range from 1 to 5, with 5 set as the empirically supported upper limit to cover the practical range of population heterogeneity. Specifically, $N_{LC}=1$ indicates no heterogeneity and $N_{LC}\ge 2$ indicates heterogeneous scenarios. With $L{C}_M$ fixed at 4 for the NHDR method, $N_{LC}\in [2,4]$ corresponds to settings where NHDR can fully identify the heterogeneity, while $N_{LC}=5$ corresponds to settings in which NHDR fails to capture the heterogeneity. The distribution parameters of ${\mathit{Z}}_{\mathit{i}}$ for the five subpopulations are specified in Supplementary Table S1. We consider two parameterizations: (1) $Dist=1$: Distributions of ${\mathit{Z}}_{\mathit{i}}$ vary systematically across subpopulations, with parameters increasing monotonically from $L{C}_i=1$ to $L{C}_i=5$; (2) $Dist=2$: Distributions of ${\mathit{Z}}_{\mathit{i}}$ vary irregularly, with parameters fluctuating without a monotone trend from $L{C}_i=1$ to $L{C}_i=5$. For $N_{LC}\ge 2$, we define $n_Z$ and $n_P$ to describe the sign configurations of the associations between ${\mathit{Z}}_{\mathit{i}}$ and $Y_i$, and between ${\mathit{Z}}_{\mathit{i}}$ and $R_i$. Specifically, $n_Z$ denotes the number of covariates whose effects differ in sign across subpopulations. For scenarios with $N_{LC}$ heterogeneous subpopulations, a minimum of $N_{LC}-1$ covariates must display sign changes to distinguish all subpopulations; thus, $n_Z\in [N_{LC}-1,5]$. Let $n_P$ denote the number of subpopulations whose covariate-effect directions are opposite to those of the remaining subpopulations for the $n_Z$ covariates. By symmetry, $n_P$ and $N_{LC}-n_P$ describe the same configuration, so $n_P\le N_{LC}/2$. To ground these definitions in a practical context, we draw on a preliminary analysis of depression risk factors. This example involves modeling depression using three covariates (gender, age, education), with two latent subgroups ($N_{LC}$ = 2) hypothesized based on lifestyle [42]. A pooled analysis of the entire population suggests uniform negative associations for all covariates with depression (Supplementary Table S2). However, stratification by lifestyle reveals that gender and education exhibit directionally opposite associations between the two subgroups. This real-world case corresponds to a scenario with $n_Z=2$ covariates showing sign reversal and $n_P=1$ subgroup displaying effect directions opposite to the rest. It illustrates how latent class structure, if ignored, can mask opposing subgroup effects and bias pooled estimates. For each combination of $(N_{LC}, n_Z, n_P)$, the corresponding sign vector $\mathit{T}={\left(t_1,t_2,t_3,t_4,t_5\right)}^T$ is specified in Supplementary Table S3.

For $N_{LC}=1$, we use the $Z_i$ distribution parameters for $L{C}_i=1$ from Supplementary Table S1, and generated data using $T$ from Supplementary Table S3 corresponding to $N_{LC}$ = 1, $n_Z$ = 0, and $n_P$ = 0. For $N_{LC}=2$, we use $L{C}_i=1$ and $L{C}_i=2$ from Table S1 as the ${\mathit{Z}}_{\mathit{i}}$ distribution parameters for the two subpopulations, and generated data using $\mathit{T}$ values corresponding to various $(n_Z,n_P)$ combinations from Table S3. The settings for $N_{LC}$ = 3–5 follow the same logic, with additional subpopulations and appropriate parameter choices.

Four sample size combinations $(n_A,n_B)$ are considered: (500, 2000), (2000, 2000), (500, 5000), and (2000, 5000). To systematically evaluate method performance across a spectrum of $\mathit{Z}-Y$ and $\mathit{Z}-R$ correlations, the correlation parameter $\rho$ was set to three levels (0.3, 0.5, 0.8), corresponding to weak, moderate, and strong correlations between the covariates $\mathit{Z}$ and both the outcome $Y$ and the inclusion mechanism $R$. This range reflects the varying explanatory power that observed covariates may hold in practical applications. Additionally, the intraclass correlation coefficient (ICC) was varied across three values (0.3, 0.5, 0.8) to induce low, moderate, and high levels of outcome heterogeneity across the latent subpopulations, thereby enabling a thorough assessment of the method’s robustness to differing degrees of within-cluster dependence.

3.3. Evaluating Criteria

Relative bias (RB) and mean squared error (MSE) are used for assessing performance of point estimate. Coverage probability (CP) and confidence interval width (CIW) are used for assessing performance of CI estimate. The four criteria are calculated as follows:

$$RB={\displaystyle \frac{1}{B}}\sum _{b=1}^B{\displaystyle \frac{\left({\widehat{\mu }}^{\left(b\right)}-{\mu }_y\right)}{{\mu }_y}}\times 100$$

$$MSE={\displaystyle \frac{1}{B}}\sum _{b=1}^B{({\widehat{\mu }}^{\left(b\right)}-{\mu }_y)}^2$$

$$CP={\displaystyle \frac{1}{B}}\sum _{b=1}^{B}I\left({\mu }_y\in C{I}^{\left(b\right)}\right)\times 100$$

$$CIW={\displaystyle \frac{1}{B}}\sum _{b=1}^B\left(U^{\left(b\right)}-L^{\left(b\right)}\right)$$

where ${\mu }_y$ denotes the true mean of the outcome. $B$ denotes the number of simulations. According to Morris et al. [43], the minimum required number of simulations is 2120. More details of the calculation can be found in Supplementary Section S8. With $B$ = 3000, the acceptable lower limit of CP is 94.2%. Values of all simulation parameters are listed in Supplementary Table S4.

3.4. Results

The proposed NHDR method demonstrates robust performance in coverage probabilities (CP) across heterogeneous scenarios, maintaining nominal levels in most scenarios where existing methods show substantial deterioration. As summarized in

Table 2. Simulated coverage probabilities (%) of different methods under $Dist=1$, $\left(n_A,n_B\right)=(500,2000)$, and $\rho =0.5$ for continuous outcomes.

ICC	${\mathit{N}}_{\mathit{L}\mathit{C}}$	${\mathit{n}}_{\mathit{Z}}$	${\mathit{n}}_{\mathit{P}}$	NHDR	DR	IPW	SP	Naïve
NA	1	0	0	95.66	95.27	100.00	95.27	0.23
0.3	2	1	1	96.20	93.83	100.00	93.87	0.60
	2	2	1	95.69	93.67	100.00	93.73	0.67
	2	3	1	96.43	87.07	99.83	86.80	1.33
	2	4	1	96.06	77.63	98.27	77.63	2.37
	2	5	1	96.21	70.70	96.03	70.57	2.37
	3	2	1	95.61	93.07	100.00	93.07	0.63
	3	3	1	95.47	89.83	99.93	89.97	0.93
	3	4	1	94.86	79.10	99.80	79.33	1.07
	3	5	1	93.96	71.10	99.13	71.60	1.20
	4	3	1	94.89	90.10	99.93	90.20	0.60
	4	3	2	95.96	91.13	99.97	91.40	0.47
	4	4	1	92.79	60.47	96.33	60.63	1.10
	4	4	2	93.75	58.80	96.23	59.00	1.03
	4	5	1	92.96	56.90	95.13	57.13	1.03
	4	5	2	93.64	51.20	91.17	51.73	1.20
	5	4	1	85.75	63.50	97.97	64.37	0.77
	5	4	2	93.18	52.07	92.90	52.30	0.93
	5	5	1	86.21	62.17	96.97	62.90	0.67
	5	5	2	93.75	45.87	89.00	46.37	1.00
0.5	2	1	1	95.49	94.83	100.00	94.97	1.07
	2	2	1	95.22	94.40	100.00	94.43	0.97
	2	3	1	96.25	88.07	99.93	88.03	2.13
	2	4	1	97.16	79.40	99.87	79.60	3.37
	2	5	1	96.53	73.87	99.43	74.00	3.23
	3	2	1	95.91	93.27	100.00	93.47	1.20
	3	3	1	95.07	91.67	100.00	91.77	1.10
	3	4	1	95.41	82.57	100.00	82.80	1.53
	3	5	1	94.87	77.70	99.97	77.90	2.33
	4	3	1	94.86	91.77	100.00	92.10	1.33
	4	3	2	94.35	90.70	100.00	90.77	1.10
	4	4	1	94.24	68.47	99.67	68.37	1.03
	4	4	2	94.78	65.03	99.73	65.63	2.03
	4	5	1	93.96	62.37	99.33	62.47	1.70
	4	5	2	94.00	55.90	98.80	56.07	1.50
	5	4	1	87.92	70.10	99.90	70.70	1.33
	5	4	2	94.53	57.93	98.97	58.27	1.57
	5	5	1	87.74	67.67	99.83	68.27	1.20
	5	5	2	94.93	52.27	98.90	52.67	1.67
0.8	2	1	1	95.35	94.03	100.00	94.07	3.47
	2	2	1	96.09	95.40	100.00	95.37	3.60
	2	3	1	95.41	93.20	100.00	93.10	5.37
	2	4	1	95.97	88.40	100.00	88.23	5.57
	2	5	1	96.48	83.60	100.00	83.67	5.53
	3	2	1	95.44	94.73	100.00	94.70	3.93
	3	3	1	95.54	94.57	100.00	94.60	3.10
	3	4	1	94.78	90.23	100.00	90.37	4.33
	3	5	1	95.37	85.67	100.00	85.77	3.90
	4	3	1	95.55	93.90	100.00	93.97	3.40
	4	3	2	94.82	93.73	100.00	93.73	3.90
	4	4	1	94.76	80.47	100.00	80.43	4.30
	4	4	2	94.90	79.77	100.00	79.87	4.13
	4	5	1	94.41	78.77	100.00	78.63	4.17
	4	5	2	95.07	69.27	100.00	69.43	4.40
	5	4	1	91.28	82.20	100.00	82.30	3.20
	5	4	2	94.81	73.67	100.00	73.97	3.97
	5	5	1	91.34	79.33	100.00	79.50	3.97
	5	5	2	95.25	69.20	100.00	69.43	3.77

for a representative setting (continuous outcomes, $\rho =0.5$, $Dist=1$, $\left(n_A,n_B\right)=(500,2000)$), NHDR consistently maintains nominal coverage across the entire spectrum of heterogeneity considered ($N_{LC}\in [1, 4]$). In stark contrast, the coverage of both DR and SP estimators deteriorates systematically upon the introduction of heterogeneity ($N_{LC}\ge 2$). Although valid under homogeneity ($N_{LC}=1$), their confidence intervals become anti-conservative, with coverage deficit increasing monotonically as a function of the complexity of the latent structure—specifically, with the number of heterogeneous populations ($N_{LC}$), the number of covariates exhibiting sign reversal ($n_Z$) and the count of subpopulations with opposing effect directions ($n_P$). This pattern stems from fundamental model misspecification: these approaches assume homogeneous covariate-outcome relationships, producing biased estimates when pooling data from subpopulations with divergent response processes. When true heterogeneity exceeds the specified model capacity ($N_{LC}=5$ with $L{C}_M=4$), NHDR shows moderate coverage decline. This is attributed to residual confounding arising from the unmodeled latent class. Nevertheless, NHDR maintains superior coverage compared to existing methods under identical conditions, demonstrating that partial adjustment for population heterogeneity provides meaningful improvements over complete omission. Simulated CP under other sample sizes and correlation values (Supplementary Tables S5–S15) follow similar trends: all methods perform similarly across $\left(n_A,n_B\right)$ and $\rho$ under $N_{LC}=1$ (Supplementary Figure S1), while DR, IPW and SP exhibit reduced CPs with larger $\left(n_A,n_B\right)$ and higher $\rho$ for $N_{LC}\ge 2$ (Supplementary Figures S2–S5).

A key strength of the NHDR estimator is its ability to deliver asymptotically unbiased estimation across all heterogeneity configurations, a property that standard methods forfeit as heterogeneity increases. Under homogeneous populations ($N_{LC}=1$), all methods demonstrate negligible relative bias (RB), as shown in Supplementary Figure S6. However, the RB performance of DR, SP, and IPW escalates predictably as a function of the latent structure complexity, specifically with the number of latent classes ($N_{LC}$) and the number of covariates with sign-reversed effects ($n_Z$). As Figure 2, Figure 3, Figure 4 and Figure 5 demonstrate, for $N_{LC}\ge 2$ and $n_Z\ge 3$, these estimators exhibit substantial bias, while NHDR maintains RB near zero. This performance advantage reflects NHDR’s capacity to properly account for latent population heterogeneity, thereby outperforming methods that assume population-level homogeneous effects. The RB patterns remain consistent across different parameter values, with greater variability observed under larger intraclass correlation coefficients (ICC) and smaller nonprobability samples (Supplementary Figures S6–S26).

NHDR achieves superior estimation efficiency under heterogeneity, as quantified by a systematically lower mean squared error (MSE) than conventional methods. This efficiency gain, as demonstrated in identical scenarios in

Table 3. Simulated mean squared error (×10³) of different methods under $Dist=1$, $\left(n_A,n_B\right)=(500, 2000)$, and $\rho =0.5$ for continuous outcomes.

ICC	${\mathit{N}}_{\mathit{L}\mathit{C}}$	${\mathit{n}}_{\mathit{Z}}$	${\mathit{n}}_{\mathit{P}}$	NHDR	DR	IPW	SP	Naïve
NA	1	0	0	2.09	2.00	2.12	2.00	24.47
0.3	2	1	1	2.23	2.47	2.63	2.47	24.68
	2	2	1	2.26	2.54	2.68	2.55	25.04
	2	3	1	2.22	4.44	4.50	4.47	18.56
	2	4	1	2.34	7.55	7.52	7.56	12.76
	2	5	1	2.33	9.73	9.76	9.72	12.78
	3	2	1	2.37	2.84	3.09	2.84	25.98
	3	3	1	2.49	3.56	3.88	3.55	26.06
	3	4	1	2.62	6.20	6.06	6.16	21.95
	3	5	1	2.74	8.26	8.02	8.21	22.93
	4	3	1	2.84	3.99	4.36	3.98	30.47
	4	3	2	2.57	3.64	4.01	3.62	29.17
	4	4	1	3.75	12.83	13.83	12.70	25.47
	4	4	2	3.53	13.64	14.06	13.52	22.33
	4	5	1	3.60	14.31	15.25	14.14	25.17
	4	5	2	3.67	17.08	17.43	16.89	21.78
	5	4	1	7.05	12.46	12.90	12.25	29.14
	5	4	2	3.88	17.42	17.81	17.26	26.00
	5	5	1	7.09	13.32	13.82	13.11	29.55
	5	5	2	3.78	20.44	20.69	20.22	25.71
0.5	2	1	1	2.71	2.68	2.91	2.68	26.90
	2	2	1	2.67	2.89	3.08	2.90	26.78
	2	3	1	2.70	4.75	4.83	4.77	20.35
	2	4	1	2.64	7.92	7.88	7.93	14.12
	2	5	1	2.70	10.36	10.42	10.35	14.23
	3	2	1	2.80	3.22	3.53	3.23	27.41
	3	3	1	2.94	3.76	4.14	3.74	28.26
	3	4	1	3.01	6.29	5.99	6.26	23.99
	3	5	1	3.12	7.92	7.54	7.88	24.35
	4	3	1	3.27	4.05	4.48	4.03	31.46
	4	3	2	3.25	4.28	4.78	4.25	31.87
	4	4	1	3.95	12.45	13.93	12.31	27.10
	4	4	2	3.82	13.60	14.27	13.47	24.43
	4	5	1	4.07	14.49	15.78	14.32	27.09
	4	5	2	3.93	17.59	18.23	17.41	24.36
	5	4	1	7.23	12.12	12.93	11.93	31.97
	5	4	2	4.17	18.00	18.65	17.79	28.56
	5	5	1	7.38	13.23	13.98	13.02	30.98
	5	5	2	4.12	20.33	20.79	20.10	27.06
0.8	2	1	1	4.89	5.28	5.75	5.27	33.71
	2	2	1	4.69	5.19	5.59	5.19	34.62
	2	3	1	4.80	6.38	6.54	6.40	28.73
	2	4	1	4.91	9.40	9.38	9.43	23.54
	2	5	1	4.86	12.54	12.63	12.53	22.66
	3	2	1	5.17	5.32	5.79	5.31	36.36
	3	3	1	5.03	5.55	6.16	5.52	37.10
	3	4	1	5.30	7.75	7.50	7.74	32.09
	3	5	1	5.31	9.99	9.30	9.97	32.88
	4	3	1	5.63	6.35	7.07	6.33	41.48
	4	3	2	5.75	6.83	7.72	6.80	41.67
	4	4	1	6.45	14.03	16.77	13.87	37.86
	4	4	2	6.25	15.40	16.58	15.24	34.30
	4	5	1	6.24	15.90	18.70	15.69	37.61
	4	5	2	6.50	21.16	22.35	20.96	32.64
	5	4	1	9.42	13.86	15.37	13.67	43.20
	5	4	2	7.03	19.48	20.99	19.28	38.94
	5	5	1	9.60	15.12	16.77	14.90	41.73
	5	5	2	7.01	22.84	23.96	22.62	37.93

, is attributable to its dual mechanism of bias correction and variance control: by modeling group-specific relationships, NHDR eliminates the bias component that grows with heterogeneity complexity ($N_{LC}$, $n_Z$, and $n_P$) in conventional methods, while its kernel smoothing mitigates the variance inflation from extreme weights that plague IPW. The IPW estimator exhibits characteristically elevated MSE due to extreme weights inducing high variance, despite maintaining adequate CP in most scenarios. This pattern confirms that IPW’s apparent validity in coverage metrics comes at the cost of estimation precision, substantially limiting its practical utility. MSE increases across methods with smaller $n_A$. With larger $\rho$, these methods show smaller MSE for $N_{LC}=1$ (Supplementary Figure S27); however, for $N_{LC}\ge 2$, MSE of three existing methods increase with $\rho$, while NHDR maintains lower values (Supplementary Figures S28–S31). Regarding confidence interval width (CIW), NHDR achieves precision comparable to DR and SP (

Table 4. Simulated confidence interval width of different methods under $Dist=1$, $\left(n_A,n_B\right)=(500, 2000)$, and $\rho =0.5$ for continuous outcomes.

ICC	${\mathit{N}}_{\mathit{L}\mathit{C}}$	${\mathit{n}}_{\mathit{Z}}$	${\mathit{n}}_{\mathit{P}}$	NHDR	DR	IPW	SP	Naïve
NA	1	0	0	0.59	0.55	1.06	0.56	0.03
0.3	2	1	1	0.62	0.59	1.38	0.59	0.04
	2	2	1	0.62	0.60	1.35	0.60	0.04
	2	3	1	0.65	0.63	1.30	0.63	0.04
	2	4	1	0.70	0.68	1.22	0.68	0.04
	2	5	1	0.71	0.71	1.17	0.71	0.04
	3	2	1	0.64	0.61	1.37	0.61	0.04
	3	3	1	0.64	0.62	1.36	0.62	0.04
	3	4	1	0.66	0.64	1.33	0.65	0.04
	3	5	1	0.66	0.65	1.30	0.66	0.04
	4	3	1	0.66	0.65	1.39	0.65	0.04
	4	3	2	0.66	0.65	1.38	0.65	0.04
	4	4	1	0.70	0.71	1.32	0.71	0.04
	4	4	2	0.71	0.72	1.30	0.72	0.04
	4	5	1	0.71	0.72	1.30	0.72	0.04
	4	5	2	0.72	0.74	1.26	0.74	0.04
	5	4	1	0.82	0.73	1.38	0.73	0.04
	5	4	2	0.75	0.76	1.34	0.76	0.04
	5	5	1	0.81	0.73	1.37	0.73	0.04
	5	5	2	0.75	0.77	1.31	0.77	0.04
0.5	2	1	1	0.68	0.64	1.76	0.64	0.04
	2	2	1	0.67	0.65	1.71	0.66	0.04
	2	3	1	0.70	0.68	1.67	0.68	0.04
	2	4	1	0.75	0.73	1.6	0.73	0.04
	2	5	1	0.75	0.76	1.56	0.76	0.04
	3	2	1	0.69	0.66	1.75	0.66	0.04
	3	3	1	0.69	0.67	1.73	0.67	0.04
	3	4	1	0.71	0.69	1.72	0.69	0.04
	3	5	1	0.73	0.70	1.69	0.71	0.04
	4	3	1	0.72	0.70	1.78	0.70	0.05
	4	3	2	0.71	0.70	1.76	0.70	0.05
	4	4	1	0.76	0.76	1.73	0.76	0.05
	4	4	2	0.76	0.78	1.72	0.77	0.05
	4	5	1	0.75	0.77	1.72	0.77	0.05
	4	5	2	0.77	0.79	1.69	0.79	0.05
	5	4	1	0.86	0.78	1.81	0.78	0.05
	5	4	2	0.81	0.81	1.78	0.81	0.05
	5	5	1	0.87	0.79	1.80	0.79	0.05
	5	5	2	0.81	0.83	1.76	0.83	0.05
0.8	2	1	1	0.90	0.88	2.99	0.88	0.07
	2	2	1	0.89	0.91	2.91	0.91	0.07
	2	3	1	0.92	0.92	2.90	0.93	0.07
	2	4	1	0.95	0.97	2.85	0.97	0.07
	2	5	1	0.96	1.00	2.82	1.00	0.07
	3	2	1	0.91	0.90	3.02	0.90	0.07
	3	3	1	0.92	0.90	3.00	0.90	0.07
	3	4	1	0.94	0.91	3.01	0.92	0.07
	3	5	1	0.94	0.93	2.98	0.94	0.07
	4	3	1	0.96	0.95	3.09	0.95	0.07
	4	3	2	0.96	0.96	3.08	0.96	0.07
	4	4	1	0.98	1.00	3.10	1.00	0.07
	4	4	2	1.00	1.01	3.08	1.01	0.07
	4	5	1	0.98	1.01	3.09	1.00	0.07
	4	5	2	1.00	1.03	3.06	1.03	0.07
	5	4	1	1.08	1.02	3.22	1.02	0.08
	5	4	2	1.04	1.04	3.21	1.04	0.07
	5	5	1	1.09	1.03	3.21	1.02	0.08
	5	5	2	1.04	1.06	3.21	1.06	0.07

), while IPW produces consistently wider intervals due to its inherent instability. This demonstrates NHDR’s optimal balance between precision and reliability, a pattern that persists across other parameter configurations (Supplementary Tables S16–S37).

All performance patterns remain consistent under alternative data-generating processes, including irregular covariate distributions ($Dist=2$; Supplementary Tables S38–S74 and Figures S32–S48) and binary outcomes (Supplementary Tables S75–S146 and Figures S49–S82). This consistency confirms NHDR’s robustness to variations in distributional assumptions and outcome types.

In summary, our simulations yield a key practical insight: NHDR effectively addresses fundamental limitations in existing methods for handling heterogeneous populations. It provides valid, unbiased, and efficient inference precisely in those complex scenarios where existing approaches show systematic deficiencies. For continuous outcomes, NHDR achieves an average reduction in relative bias of 1.8%, 2.1%, 4.5%, and 3.8% compared to the best-performing existing method (DR) under scenarios with $N_{LC}$ = 2, 3, 4, and 5, respectively. Corresponding reductions in mean squared error are 5.1, 5.7, 15.5, and 11.9. The method’s performance advantage is most pronounced under moderate to strong heterogeneity ($N_{LC}\in [2,4]$) with multiple divergent covariates ($n_Z\ge 3$). Even when heterogeneity is underestimated ($N_{LC}=5$), NHDR maintains a competitive advantage over all existing methods, demonstrating the value of partial heterogeneity adjustment. Notably, NHDR achieves these improvements while introducing minimal efficiency penalty under homogeneous conditions.

4. Application to Health Monitoring SMS Survey Data

We applied our proposed method to data from Health Monitoring SMS Survey (HMSS), a nonprobability online survey targeting Guangzhou residents aged 18–60 in China. The HMSS is administered via the Guangzhou Health Hotline 12320, a government-operated platform focusing on health education and promotion. The 12320 system randomly generated mobile phone numbers based on the demographic distribution across Guangzhou’s 11 districts. Data collection proceeded in two steps: invitations were first sent by SMS to the randomly selected phone numbers; followed by distribution of the survey link over the next day to collect responses via an electronic questionnaire. For this analysis, we use data collected in October 2024 to describe health-monitoring indicators among adult residents of Guangzhou, with a sample size of $n_A$ = 1527 participants. Nine health monitoring indicators are analyzed, with corresponding survey questions listed in Supplementary Table S147.

For the reference sample, we used data from the seventh Guangdong Health Service Survey (GHSS-7) for Guangzhou, whose surveyed period was close to that of the HMSS. The GHSS-7 employed a multi-stage stratified cluster sampling design. We denote Guangzhou data from GHSS-7 as GHSS-GZ-7. After excluding individuals aged under 18 or over 60, the final GHSS-GZ-7 sample comprises $n_B$ = 9299 individuals. Common covariates $\mathit{Z}$ between HMSS and GHSS-GZ-7 include sex, age, employment status, marital status, educational level, and district.

We first compared the distributions of the common variables between the HMSS and GHSS-GZ-7 datasets using chi-square tests.

Table 5. Distributions of common variables in HMSS and GHSS-GZ-7 (%).

	HMSS	GHSS-GZ-7	p
	N = 1527	N = 9299
Age (Mean ± SD)	37.6 ± 9.16	42.7 ± 11.0	<0.001
Sex			0.0110
Male	44.86	48.42
Female	55.14	51.58
Employment status			<0.001
Employed	81.73	75.52
Student	2.16	1.74
Retired	4.98	9.18
Unemployed or others	11.13	13.55
Marital status			<0.001
Married	62.67	79.29
Unmarried	30.98	16.81
Divorced or others	6.35	3.90
Educational level			<0.001
Primary school or below	0.52	10.64
Middle school	4.32	27.10
High school or vocational high school	12.18	20.08
college or above	82.97	42.19
District			<0.001
Central urban districts	51.3	29.8
Peripheral districts	48.7	70.2

SD, standard deviation.

shows that all covariates differ statistically significant between the two samples. Compared to GHSS-GZ-7, the HMSS sample has higher proportions of individuals who are female, aged younger, employed, unmarried, and who had attained a college degree or higher. The HMSS sample is more concentrated in central urban districts.

We then applied our proposed NHDR method to estimate health indicators using the HMSS dataset, setting the maximum number of latent classes $L{C}_M$ to 5 as supported by existing studies [25,40,41]. The FMM within the NHDR procedure was fitted using the six common covariates available in both samples. Electronic screen use was treated as continuous outcome, while all other indicators were binary. For comparison, we include estimates from three existing methods (DR, IPW, and SP) as well as a naïve method for comparison. Finite mixture modelling identified five socio-demographically distinct latent classes, with significant differences across all covariates (all p < 0.001; Supplementary Table S148). The distribution of these classes was uneven, ranging from 8.8% (older retired women) to 33.6% (prime aged married residents in peripheral districts). Point estimates, 95%CI, and CIWs are summarized in

Table 6. Point estimate, 95%CI, and CIW of nine health indicators among adult residents of Guangzhou in October 2024.

	Naïve			DR			IPW			SP			NHDR
	Estimate	95%CI	CIW	Estimate	95%CI	CIW	Estimate	95%CI	CIW	Estimate	95%CI	CIW	Estimate	95%CI	CIW
Electronic screen use	11.72	(11.49, 11.95)	0.460	9.62	(9.10, 10.14)	1.041	9.60	(8.70, 10.50)	1.803	9.58	(9.04, 10.13)	1.091	8.85	(7.83, 9.87)	2.044
Poor/very poor self-rated health (%)	7.53	(6.26, 8.97)	0.027	10.75	(7.47, 14.03)	0.066	10.59	(6.28, 14.90)	0.086	10.84	(7.20, 14.47)	0.073	8.04	(3.29, 12.78)	0.095
Weekly alcohol consumption over 14 units (%)	6.35	(5.18, 7.69)	0.025	6.27	(4.12, 8.43)	0.043	6.02	(3.55, 8.48)	0.049	6.84	(4.61, 9.07)	0.045	6.20	(4.34, 8.06)	0.037
Exercise over 150 min per week (%)	17.22	(15.36, 19.21)	0.039	19.24	(15.59, 22.90)	0.073	20.61	(15.06, 26.16)	0.111	18.35	(13.93, 22.76)	0.088	16.50	(9.81, 23.2)	0.134
Daily sleep duration less than 5 h (%)	8.38	(7.04, 9.89)	0.029	7.91	(5.51, 10.32)	0.048	7.86	(4.69, 11.03)	0.063	7.95	(5.40, 10.50)	0.051	7.40	(5.01, 9.80)	0.048
Poor/very poor sleep quality (%)	14.21	(12.5, 16.06)	0.036	15.83	(12.43, 19.23)	0.068	15.23	(10.76, 19.69)	0.089	16.50	(12.66, 20.34)	0.077	11.59	(7.70, 15.48)	0.078
Depression (%)	37.39	(34.96, 39.88)	0.050	34.77	(30.44, 39.10)	0.087	35.40	(29.31, 41.5)	0.122	35.78	(31.13, 40.43)	0.093	42.50	(25.93, 59.07)	0.331
Anxiety (%)	26.26	(24.07, 28.54)	0.045	24.67	(20.79, 28.55)	0.078	22.91	(17.75, 28.06)	0.103	25.62	(21.56, 29.68)	0.081	19.33	(14.90, 23.76)	0.089
High/very high stress (%)	64.51	(62.05, 66.91)	0.049	59.23	(54.93, 63.54)	0.086	57.76	(51.36, 64.15)	0.128	59.37	(54.32, 64.42)	0.101	62.57	(48.61, 76.52)	0.279

Random seed of NHDR: 20251202.

. The adjusted estimates differ from the naïve estimates, confirming that covariates used for adjustment are correlated with both the health indicators and inclusion in HMSS. Estimates from DR, IPW, and SP are generally similar, while NHDR produces lower prevalence estimates for exercise over 150 min per week and poor sleep quality, and higher estimates for depression and stress. No notable differences are observed across methods for the remaining outcomes. These patterns suggest the presence of population heterogeneity underlying the outcome model. For example, regression coefficients of covariates for depression obtained from a mixed-effects model that accounts for latent classes (Supplementary Table S149) reveal divergent effects of all covariates across the five classes. Such heterogeneity is obscured in a pooled analysis of the full sample, which would lead to biased inference. In terms of precision, NHDR showed a mixed pattern relative to DR and SP: confidence intervals were similar or slightly narrower for some outcomes (e.g., weekly alcohol assumption, short sleep) but moderately wider for others, including depression and stress. The NHDR estimates remained nearly identical across analyses conducted with different random seeds (Supplementary Table S150), indicating stability with respect to initialization. Regarding computational cost, the average runtimes for DR, IPW, SP, and NHDR were 0.31 s, 0.25 s, 0.11 s, and 1765.33 s (approximately 29.4 min), respectively, reflecting the higher computational demand of the proposed method.

5. Discussion

This study introduces a Nonprobability Heterogeneity-adjusted Doubly Robust (NHDR) estimator. Its core methodological contribution is the formal integration of latent-class discovery—implemented via finite mixture modeling—into the doubly robust estimation framework, thereby addressing the well-documented failure of standard methods under latent population heterogeneity. Unlike conventional approaches that presume global homogeneity in the covariate–outcome/selection relationships, NHDR explicitly accommodates heterogeneity through a principled, three-stage procedure: (1) identification of latent subpopulations using finite mixture modeling; (2) incorporation of the estimated class memberships as a grouping variable in mixed-effects models for both the propensity score and the outcome projection; and (3) construction of a kernel-smoothed doubly robust estimator. This architecture preserves the double-robustness property within each latent stratum, ensuring consistency when either the stratum-specific propensity score or outcome model is correctly specified. Extensive simulations substantiate this theoretical advance. In heterogeneous settings where standard estimators exhibit systematic under-coverage and bias, NHDR robustly maintains nominal coverage and approximate unbiasedness. Furthermore, it delivers uniformly lower mean squared error and confidence intervals of comparable or superior precision, demonstrating a more favorable bias–variance trade-off. Critically, this performance profile is corroborated by applied evidence: estimates of health indicators from the Health Monitoring SMS Survey obtained via NHDR not only align with the simulation-based performance trends but also differ substantively from those produced by unadjusted or conventional adjustment methods—a pattern consistent with the presence of latent subgroups. Collectively, these findings establish NHDR as a valid and efficient inferential tool for nonprobability samples drawn from heterogeneous populations.

The primary contribution of NHDR lies in its identification and adjustment for the population heterogeneity. By detecting heterogeneous subpopulations via FMM, NHDR effectively reduces bias stemming from heterogeneous relationships between covariates and both the outcome ($Y$) and selection mechanism ($R$). This is achieved by integrating the estimated latent class structure into both the propensity score and outcome projection models. The method requires pre-specifying the maximum number of latent classes ($L{C}_M$), with the optimal number ($L{C}^{\ast }$) determined using BIC. However, if $L{C}_M$ is set lower than the true number of classes, NHDR’s performance may deteriorate, as observed in simulations when $N_{LC}=5$. Thus, when prior knowledge is available, it should guide the choice of $L{C}_M$; in the absence of such information, we recommend a conservative value of $L{C}_M$ as suggested in literature [33] to improve detection of underlying heterogeneous subpopulations.

To simulate the heterogeneous scenarios, we draw upon empirical studies that typically report fewer than five distinct subgroups in population-level data [25,40,41]. Accordingly, we varied the number of heterogeneous populations ($N_{LC}$) from 1 to 5, where $N_{LC}=1$ corresponds to the homogeneity scenario, and $N_{LC}\in [2, 5]$ represents the heterogeneous scenarios. For each value of $N_{LC}$, we further diversify the heterogeneity patterns using two parameters: $n_Z$ (number of covariates exhibiting subpopulation-specific effects) and $n_P$ (number of subpopulations demonstrating reversed effect directions). The degree of between-subpopulation heterogeneity was rigorously controlled through the intraclass correlation coefficient (ICC), set as low, moderate, and high levels for each combination of $N_{LC}$, $n_Z$ and $n_P$. This comprehensive simulation design ensures a realistic representation of heterogeneous conditions. The robust performance of NHDR across these simulated scenarios confirms its reliability and applicability.

We also observe that larger nonprobability sample sizes ($n_A$) lead to marked declines in CPs for the three existing methods. These underscores a critical pitfall in the analysis of nonprobability samples: increased sample size does not automatically mitigate selection bias and may even amplify it, resulting in more severely biased estimates and erroneous conclusions [44]. Moreover, a stronger association between covariates ($\mathit{Z}$) and the outcome or selection mechanism (as represented by higher $\rho$) further impair the performance of existing methods. This suggests that even high informative auxiliary variables cannot correct for bias if underlying heterogeneity in the relationship between $\mathit{Z}-Y$ and $\mathit{Z}-R$ is ignored. Therefore, accurately accounting for population heterogeneity is essential for valid inference from nonprobability samples.

In the applied analysis, the proposed NHDR estimator produced moderately wider confidence intervals for several outcomes compared to conventional methods—a pattern not fully mirrored our simulation results. This discrepancy can be attributed to how latent heterogeneity is formally incorporated within the NHDR framework. Specifically, NHDR introduces the latent-class variable as a second-level grouping factor in mixed-effects models, thereby explicitly modeling between-class heterogeneity in both the outcome-generating ($\mathit{Z}-Y$) and selection ($\mathit{Z}-R$) mechanisms. This modeling strategy introduces an additional variance component that naturally inflates the estimator’s variance when underlying heterogeneity exists, resulting in wider intervals that more accurately reflect the underlying sampling uncertainty in a heterogeneous population. The variance estimator for NHDR conditions on the point-estimated latent-class assignments from the first-stage finite mixture model—a pragmatic simplification typical of two-step frameworks to preserve computational feasibility. Critically, our simulation studies demonstrated that NHDR consistently achieved nominal coverage across diverse heterogeneous scenarios. This indicates that, for the settings considered, the practical consequence of this simplification on inference validity is minimal. Thus, the wider intervals observed in practice are a coherent and expected outcome of modeling latent subgroups, not an indication of instability. They represent a more statistically honest quantification of uncertainty when population heterogeneity is present. Future work could explore variance estimators that more fully integrate clustering uncertainty.

The NHDR framework provides a principled approach for selection bias adjustment under latent heterogeneity. Its effective implementation involves several methodological trade-offs. First, NHDR’s performance depends on whether the common covariate set $\mathit{Z}$ can capture latent heterogeneity. Simulation results show that even with moderate population-level correlation (e.g., $\rho =3$) between $\mathit{Z}$ and the outcome/selection mechanism, NHDR retains a clear advantage over methods that ignore heterogeneity, as it explicitly models subgroup-specific $\mathit{Z}-Y$ and $\mathit{Z}-R$ relationships. Only when $\mathit{Z}$ is entirely uninformative for both latent classes and direct associations would NHDR offer no improvement over conventional approaches. Second, NHDR is sensitive to the number of latent classes specified. Under-specification (e.g., setting $L{C}_M=4$ when $N_{LC}=5$) leads to residual confounding and reduced coverage, as our simulations confirm. Over-specification, while less harmful to bias, increases model complexity and may destabilize mixed-effects estimates. Empirical evidence suggests that substantively interpretable latent subgroups in regional populations are typically limited in number; thus, instability from moderate over-specification is of low practical concern. The key recommendation is to avoid under-specifying the number of classes. Third, NHDR relies on the conditional-independence assumption of covariates within latent classes—a standard but strong simplification for public-health data. The estimator’s dependence on between-class distributional differences suggests potential robustness to mild violations; however, strongly correlated covariates may affect latent-class recovery and estimate stability. Future work should explicitly examine performance under correlated covariate structures. Finally, practitioners should recognize an inherent efficiency–complexity trade-off: improved inferential validity under heterogeneity comes with increased computational cost. NHDR is therefore most appropriate when latent heterogeneity is plausible and substantively justified covariates are available.

The NHDR method has several advantages. First, by leveraging covariates to identify latent population structure, it adjusts for heterogeneity in both the propensity score and outcome projection models, improving inferential accuracy. Second, although the variance estimator for NHDR does not fully account for the additional uncertainty introduced by the random effects in our mixed-effects models, the empirically robust simulated coverage probabilities provide strong reassurance against this theoretical limitation. Third, FMM accommodates mixed variable types (binary, multinomial and continuous), enhancing flexibility and practical utility compared to traditional methods such as latent class analysis or latent profile analysis, which are limited to single type of variable [45].

Nonetheless, there are some limitations in the NHDR method. First, our current approach assumes the same latent population structure for both the propensity score and outcome projection models. However, this may not hold true in practice since relationships for $\mathit{Z}-Y$ and $\mathit{Z}-R$ may be different. Thus, identification of outcome-specific latent population structure will be our future research interest. Second, the assumed linearity in the mixed-effects models may fail to capture more complex relationships. Future research could extend this method to accommodate non-linear associations for further improvement. Third, the current simulation study did not examine more complex conditions—such as correlated covariates and imbalanced class sizes—that are often encountered in real-world survey data. Further research is therefore needed to evaluate the method’s robustness under these more realistic and challenging scenarios. Fourth, the HMSS relies on self-reported measures, which are subject to recall and social-desirability biases. These data-collection limitations do not affect the relative performance of the methods compared but caution against over-generalizing the absolute prevalence estimates beyond the study context. Fifth, the NHDR method incurs a substantially higher computational cost than conventional adjustment methods, reflecting the complexity of its two-stage modeling framework. Future work should focus on algorithmic improvements to enhance scalability.

6. Conclusions

With the increasing use of nonprobability sampling in epidemiologic research, selection bias poses a critical threat to the validity and accuracy of inference. A common shortcoming of existing adjustment methods is their neglect of the latent heterogeneity in the covariate-outcome and covariate-inclusion relationships, which can bias conclusions drawn from nonprobability samples. To address this gap, we propose the Nonprobability Heterogeneity-adjusted Doubly Robust (NHDR) method, which explicitly accounts for population heterogeneity during bias adjustment by identifying and integrating latent subpopulations into the adjustment procedure. The core contribution of NHDR lies in its formal integration of latent-class-based population structure into a doubly robust framework, enabling more reliable inference under heterogeneous population structures. Simulation results show that the proposed method control the coverage probabilities well in most scenarios. Under the simulated scenarios with population heterogeneity, the proposed method consistently exhibits lower relative bias and mean squared error compared to existing methods. We apply the proposed method to Health Monitoring SMS Survey data in Guangzhou China, results of which are generally consistent with the simulation evidence. Therefore, when population heterogeneity is present, NHDR provides more accurate inference than existing approaches and represents a recommended option for analysis. The proposed method is subject to several limitations. First, inference depends on a limited set of common covariates available across samples and may be sensitive to misspecification of the latent class structure. Additionally, further investigation is needed to assess its performance in more complex settings—such as high-dimensional data and scenarios where the conditional independence assumption is violated.