Estimating General Parameters from Non-Probability Surveys Using Propensity Score Adjustment

Abstract

This study introduces a general framework on inference for a general parameter using nonprobability survey data when a probability sample with auxiliary variables, common to both samples, is available. The proposed framework covers parameters from inequality measures and distribution function estimates but the scope of the paper is broader. We develop a rigorous framework for general parameter estimation by solving survey weighted estimating equations which involve propensity score estimation for units in the non-probability sample. This development includes the expression of the variance estimator, as well as some alternatives which are discussed under the proposed framework. We carried a simulation study using data from a real-world survey, on which the application of the estimation methods showed the effectiveness of the proposed design-based inference on several general parameters.

1. Introduction

Nonprobability samples are increasingly common in empirical sciences. The rise of online and smartphone surveys, along with the decrease of response rates in traditional survey modes, have contributed to the popularization of volunteer surveys where sampling is non-probabilistic. Moreover, the development of Big Data involves the analysis of large scale datasets whose obtention is conditioned by data availability and not by a probabilistic selection, and therefore they can be considered large nonprobability samples of a population [1].

The lack of a probability sampling scheme can be responsible for selection bias. Following the description from [1,2], we can distinguish the target population, $U_T$, the subpopulation that a given selection method can potentially cover, $U_{pc}$, and the fraction of the subpopulation that is finally covered, $U_{fc}$, and whose individuals might participate in the survey. Selection bias occurs when the characteristics of the individuals in $U_{fc}$ differ significantly from those in $U_T$ in a way that could affect final estimates. Typically, differences between individuals in $U_T$ and individuals in $U_{pc}$ are caused by a lack of coverage induced by the survey administration mode (for example, an online questionnaire cannot be administered to the population without internet access), while differences between $U_{pc}$ and $U_{fc}$ are caused by the variability in the propensities to participate between social-demographic groups (for example, an online questionnaire accesible in a thematic website might only be fulfilled by visitors of the website who have a specific interests that could influence the results).

Following the rise of nonprobability samples, a class of methods for reducing selection bias have been proposed in the last decades. These methods were developed from different perspectives according to the availability of auxiliary information. We can mention calibration, Propensity Score Adjustment (PSA), Statistical Matching and superpopulation modelling as the most relevant techniques to mitigate selection bias produced by coverage and self-selection errors.

Calibration weighting was originally developed by [3] as a method to correct representation issues in samples with coverage or non-response errors. It only requires a vector of auxiliary variables available for each individual of the sample and the population totals of those variables. Calibration is able to remove selection bias in nonprobability samples if the selection mechanism is ignorable [4], and despite being originally developed for parametric estimation, further work [5,6,7] has extended calibration to distribution function, quantile and poverty measures estimation.

Propensity Score Adjustment (PSA) and Statistical Matching require, apart from the nonprobability sample, a probability sample to do the adjustments. PSA was originally developed for balancing groups in non-randomized clinical trials [8] and it was adapted for non-response adjustments shortly after [9,10]. The application of PSA for removing bias in nonprobability surveys was theoretically developed in [11,12]. Statistical Matching was firstly proposed in [13] and extended in [14] for non-response adjustments. The difference between both methods is the sample used in the estimators: PSA estimates the propensity of each individual of the nonprobability sample to participate in the survey and then this propensity is used to construct the weights of the estimators, while Statistical Matching adjusts a prediction model using data from the nonprobability sample, applies it in the probability sample to predict their values for the target variable y and uses them in the parametric estimators. To the best of our knowledge, PSA and Statistical Matching has not been developed for nonparametric estimation.

Superpopulation modelling requires data from the complete census of the target population for the covariates used in the adjustment, which is assumed to be a realization (sample) of a superpopulation where the (unknown) target values follow a model. It is based on the works by [15,16], where the main idea is to fit a regression model on the target variable with data from the nonprobability sample, and use the model to predict the values of the target variable for each individual in the population. The prediction can be used for estimation using a model-based approach or some alternative versions such as model-assisted and model-calibrated. LASSO models [17] and Machine Learning predictors [18,19] have been studied as alternatives to ordinary least squares regression in superpopulation modelling.

The interest of society on poverty and inequality has increased in the last decades given the successive economic cycles and crisis. In such a context, official poverty rates and the percentage of people in poverty (or under a poverty threshold) are some important measures of a country’s wealth. The common characteristic of many poverty measures is their complexity. The literature on survey sampling is usually focused on the goal of estimating linear parameters. However, it is usual that the variable of interest in poverty studies is a measure of wages or income, where the distribution function becomes a relevant tool because it is required to calculate the proportion of people with low income, the poverty gap and other measures. Estimators for the cumulative distribution function, quantiles [20,21] and poverty measures [22] can be found in literature regarding probability samples, but there is hardly any work on the estimation of these parameters when the samples are obtained from volunteers.

In this paper, we aim to develop a framework for statistical inference on a general parameter with non probability survey samples when a reference probability sample is available. After introducing the problem of the mean estimation for volunteer samples in Section 2, in Section 3, we consider the problem of the estimation for a general parameter through general estimating equations. Section 4 presents a new estimator for a general parameter through the use of PSA to estimate the propensity score of each individual in the survey weighted estimating equation and major theoretical results are presented. Results from simulation studies are reported in Section 5 and Section 6 presents the concluding remarks.

2. Approaches to Estimation of a Mean for Volunteer Online Samples

Let $U_T$ be the target population with N elements and $s_v$ a nonprobability sample drawn from a subset of $U_T$, $U_v$, with a size of $n_v\le N$. Let y be the target variable of the survey, whose mean in the population $U_T$ is denoted as $\overline{Y}$. The sample estimation of $\overline{Y}$, $\widehat{\overline{Y}}$, is done using the Horvitz-Thompson estimator:

$${\widehat{\overline{Y}}}_{HT}=\frac{{\sum }_{i\in s_v}{w}_i{y}_i}{{\sum }_{i\in s_v}{w}_i}$$

(1)

where w is a vector of weights that accounts for the lack of representativity of $s_v$ caused by selection bias. If no auxiliary information is given, the weight would be the same for every unit, $w_i=N/n_v$, which requires to assume that the sample was drawn under a simple random sampling scheme. This is a naïve assumption given that $s_v$ is not probabilistic, this is, the probability of being in the sample is unknown and/or null for any of the units in $U_T$.

Let x be a matrix of covariates measured in $s_v$ along with y. If the population totals of the covariates, $\mathbf{X}$, are available, it is possible to estimate the mean using a vector of weights obtained with calibration, $w^{CAL}$. The calibration weights aim to minimize the distance between the original and the new weights

$$\underset{w_i^{CAL}}{min}E\left[\sum _{i\in s_v}G(w_i,w_i^{CAL})\right]$$

(2)

while respecting the calibration equations

$$\sum _{i\in s_v}{w}_i^{CAL}{\mathbf{x}}_i=\mathbf{X}.$$

(3)

Some choices for the distance $G(.,.)$ were listed in [3], along with the resulting estimators. Calibration weighting for selection bias treatment was studied in [4], where post-stratification, which is a special case of calibration [23], was used to mitigate the bias caused by different selection mechanisms, showing its efficacy when the selection of the units of $s_v$ is Missing At Random (MAR).

If a reference sample, $s_r$, drawn from the population $U_T$ is available and a number of covariates $\mathbf{x}$ have been measured both in $s_v$ and $s_r$, two procedures can be done to reduce selection bias present in $s_v$. Let $I_v$ be an indicator variable of an element being in $s_v$, this is

$$I_{vi}=\left\{\begin{array}{cc}1\hfill & i\in s_v\hfill \\ 0\hfill & i\notin s_v\hfill \end{array}\right.$$

(4)

Propensity Score Adjustment (PSA) assumes that each element of $U_T$ has a probability (propensity) of being selected for $s_v$ which can be formulated as

$${\pi }_i^v=Pr(I_{vi}=1|{\mathbf{x}}_i,y_i)$$

(5)

where ${\pi }_i^v$ is the propensity of the i-th individual to participate in $s_v$. The random mechanism behind this probability is the selection mechanism that governs the nonprobability sample. If the selection is Missing Completely At Random (MCAR), then ${\pi }_i^v=Pr(I_{vi}=1)$ and the selection bias is null, while if the selection is MAR then ${\pi }_i^v=Pr(I_{vi}=1|{\mathbf{x}}_{\mathbf{i}})$ and the selection mechanism is considered ignorable. This does not mean that the selection bias should be ignored but rather it can be treated with the right techniques.

In PSA, we consider the situation where true propensities are not known and therefore have to be estimated; we do it by combining $s_v$ and $s_r$ into a sample. The probability that $I_v=1$ is then estimated using a prediction model, traditionally a logistic regression one:

$${\widehat{\pi }}_i^v=\frac{1}{1+exp\{-\beta {\mathbf{x}}_{\mathbf{i}}\}}$$

(6)

Alternative models, such as non-linear regression and Machine Learning classification algorithms, have been studied in literature as a substitute of logistic regression (see [24] for a review). The resulting propensities can be used to adjust new weights, $w^{PSA}$, with different alternatives:

• A simple inverse probability weighting is proposed by [25]

  $$w_i^{PSA1}=\frac{w_i}{{\widehat{\pi }}_i^v}$$

  (7)

  which is a similar approach to the formula used in [26]

  $$w_i^{PSA2}=\frac{1-{\widehat{\pi }}_i^v}{{\widehat{\pi }}_i^v}$$

  (8)
• Alternatively, individuals of the combined sample ($s_v\cup s_r$) can be grouped in g equally-sized strata of similar propensity scores from which an average propensity is calculated for each group. Let ${\overline{\pi }}_g$ be the mean propensities of the g-th strata. [2] use the means as in (7) to calculate the new weights:

  $$w_i^{PSA3}=\frac{w_i}{{\overline{\pi }}_{g_i}}$$

  (9)

  where $g_i$ refers to the strata to which the i-th individual of $s_v$ belongs.
• A similar approach can be found in [12], but instead of using the means, a factor is calculated for each strata:

  $$f_g=\frac{{\sum }_{k\in s_{r_g}}{\tilde{w}}_k/{\sum }_{k\in s_r}{\tilde{w}}_k}{{\sum }_{i\in s_{v_g}}{w}_i/{\sum }_{i\in s_v}{w}_i}$$

  (10)

  where $s_{r_g}$ and $s_{v_g}$ are respectively the individuals from the probability and nonprobability sample that belong to the g-th strata, and $\tilde{w}$ is the vector of design weigths of the reference sample. The final weights are obtained by multiplying the original weights and the correction factor:

  $$w_i^{PSA4}=w_i·f_{g_i}$$

  (11)

PSA has been proven to successfully remove selection bias when prognostic covariates are chosen [11] and further adjustments, such as calibration, are applied in the estimations [2,12,27]. A recent paper [28] shows a real application of PSA in web panel surveys where the reductions in bias, although present, were not large enough to consider the estimates as unbiased.

As an alternative to PSA, Statistical Matching is another method to mitigate selection bias when a reference sample is available. For the matter, a prediction model for y using $\mathbf{x}$ as the dependent variables is built using data from $s_v$. The model is subsequently applied on the reference sample to obtain the estimates from the predicted values of y in $s_r$, $\widehat{y}$:

$$\widehat{\overline{Y}}=\sum _{k\in s_r}{w}_k{\widehat{y}}_k$$

(12)

The choice of prediction models has been studied in literature; the usual method is linear regression but other approaches such as donor imputation [13] or Machine Learning algorithms [19,29] have been listed as alternatives. Under certain conditions, Statistical Matching can reduce bias and mean square error to a greater extent than PSA [29].

When a complete census of the entire target population is available, with information on the covariates present in $s_v$, superpopulation modelling can be applied to remove selection bias [19]. In this paper we consider the case when auxiliary information is available only from a reference probability survey.

3. Estimation of a General Parameter by Using PSA

Let y be the variable of interest in a survey and $y_i$ be the value of the i-th unit in that variable, $i=1,...,N$. Suppose we want to estimate a finite population parameter ${\theta }_N$ of dimension $p\ge 1$ defined as the solution of the census estimating equations:

$$U\left({\theta }_N\right)=\frac{1}{N}\sum _U{u}_i(y_i,{\theta }_N)=0$$

(13)

where $u_i(y_i,{\theta }_N)$ is be a function of ${\theta }_N$. Some unidimensional parameters of interest can be:

• the population total $T_y$ for $u_i=(y_i-{\theta }_N/N)$,
• the population mean $\overline{Y}$ for $u_i=(y_i-{\theta }_N)$,
• the population distribution function $F_y\left(t\right)$ for $u_i=(1(y_i\le t)-{\theta }_N)$ with $1(·)$ being the indicator function,
• the finite population quantile of order j, $Q_j$ for $u_i=(1(y_i\le {\theta }_N)-j$, where $0<j<1$,

We denote by $\widehat{\theta }$ the solution of the equation:

$$\widehat{U}\left({\theta }_N\right)=\sum _U{I}_{vi}{u}_i(y_i,{\theta }_N)/{\pi }_i^v=\sum _{s_v}{u}_i(y_i,{\theta }_N)/{\pi }_i^v=0.$$

(14)

It is clear the $E_r\left(\widehat{U}\left({\theta }_N\right)\right)=U\left({\theta }_N\right)$ where r stands for the model of the selection mechanism for the sample $s_v$, this is, the true model that fits propensity scores. If ${\pi }_i^v$ are known we can get the consistent estimator of ${\theta }_N$ by solving the equation above. For the study of the properties of this estimator we consider a quasi-probability approach or pseudo-design-based inference ([19]) and we treat the volunteer sample as a realization of a Poisson sampling with probabilities ${\pi }_i^v$.

For any sample design that verifies certain regularity conditions, the solution to $\widehat{U}\left(\theta \right)=0$ provides a consistent estimator for the parameter ${\theta }_N$ (see [30]). Poisson sampling verifies these conditions, so that the consistency of the estimator is obtained immediately from the result of [30]. The normality of the estimator is demonstrated by [31], who also obtains the asymptotic variance of the estimator. From said expression and taking into account that in Poisson sampling the extractions are independent and therefore the probability of second order is given by ${\pi }_{ij}^v={\pi }_i^v{\pi }_j^v$ we can obtain the variance of $\widehat{\theta }$:

$$V\left(\widehat{\theta }\right)=J{\left(\widehat{\theta }\right)}^{-1}var\left(\widehat{U}\left(\theta \right)\right)J^{\prime }{\left(\widehat{\theta }\right)}^{-1}$$

(15)

being $J\left(\theta \right)=\frac{1}{N}{\sum }_U\partial u_i/\partial \theta$ and $var\left(\widehat{U}\left(\theta \right)\right)={\sum }_U(1-{\pi }_i^v)u_i^2/{\left({\pi }_i^v\right)}^2$

4. Estimation of a General Parameter with Estimated Propensities

The propensity scores ${\pi }_i^v$ are not known are impossible to estimate using the nonprobability sample $s_v$ alone, so additional information must be included. Let $s_r$ be a reference probability sample, of size $n_r$, selected from $U_T$ under a sampling design $(s_d,p_d)$ where the first order inclusion probabilities, ${\pi }_i^p={\sum }_{s_r\ni i}{p}_d\left(s_r\right),i=1,...,n_r$, are known and non-null.

The covariates of the propensity model $\mathbf{x}$ have been measured both in $s_v$ and $s_r$, while the variable of interest y is only available for those individuals in $s_v$.

Suppose that the propensity scores can be modelled parametrically as

$${\pi }_i^v=P(I_{vi}=1/{\mathbf{x}}_i)=m({\lambda }_o,{\mathbf{x}}_i)i=1,...,N$$

(16)

for some known function $m(·)$ with second continuous derivatives with respect to an unknown parameter ${\lambda }_o$.

We estimate the propensity scores by using data of both the volunteer and the probability sample. The maximum likelihood estimator (MLE) of ${\pi }_i^v$ is $m(\widehat{\lambda },{\mathbf{x}}_i)$ where $\widehat{\lambda }$ corresponds to the value of lambda that maximizes the log-likelihood function:

$$l\left(\lambda \right)=\sum _U(I_{vi}log\left(m(\lambda ,{\mathbf{x}}_i)\right)+(1-I_{vi})log(1-m(\lambda ,{\mathbf{x}}_i))=$$

$$\sum _{s_v}log\frac{m(\lambda ,{\mathbf{x}}_i)}{1-m(\lambda ,{\mathbf{x}}_i)}+\sum _{U}log(1-m(\lambda ,{\mathbf{x}}_i)).$$

(17)

As it is usual in survey sampling, we consider the pseudo-likelihood given that some units of the population have not been sampled:

$$\tilde{l}\left(\lambda \right)=\sum _{s_v}log\frac{m(\lambda ,{\mathbf{x}}_i)}{1-m(\lambda ,{\mathbf{x}}_i)}+\sum _{s_p}\frac{1}{{\pi }_i^p}log(1-m(\lambda ,{\mathbf{x}}_i)).$$

(18)

We propose thus a two phase procedure in this manner:

• Step 1: Calculate ${\widehat{\lambda }}_{pl}$ by solving the score equations:

  $$\partial \tilde{l}({\mathbf{x}}_i,\lambda )/\partial \lambda =0$$
• Step 2: Calculate ${\widehat{\theta }}_v$ as the solution of the estimating function:

  $${\widehat{U}}_V\left(\theta \right)=\sum _U{I}_{vi}{u}_i(y_i,\theta )\frac{1}{m({\widehat{\lambda }}_{pl},{\mathbf{x}}_i)}=0$$

  (19)

We consider the following asymptotic framework for theoretical development, which is equivalent to the framework in [32]. Let $U_{T\nu }$ be a sequence of finite populations of size $N_{\nu }$. Each $U_{T\nu }$ has an associated non-probability sample $s_{v\nu }$ of size $n_{v\nu }$ and an associated probability sample $s_{p\nu }$ of size $n_{p\nu }$. We consider that the population size $N_{\nu }\to \infty$, the nonprobability sample size $n_{v\nu }\to \infty$ and the probability sample size $n_{p\nu }\to \infty$ as $\nu \to \infty$. For notational simplicity the index $\nu$ is suppressed for the rest of the paper. The properties of the estimator ${\widehat{\theta }}_v$ are developed under both the model for the propensity scores and the survey design for the probability sample.

We make the following assumptions:

• A.1. The estimating function $u_i(y_i,\theta ,\lambda )$ is twice differentiable with respect to $\theta$ and $\lambda$.
• A.2. The propensities and the sampling design ensure that ${\widehat{U}}_V\left(\theta \right)-U\left(\theta \right)=O_p\left(n^{-1/2}\right)$ for any $\theta \in \Theta$.
• A.3. The propensities and the sampling design ensure that ${\widehat{U}}_V\left(\theta \right)$ is asymptotically Normal with mean $\mathbf{U}\left(\theta \right)$ and entries of the variance at the order $O\left(n^{-1}\right)$ for any fixed $\theta \in \Theta$.

Theorem 1.

Under the conditions A.1, A.2 and A.3, ${\widehat{\theta }}_v$ is a consistent and asymptotically normal estimator for θ.

Proof. 

Under assumed conditions,

${\widehat{U}}_V\left(\theta \right)=U\left(\theta \right)+O_p\left(n^{-1/2}\right)$, thus by using the mean value theorem, ${\widehat{\theta }}_v$ has the same asymptotic behaviour that $\widehat{\theta }$ which is consistent for $\theta$ and asymptotically normal distributed (see Section 3). □

Variance estimation for ${\widehat{\theta }}_v$ can be handled by combining the two estimating equations, $\tilde{l}$ and ${\widehat{U}}_v$, into a single system as it is done in [33].

The MLE of $\lambda$, ${\widehat{\lambda }}_{pl}$ is the solution to the equations:

$$U_2\left(\lambda \right)=\sum _{s_v}\partial log\frac{m(\lambda ,{\mathbf{x}}_i)}{1-m(\lambda ,{\mathbf{x}}_i)}/\partial \lambda +\sum _{s_p}\partial \frac{1}{{\pi }_i^p}log(1-m(\lambda ,{\mathbf{x}}_i))/\partial \lambda =0$$

and the PSA estimator of ${\theta }_N$ is the solution to the estimating equations

$$U_1(\theta ,\lambda )=\sum _{s_v}{u}_i(y_i,\theta )\frac{1}{m({\lambda }_{pl},{\mathbf{x}}_i)}=\sum _{s_v}{g}_1(y_i,{\mathbf{x}}_i,\theta ,\lambda )=0.$$

Let $\mathbf{U}(\theta ,\lambda )={(U_1^{\prime }(\theta ,\lambda ),U_2^{\prime }\left(\lambda \right))}^{\prime }$. Let $\mathbf{\psi }={({\theta }_N^{\prime },{\lambda }_o)}^{\prime }$ be the true parameter values defined through the census estimating equations and $\widehat{\psi }={({\widehat{\theta }}_N^{\prime },{\widehat{{\lambda }_o}}^{\prime })}^{\prime }$ the solutions to $\mathbf{U}(\theta ,\lambda )=0$.

We need an additional assumption:

• A.4. The propensities, the sampling design and the estimating function satisfy $\partial \widehat{\mathbf{U}}/\partial \mathbf{\psi }=O_p\left(1\right)$ and ${\partial }^2\widehat{\mathbf{U}}/\partial \mathbf{\psi }\partial {\mathbf{\psi }}^{\prime }=O_p\left(1\right)$.

Theorem 2.

Under the conditions A.1, A.2, A.3 and A.4, the asymptotic variance-covariance matrix of $\widehat{\mathbf{\psi }}$ is given by the expression:

$$\mathbf{V}\left(\widehat{\mathbf{\psi }}\right)={\mathbf{H}}^{-1}V\left(\widehat{\mathbf{U}}(\theta ,\lambda )\right){\mathbf{H}}^{\prime -1}$$

(20)

with $\mathbf{H}=\left(\begin{array}{cc}{H}_{11}& H_{12}\\ 0& H_{22}\end{array}\right)$

$$H_{11}=E\left\{\frac{\partial }{\partial \theta }{U}_1({\theta }_N,\lambda )\right\}$$

$$H_{21}=E\left\{\frac{\partial }{\partial \lambda }{U}_1({\theta }_N,\lambda )\right\}$$

$$H_{22}=E\left\{\frac{\partial }{\partial \lambda }{U}_2\left(\lambda \right)\right\}$$

Proof. 

Since ${\widehat{\theta }}_v$ and $\widehat{\lambda }$ are consistent estimator of respective parameters, we can write $\widehat{\mathbf{\psi }}=\mathbf{\psi }+O_p\left(1\right)$ and the Taylor series expansion gives:

$$\widehat{\mathbf{\psi }}=\mathbf{\psi }-{\mathbf{H}}^{-1}\widehat{\mathbf{U}}(\theta ,\lambda )+O_p(\parallel \widehat{\mathbf{\psi }}-\mathbf{\psi }{\parallel }^2),$$

Thus the asymptotic variance of $\widehat{\psi }$ is given by:

$$\mathbf{V}\left(\widehat{\mathbf{\psi }}\right)={\mathbf{H}}^{-1}V\left(\widehat{\mathbf{U}}(\theta ,\lambda )\right){\mathbf{H}}^{\prime -1}.$$

Taking into account the two random mechanisms, and the probabilities of the conditional expectation, we have $V\left(\widehat{\mathbf{U}}(\theta ,\lambda )\right)=V_p{E}_r\left(\widehat{\mathbf{U}}(\theta ,\lambda )\right)+E_p{V}_r\left(\widehat{\mathbf{U}}(\theta ,\lambda )\right)$ where r stands for the model of the selection mechanism for the sample $s_v$ and p refers to the probability sampling design for $s_p$. □

The asymptotic variance of $\widehat{\mathbf{\psi }}$ depends on the probability of selecting the sample $s_p$ under the given sampling design and the selection mechanism described by the propensity model. Plug-in estimators can be used to construct variance estimators for all the required components but it is not a simple issue.

In practice, and as described in [7], the use of jackknife [34] and bootstrap techniques [35] in the variance estimation for nonlinear parameters should be more advantageous because of their wide applicability for different cases and conditions. Direct applications of bootstrap methods for estimating the variance-covariance matrix of $\widehat{\mathbf{\psi }}$ involve solving the equation $\mathbf{U}(\theta ,\lambda )=0$ repeatedly for each bootstrap sample. Multiplier Bootstrap with Estimating Functions was proposed by [36].

5. Simulation Study

5.1. Data

Data for the simulation study come from a wave of the Spanish Living Conditions Survey collected between 2011 and 2012 [37], which contains an annual thematic module that, in 2012, was dedicated to household conditions. The survey sampling follows a two-phase cluster sampling, where the primary units are the households and the secondary units are their members. In 2012, the final sample included 33,573 individuals. For this study, the dataset was filtered to rule out individuals and variables with high quantities of missing data. After this procedure, the dataset employed as pseudopopulation of the study had a size of N = 28,210 individuals and p = 60 available variables.

From this pseudopopulation, two probability samples of size $n_r$ were drawn according to the following sampling strategies:

• The first sample, $s_{r1}$, was drawn with a stratified cluster sampling, where the strata were defined by the Autonomous Communities (NUTS2 regions) and the clusters were the households, which were drawn with probabilities proportional to the household size. The number of households to be selected, m, was estimated dividing $n_r$ by the medium household size in order to reach the aforementioned size of $n_r=2000$, resulting in $m=902$ households. The final sample size of $s_{r1}$ was $n_{r1}=2003$.
• The second sample, $s_{r2}$, was drawn with an unequal probability sampling, where probabilities were proportional to the minimum income of the individual’s household to make ends meet (variable HS130 in [37]).

The extraction of the nonprobability sample, $s_v$, was done with unequal probability sampling from the full pseudopopulation, where the probability of selection for the i-th individual, $p_i$, was given by the formula:

$$p_i=\frac{1}{1+exp(-2x_i^1+0.2x_i^2+0.01x_i^3+0.2x_i^{41}+0.4x_i^{42})}$$

(21)

where

• $x_i^1=1$ when the i-th sampled individual has a computer at home, and $x_i^1=0$ otherwise.
• $x_i^2=1$ when the i-th sampled individual is a man, and $x_i^2=0$ otherwise.
• $x_i^3$ is the age (in years) of the i-th sampled individual.
• $x_i^{41}=1$ when the i-th sampled individual lives in a medium population density area, and $x_i^{41}=0$ otherwise.
• $x_i^{42}=1$ when the i-th sampled individual lives in a low population density area, and $x_i^{42}=0$ otherwise.

The reasoning behind this sampling procedure is to take into account more similar mechanisms to self-selection procedures that take place in real nonprobability surveys.

We have considered three different sample sizes, $n_v$ = 2000, 4000, 6000. 1000 simulation runs were performed for each procedure and sample size, drawing a sample in each run.

5.2. Simulation

In each simulation, the parameters to be estimated were the following:

• The Gini coefficient [38], which measures the income inequality, estimated as

  $${\widehat{G}}_y=\frac{{\sum }_{k\in s_v}\frac{1}{{\pi }_k}(2{\widehat{F}}_y\left(y_k\right)-1)y_k}{{\sum }_{k\in s_v}{y}_k/{\pi }_k}$$
• The proportion of individuals with a disposable income below the at-risk-at-poverty threshold. This measure can be referred to as poverty incidence, poverty proportion, poverty risk or HCI ([39] and is estimated as

  $$\widehat{HCI}=\frac{1}{N}\sum _{k\in s_v}\frac{1}{{\pi }_k}I(y<0.6Q_{0.5})$$
• The interquartile range, estimated as

  $$\widehat{IQR}=\frac{{\widehat{Q}}_{0.75}}{{\widehat{Q}}_{0.25}}$$
• The interdecile range, estimated as

  $$\widehat{IDR}=\frac{{\widehat{Q}}_{0.9}}{{\widehat{Q}}_{0.1}}.$$

Every parameter was estimated with and without applying PSA so we could evaluate its performance. In order to estimate the propensities, a logistic regression model was chosen:

$$m(\widehat{\lambda },{\mathbf{x}}_i)=\frac{exp\left({\widehat{\lambda }}^{\mathsf{T}}{\mathbf{x}}_i\right)}{1+exp\left({\widehat{\lambda }}^{\mathsf{T}}{\mathbf{x}}_i\right)}$$

1000 simulations were executed for each context. The resulting mean bias, standard deviation and Root Mean Square Error were measured in relative numbers to make them comparable across different scenarios. The formulas used for their calculation can be found below:

$$RBias(\%)=\left|\frac{{\sum }_{i=1}^{1000}{\widehat{\theta }}^{\left(i\right)}}{1000}-{\theta }_N\right|·\frac{100}{{\theta }_N}$$

(22)

$$RStandarddeviation(\%)=\sqrt{\frac{{\sum }_{i=1}^{1000}{({\widehat{\theta }}^{\left(i\right)}-\widehat{\overline{\theta }})}^2}{999}}·\frac{100}{{\theta }_N}$$

(23)

$$RMSE(\%)=\sqrt{RBia{s}^2+RS{D}^2}$$

(24)

with ${\widehat{\theta }}^{\left(i\right)}$ the estimation in the i-th simulation and $\widehat{\overline{\theta }}$ the mean of the 1000 estimations.

5.3. Results

The relative mean bias of the estimations can be observed in

Table 1. Relative mean bias (%) of each parameter without applying PSA.

Size	Gini	HCI	IQR	IDR
6000	6.7	80.4	7.9	9.4
2000	3.2	93	3.8	3.1
4000	3.1	86	3.7	3
6000	3	79	3.5	3

,

Table 2. Relative mean bias (%) of each parameter applying PSA with the stratified reference sample.

Size	Gini	HCI	IQR	IDR
2000	1.7	3	2.5	1
4000	2.1	3.3	2.7	1
6000	2.2	3.1	2.7	0.9

and

Table 3. Relative mean bias (%) of each parameter applying PSA with the proportional reference sample.

Size	Gini	HCI	IQR	IDR
2000	0.3	1.1	0	0.2
4000	0.1	1.3	0.1	0.3
6000	0	1.1	0.1	0.5

. We can observe that PSA reduces the bias in all situations, specially in the estimation of HCI. PSA using the reference sample drawn with probabilities proportional to the income, $s_{r2}$, provided much less biased estimates overall.

The relative standard deviation of the estimations can be observed in

Table 4. Relative standard deviation (%) of each parameter without applying PSA.

Size	Gini	HCI	IQR	IDR
2000	1.6	0.2	2.2	4.2
4000	1.1	0.3	1.5	2.9
6000	0.8	0.4	1.2	2.2

,

Table 5. Relative standard deviation (%) of each parameter applying PSA with the stratified reference sample.

Size	Gini	HCI	IQR	IDR
2000	1.7	4.1	2.7	4
4000	1.1	2.8	1.8	2.6
6000	0.9	2.2	1.4	2

and

Table 6. Relative standard deviation (%) of each parameter applying PSA with the proportional reference sample.

Size	Gini	HCI	IQR	IDR
2000	1.3	3.9	2.1	3.2
4000	0.9	2.8	1.5	2.2
6000	0.8	2.3	1.2	2.3

. The standard deviation remained stable across estimates of Gini coefficient, IQR and IDR, even with small gains for the latter when using the reference sample with probabilities proportional to the minimum income to make ends meet, $s_{r2}$, but increased after applying PSA in the estimation of HCI.

The relative Root Mean Square Error of the estimations can be observed in

Table 7. Relative RMSE (%) of each parameter without applying PSA.

Size	Gini	HCI	IQR	IDR
2000	3.6	93	4.4	5.2
4000	3.3	86	4	4.2
6000	3.1	79	3.7	3.7

,

Table 8. Relative RMSE (%) of each parameter applying PSA with the stratified reference sample.

Size	Gini	HCI	IQR	IDR
2000	2.4	5.1	3.7	4.2
4000	2.4	4.3	3.2	2.8
6000	2.4	3.8	3	2.2

and

Table 9. Relative RMSE (%) of each parameter applying PSA with the proportional reference sample.

Size	Gini	HCI	IQR	IDR
2000	1.4	4.1	2.1	3.2
4000	0.9	3.1	1.5	2.2
6000	0.8	2.5	1.2	2.4

. As a result of the stability of standard deviation and the reduction in bias, the RMSE of the estimates of the four parameters has a similar pattern than the observed for bias. Although RMSE is reduced after applying PSA in all cases, PSA was more efficient when the reference sample was drawn with probabilities proportional to the minimum income to make ends meet, $s_{r2}$.

PSA performance could be deeply affected by the selection mechanisms, which could lead to model misspecifications in propensity estimations. To test limitation and robustness of the proposed approach we have repeated the simulation with different patterns of non-response. The selection procedures can be described as follows:

NP1

Simple Random Sampling Without Replacement (SRSWOR) from the population fraction of individuals with a computer at home, $U_v$.

NP2

The probability of selection for the i-th individual, is given by

$$p_i=\frac{1}{1+exp(-2x_i^1+0.2x_i^2+0.01x_i^3+0.2x_i^{41}+0.4x_i^{42})}$$

(25)

NP3

The probability of selection for the i-th individual, is given by

$$p_i={(x_i^3-1925)}^3/{(1995-1925)}^3$$

(26)

NP4

The probability of selection for the i-th individual, is given by

$$p_i=0.35+0.1\ast x_i^1-cos((2012-x_i^3)/5)/3$$

(27)

The procedure 1 is a typical case of coverage error (which is a type of selection bias itself [1]). The third scheme represents a cubic relationship between age and the probability of selection, with young people being the individuals with the highest probabilities and decreasing as age increases. The last scheme has two components: one dichotomous and the other cosine-shaped.

Table 10. Relative mean bias (%) for estimating HCI without and with applying PSA.

	Unadjusted	PSA with Stratified Sample	PSA with Proportional Sample
NP1 2000	93.5	1.7	4.5
NP1 4000	86.9	1.8	4.5
NP1 6000	80.4	1.9	4.5
NP2 2000	93	3	1.1
NP2 4000	86	3.3	1.3
NP2 6000	79	3.1	1.1
NP3 2000	92.9	1.3	1.3
NP3 4000	85.8	0.1	0.2
NP3 6000	78.7	0.5	0.5
NP4 2000	92.8	3	1.4
NP4 4000	85.5	3.2	1.5
NP4 6000	78.3	3.2	1.4

and

Table 11. Relative RMSE(%) for estimating HCI without and with applying PSA.

	Unadjusted	PSA with Stratified Sample	PSA with Proportional Sample
NP1 2000	93.5	3.6	5.3
NP1 4000	86.9	2.8	4.9
NP1 6000	80.4	2.5	4.7
NP2 2000	93	5.1	4.1
NP2 4000	86	4.3	3.1
NP2 6000	79	3.8	2.5
NP3 2000	92.9	9.6	8.6
NP3 4000	85.8	6.4	5.6
NP3 6000	78.7	5	4.3
NP4 2000	92.8	4.7	4
NP4 4000	85.5	4.1	3.1
NP4 6000	78.3	3.6	2.4

show the results of bias and relative ecm for the HCI parameter, where the selection bias of the unweighted estimator is large.

The results show a large decrease in bias and MSE for all response patterns for both PSA methods, which shows the robustness of the adjustment method. The reduction in bias and MSE is different across them. Using PSA with the reference sample drawn under a stratified design, $s_{r1}$, provided less RMSE when the convenience sample was drawn using NP1. On the other hand, PSA using the reference sample drawn with probabilities proportional to the income, $s_{r2}$ provided much less biased estimates overall when the selection mechanism depended on NP2, NP3 or NP4.

6. Conclusions

Technological development has made large amounts of inexpensive data (commonly known as Big Data) available for researchers to be used for inference. New survey administration methods have also favoured the rise of data from nonprobability samples. Inferences from Big Data and nonprobability surveys have important sources of error ([4,24,28], ...). Given the characteristics of these data collection procedures, selection bias is particularly relevant.

Despite the growing interest raised by nonprobability data (both coming from Big Data or nonprobability surveys), there is still a lack of rigorous theory to make statistical inferences for general parameters through estimating equations. The current paper aims to fill this gap by establishing a theoretical framework for estimation of general parameters with nonprobability samples.

Results observed in our simulation study provide strong evidence on the efficiency of methods based in estimating equations with estimated propensities. However, it must be noted that the efficiency depends on the selection mechanisms of nonprobability samples and the availability of covariates for propensity estimation. In our simulations, results showed that Propensity Score Adjustment is more efficient when the propensity of being in the nonprobability sample is less related to the variable of interest. This behavior has been observed in literature regarding PSA for parametric estimation [11,24].

We used parametric methods to obtain the estimated propensities but we could use machine learning techniques as regression trees, spline regression, random forests etc. Recently [24,29] presented simulation studies where decision trees, k-nearest neighbors, Naive Bayes, Random Forest, Gradient Boosting Machine and Model Averaged Neural Networks are used for propensity score estimation. These studies compare the empirical efficiency of the use of linear models and Machine Learning prediction algorithms in estimation of linear parameters, but the theory is more complex and has not yet been developed. Other way to reduce the bias of the PSA estimates is to combine the PSA technique with other techniques as Statistical Matching or calibration. [27] apply a combination of propensity score adjustment and calibration on auxiliary variables in a real volunteer survey aimed to a population for which a complete census was available. [32] propose a doubly robust estimator for population mean estimation by incorporating the model-based estimator framework to PSA methods, improving their efficiency and making it robust to model misspecifications. Further research should focus on extensions of those methods for general parameter estimation.