Imputing Missing Data in One-Shot Devices Using Unsupervised Learning Approach

Abstract

One-shot devices are products that can only be used once. Typical one-shot devices include airbags, fire extinguishers, inflatable life vests, ammo, and handheld flares. Most of them are life-saving products and should be highly reliable in an emergency. Quality control of those productions and predicting their reliabilities over time is critically important. To assess the reliability of the products, manufacturers usually test them in controlled conditions rather than user conditions. We may rely on public datasets that reflect their reliability in actual use, but the datasets often come with missing observations. The experimenter may lose information on covariate readings due to human errors. Traditional missing-data-handling methods may not work well in handling one-shot device data as they only contain their survival statuses. In this research, we propose Multiple Imputation with Unsupervised Learning (MIUL) to impute the missing data using Hierarchical Clustering, k-prototype, and density-based spatial clustering of applications with noise (DBSCAN). Our simulation study shows that MIUL algorithms have superior performance. We also illustrate the method using datasets from the Crash Report Sampling System (CRSS) of the National Highway Traffic Safety Administration (NHTSA).

1. Introduction

One-shot devices are products that can only be used once. For example, airbags deploy during collision. They have to be replaced as they cannot deploy again. EpiPens are auto-injectors that are capable of injecting injections of epinephrine into patients who are experiencing life-threatening allergic reactions. These pens need to be replaced because they cannot be deployed again, obviously. Since most one-shot devices are life-saving, they have to be extremely dependable in times of emergency.

As most products come with warranties and insurance, any small systematic errors in one-shot device production would lead to colossal life and financial losses. The recent bankruptcy of the Takata airbag company is an infamous example of this [1]. The quality control of those products and predicting their reliability over time is critically important. Accelerated Life Tests (ALTs) are popular procedures to assess and analyze the quality of one-shot devices, and they depend heavily on the extrapolation of the life model from the high-stress levels during the experiment to the user conditions. Any biases in the model estimates would be amplified during the extrapolation.

Under parametric settings, the one-shot device data with N observations would have the likelihood function,

$$\begin{array}{c}\hfill L(\theta ;\mathit{x})=\prod _{i=1}^N{\left(F(t_i;\theta ,{\mathit{x}}_i)\right)}^{1-{\delta }_i}{(1-F(t_i;\theta ,{\mathit{x}}_i))}^{{\delta }_i},\end{array}$$

(1)

where $\theta$ is the parameter of the lifetime distribution with cumulative distribution function $F(·)$, ${\mathit{x}}_i$ is the vector of covariates of the ith subject, $\mathit{x}=\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_N\}$, $t_i$ is the observed time when the ith one-shot device is activated and ${\delta }_i$ is the indicator equal to 1 if the device functions correctly.

Manufacturers often carry out these tests before products enter the market, and we rarely test them in actual user conditions. To address this issue, we can refer to publicly available datasets that collect information from actual usage and retrospectively analyze the reliability of those products. For example, the Crash Report Sampling System (CRSS) of the National Highway Traffic Safety Administration (NHTSA) [2] has collected national crash data since 2016. It collects details of car crashes, including the status of airbag deployment. It provides a good data source for evaluating different car safety systems during the actual user conditions. This kind of retrospective study is vitally important as it can indicate potential problems in safety devices early and avoid disastrous replacements similar to the Takada airbag recall, see [1].

In most reliability testing, there are seldom any missing data as the experiments are under well-controlled conditions. However, datasets used in retrospective studies often have missing data. For example, in CRSS, the proportion of missing observations in the datasets is substantial (at least 16.43% in 2017, 13.39% in 2018, 10.00% in 2019 and 11.78% in 2020). This is due to various difficulties in data collection, such as lack of human resources, vehicle conditions or the severity of the accidents. Most popular statistical methods, like multiple imputations, to handle missing data often require the missingness not to depend on the response [3,4,5], which may not be reasonable in a retrospective study. This is because some latent factors not recorded in the datasets may affect both the responses and the missing mechanism. One obvious latent factor is whether the car was parked in a covered space, which is a factor of the airbag lifetime and overall car condition but is not reported in the CRSS. The existence of latent variables leads to heterogeneous data and produces biased imputation results.

Unsupervised learning (UL) techniques such as k-prototype, DBSCAN, and Hierarchical Clustering can effectively discover hidden variables in various domains. In the medical field, refs. [6,7] utilize these methods to classify patients more precisely. They improve the categorization of primary breast cancer and heart failure with maintained ejection fraction by finding separate patient clusters with diverse clinical profiles and outcomes. In engineering, ref. [8] apply the algorithms on railway vibration data to extract useful features for defect detection. UL seems helpful in categorizing the datasets into homogeneous subsets, the observations of which should have similar latent factors.

To analyze the one-shot device data with missing observations and latent factors, we propose using an unsupervised learning algorithm to form clusters, of which the data are considered homogeneous, and a standard statistical imputation technique should be possible. This paper is organized as follows: Section 2 introduces the missing data in one-shot data analysis and the inverse probability weighting (IPW). We review the traditional imputation methods in Section 3. We propose the novel multiple imputations with unsupervised learning (MIUL) in Section 4. We compare the proposed algorithm with some traditional missing data-handling methods in Section 5 and illustrate the usefulness of using CRSS datasets in Section 6. We provide concluding remarks and future research directions in Section 7.

2. Missing Mechanisms and IPW

There are many reasons missing data exist in one-shot device analysis. In particular, when we analyze data from consumers, covariates are often lost for various reasons. For example, customers may not recall certain variables or outcomes, or data entries may be deleted due to human error.

2.1. Missing Data Mechanisms

The missing data mechanism describes how the missingness appears in the datasets. Denote $d_i$ to be the missing indicator for the ith observation, and $d_i=1$ if all the covariates, age of the devices at the instance and the survival statuses are observed, and $d_i=0$ otherwise. We denote $X^{*}$ to be the variables that are completely observed for all data entries, and $X_i^{*}$ is the corresponding value of the ith observation. If $Pr(d_i=0|X_i)=Pr(d_i=0)$ which is a constant, the missing mechanism is referred to as missing completely at random (MCAR); if $Pr(d_i=0|X_i)$ is not constant but independent of the missing values, it is referred to as missing at random (MAR). Otherwise, we say the mechanism is missing not at random (MNAR).

If data are MCAR, analysis based on complete cases remains valid with respect to yielding unbiased results but loses statistical power [9,10,11]. The analysis may yield biased results if data are MAR, but this can be overcome using appropriate statistical methods [12]. If data are MNAR, the probability that a variable value is missing depends on the missing value and cannot be fully explained by the remaining observed variables. In that case, analysis tends to yield biased results if missing data are not appropriately handled, and sensitivity analyses are usually recommended to examine the effect of different assumptions about the missing data. In most cases, the missing mechanism is not MCAR, and handling the missing data properly is essential for valid estimation results.

2.2. Inverse Probability Weighting

Dropping the observations with missing information (i.e., complete case analysis) is generally not recommended as it removes much information and may also introduce biases in the model estimation. Therefore, in the analysis of missing data in the one-shot device with completing risks, extra caution is required to assess the extent and types of missing data before analysis, explore potential mechanisms that contribute to the missing data and use appropriate missing-data strategies to handle the missing data and conduct sensitivity analysis to assess the robustness of research findings.

The inverse probability weighting (IPW) method is a popular statistical approach to handling missing data, see [13]. IPW is a technique that can correct the bias resulting from complete data analysis and is also utilized to adjust for unequal fractions in missing data. IPW addresses this issue by assigning weights to each individual in the analysis based on the inverse of the corresponding probability of being a complete case. These weights balance the distribution of observed data to resemble a distribution with no missing data, thus reducing the bias introduced in a complete-case analysis.

Usually, we denote the non-missing probability as $p_i=p\left({\mathit{x}}_i^{*}\right)$, where ${\mathit{x}}^{*}$ are the covariates observable for all observations and $X_i^{*}$ are the corresponding values for the ith entry. Under the IPW method, the log-likelihood function of one-shot device data is

$$\begin{array}{c}\hfill {\mathcal{l}}^{IPW}(\theta ;\mathit{x})=\sum _{i=1}^N{\displaystyle \frac{d_i}{p_i}}\left((1-{\delta }_i)ln\left(F(t_i;\theta ,{\mathit{x}}_i)\right)+{\delta }_{i}ln(1-F(t_i;\theta ,{\mathit{x}}_i))\right),\end{array}$$

(2)

and the score function is

$$\begin{array}{c}\hfill S_{\theta }\left(\theta \right)={\displaystyle \frac{\partial {\mathcal{l}}^{IPW}(\theta ;\mathit{x})}{\partial \theta }}=\sum _{i=1}^N{\displaystyle \frac{d_i}{p_i}}\left({\displaystyle \frac{1-{\delta }_i}{F(t_i;\theta ,{\mathit{x}}_i)}}-{\displaystyle \frac{{\delta }_i}{1-F(t_i;\theta ,{\mathit{x}}_i)}}\right){\displaystyle \frac{\partial F(t_i;\theta ,{\mathit{x}}_i)}{\partial \theta }}.\end{array}$$

(3)

If we assume that a parametric model for the non-missing probability, $p_i=p({\mathit{x}}_i^{*};\alpha )$, the score function for estimating $p_i$ would be

$$\begin{array}{c}\hfill S_{\alpha }\left(\alpha \right)=\sum _{i=1}^N\left({\displaystyle \frac{d_i}{p({\mathit{x}}_i^{*};\alpha )}}-{\displaystyle \frac{1-d_i}{1-p({\mathit{x}}_i^{*};\alpha )}}\right){\displaystyle \frac{\partial p({\mathit{x}}_i^{*};\alpha )}{\partial \alpha }},\end{array}$$

(4)

where $S_{\alpha }\left(\widehat{\alpha }\right)=0$ and ${\widehat{p}}_i=p({\mathit{x}}_i^{*},\widehat{\alpha })$ is the corresponding estimate.

3. Literature Review on Imputation

Besides the IPW method, imputation strategies provide alternative ways to handle missing data. They are more intuitive as they fill out the missing data and give “complete datasets”. Researchers can use the datasets directly without complicated mathematical formulas. Here are a few popular methods for imputing missing data.

3.1. Single Imputations

3.1.1. Mean (or Mode) Imputation

Imputation is a common strategy for dealing with missing data. It fills in the blanks with an appropriate value to create a “completed” dataset that can be analyzed using traditional statistical procedures. The most direct and intuitive strategy for continuous variables is to replace missing observations with the mean of the observed sections, known as mean imputation, see [14]. Similarly, unobserved categorical and ordinal variables could be substituted by the mode of the observed ones (mode imputation). However, because this method does not take into account association across variables [15,16], it also produces biased and over-fitting results [17]. In Figure 1, we show how to perform a mean imputation. The variable Region represents the region of a car accident, the variable Age represents the age of the car involved, and the variable Success? represents whether the airbags of the car function properly.

Figure 1. Illustration on how to perform a mean imputation. The highlighted entries are the missing observations, and the red numbers represent the values imputed by the mean imputation. The table on the left is complete data, and the table in the middle is the data with missing observations. The table on the right has the missing data imputed by the mean of observed values.

3.1.2. Expectation Maximization Imputations

The expectation maximization (EM) imputation is a type of single imputation technique using the EM algorithm, and it is extensively studied by various literature [5,18]. Refs. [19,20] also consider using EM for the masked causes of one-shot devices. The EM algorithm is an iterative procedure of expectation (E-steps) and maximization steps (M-steps). In one iteration, the expected values of the missing responses or covariates would be updated as the mean conditional on the observed covariates, such as the components’ manufacturer and current stress levels (E-step). The complete data formulas will then be applied to the dataset with missing data filled from the E-step to obtain the updated estimates by maximizing the complete likelihood function (M-step). The E-step and M-step will be performed iteratively until the imputed values converge. To illustrate how EM imputation works, we use the data example in Figure 1. We assume the $Age$ follows the exponential distribution with rate = $\lambda$ and the probability of Success, $S=1$, given the $Age$ of the car is

$$P(S=1)=\left\{\begin{array}{cc}\hfill \beta ,& Age\le 6\hfill \\ \hfill 0.5\beta ,& Age>6\hfill \end{array}\right.,for0<\beta <1.$$

The observed likelihood function is

$$L(\lambda ,\beta )=0.5{\beta }^3{(1-\beta )}^2(1-0.5\beta ){\lambda }^{7}exp(-33.5\lambda )(1-0.5exp(-6\lambda ))\beta ,$$

and the complete log-likelihood is

$$\begin{array}{cc}\hfill {\mathcal{l}}_c\left(\lambda ,\beta \right)=& \left(1+I\left(Ag{e}_5>6\right)\right)ln0.5+4\beta +2ln\left(1-\beta \right)+ln\left(1-0.5\beta \right)\hfill \\ \hfill & +8ln\lambda -\left(33.5+Ag{e}_5\right)\lambda \hfill \end{array}$$

with relevant expected values given the current parameter estimates ${\lambda }^{\left(t\right)}$ and ${\beta }^{\left(t\right)}$ being

$$E\left(I\left(Ag{e}_5>6\right)\right)=exp\left(-{6\lambda }^{\left(t\right)}\right)andE\left(Ag{e}_5\right)=1/{\lambda }^{\left(t\right)}.$$

We then maximize $Q_t(\lambda ,\beta )=E\left({\mathcal{l}}_c\left(\lambda ,\beta \right)|{\lambda }^{\left(t\right)},{\beta }^{\left(t\right)}\right)$ to obtain the next parameter estimates, ${\lambda }^{(t+1)}$ and ${\beta }^{(t+1)}$. Figure 2 illustrates how the EM imputation can be implemented.

Figure 2. An illustration on how we implement EM algorithm imputation. The highlighted entries are the missing observations, and the red numbers represent the values imputed during the imputation procedure. The initial parameter estimates are ${\lambda }^{\left(0\right)}=0.1,{\beta }^{\left(0\right)}=0.5$. We impute the missing data with the expectation based on the current parameter estimates, ${\lambda }^{\left(t\right)},{\beta }^{\left(t\right)}$.

3.2. Multiple Imputations

Multiple imputations have been considered as a gold standard to account for missing data, and more specifically, the fully conditional specification (FCS) method (also labeled the multiple imputations by chained equations (MICE) algorithm) [21]. The multiple imputations by FCS impute multivariate missing data on a variable-by-variable basis, which is particularly useful for studies with large datasets and complex data structures, see [22]. It requires a specification of an imputation model for each incomplete variable and iteratively creates imputations per variable [23]. Compared to a single imputation, MI replaces each missing value with multiple plausible values, allowing uncertainty about the missing data to be considered. MI consists of two stages. The first stage involves creating multiple imputed datasets. In practice, we can first apply Box-Cox transformation on the numerical covariates such that they follow a multivariate normal distribution approximately. The MI method imputes the missing responses or the covariates stochastically based on the distribution of the observed data. We will repeat the process five to ten times to create ten imputed datasets. Once again, we use the data example in Figure 1 to show how multiple imputation works. We assume the variable Age has a linear relationship with the variables Region and Success and the log-odds of Success are linearly related to the variables Region and Age. We first impute the missing data with the average of observed values. Then, we form a regression model on the variable under imputation and predict the missing value with the model. If the variable is numerical, we often use the predictive mean matching method, which imputes the missing entries with the closest observed values to the predictions. For binary variables, we can perform a logistic regression and draw Bernoulli random variables with the predicted probabilities to impute the missing variables. Figure 3 shows how multiple imputation works step by step. For brevity reasons, we do not consider the model coefficient uncertainty in this example.

Figure 3. An illustration of how the multiple imputation procedure works. The highlighted entries are the missing observations, and the red numbers represent the values imputed during the procedure. In this example, we assume there is no model coefficient uncertainty.

In the second stage, we will analyze each of the five or ten imputed datasets separately using standard statistical models and then combine the results from the five or ten analyses to report the conclusion; see [24]. One popular choice is to use Rubin’s Rules [25,26] to combine the estimates ${\tilde{\beta }}^{\left(m\right)},m=1,\dots ,M$ from multiple imputed datasets, and obtain the the pooled estimate and variance,

$$\begin{array}{c}\hfill \widehat{\beta }={\displaystyle \frac{1}{M}}\sum _{i=1}^M{\tilde{\beta }}^{\left(m\right)}\mathrm{a}\mathrm{>n}\mathrm{>d}{V}_{Total}=V_W+V_B+V_B/M,\end{array}$$

respectively, where

$$\begin{array}{c}\hfill V_W={\displaystyle \frac{1}{M}}\sum _{i=1}^M{\left(SE\left({\tilde{\beta }}^{\left(m\right)}\right)\right)}^2\mathrm{>a}\mathrm{>n}\mathrm{>d}{V}_B={\displaystyle \frac{1}{M-1}}\sum _{i=1}^M{\left({\tilde{\beta }}^{\left(m\right)}-\widehat{\beta }\right)}^2,\end{array}$$

$SE\left({\tilde{\beta }}^{\left(m\right)}\right)$ is the standard error of of the estimate from the m imputed dataset.

4. Multiple Imputation with Unsupervised Learning

This research proposes new imputation methods to handle missing data in one-shot devices that hybridize multiple imputations with unsupervised learning (MIUL). Traditional imputation assumes the data are from a homogenous group, which is only sometimes valid in the actual ALTs or observational studies on a one-shot device. The components of devices/systems may be coupled in the manufacturing process or assembly, so the components within the device may have interrelationships, leading to data with latent heterogeneity and dependence, which can be described by frailty models [27,28].

It is well-known that the latent variables will affect the model coefficients when they are correlated with the observed variables. Since the latent variables are not observed, an unsupervised ML algorithm would be a powerful way to discover hidden clusters of the devices based on the observed variable [29]. We cluster one-shot devices into groups with different characteristics using machine-learning techniques. Within each group, their latent factors should be similar to each other. Therefore, the unknown latent factors are “controlled”, and the regression model parameters would be correctly adjusted. We can apply several conventional unsupervised clustering techniques to the observed covariates to discover hidden structures. For example, the k-means clustering technique is a popular method for clustering analysis in data mining. It is an unsupervised ML method partitioning the dataset into k clusters according to the distance of each observation to the cluster means. Once we cluster the data, we can impute the data using the following Algorithm 1.

Algorithm 1: Multiple imputations with unsupervised learning (MIUL)

This algorithm describes our proposed methods: multiple imputations with unsupervised learning (MIUL). Ref. [30] also discuss similar ideas, but they mainly focus on imputing datasets for clustering, while this research study focuses on clustering the datasets for imputation. There are several popular unsupervised learning algorithms, and they are listed below.

4.1. Hierarchical Clustering

Hierarchical Clustering is a technique that creates an ordered sequence of data clusters using a specific dissimilarity measure [31]. The concept of grouping hierarchically was first introduced by [32]. The term “Hierarchical Clustering” was first coined by [33], who discussed the approaches and the choice of distance measure. Ref. [34] gives further details on how Hierarchical Clustering can be done. It is also an unsupervised ML method that tries to build a hierarchy of clusters. In the agglomerative approach, each observation is first treated as a distinct cluster. These single observation clusters gradually merge depending on the smallest dissimilarity. Given the distance measure between two data points $X_i,X_{i^{\prime }}$ is $dist(X_i,X_{i^{\prime }})$, there are three common dissimilarity measures between two clusters [35]: suppose G and H are two groups during the clustering process:

Single linkage (SL) dissimilarity,

$$d_{SL}(G,H)=\underset{\{X_i\in G,X_{i^{\prime }}\in H\}}{min}dist(X_i,X_{i^{\prime }}),$$

complete linkage dissimilarity,

$$d_{CL}(G,H)=\underset{\{X_i\in G,X_{i^{\prime }}\in H\}}{max}dist(X_i,X_{i^{\prime }}),$$

and group average (GA),

$$d_{GA}(G,H)={\displaystyle \frac{1}{N_G{N}_H}}\sum _{X_i\in G}\sum _{X_{i^{\prime }}\in H}dist(X_i,X_{i^{\prime }}),$$

where $N_G$ and $N_H$ are the numbers of data points in Groups G and H, respectively. Similarly, in the divisive approach, all the observations are treated as one cluster and split into clusters based on dissimilarity until each observation is treated as a distinct cluster. Both methods create multi-tiered hierarchies as the procedure goes on. They span the extremes of each observation existing as their cluster and all observations combining into a single, all-encompassing cluster. A dendrogram is a visual representation of the cluster-merging process. The process can stop at k clusters if the average dissimilarity between a pair of clusters changes substantially. Factors like many irrelevant variables may obscure clusters, as they only exist within a small subset of variables. Thus, despite seeming close due to relevant variables, the curse of dimensionality can create a vast distance between observations in a high-dimensional space, which challenges the effectiveness of dissimilarity measures and the accuracy of the resulting clusters.

4.2. K-Prototype

K-prototype was initially proposed by [36]. The term “prototypes” refers to the centroids or representative points of clusters in a dataset. It extends the well-known k-means clustering algorithm to the categorical variables or attributes with mixed numerical and categorical values. The K-means clustering algorithm first normalizes each variable of the ith observation, $X_i^{*}=(x_{i1}^{*},\cdots ,x_{ij}^{*},\cdots ,x_{iJ}^{*})$, by

$$x_{ij}={\displaystyle \frac{x_{ij}^{*}-min(x_{1j}^{*},\cdots ,x_{Ij}^{*})}{max(x_{1j}^{*},\cdots ,x_{Ij}^{*})-min(x_{1j}^{*},\cdots ,x_{Ij}^{*})}},$$

where $X_i=(x_{i1},\cdots ,x_{ij},\cdots ,x_{iJ})$ It, then, partitions numerical dataset variables $\mathit{X}=\{X_1,X_2,\cdots ,X_n\}$ into k clusters by minimizing the within groups sum of squared error (WGSS),

$$\begin{array}{c}\hfill P(W,\mathit{Q})=\sum _{l=1}^k\sum _{i=1}^n{w}_{i,l}dist(X_i,Q_l),\end{array}$$

(5)

where $w_{i,l}\in \{0,1\},{\sum }_{l=1}^k{w}_{i,l}=1,1\le i\le n$, is the indicator of the ith observation being in the lth cluster, $Q_l$, $\mathit{Q}=\{Q_1,Q_2,\dots ,Q_k\}$ is a set of observations in the same cluster, and $dist(X_i,Q_l)$ is the squared Euclidean distance between $X_i$ and the cluster center of $Q_l$,

$$\begin{array}{c}\hfill dis{t}_1(X_i,Q_l)=\sum _{j=1}^p{(x_{i,j}-q_{l,j})}^2,\end{array}$$

(6)

where $x_{i,j}$ is the value of the jth variable of the ith observations and there are p numerical variables. We adopt the elbow method to determine k, which gives the reduction of minimized WGSS less than 10%.

However, datasets often contain both numerical and categorical variables. Suppose the first p variables are numerical and the remaining $m-p$ variables are categorical in a dataset. The k-prototype algorithm replaces $d(X_i,Q_l)$ by the following dissimilarity measure,

$$\begin{array}{c}\hfill dis{t}_2(X_i,Q_l)=\sum _{j=1}^p{(x_{i,j}-q_{l,j})}^2+\gamma \sum _{j=p+1}^m\delta (x_{i,j},q_{l,j}),\end{array}$$

(7)

where

$$q_{l,j}=\left\{\begin{array}{cc}{\displaystyle \frac{1}{N}}\sum _{i\in Q_l}{x}_{i,j},\hfill & \mathrm{for} 1\le j\le p\hfill \\ c_{i,j},\hfill & \mathrm{for} p+1\le j\le m\hfill \end{array}\right.,$$

and $c_{l,j}$ is the most frequent category of jth variable in the cluster. $Q_l$ and $\delta (x_{i,j},c_{l,j})=1$ if $x_{i,j}=q_{l,j}$ and zero otherwise. The weight $\gamma$ is chosen to balance the effects of numerical and categorical attributes. Details on how to select $\gamma$ can be found in [37]. Ref. [38] reduces the misclassification of data points near the boundary region by considering the distribution centroids for the categorical variables in a cluster, and [39] further improves it by proposing a new dissimilarity measure between data objects and cluster centers.

4.3. Density-Based Spatial Clustering of Application with Noise

Density-Based Spatial Clustering of Application with Noise (DBSCAN) was originally proposed by [40]. The algorithm is uniquely designed to discover clusters of arbitrary shape, and it requires minimal domain knowledge to determine input parameters with very general assumptions [41]. It is efficient for large databases, making it a practical choice for various applications. The effectiveness and efficiency of DBSCAN were evaluated using synthetic and real data.

The concept of density-based clusters is central to the operation of DBSCAN. The Eps-neighborhood of a point p in the dataset D is

$$N_{Eps}\left(p\right)=\{q\in D|dist(p,q)\le Eps\},$$

where $dist(p,q)$ is a distance measure between points p and q. If, $p\in N_{Eps}\left(q\right)$ and the neighborhood size, $|N_{Eps}\left(q\right)|$ is greater than a preset minimum number of points, $MinPts$, then p is directly density-reachable to q. We also define p as density-reachable from q if there is a chain of points $p_1=p,p_2,\dots p_n=q$ which are directly density-reachable consecutively. p and q are density-connected if they are both density-reachable from a common point o. The cluster here is defined as the set of points that are density-reachable from each other (maximality) and density-connected (connectivity). The set of points not belonging to any cluster are considered to be noise. Ref. [42] addresses the issue of detecting the cluster in data of varying densities by assigning the noise points to the closest eligible cluster. Ref. [43] revisits the DBSCAN algorithm and discusses how to obtain the appropriate parameters for DBSCAN. Ref. [44] extends the algorithm to produce DBSCAN clusters in a nested, hierarchical way. Ref. [45] implements the hierarchical DBSCAN in Python that gives the best DBSCAN parameters.

4.4. Gower’s Distance

Most dissimilarity measures for numerical variables are based on the Euclidean distance, the square root of (6). If discrete variables exist, the k-prototype algorithm adopts the dissimilarity measure states in (7). However, both of them require the variables to be completely observed. We may use Gower’s distance proposed by [46] to handle partially observed data entries. It provides another similarity measure between the ith and kth observations,

$$\begin{array}{c}\hfill dis{t}_G(i,k)={\displaystyle \frac{{\sum }_{j=1}^m{w}_j{d}_{ik}^{\left(j\right)}dis{t}_{ik}^{\left(j\right)}}{{\sum }_{j=1}^m{w}_j{d}_{ik}^{\left(j\right)}}},\end{array}$$

(8)

where $d_{ik}^{\left(j\right)}$ is the indicator if the jth variable is observed for both the ith and kth data entries, $dis{t}_{ik}^{\left(j\right)}$ is the distance measure of the jth variable between ith and kth observations, $x_i^{\left(j\right)}$ and $x_k^{\left(j\right)}$. If jth variable is binary or nominal, $dis{t}_{ik}^{\left(j\right)}=1$ if $x_i^{\left(j\right)}\ne x_k^{\left(j\right)}$ and 0 otherwise. If the jth variable is numerical or ordinal, $dis{t}_{ik}^{\left(j\right)}$ would be the absolute difference, standardized by the range $|x_i^{\left(j\right)}-x_k^{\left(j\right)}|/range\left(x_i^{\left(j\right)}\right)$. The weight for the jth variable, $w_j$, indicating the importance of the variable, is usually set to one, based on the Adansonian principle of equal weighting, see [47].

5. Simulation Study

5.1. Simulation Settings

We develop a simulation study to compare various methods when handling missing data in one-shot devices. Suppose we are monitoring the quality of airbags in the United States. For simplicity, we assume that the airbags have exponential lifetimes with the hazard rate, $\lambda \left(\mathit{x}\right),$

$$f(t_i;{\mathit{x}}_i)=\lambda \left({\mathit{x}}_i\right)exp(-\lambda \left({\mathit{x}}_i\right)t_i),\mathrm{for} t_i>0,$$

where ${\mathit{x}}_i$ is the variables that are associated with the airbag quality, the car make (A, B, and C), the car registration region (Northeast, Midwest, South, and West) and parking location (indoor or outdoor). To ensure the value is positive, we adopt a log-linear function.

$$\begin{array}{c}\hfill {\lambda }_{actual}\left({\mathit{x}}_i\right)=exp({\beta }_0+{\beta }_{make}+{\beta }_{region}+{\beta }_{park}),\end{array}$$

(9)

where ${\beta }_0$ is the baseline log-hazard for all airbags, equal to −6, −5 and −4 representing high, medium and low reliability (Relb.) levels of the devices, around 62%, 82% and 93%, respectively. ${\beta }_{make}^B=-0.4$ and ${\beta }_{make}^C=-0.8$ correspond to the log-hazard rates of airbags from the car makes B and C relative to A. $({\beta }_{region}^{MW},{\beta }_{region}^S,{\beta }_{region}^W)=(-0.6,0.4,0.8)$ represents the log-hazard rates of cars from Midwest, South and West relative to those from Northeast, accordingly.

The effect of outdoor parking is assumed to be ${\beta }_{park}^{out}=0.6$ relative to parking inside. Parking location is quite possibly related to various factors, namely, car owner’s location (urban or rural), car body type (sedan, Sport Utility Vehicles and others), whether the car is in the West region, and whether the driver is an alcohol drinker. Here, we assume the probability of outdoor parking is linearly related to the factors mentioned. Parking location is unlikely to be recorded, and we consider it a hidden variable. Therefore, the regression equation would be limited to

$$\begin{array}{c}\hfill {\lambda }_{model}\left({\mathit{x}}_i\right)=exp({\beta }_0+{\beta }_{make}+{\beta }_{region}),\end{array}$$

(10)

which resembles the model misspecification in actual modeling.

Compared to other reliability experiments, this is an observational study, and some data are inevitably missed during the data collection. We simulate two different types of missing indicators. One is unrelated to the one-shot device response, nor any hidden variable corresponding to the MAR mechanism. We also generate another missing indicator related to the hidden variable, parking location, representing the MNAR scenarios. IPW uses logistic regression with all the fully observed variables to model the probability of non-missing. MI and all the MIUL methods use all the variables in the datasets for imputation. Details of the parameter setting can be found in the Appendix A.

In the simulation, we consider the scenarios with 1000, 2000, and 4000 cars we examined, representing small, medium, and large datasets. We also consider three missingness levels: around 10%, 15%, and 25%. The simulation study aims to analyze car makes’ coefficients to see which one has potential problems.

5.2. The Competitors’ Details

5.2.1. Traditional Methods

In the simulation, we compare six missing handling methods. The first three are mean imputation (Mean.I), IPW and MI, representing traditional and popular missing handling methods.

In Mean.I, we impute the missing numerical observations by the average of the observed variables. If the variables are categorical, it would be intuitive to use the mode (the most frequently observed categorical values). However, it always imputes the survival status as “success” as the one-shot devices are highly reliable. To have a more sensible imputation, we impute categorical variables by linear discrimination analysis, for which linear equations are formed based on other variables to discriminate the categorical responses.

For IPW, we use all the fully observed variables to perform logistic regression on non-missing indicators (1- the missing indicator of the missing variable). We then predict the probabilities of non-missingness, $p_i$ and weight the score function by $1/p_i$ when we estimate the parameters for a one-shot device.

For MI, we use the default setting of mice() function from the mice package [48] in R program [49]. By default, the program creates linear equations for imputing missing data. Numerical variables are imputed by the predictive mean matching method (which gives the closest observed values to the missing value prediction based on a linear model). It also imputes the categorical values by the prediction from logistic regression, multinomial regression and proportional odds model regression for binary, nominal and ordinal variables, respectively. We then estimate the parameters, ${\tilde{\beta }}_{make}^{\left(m\right)}$ for the mth imputed data. We then use Rubin’s Rules [25,26] mentioned in Section 3.2 to obtain the estimate and variance.

5.2.2. Our Proposed Methods

The other three methods are based on our proposed method MIUL with the unsupervised learning algorithms, k-prototype (K.MI), DBSCAN with Gower’s distance (DB.MI) and Hierarchical Clustering with Gower’s distance (HC.MI). Before clustering, we calculate the distance between data points by daisy function in the cluster package [50].

For k-prototype, we use the kproto function from the clustMixType package by [51]. We first group the data with the number of clusters $k=3$ and then keep increasing the cluster numbers until the reduction of the within-cluster sum of squares is less than 10%.

For DBscan, we modify the suggestion by [52] and select the $Eps$ considering the distance from each point to its third-nearest neighbor and define the minimum data points for a cluster, $MinPts$ as 50. We, then, use the dbscan function fpc package by [53] to perform DBSCAN clustering.

For Hierarchical Clustering, we use the hclust from the stats package by [49] and we cut trees to form five clusters.

After clustering, we use the MI methods mentioned above to impute data within each cluster. We combine the clusters as imputed datasets and follow Rubin’s Rules again to infer the estimation.

5.3. Simulation Results

To evaluate the traditional methods and our proposed methods, we compare the biases and the mean squared error (MSE) of the car make coefficient ${\beta }_{make}$, which are defined as

$$\begin{array}{c}\hfill Bias={\displaystyle \frac{1}{n}}\sum _{i=1}^n({\widehat{\beta }}_{make,i}-{\beta }_{make}),MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{({\widehat{\beta }}_{make,i}-{\beta }_{make})}^2,\end{array}$$

(11)

for the simulation size n, which is 1000 in our simulation study. We have six scenarios focusing on different missing data cases.

Scenario 1 focuses on the response variable, the survival statuses of one-shot devices, being missing at random. Table A2, Table A3, Table A4 and Table A5 show the bias and mean squared error of ${\widehat{\beta }}_{make}^B$ and ${\widehat{\beta }}_{make}^C$ when the response variable is missing at random. Although mean imputation generally has lower biases when the one-shot devices have higher reliability, our proposed methods become more accurate when we have more failure responses. The reason is quite apparent: the majority of the devices are successful and the mean imputation imputes the missing response as a success, which is the mode of the response. As a result, the imputed responses are most likely in favor of the success case and result in worse biases. When the reliability is high, the multiple imputations with unsupervised learning (MIUL) are generally better. However, when we consider the mean squared errors of ${\widehat{\beta }}_{make}^B$ and ${\widehat{\beta }}_{make}^C$, the traditional statistical methods do not have any advantage. The DB.MI performs better when the product reliability is high, while the K.MI works better when the product reliability is high.

Scenario 2 considers the survival statuses of one-shot devices being MNAR due to hidden variables. The simulation results are presented in Table A6, Table A7, Table A8 and Table A9. Considering the biases, they have similar patterns to the previous scenario. The mean imputation has smaller biases with low product reliability, while the MIUL algorithm gains an advantage when product reliability decreases. For MSE, MIUL algorithms generally perform better and, in addition, K.MI has superior performance when product reliability is low.

Scenario 3 considers the age of one-shot devices being missing at random. The simulation results are presented in Table A10, Table A11, Table A12 and Table A13. Similar to Scenarios 1 and 2, Mean.I has the most negligible bias when product reliability is low. However, as the reliability becomes high, DB.MI’s performance suppresses other methods and HC.MI comes next. When we look at MSE, DB.MI has the smallest and HC.MI has the second smallest. This indicates that DB.MI and HC.MI outperforms K.MI when the missing variable is numeric instead of categorical.

Scenario 4 considers the age of one-shot devices being missing not at random due to the hidden variable and Table A14, Table A15, Table A16 and Table A17 summarize the results. Here, we have a similar conclusion as Scenario 3, that Mean.I has a low bias when the reliability is low. DB.MI works best regarding bias and MSE when the reliability is high and HC.MI is the second best. Occasionally, MI has less MSE compared to others.

Scenario 5 represents the case when the covariate Region is MAR and the simulation results are shown in Table A18, Table A19, Table A20 and Table A21. The mean imputation usually has the most insignificant biases when the product reliability is low. However, our proposed MIUL methods work relatively well for most cases and HC.MI is slightly better than other MIUL methods.

Scenario 6 represents the case when the covariate Region is missing not at random and we present the results in Table A22, Table A23, Table A24 and Table A25. Again, the Mean.I has the most negligible bias when the reliability of the one-shot device is low, while MIUL methods, especially HC.MI, have better performance in bias for low-reliability products. When we look at the MSE, DB.MI seems the best when the reliability is low to medium, while MI performs pretty well when the reliability is low.

The simulation shows that MIUL algorithms consistently outperform the traditional approach, namely the mean imputation, IPW and MI approaches for products. This tendency is more obvious when the products have a medium or high reliability. This is because high-reliability products produce fewer failure cases. Then, traditional methods tend to weight success cases higher and create biases as they cannot observe the latent variables in the dataset.

With MIUL, the clustering of the observations attempts to “discover” the latent variables and gives more flexible equations for imputing the missing variable. The target parameters ${\beta }_{make}^B$ and ${\beta }_{make}^C$ are estimated accurately with different variables under various missing mechanisms. Generally speaking, when the survival statuses of one-shot devices are missing, K.MI is the best; when the ages of one-shot devices are missing, both DB.MI and HC.MI work well; when a covariate is missing, HC.MI works well in most cases. This gives HC.MI the best overall performance among all MIUL algorithms.

To summarize, MIUL algorithms suppress the traditional missing data-handling methods, especially when the product reliability is medium to high, which is valid for most one-shot devices. The missing mechanism does not impact their relative performance much, suggesting that we prefer MIUL most of the time.

6. Applications with the CRSS Data

6.1. CRSS Datasets

In this section, we illustrate the application of the MIUL algorithm on the Crash Report Sampling System (CRSS) datasets 2016–2020. National Highway Traffic Safety Administration (NHTSA) developed and implemented CRSS to reduce the motor vehicle crash experience in the United States [54]. It is an annual survey designed independent of other NHTSA surveys. It is a valuable instrument for comprehending and analyzing crash data, yielding vital insights that can be used to enhance road safety. For simplicity reasons, we ignore the complex sample design and treat the data as independent in this illustration.

6.2. Defining the Airbag Success in CRSS Data

In CRSS, the airbag variable indicates when the airbag deploys or not during the accident. It does not explicitly imply whether the airbags function correctly. Here, we make a few modifications to the variable. First, we define the situation when the airbags should be deployed. It is reasonable to assume that an airbag should be deployed when the driver or passengers are critically injured. After all, airbags are designed to reduce the severity of injury. As the airbag sensor is at the front and due to the size of a car, the airbag should be deployed if the areas are impacted. We also assume the airbag should be deployed if the car has to be towed after the accident, as the collision is severe. We present the details of how such a a situation is exactly defined in Appendix B. Then, we can define the airbags as functioning correctly during the accident if the airbags deploy when they should or they do not deploy when they should not. Figure 4 presents the airbag success rate by manufacturing year for different accident years. The success rates are around 95%, which is reasonable. Each manufacturing year’s airbag success rates are consistent with each other for different accident years. Thus, we can conclude that such a definition for airbag success looks reasonable.

Figure 4. Airbag success rate by manufacturing year for the accidents that happened between 2017 and 2020.

6.3. Data Analysis

To simplify the modeling process, we use the same model (10) in the simulation study and we limit our target to comparing the airbag’s reliability of the car makes’ origins, namely, America, Asia and Europe. Throughout the accident years 2017–2020, the missing rates are 16.43%, 13.39%, 10.00% and 11.78%, respectively, and they are close to medium to low missing levels in the simulation study. Excluding the observations with missing variables, the success rate of the airbags is about 95%, as shown in Figure 4, confirming that airbags are highly reliable products. The CRSS generally publishes around 95,000 observations each year. Due to technical and resource limitations, we usually work on a dataset with sizes smaller than CRSS. In our illustration, we only sample 1000 (small), 2000 (medium) and 4000 (large) from the CRSS dataset. Then, we repeat this process 1000 times and record the estimates of the log hazard rate of airbags in Asian and European cars relative to American cars. There are two reasons for the sub-sampling. First, as the sample size increases, while there are some hidden variables that we cannot observe, the biases would increase as sample size increases which can be observed in the simulation study above. Therefore, we sample observations from large datasets and see if the estimate averages vary for different sample sizes. Second, it would allow a more straightforward confidence interval calculation using the bootstrap method while fewer computational resources are required. It also resembles the standard airbag tests of particular makes and models, as the sample sizes rarely exceed 4000. We report the means, standard errors (SEs) and the 95% confidence intervals (lower limit, CI.L and upper limit, CI.U) in Table 1 and Table 2.

Table 1. The mean, standard error (SE), 95% confidence interval lower and upper limits (CI.L, CI.U) estimates of ${\beta }_{make}^{Asian}$ under various imputation methods with bootstrap samples of 1000 (Sml), 2000 (Med) and 4000 (Lrg).

			Methods
Year	Size	Stat	Mean.I	IPW	MI	K.MI	DB.MI	HC.MI
2017	Sml	Mean	−0.00520	−0.00883	−0.01023	0.00636	−0.01036	−0.01041
		SE	0.35721	0.35774	0.35517	0.32478	0.35749	0.35748
		CI.L	−0.73138	−0.74240	−0.74146	−0.65902	−0.73862	−0.73884
		CI.U	0.70197	0.68470	0.69647	0.62163	0.68764	0.68775
	Med	Mean	0.01484	0.01049	0.00930	0.01770	0.01030	0.01026
		SE	0.23614	0.24019	0.23639	0.22898	0.23549	0.23552
		CI.L	−0.45937	−0.47115	−0.46515	−0.43956	−0.46083	−0.46396
		CI.U	0.46983	0.46826	0.45999	0.44759	0.46399	0.46300
	Lrg	Mean	0.02493	0.02057	0.01909	0.02714	0.02097	0.02092
		SE	0.16560	0.16903	0.16707	0.16564	0.16536	0.16541
		CI.L	−0.29847	−0.31457	−0.31544	−0.31185	−0.31176	−0.31219
		CI.U	0.33600	0.33924	0.33859	0.34427	0.33320	0.33312
2018	Sml	Mean	0.01970	0.02723	0.01685	0.02073	0.01631	0.01623
		SE	0.33288	0.33681	0.33263	0.30876	0.33149	0.33135
		CI.L	−0.63370	−0.62636	−0.63507	−0.57207	−0.63620	−0.63582
		CI.U	0.65528	0.67321	0.66805	0.58658	0.64048	0.64060
	Med	Mean	0.03063	0.03812	0.03144	0.03497	0.02886	0.02874
		SE	0.23144	0.23279	0.22925	0.22562	0.23077	0.23081
		CI.L	−0.41830	−0.40815	−0.42225	−0.41337	−0.41929	−0.41835
		CI.U	0.48970	0.49713	0.48528	0.47663	0.47902	0.47678
	Lrg	Mean	0.01974	0.02741	0.01967	0.02583	0.01745	0.01735
		SE	0.16032	0.16143	0.15951	0.15919	0.15976	0.15975
		CI.L	−0.30033	−0.29550	−0.30266	−0.30400	−0.30021	−0.30070
		CI.U	0.32769	0.34210	0.31599	0.33177	0.32383	0.32292
2019	Sml	Mean	0.02322	0.02289	0.02726	0.02150	0.02531	0.02537
		SE	0.30592	0.30570	0.30348	0.29186	0.30543	0.30532
		CI.L	−0.56986	−0.57100	−0.57995	−0.54529	−0.56919	−0.56984
		CI.U	0.60067	0.60591	0.61962	0.58495	0.60904	0.60940
	Med	Mean	0.03005	0.02988	0.03560	0.03289	0.03252	0.03247
		SE	0.22268	0.22280	0.22270	0.21788	0.22174	0.22169
		CI.L	−0.41365	−0.41014	−0.39257	−0.40485	−0.39461	−0.39340
		CI.U	0.45068	0.44856	0.46157	0.44709	0.45015	0.45006
	Lrg	Mean	0.03713	0.03695	0.04328	0.04274	0.03957	0.03952
		SE	0.14095	0.14145	0.14193	0.14064	0.14053	0.14055
		CI.L	−0.23866	−0.25076	−0.23560	−0.22885	−0.23287	−0.23235
		CI.U	0.31660	0.31528	0.32764	0.32541	0.32208	0.32310
2020	Sml	Mean	−0.05207	−0.05195	−0.04961	−0.04693	−0.04882	−0.04900
		SE	0.32679	0.32686	0.32499	0.31155	0.32559	0.32565
		CI.L	−0.75252	−0.75609	−0.72875	−0.71641	−0.73368	−0.73715
		CI.U	0.62621	0.61703	0.58905	0.55351	0.60640	0.61026
	Med	Mean	−0.05094	−0.05105	−0.04930	−0.04889	−0.04853	−0.04860
		SE	0.23119	0.23157	0.23300	0.22611	0.23053	0.23046
		CI.L	−0.49147	−0.49482	−0.50387	−0.47786	−0.48809	−0.48786
		CI.U	0.39535	0.40019	0.40279	0.40522	0.39801	0.39755
	Lrg	Mean	−0.06173	−0.06173	−0.06056	−0.05789	−0.05855	−0.05867
		SE	0.15746	0.15778	0.15726	0.15485	0.15631	0.15630
		CI.L	−0.40157	−0.40060	−0.39700	−0.39529	−0.38936	−0.38874
		CI.U	0.25195	0.25543	0.24474	0.23698	0.24169	0.23980

Table 2. The mean, standard error (SE), 95% confidence interval lower and upper limits (CI.L, CI.U) estimates of ${\beta }_{make}^{European}$ under various imputation methods with bootstrap samples of 1000 (Sml), 2000 (Med) and 4000 (Lrg).

			Methods
Year	Size	Stat	Mean.I	IPW	MI	K.MI	DB.MI	HC.MI
2017	Sml	Mean	−0.02011	−0.03289	0.31172	0.70079	0.17739	0.17519
		SE	1.80428	1.86420	1.14748	0.56135	1.46658	1.46665
		CL.L	−8.06343	−8.22431	−1.48400	−0.39785	−1.84854	−1.84319
		CL.U	1.42867	1.53936	1.54589	1.62922	1.49423	1.48724
	Med	Mean	0.36758	0.38603	0.49093	0.67242	0.44369	0.44238
		SE	0.53164	0.59659	0.43307	0.37890	0.43006	0.43005
		CL.L	−0.64688	−0.86281	−0.41189	−0.12741	−0.46466	−0.48352
		CL.U	1.20380	1.25691	1.25447	1.34870	1.17280	1.17846
	Lrg	Mean	0.38532	0.41606	0.48753	0.60236	0.45673	0.45434
		SE	0.29940	0.34218	0.28816	0.27733	0.28529	0.28575
		CL.L	−0.24345	−0.32304	−0.13967	−0.01447	−0.15739	−0.15425
		CL.U	0.90793	1.03841	0.97901	1.08562	0.95691	0.95491
2018	Sml	Mean	0.16647	0.15027	0.35415	0.62988	0.25679	0.25582
		SE	1.34332	1.37701	0.92730	0.58264	1.13752	1.13778
		CL.L	−1.36720	−1.77539	−1.06060	−0.48064	−1.17999	−1.22472
		CL.U	1.38871	1.41256	1.41676	1.52952	1.36457	1.36863
	Med	Mean	0.36699	0.36244	0.45730	0.60685	0.41407	0.41211
		SE	0.41562	0.43450	0.40177	0.36590	0.39731	0.39717
		CL.L	−0.47720	−0.51849	−0.41055	−0.11368	−0.42388	−0.41376
		CL.U	1.07151	1.07604	1.15827	1.29320	1.10838	1.10896
	Lrg	Mean	0.37497	0.37219	0.45642	0.54508	0.41874	0.41656
		SE	0.28811	0.30315	0.27793	0.26883	0.27054	0.27107
		CL.L	−0.24697	−0.29888	−0.16556	−0.04139	−0.18245	−0.18405
		CL.U	0.86209	0.89625	0.94845	1.01856	0.89849	0.90448
2019	Sml	Mean	0.12799	0.14751	0.31003	0.59799	0.23611	0.23435
		SE	1.12483	1.14024	0.89727	0.50208	0.97898	0.97823
		CL.L	−1.28459	−1.37239	−1.07873	−0.47576	−1.17632	−1.18715
		CL.U	1.24107	1.27065	1.30010	1.43291	1.24504	1.24685
	Med	Mean	0.25313	0.27137	0.36703	0.51753	0.33018	0.32845
		SE	0.40623	0.41212	0.37021	0.34687	0.36915	0.36883
		CL.L	−0.62547	−0.55190	−0.41421	−0.19964	−0.43137	−0.42785
		CL.U	0.93457	0.94905	1.01682	1.14174	0.97396	0.97342
	Lrg	Mean	0.30675	0.32608	0.41438	0.50307	0.38252	0.37995
		SE	0.24617	0.24904	0.23975	0.23126	0.23244	0.23235
		CL.L	−0.21820	−0.18353	−0.07694	0.02680	−0.08056	−0.08262
		CL.U	0.75815	0.79900	0.85556	0.90354	0.79991	0.80472
2020	Sml	Mean	0.14460	0.16218	0.36551	0.64016	0.28710	0.28645
		SE	1.07676	1.08771	0.73292	0.56255	0.82508	0.82700
		CL.L	−1.23825	−1.31103	−0.84157	−0.40260	−0.96034	−1.03690
		CL.U	1.31753	1.31334	1.37122	1.51630	1.27246	1.29429
	Med	Mean	0.27679	0.29828	0.40339	0.57873	0.36809	0.36792
		SE	0.41222	0.41826	0.38554	0.36151	0.37847	0.37922
		CL.L	−0.61672	−0.55402	−0.40512	−0.14819	−0.46587	−0.47197
		CL.U	0.97036	0.99896	1.03570	1.20528	1.02671	1.01124
	Lrg	Mean	0.30138	0.32174	0.41495	0.52914	0.38813	0.38716
		SE	0.27387	0.27711	0.25597	0.25260	0.24996	0.24982
		CL.L	−0.28459	−0.27273	−0.14651	0.01791	−0.11541	−0.12547
		CL.U	0.78358	0.81176	0.88905	0.98116	0.83944	0.83161

Table 1 presents the estimates of ${\beta }_{make}^{Asian}$ under various imputation methods. The estimates are consistent for various sample sizes, indicating that a sample size of 2000 to 4000 should be enough for the parameter estimation. All the ${\widehat{\beta }}_{make}^{Asian}$ are most likely around zero. Therefore, we can safely claim that the airbag motility of Asian brands is pretty much similar to that of American ones.

Next, we present the estimates of ${\beta }_{make}^{European}$ in Table 2. The estimates increase when the sample size increases for traditional methods, while the estimates from the proposed MIUL methods remain relatively stable. This suggests that the MIUL methods are more robust to the sample sizes. The CIs of ${\widehat{\beta }}_{make}^{European}$ do not cover the zero under K.MI when the sample size is large for accident years 2019 and 2020 with the corresponding Wald statistics ${\displaystyle \frac{0.503}{0.231}}=2.175$ (p-value $=0.030$) and ${\displaystyle \frac{0.529}{0.253}}=2.095$ (p-value $=0.036$). Since K.MI usually has the most negligible bias and MSE, especially when the sample size and product reliability are high with medium to low missing levels, the quality of airbags of European manufacturers is likely significantly worse than the American ones, requiring further investigation.

7. Conclusions and Discussions

Retrospective studies are important in reliability analysis as they measure the actual lifetimes of the one-shot devices under the actual users’ conditions. They can detect if the one-shot devices are designed and manufactured correctly and give early warnings if there are some systematic defaults in one-shot devices. This is crucial as most safety and life-saving products are one-shot devices. Early detection of potential flaws can save a lot of lives and casualties. In this study, we have proposed a retrospective way to analyze the reliability of one-shot devices through publicly available data.

Different from the usual reliability experiment, those datasets are not collected in a controlled environment, and therefore missing observations are inevitable. The traditional statistical methods may not handle the missing data properly since the observations may not be missing at random. With hidden variables, like parking locations and maintenance habits, those methods may not work well as the model cannot be specified correctly.

When machine learning algorithms are applied to impute the missing data, unsupervised learning may be useful to impute the missing observations. Still, it is not intuitive how the imputation can be carried out. Our study proposed an innovative way for missing data imputation using unsupervised learning, and it works reasonably well when hidden variables are present in the dataset under a missing-not-at-random assumption. With an accurate imputation strategy, retrospective studies on one-shot devices become possible.

Using a simulation study, we have shown that the MIUL methods perform superior to the traditional methods in the context of one-shot device reliability evaluation. We illustrate the methods using the CRSS datasets provided by NHTSA. Under a definition of airbag success, we find out that the airbags made by European cars may be significantly worse than those made by American manufacturers.

The CRSS datasets are collected through a complex survey design, which this research ignores for simplicity reasons. Therefore, in future studies, we could incorporate the survey design structure in the MIUL, enhancing the estimation accuracy. We could also extend the unsupervised learning part to more advanced algorithms like auto-encoder, and the self-organizing map, which may provide more precise results. It would also be interesting to see their estimation performance when the one-shot devices have Weibull and gamma lifetimes with frailty, which are more realistic than the exponential distribution.

This manuscript also provides a potential way to detect possible manufacturing issues with one-shot devices using public data. It would be interesting to regularly track the number of airbag failures and report on any defective models. One method is to monitor failures using control charts. Control charts are commonly used in quality control to identify underlying problems in industrial processes; see [55,56,57]. Modifying control charts for public datasets would be an intriguing issue worthy of more investigation.

Appendix A.1. One-Shot Device Model Setting

The reliability function of one-shot devices given covariate ${\mathit{x}}_i$:

$$R\left(t\right|\mathit{x})=Pr(T>t|\mathit{x})=exp(-t{\lambda }_{actual}\left(\mathit{x}\right))$$

The actual hazard rates of one-shot devices:

$${\lambda }_{actual}\left(\mathit{x}\right)=exp({\beta }_0+{\beta }_{make}+{\beta }_{region}+{\beta }_{park}),$$

where ${\beta }_0=-6,-5$ and $-4$ represent high, medium and low reliability of the one-shot devices, respectively, $({\beta }_{make}^B,{\beta }_{make}^C)=(-0.4,-0.8)$, $({\beta }_{region}^{MW},{\beta }_{region}^S,{\beta }_{region}^W)=(-0.6,0.4,0.8)$ and ${\beta }_{park}=0.6$.

The probability of parking outside is

$$Pr(Park=Out|\mathit{x})=0.2+0.2\ast I_{Rural}+0.2\ast I_{West}-0.1\ast I_{Sedan}+0.2\ast I_{Alcohol},$$

where the meanings of indicators are $I_{Rural},Pr(I_{Rural}=1)=23.5\%$, the car is in a rural area, $I_{West},Pr(I_{West}=1)=16.9\%$, the car is in the West region, $I_{Sedan},Pr(I_{Sedan}=1)=47.5\%$, the car is a Sedan, $I_{Alcohol},Pr(I_{Alcohol}=1)=6\%$, the driver is an alcohol drinker. We also assume that the driver’s age following gamma distribution with shape and rate parameters is $(\alpha ,\beta )=(20.01,0.83)$, respectively, if the cars are parked outdoors or $(\alpha ,\beta )=(12.04,0.24)$ otherwise. Finally, Car_Age, the car age at the time of the accident, follows the following multinomial distribution.

Table A1. The distribution of car age at the time of the accident in the simulation study.

Car_Age	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5	9.5	10.5	11.5	12.5	13.5	14.5	15.5	16.5
Prob.	0.36%	5.02%	7.65%	7.39%	7.42%	7.22%	4.94%	4.44%	4.08%	3.95%	4.11%	4.31%	4.61%	4.88%	4.64%	4.24%	3.79%
Car_Age	17.5	18.5	19.5	20.5	21.5	22.5	23.5	24.5	25.5	26.5	27.5	28.5	29.5	30.5	31.5	32.5	33.5
Prob.	3.37%	2.87%	2.36%	1.93%	1.56%	1.18%	0.88%	0.66%	0.47%	0.33%	0.28%	0.21%	0.16%	0.13%	0.11%	0.07%	0.06%
Car_Age	34.5	35.5	36.5	37.5	38.5	39.5	40.5	41.5	42.5	43.5	44.5	45.5	46.5	47.5	48.5	49.5	50+
Prob.	0.06%	0.04%	0.03%	0.02%	0.03%	0.02%	0.02%	0.02%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%

Table A1. The distribution of car age at the time of the accident in the simulation study.

Car_Age	0.5	1.5	2.5	3.5	4.5	5.5	6.5	7.5	8.5	9.5	10.5	11.5	12.5	13.5	14.5	15.5	16.5
Prob.	0.36%	5.02%	7.65%	7.39%	7.42%	7.22%	4.94%	4.44%	4.08%	3.95%	4.11%	4.31%	4.61%	4.88%	4.64%	4.24%	3.79%
Car_Age	17.5	18.5	19.5	20.5	21.5	22.5	23.5	24.5	25.5	26.5	27.5	28.5	29.5	30.5	31.5	32.5	33.5
Prob.	3.37%	2.87%	2.36%	1.93%	1.56%	1.18%	0.88%	0.66%	0.47%	0.33%	0.28%	0.21%	0.16%	0.13%	0.11%	0.07%	0.06%
Car_Age	34.5	35.5	36.5	37.5	38.5	39.5	40.5	41.5	42.5	43.5	44.5	45.5	46.5	47.5	48.5	49.5	50+
Prob.	0.06%	0.04%	0.03%	0.02%	0.03%	0.02%	0.02%	0.02%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%	0.01%

Appendix A.2. The Missing Mechanism for Different Scenarios

Scenario 1 in Section 5 stands for the survival statuses of one-shot devices being MAR,

$$Pr(I_{Mis{s}_i}=1)={\alpha }_0+0.05\ast I_{Other}+0.01\ast log(Car\_Age),$$

where ${\alpha }_0=0.05,0.1$ and $0.2$ represent low, medium and high missing levels, respectively, and the variable $I_{Other},Pr(I_{Other}=1)=32.7\%$ indicates that the car is not a Sedan nor a Utility Vehicle.

Scenario 2 in Section 5 stands for the survival statuses of one-shot devices being MNAR,

$$Pr(I_{Mis{s}_i}=1)={\alpha }_0+0.05\ast I_{Other}+0.01\ast log(Car\_Age)+0.1\ast I_{Out},$$

where ${\alpha }_0=0.05,0.1$ and $0.2$ represent low, medium and high missing levels, respectively, and the indicator $I_{Other}=1$ if the car is not a Sedan nor a utility vehicle and $I_{Out}=1$ if the car is parked outdoors.

Scenario 3 in Section 5 stands for the $Car\text{\_}Age$ being MAR,

$$Pr(I_{Mis{s}_i}=1)={\alpha }_0+0.01\ast I_{South}+0.05\ast I_{Rural},$$

where ${\alpha }_0=0.05,0.1$ and $0.2$ represent low, medium and high missing levels, respectively, and the indicators, $I_{South},Pr(I_{South}=1)=53.6\%$, the accident location in the South region and $I_{Rural}$ the car is in a rural area.

Scenario 4 in Section 5 stands for the $Car\text{\_}Age$ being MNAR,

$$Pr(I_{Mis{s}_i}=1)={\alpha }_0+0.01\ast I_{South}+0.05\ast I_{Rural}+0.1\ast I_{Out},$$

where ${\alpha }_0=0.05,0.1$ and $0.2$ represent low, medium and high missing levels, respectively, and the indicators $I_{South}=1$ if the accident is located in the South region, $I_{Rural}=1$ if the accident location is in a rural area and $I_{Out}=1$ if the car is parked outdoors.

Scenario 5 in Section 5 stands for the $Region$ being MAR,

$$Pr(I_{Mis{s}_i}=1)={\alpha }_0+0.05\ast I_{Rural},$$

where ${\alpha }_0=0.05,0.1$ and $0.2$ represent low, medium and high missing levels, respectively, and the indicator $I_{Rural}=1$ means the car is in a rural area.

Scenario 6 in Section 5 stands for the $Region$ being MNAR,

$$Pr(I_{Mis{s}_i}=1)={\alpha }_0+0.01\ast I_{South}+0.05\ast I_{Rural}+0.1\ast I_{Out},$$

where ${\alpha }_0=0.05,0.1$ and $0.2$ represent low, medium, and high missing levels, respectively, and the meanings of indicators are $I_{South}=1$ if the accident location is in the South region, $I_{Rural}=1$ if the car is in a rural area and $I_{Out}=1$ if the car is parked outdoors.