Finite Mixture Models: A Key Tool for Reliability Analyses

Abstract

As system complexity increases, accurately capturing true system reliability becomes increasingly challenging. Rather than relying on exact analytical solutions, it is often more practical to use approximations based on observed time-to-failure data. Finite mixture models provide a flexible framework for approximating arbitrary probability density functions and are well suited for reliability modelling. A critical factor in achieving accurate approximations is the choice of parameter estimation algorithm. The REBMIX&EM algorithm, implemented in the rebmix R package, generally performs well but struggles when components of the finite mixture model overlap. To address this issue, we revisit key steps of the REBMIX algorithm and propose improvements. With these improvements, we derive parameter estimators for finite mixture models based on three parametric families commonly applied in reliability analysis: lognormal, gamma, and Weibull. We conduct a comprehensive simulation study across four system configurations, using lognormal, gamma, and Weibull distributions with varying parameters as system component time-to-failure distributions. Performance is benchmarked against five widely used R packages for finite mixture modelling. The results confirm that our proposal improves both estimation accuracy and computational efficiency, consistently outperforming existing packages. We also demonstrate that finite mixture models can approximate analytical reliability solutions with fewer components than the actual number of system components. Our proposals are also validated using a practical example from Backblaze hard drive data. All improvements are included in the open-source rebmix R package, with complete source code provided to support the broader adoption of the R programming language in reliability analysis.

1. Introduction

Estimating system reliability is a complex yet essential task [1,2,3]. It is crucial because system reliability often plays a key role in further analyses, such as calculating the mean time to failure (MTTF) [1,3], determining remaining useful life (RUL) [4], assessing hazard rates [5,6], conducting safety evaluations [7,8], and so on. Accurate reliability estimation enables better decision-making and enhances safety and longevity.

In a complex system consisting of several components, reliability estimation typically involves several steps [9,10]. These include collecting relevant time-to-failure data sets, selecting suitable parametric families for the component time-to-failure distributions, estimating their parameters and finally deriving the overall system reliability model. The application of these steps can be quite challenging even for smaller systems [8]. In addition, in order to obtain meaningful estimates, various assumptions often need to be made that simplify the derivation of the final, more sophisticated reliability models.

On the other hand, surrogate modelling has gained significant popularity for reliability estimation [11,12]. In probabilistic modelling, system reliability is often represented using exponential or Weibull parametric families [5,13]. However, such oversimplifications typically result in poor estimates that are inadequate for complex systems [14,15]. One of the gold-standard methods for reliability estimation is finite mixture modelling [14,16,17,18]. The applicability of finite mixture models (FMMs) for reliability data is well established in the literature, either in the separation of system failure modes [14,19] or component-level heterogeneity [16,17,18]. Additionally, this semi-parametric approach is capable of modelling a wide range of random phenomena beyond reliability data [20,21,22,23,24]. While FMMs are a part of probabilistic modelling, their inherent properties also make them useful for uncovering additional meaningful insights about the systems being analysed [14,19].

Despite significant progress, the development of a suitable parameter estimation algorithm for different FMMs remains elusive. Research on normal FMMs, where the normal parametric family serves as the component distribution, is abundant and includes well-established parameter estimation algorithms along with software implementations [25]. In contrast, similar studies addressing FMMs with alternative parametric families, such as lognormal, gamma, or Weibull, are lacking, likely due to their limited use and the practical difficulties associated with their derivation and implementation [26,27,28]. Nonetheless, these parametric families are particularly relevant in reliability analysis, as they often provide a better fit for modelling the time-to-failure distributions of system components [16,29,30,31]. Furthermore, readily available software implementations of these FMMs are often lacking or, at the very least, remain unknown to the wider public. Addressing this gap is essential for the wider adoption of FMMs in practical reliability studies, particularly where the flexibility to incorporate diverse parametric families as component distributions is critical for accurately capturing system behaviour.

The Expectation–Maximisation (EM) algorithm is widely considered the most promising approach in terms of accuracy and computational efficiency for estimating parameters in FMMs [18]. However, since the EM algorithm requires proper initialisation, recent studies have highlighted that the Rough-Enhanced-Bayes (REBMIX) algorithm can also effectively tackle this issue, albeit with certain limitations [25,32]. More specifically, recent findings in [25] suggest that REBMIX struggles when components in the FMM are highly overlapping. To address these challenges, we briefly review the REBMIX algorithm and propose improvements to its rough component parameter estimation and Bayesian mixture parameter optimisation steps. Furthermore, we derive comprehensive parameter estimation formulas for both the EM and REBMIX algorithms for lognormal, Weibull, and gamma parametric families, and we provide ready-to-use software implementations within the rebmix R package. We also review other popular R packages that can be used to estimate parameters of lognormal, Weibull, and gamma FMMs. All corresponding source code is included as Supplementary Material to ensure full reproducibility of this work.

Finally, to obtain conclusive benchmarks, we designed a comprehensive simulation study involving four different system configurations. For each configuration, the exact system reliability and corresponding probability density function were analytically derived. The lognormal, gamma, and Weibull parametric families were used to model the time-to-failure distributions of individual system components. The parameters for each component’s distribution were generated to ensure significant overlap between them. This process was repeated for 20 different random seeds to thoroughly evaluate: (a) the performance of the improved REBMIX&EM parameter estimation algorithm, (b) its benchmarking against state-of-the-art estimation methods implemented in freely available R packages, and (c) the effectiveness of different FMMs in approximating the true system reliability.

The remainder of this paper is organized as follows. Section 2 presents all the improvements made to the parameter estimation procedures. Section 3 outlines the simulations. Section 4 discusses the results, while Section 5 gives the practical real-life example. Section 6 provides the concluding remarks. Appendix A contain the derivations of the EM and REBMIX parameter estimation methods for the lognormal, Weibull and gamma FMMs.

2. Methodology

In this section, recent improvements to the REBMIX algorithm are presented, mainly inspired by the findings in [25]. Chassagnol et al. [25] demonstrated that REBMIX initialisation provides the most accurate estimates when working with well-separated components or a small number of components with clearly distinguishable modes. Furthermore, the study indicated that for data sets with highly overlapping components, k-means [33] or random initialisation [34] gives more accurate results. The advances presented in this article are intended to demonstrate improved performance in cases with moderately overlapping components as well as increased accuracy and calculation speed. The scope of this study is restricted to lognormal, Weibull and gamma FMMs.

2.1. Lognormal, Weibull and Gamma FMMs

Let $y_1,\dots ,y_j,\dots ,y_n$ be an observed $d=1$ dimensional data set of size n of continuous observations. Each observation is assumed to follow the predictive FMM density

$$f\left(y\right|c,\mathit{w},\mathbf{\Theta })=\sum _{l=1}^c{w}_{l}f\left(y\right|{\mathsf{\theta }}_l).$$

(1)

The objective of the analysis is to obtain the number of components c, the component weights $w_l$, which sum to 1, and the component parameters ${\mathsf{\theta }}_l$. It is assumed that the component time-to-failure probability densities come from the lognormal

$$\mathrm{lognormal}\left(y\right|{\mathsf{\theta }}_l)={\displaystyle \frac{1}{\sqrt{2\pi }{\sigma }_{l}y}}exp\left\{-{\displaystyle \frac{1}{2}}{\displaystyle \frac{{(log\left(y\right)-{\mu }_l)}^2}{{\sigma }_l^2}}\right\},$$

(2)

Weibull

$$\mathrm{Weibull}\left(y\right|{\mathsf{\theta }}_l)={\displaystyle \frac{{\beta }_l}{{\theta }_l}}{\left({\displaystyle \frac{y}{{\theta }_l}}\right)}^{{\beta }_l-1}exp\left\{-{\left({\displaystyle \frac{y}{{\theta }_l}}\right)}^{{\beta }_l}\right\}\mathrm{or}$$

(3)

gamma

$$\mathrm{gamma}\left(y\right|{\mathsf{\theta }}_l)={\displaystyle \frac{1}{\Gamma \left[{\beta }_l\right]y}}{\left({\displaystyle \frac{y}{{\theta }_l}}\right)}^{{\beta }_l}exp\left\{-{\displaystyle \frac{y}{{\theta }_l}}\right\}$$

(4)

parametric family.

2.2. Parameter Estimation

The REBMIX algorithm [35] was originally developed as a sequential method in which the components are estimated individually. The algorithm is composed of several steps, which are shown in Figure 1. However, a significant limitation is the inability to reach the optimal solution. Thus, recent studies [32] propose to merge the algorithm with the EM algorithm [36] to form REBMIX&EM.

In this case, REBMIX first provides estimates for the number of components c, the weights $\mathit{w}$ and the parameters $\mathbf{\Theta }$, which are usually close to the optimum. The EM algorithm then refines these estimates to converge to the optimal solution.

Nevertheless, the introduction of the EM algorithm into the estimation procedure did not solve the problems of optimising the spurious optima with overlapping components in the FMM. We have found that the main problems lie in the rough parameter estimation step and the Bayesian mixture parameter optimisation step, which we will discuss in more detail in the following text.

2.2.1. Rough Parameter Estimation

The rough parameter estimation phase is the most important step in the REBMIX algorithm. In this phase, rough parameters for the lth component are estimated based on the global mode position $y_m$ and its probability density $f_{lm}$. These parameters are then used to cluster observations between the lth component and the residue, as detailed in [35,37]. Unlike algorithms such as k-means, which depends exclusively on the positions of observations and their distances, REBMIX utilizes additional information from the data set. It not only considers the positions of observations but also identifies the global mode, a distinctive feature of the data set. The determination of the empirical densities $f_{lj}$ by histogram preprocessing or external binning of the observations into $j=1,\dots ,v$ nonempty bins also substantially reduces computational time, especially for large data sets. For small data sets, kernel density estimation or k-nearest neighbour may be preferable for preprocessing the observations.

However, a fundamental problem, illustrated in Figure 2, remains unresolved. Figure 2 presents three plots, where rough component parameter estimation is depicted on three different estimation cases, namely a FMM with non-overlapping components (the plot on the left), moderately (the middle plot) and heavy overlapping components (the plot on the right). In all three plots, the residue is shown in green, the lth component in red and the FMM probability density as a black line. The global mode is indicated by a black circle, while the mode of the true lth component is indicated by a red circle. The true residue probability density at the global mode is shown by a green circle. The left and right global minima are represented by a black triangle pointing upwards and a triangle pointing downwards, respectively. The rough probability density of the lth component is depicted by a thin red solid line, and the loose probability density of the lth component is shown by a thin blue line.

The empirical density of the global mode $f_{lm}$ belongs exclusively to the lth component only if the components are well separated (see red circle in left plot of Figure 2), which overlaps the black circle. This is generally not the case for moderately or heavily overlapping components, as shown in the middle and right plots of Figure 2. In these cases, the red and black circles are misaligned. The approach proposed in [38], which is implemented in the package rebmix version 2.15.0 and older remains accurate only under conditions where components are well separated, as shown in the left plot of the Figure 2, or if a single component of the residue contributes to $f_{lm}$, as shown in the middle plot of the Figure 2. Cases where multiple components contribute to the lth component, as illustrated in the right plot of the Figure 2, may lead to estimation errors in $f_{lm}$. According to [38], the wider component of the residue in the middle plot of the Figure 2 (mean $\mu =75$, standard deviation $\sigma =7$) is extracted before the dominant lth narrower component ($y_m=\mu =80$, $\sigma =2$), which is suboptimal. Ideally, the dominant component in red, centred on $y_m$, should be extracted first. The refined rough parameter estimation, which addresses the the aforementioned issues, is outlined below.

After preprocessing or external binning of observations, the empirical probability densities $f_{lj}$ (black solid line) are obtained, from which the global mode position $y_m$ and the corresponding value $f_{lm}$ can be readily identified. The true component probability density $f\left(y_m\right|{\mathsf{\theta }}_l)$, marked by the red circle, and the true probability density of the residue

$$f_{\mathrm{res}}\left(y_m\right|c,\mathit{w},\mathbf{\Theta })=f\left(y_m\right|c,\mathit{w},\mathbf{\Theta })-w_{l}f\left(y_m\right|{\mathsf{\theta }}_l),$$

(5)

marked by the green circle, are not available at this stage. Assume that the weight of the lth component is $w_l=1$ and that $f\left(y_m\right|{\mathsf{\theta }}_{l\mathrm{rigid}})=f_{lm\mathrm{rigid}}=f_{lm}$. In other words, the thin red line representing the envelope under which the entire lth component is believed to be contained can be calculated and visualised. The component parameters ${\mathsf{\theta }}_{l\mathrm{rigid}}$ of the red thin lines in Figure 2 were connected to rigid restraints for the first time in [38]. It was further demonstrated in [38] that the rigid restraints alone are not sufficient to accurately restore the components of the residue when they overlap. Furthermore, an iterative procedure for defining the rough component parameters based on loose restraints was proposed in [38].

If we carefully observe the area between the red thin lines and the black lines in Figure 2, we can see that there is a global minimum $y_a$ (triangle pointing upwards) to the left of $y_m$ and another $y_b$ (triangle pointing downwards) to the right of the global mode. At the beginning let $y_a=y_b=y_m$ or equivalently $a=b=m$. Then

$$\begin{array}{ccc}\hfill a& =& \underset{j}{argmin}{f}_{lj}\mathrm{subject}\mathrm{to}f\left(y_j\right|{\mathsf{\theta }}_{l\mathrm{rigid}})>f_{lj}\mathrm{and}{y}_j<y_m,\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill b& =& \underset{j}{argmin}{f}_{lj}\mathrm{subject}\mathrm{to}f\left(y_j\right|{\mathsf{\theta }}_{l\mathrm{rigid}})>f_{lj}\mathrm{and}{y}_j>y_m.\hfill \end{array}$$

(7)

The sum of the differences between the empirical densities and the corresponding probability densities of the lth component

$$\begin{array}{cc}\hfill W_l& =h\sum _{j=1}^v\left(f_{lj}-f\left(y_j\right|{\mathsf{\theta }}_{l\mathrm{rigid}})\right)\hfill \\ \hfill & =1-\sum _{j=1}^{v}f\left(y_j\right|{\mathsf{\theta }}_{l\mathrm{rigid}})h\hfill \end{array}$$

(8)

is also calculated, where h stands for the bin widths. Next, the number $n_{\mathrm{res}}$ and the density $f_{\mathrm{res}}$ are both initialised to zero. If $W_l\le 0$ and $y_a<y_m$, $n_{\mathrm{res}}$ is incremented and $f_{\mathrm{res}}$ is increased by $f_{la}$. Similarly, if $W_l\le 0$ and $y_b>y_m$, $n_{\mathrm{res}}=n_{\mathrm{res}}+1$ and $f_{\mathrm{res}}=f_{\mathrm{res}}+f_{lb}$. Finally, if $n_{\mathrm{res}}>0$, $f_{\mathrm{res}}=f_{\mathrm{res}}/n_{\mathrm{res}}$. The loose restraints then lead to

$$f_{lm\mathrm{loose}}=max\{f_{lm\mathrm{rigid}}-f_{\mathrm{res}},f_{\mathrm{res}}\},$$

(9)

from which the loose rough component parameters, ${\mathsf{\theta }}_{l\mathrm{loose}}$, are calculated as explained in Appendix A.1. For simpler notation, let $f_{lm}=f_{lm\mathrm{loose}}$ and ${\mathsf{\theta }}_l={\mathsf{\theta }}_{l\mathrm{loose}}$. In Figure 2, the probability densities of the lth component associated with the loose restraints are shown as thin blue lines.

Equation (8) is valid only for non-overlapping bins. For kernel density estimation, the histogram is constructed with one bin aligned with $y_m$. Similarly, for k-nearest neighbour, the histogram is constructed with one bin aligned with $y_m$. Here, $h=2R_{\mathrm{m}}$, where $R_{\mathrm{m}}$ denotes the mean Euclidean distance to the first non-coincident nearest neighbour. Further details on the estimation of the rough component parameters can be found in [35,37].

2.2.2. Bayesian Mixture Parameter Optimisation

The REBMIX algorithm clusters an observed data set of size n into c clusters of sizes $n_1,\dots ,n_c$ associated with component probability densities $f\left(y_j\right|{\mathsf{\theta }}_l)$ and component weights $w_l=n_l/n$. The clustering is designed to minimise both the sum of the positive relative deviations and the information criterion. As a result, the sum of the component weights satisfies ${\sum }_{l=1}^c{w}_l\le 1$, meaning that some observations may remain unassigned.

At the beginning, the number of outliers N is set to zero and the initial first moments $m_l$ and second moments $V_l$ are taken from Appendix A.3. The remaining observations are then allocated to the existing components based on the Bayes decision rule

$$l=\underset{l}{argmax}{w}_{l}f\left(y_j\right|{\mathsf{\theta }}_l).$$

(10)

For each observation, it is also determined whether it is an outlier. Outliers are identified as those that fall into the left or right tail of the probability density with a probability of $p=0.0001$. If the observation is recognised as an outlier, then $N=N+k_{lj}$. Otherwise

$$w_l=w_l+{\displaystyle \frac{k_{lj}}{n}},$$

(11)

$$m_l=m_l+{\displaystyle \frac{k_{lj}(y_j-m_l)}{n{w}_l}},$$

(12)

$$V_l=V_l+{\displaystyle \frac{k_{lj}(y_j^2-V_l)}{n{w}_l}}.$$

(13)

This ensures that the outliers do not influence $m_l$ and $V_l$. A detailed explanation of the frequencies $k_{lj}$ associated with the densities $f_{lj}$ can be found in [35].

After processing v bins or n observations, depending on the type of preprocessing, the component weights are recalculated

$$w_l={\displaystyle \frac{n{w}_l}{n-N}}, l=1,\dots ,c$$

(14)

to fulfil ${\sum }_{l=1}^c{w}_l=1$. The initial FMM parameters for the EM are finally obtained by inverting Equations (A22)–(A27). The lognormal parameters result from

$${\mu }_l=2log\left(m_l\right)-{\displaystyle \frac{1}{2}}log\left(V_l\right),$$

(15)

$${\sigma }_l^2=log\left(V_l\right)-2log\left(m_l\right).$$

(16)

The Weibull parameters are a solution of

$$log(m_l^2/V_l)+log(\Gamma [1+2/{\beta }_l])-2log(\Gamma [1+1/{\beta }_l])=0,$$

(17)

$${\theta }_l={\displaystyle \frac{m_l}{\Gamma [1+1/{\beta }_l]}}.$$

(18)

To avoid numerical problems, the minimum value of ${\beta }_l$ was set to ${\beta }_{\min }=0.15$. If Equation (17) returns a value less than or equal to zero, then ${\beta }_l$ is set to ${\beta }_{\min }$. Otherwise, ${\beta }_l={\beta }_{\min }$ is used as the initial estimate. The gamma parameters are given by

$${\beta }_l={\displaystyle \frac{1}{\frac{V_l}{m_l^2}-1}},$$

(19)

$${\theta }_l={\displaystyle \frac{m_l}{{\beta }_l}}.$$

(20)

This section introduces an important improvement with the inclusion of outliers, as compared to [14].

3. Simulations

To demonstrate the improvements of improved rough component parameter estimation and enhanced Bayes classification of unassigned observations, the R package rebmix [37] (version 2.16.0) was compared to five widely used alternatives: flexmix [39,40] (version 2.3-20), ltmix [41] (version 0.2.2), mixdist [42] (version 0.5-5), mixR [43] (version 0.2.1), and mixtools [44,45] (version 2.0.0.1. The comparison focused on three common parametric families in the reliability domain: lognormal, Weibull and gamma. In addition, four system configurations, as shown in Figure 3, were analysed. To cover a wide range of sample sizes, data sets with 50, 100, 500 and 1000 observations were used. The parameters for the probability density functions were generated using random seeds ranging from 1 to 20.

The Weibull and gamma parameters were simulated according to the following equations

$${\beta }_l=1+9\mathcal{U}(0,1),$$

(21)

$${\theta }_l=10+90\mathcal{U}(0,1),$$

(22)

where $\mathcal{U}(0,1)$ denotes a uniform distribution in the range $[0,1]$. The lognormal distribution parameters were simulated as

$${\mu }_l=log(10+90\mathcal{U}(0,1)),$$

(23)

$${\sigma }_l={\displaystyle \frac{1}{1+9\mathcal{U}(0,1)}}.$$

(24)

Here, the index l stands for the number of the component, $l=1,\dots ,5$. Each package was thus evaluated on $3\mathrm{parametric}\mathrm{families}\times 20\mathrm{random}\mathrm{seeds}\times 4\mathrm{system}\mathrm{configurations}\times 4\mathrm{sample}\mathrm{sizes}=960$ well distinguishable data sets.

The R packages were evaluated according to the Bayesian Information Criterion (BIC) and the calculation time. The BIC is commonly regarded as the standard information criterion to assess how well the estimated FMM fits a data set.

On the other hand, the calculation time is of key importance when numerous data sets or large data sets have to be processed. This is particularly important as FMMs are often associated with solving complex regression, clustering, classification or other data mining tasks.

All of the aforementioned packages are based on the EM algorithm, although they use different strategies to determine the initial set of FMM parameters. We also performed initial tests with the package ForestFit (version 2.4.3) [46], which was significantly slower than the others. Considering the computational challenges of processing 960 data sets within a reasonable time frame, ForestFit was not further evaluated.

Assuming that the time-to-failure probability densities $f_l\left(t\right)$ and the corresponding reliabilities $R_l\left(t\right)$ of the five components in Figure 3 are known, the exact system reliability for the linked system configuration (d) can be expressed as follows

$$\begin{array}{cc}\hfill R_{\mathrm{s}}& =R_1{R}_3+R_2{R}_4+R_1{R}_4{R}_5+R_2{R}_3{R}_5\hfill \\ \hfill & -R_1{R}_2{R}_3{R}_4-R_1{R}_2{R}_3{R}_5-R_1{R}_2{R}_4{R}_5\hfill \\ \hfill & -R_1{R}_3{R}_4{R}_5-R_2{R}_3{R}_4{R}_5+2R_1{R}_2{R}_3{R}_4{R}_5.\hfill \end{array}$$

(25)

For the sake of simplicity, the dependency on the variable t is omitted from the notation. The exact system time-to-failure probability density for the linked system configuration (d) is then

$$\begin{array}{cc}\hfill f_{\mathrm{s}}& =-d{R}_{\mathrm{s}}/dt=f_1{R}_3+R_1{f}_3+f_2{R}_4+R_2{f}_4+f_1{R}_4{R}_5\hfill \\ \hfill & +R_1{f}_4{R}_5+R_1{R}_4{f}_5+f_2{R}_3{R}_5+R_2{f}_3{R}_5+R_2{R}_3{f}_5\hfill \\ \hfill & -f_1{R}_2{R}_3{R}_4-R_1{f}_2{R}_3{R}_4-R_1{R}_2{f}_3{R}_4-R_1{R}_2{R}_3{f}_4\hfill \\ \hfill & -f_1{R}_2{R}_3{R}_5-R_1{f}_2{R}_3{R}_5-R_1{R}_2{f}_3{R}_5-R_1{R}_2{R}_3{f}_5\hfill \\ \hfill & -f_1{R}_2{R}_4{R}_5-R_1{f}_2{R}_4{R}_5-R_1{R}_2{f}_4{R}_5-R_1{R}_2{R}_4{f}_5\hfill \\ \hfill & -f_1{R}_3{R}_4{R}_5-R_1{f}_3{R}_4{R}_5-R_1{R}_3{f}_4{R}_5-R_1{R}_3{R}_4{f}_5\hfill \\ \hfill & -f_2{R}_3{R}_4{R}_5-R_2{f}_3{R}_4{R}_5-R_2{R}_3{f}_4{R}_5-R_2{R}_3{R}_4{f}_5\hfill \\ \hfill & +2(f_1{R}_2{R}_3{R}_4{R}_5+R_1{f}_2{R}_3{R}_4{R}_5+R_1{R}_2{f}_3{R}_4{R}_5\hfill \\ \hfill & +R_1{R}_2{R}_3{f}_4{R}_5+R_1{R}_2{R}_3{R}_4{f}_5).\hfill \end{array}$$

(26)

If $R_5=1$ and $f_5=0$, then the last two equations correspond to the low-level redundant system configuration (c). High-level redundant system configuration (b) is obtained when $R_5=0$ and $f_5=0$. Serial system configuration (a) is achieved if $R_l=0$ and $f_l=0$ for $l\in \{2,4,5\}$.

Although the systems under consideration consist of no more than five components, Equations (25) and (26) are already quite complex. For systems with a large number of components, it is no longer feasible to derive analytical equivalents for these equations. In addition, the structure of Equation (26) differs significantly from that of Equation (1). Nevertheless, it is possible to show that the FMM is an excellent and compact approximation to the exact system time-to-failure probability density.

Figure 4 illustrates the time-to-failure probability densities for components 1 to 5, which are colour-coded as follows: Component 1 in red, 2 in green, 3 in blue, 4 in cyan and 5 in purple. The components have significantly different times to failure. The exact time-to-failure probability density of the system, shown in black, shows two different modes. The first mode lies between the modes of components 1 and 3, while the second mode closely matches the mode of component 2. Figure 5 also shows that components 4 and 5 do not contribute significantly to the reliability of the system.

The times to system failure $T_{\mathrm{s}}$ were randomly generated using Monte Carlo simulation, based on the known time-to-failure probability densities of the individual components. In order to take different system configurations into account, each component was assigned an indicator $I_l$. Specifically, $I_l=1$ indicates that $R_l=1$, $I_l=0$ means that $R_l=0$, and $I_l=0.5$ represents $R_l\left(t\right)$. The times to failure of the component l were generated for the parametric Weibull family as follows

$$T_l\sim \left\{\begin{array}{cc}\infty \hfill & I_l=1\hfill \\ \mathrm{Weibull}({\beta }_l,{\theta }_l)\hfill & I_l=0.5\hfill \\ 0\hfill & I_l=0.\hfill \end{array}\right.$$

(27)

The function of the random number generator used to model the times to failure is either $\mathrm{lognormal}({\mu }_l,{\sigma }_l)$, $\mathrm{Weibull}({\beta }_l,{\theta }_l)$ or $\mathrm{gamma}({\beta }_l,{\theta }_l)$. The different system configuration, illustrated in Figure 3, is thus uniquely defined by an indicator array $\mathbf{I}=\left\{I_l\right\}\forall l\in \{1,2,3,4,5\}$, where $I_l\in \{0,0.5,1\}$ is the lth system component indicator variable as described in the paragraph above. For example, the indicator array for a low-level redundant system, illustrated in Figure 3c, is $\mathbf{I}=\{0.5,0.5,0.5,0.5,1\}$ and the indicator array for a linked system, illustrated in Figure 3d, is $\mathbf{I}=\{0.5,0.5,0.5,0.5,0.5\}$. Finally, the time until system failure is then given by

$$T_{\mathrm{s}}=max\{min\{T_1,T_3\},min\{T_2,T_4\},min\{T_1,T_4,T_5\},min\{T_2,T_3,T_5\}\}.$$

(28)

In other words, the time to system failure depends on the four minimal paths of the system. In this way, data sets of different lengths were created. The pseudo code for data set generation is given in Algorithm 1.

The R files flexmix.R, ltmix.R, mixdist.R, mixR.R, mixtools.R and rebmix.R are provided as Supplementary Materials to enable the reproduction of all results presented in this article. Please replace the working directory setwd() and make sure that the required packages are installed before executing the above *.R files. The R programming environment has undoubtedly become the leading platform for statistical computing. Figures such as Figure 4 and Figure 5 can be reproduced by running the figures.R script.

Algorithm 1: System time-to-failure data set generation.

Input: Number of data points to generate n, parametric family of system components $\mathrm{pdf}$, Parameters for system component distributions $\mathbf{\Theta }$, Indicator array $\mathbf{I}$; Output: Times to system failure $\mathbf{T}$; 1: Create empty array for data set $\mathbf{T}=\left\{\right\}$; 2: foreach $i\in \{1,\dots n\}$do: 3:       Create empty array for system component time to failure $\mathbf{t}=\left\{\right\}$; 4:       foreach $l\in \{1,\dots 5\}$ do: 5:             Select $I_l$ from lth index in indicator array $\mathbf{I}$; 6:             Set parametric family defined in $\mathrm{pdf}$ with distribution parameters from $\mathbf{\Theta }$; 7:             Generate time to failure $T_l$ of lth system component using Equation (27). 8:             Insert $T_l$ in array $\mathbf{t}$ at position l; 9:         end 10:       Select $T_1,T_2,T_3,T_4,T_5$ from $\mathbf{t}$; 11:       Determine system time to failure $T_s$ using Equation (28); 12:       Insert $T_s$ in array $\mathbf{T}$ at position i; 13:   end

4. Results and Discussion

The package rebmix was developed for FMM parameter estimation, with a particular focus on large data sets and the Weibull parametric family [35,38]. It is primarily characterized by its distinctive approach of sequential rough parameter estimation, which significantly reduces computational time relative to other EM-based packages. In contrast, where these alternatives rely on simultaneous parameter estimation and require initial estimates for all parameters, rebmix achieves greater efficiency through its streamlined process.

However, a major disadvantage of rebmix was its lower accuracy, as indicated by higher values for the information criteria, where other packages performed better. To address this limitation, an EM step was incorporated [32] to fine-tune the FMM parameters. Although this improvement reduced computational speed, it enhanced accuracy. The effects of the latest improvements to the package are demonstrated below. The computational procedure for obtaining the results presented in this section and for further analysis is shown in Figure 6.

The 960 data sets were initially processed by all six packages. Since the system is composed of five components, the optimal solution was searched for $c=1$ to 5, where c is the number of components. The package rebmix offers three types of preprocessing: histogram, kernel density estimation (KDE) and k-nearest neighbour (KNN). Histogram preprocessing is particularly suitable for larger data sets, while KDE and KNN are more suitable for smaller data sets. The KDE&histogram option in

Table 1. Comparison of packages for different preprocessing based on mean BIC (bold values represent the best values obtained).

Preprocessing	flexmix	ltmix	mixdist	mixR	mixtools	rebmix
histogram	3518	3516	3520	3533	4671	3510
KDE&histogram						3510
KDE						3518
binned						3516

indicates that the histogram preprocessing was used if the number of observations $n>100$, otherwise KDE was used. The package rebmix provided the most favourable BIC values. The rebmix can also use a histogram as input by first converting the data set using the methods fhistogram or chistogram. See rebmix help for details. This option is referred to as binned in the table.

As can be seen from

Table 2. Comparison of packages for different preprocessing based on mean calculation time [s] (bold values represent the best values obtained).

Preprocessing	flexmix	ltmix	mixdist	mixR	mixtools	rebmix
histogram	2.41	12.13	2.74	5.41	10.29	0.27
KDE&histogram						0.27
KDE						0.49
binned						0.01

, the package rebmix was on average 241 times faster than flexmix and 1029 times faster than mixtools. For further analysis, we chose the binned option, which resulted in an average BIC that was lower than that of the second-best package ltmix. Details on how to use this option can be found in the file rebmix.R.

Studying the mean values alone is insufficient. For

Table 3. Comparison of rebmix using binned input with other packages for different parametric families based on $\Delta \mathrm{BIC}$ (bold values represent the best values obtained).

Parametric Family	flexmix	ltmix	mixdist	mixR	mixtools
Weibull	90	86	148	69	NA
gamma	−14	-26	6	−30	−14
lognormal	20	-20	62	8	NA

and

Table 4. Comparison of rebmix using binned input with other packages for different parametric families based on $\Delta \mathrm{time}$ (bold values represent the best values obtained).

Parametric Family	flexmix	ltmix	mixdist	mixR	mixtools
Weibull	320	320	320	315	NA
gamma	320	320	320	320	320
lognormal	320	320	320	318	NA

, the values of $\Delta \mathrm{BIC}={\mathrm{BIC}}_{\mathrm{rebmix}}-{\mathrm{BIC}}_p$ and $\Delta \mathrm{time}={\mathrm{time}}_{\mathrm{rebmix}}-{\mathrm{time}}_p$ were calculated for all 960 data sets, where $p\in \{\mathrm{flexmix},\mathrm{ltmix},\mathrm{mixdist},\mathrm{mixR},\mathrm{mixtools}\}$. The summation was performed as follows: for $\Delta \mathrm{BIC}<0$, a value of one was added; for $\Delta \mathrm{BIC}>0$ a value of minus one was added; and for $\Delta \mathrm{BIC}=0$ zero was added to the sum. This process was carried out separately for each parametric family. The same logic was applied to $\Delta \mathrm{time}$.

The values in the tables can range from − 320 to 320, with a positive value indicating that rebmix outperformed package p. It is obvious that the improved REBMIX&EM algorithm not only worked faster, but also provided better BIC estimates compared to the other packages. For the gamma parametric family, however, ltmix flexmix, mixR and mixtools performed slightly better. In terms of speed, rebmix was faster in almost all cases. The NA entries in the tables indicate that mixtools could not process the lognormal and Weibull parametric families.

Next, the effects of the parametric family and the sample size on the BIC and the calculation time were analysed. The

Table 5. Comparison of packages for different parametric families and sample sizes based on mean BIC (bold values represent the best values obtained).

Parametric Family	n	flexmix	ltmix	mixR	mixdist	mixtools	rebmix
Weibull	50	365 (63)	365 (63)	NA (NA)	367 (63)	NA (NA)	362 (63)
	100	751 (105)	750 (104)	NA (NA)	751 (105)	NA (NA)	747 (104)
	500	3566 (634)	3564 (633)	3564 (633)	3568 (634)	NA (NA)	3563 (633)
	1000	7263 (1063)	7262 (1062)	7262 (1063)	7270 (1066)	NA (NA)	7267 (1063)
gamma	50	578 (55)	577 (55)	576 (55)	578 (55)	578 (55)	576 (55)
	100	1162 (107)	1162 (107)	1162 (108)	1163 (107)	1162 (107)	1162 (109)
	500	5600 (625)	5600 (625)	5600 (625)	5600 (624)	5600 (625)	5596 (619)
	1000	11,346 (1099)	11,346 (1099)	11,346 (1099)	11,346 (1099)	11,346 (1099)	11,334 (1099)
lognormal	50	364 (69)	356 (75)	364 (69)	365 (70)	NA (NA)	363 (69)
	100	709 (130)	704 (132)	709 (130)	710 (130)	NA (NA)	708 (130)
	500	3550 (616)	3545 (614)	3547 (612)	3550 (614)	NA (NA)	3550 (615)
	1000	6962 (1085)	6961 (1085)	6961 (1085)	6976 (1110)	NA (NA)	6966 (1085)

and

Table 6. Comparison of packages for different parametric families and sample sizes based on mean calculation time [s] (bold values represent the best values obtained).

Parametric Family	n	flexmix	ltmix	mixR	mixdist	mixtools	rebmix
Weibull	50	1.27 (0.37)	1.86 (1.00)	NA (NA)	2.97 (0.76)	NA (NA)	0.01(0.01)
	100	1.74 (0.64)	3.72 (2.23)	NA (NA)	3.35 (0.70)	NA (NA)	0.01 (0.02)
	500	4.22 (2.61)	21.66 (9.63)	12.49 (1.62)	3.46 (0.79)	NA (NA)	0.01 (0.01)
	1000	6.05 (4.46)	35.77 (15.81)	30.25 (44.93)	2.91 (1.00)	NA (NA)	0.01 (0.01)
gamma	50	1.55 (0.60)	2.13 (3.06)	2.13 (5.09)	2.30 (0.60)	2.69 (0.23)	0.01 (0.01)
	100	2.13 (1.04)	3.12 (1.76)	0.87 (2.90)	2.67 (0.51)	3.68 (0.16)	0.01 (0.01)
	500	4.54 (3.99)	14.77 (4.83)	4.22 (0.97)	3.05 (0.36)	12.05 (1.27)	0.01 (0.01)
	1000	5.54 (5.28)	23.88 (7.91)	9.24 (1.04)	3.12 (0.33)	22.75 (1.74)	0.01 (0.01)
lognormal	50	0.42 (0.13)	1.32 (0.71)	0.07 (0.06)	1.96 (0.56)	NA (NA)	0.01 (0.01)
	100	0.43 (0.18)	2.86 (1.50)	0.16 (0.09)	2.13 (0.46)	NA (NA)	0.01 (0.01)
	500	0.55 (0.29)	11.90 (4.93)	1.09 (0.39)	2.51 (0.44)	NA (NA)	0.01 (0.01)
	1000	0.48 (0.33)	22.58 (9.07)	2.09 (0.17)	2.51 (0.29)	NA (NA)	0.01 (0.01)

show the mean values and standard deviations (in brackets) for the information criterion and the calculation time. In terms of BIC, rebmix performed best in seven cells of Table 5, followed by ltmix, which performed best in six cells, and mixR, which performed best in four cells. The other packages only achieved the best BIC in one cell. In terms of calculation time, rebmix performed better than all other packages in all cells of Table 6.

To summarise, the accuracy of the rebmix package has improved significantly compared to its previous versions, while maintaining its superiority in calculation time. This progress is particularly important as it increases the potential of rebmix, not only for FMM parameter estimation, but also for applications in image segmentation [32] and in combination with neural networks for image processing.

The effect of packages on different parametric families was analysed based on the estimated optimal number of components. As shown in Figure 7, the FMM in the case of mixdist includes only one component in 796 out of 960 cases ($243+245+308=796$). This indicates that, although the systems may consist of up to five components, a single-component FMM is sufficient to reliably represent the system in $82.9\%$ of cases, despite the variability in component time-to-failure probability densities shown in Figure 8 and Figure 9. Two components were required in $14.9\%$ of cases, three components in $1.5\%$, four components in $0.4\%$ and five components in only $0.3\%$ of cases.

This result indicates that the complexity of the FMM does not necessarily increase with the increasing complexity of the system. However, the complexity of the exact system reliability increases with the complexity of the system. This supports the hypothesis that the FMM is an excellent approximation of the exact system reliability function, as it allows a very compact formulation of the system reliability and the system time-to-failure probability density. All tables and figures in this section can be reproduced by executing the file tables.R.

5. Practical Example

To demonstrate the practical usefulness of our proposed approach, we utilized hard disk drive (HDD) and solid-state drive (SSD) data from the Backblaze data center [47]. The time-to-failure data was extracted from the 2024 Q4 data set, spanning the period from 1 October to 31 December 2024. For each day within this period, a snapshot was available containing information on both operational and failed HDDs and SSDs. We identified all drives that failed on a given day and extracted their power-on hours, represented by the S.M.A.R.T. 9 attribute [48], as a proxy for time-to-failure. Using this method, we compiled a total of 1030 time-to-failure data points from HDDs and SSDs that failed during Q4 2024. A histogram illustrating the distribution of these time-to-failure values is shown in Figure 10.

All packages, namely rebmix, flexmix, ltmix, mixdist, mixR, and mixtools, used in the simulation study were also applied here to estimate the parameters of the FMMs. For flexmix, ltmix, mixdist, mixR, and mixtools, we used the same package-specific settings as before. In the case of rebmix, we selected only the histogram preprocessing method, as it yielded the fastest and most accurate results.

Table 7. BIC values and corresponding estimated number of components in the FMMs for different packages and parametric families (bold values represent the best values obtained).

Parametric Family	flexmix	ltmix	mixR	mixdist	mixtools	rebmix
Weibull	22,797 (3)	22,759 (4)	22,692 (5)	22,854 (4)	NA (NA)	22,692 (5)
gamma	22,727 (5)	22,738 (5)	22,691 (5)	22,774 (5)	22,768 (5)	22,676 (5)
lognormal	22,819 (4)	22,749 (5)	22,689 (5)	23,180 (2)	NA (NA)	22,689 (5)

reports the BIC values of the FMMs estimated using different packages, and

Table 8. Computation times across different packages and parametric families (bold values represent the best values obtained).

Parametric Family	flexmix	ltmix	mixR	mixdist	mixtools	rebmix
Weibull	9.446	14.59	0.730	1.507	NA	0.200
gamma	15.261	12.57	0.631	0.892	13.131	0.072
lognormal	0.384	5.141	0.130	1.000	NA	0.034

presents the corresponding computation times. Additionally, Table 7 reports the number of components estimated in the FMMs for each package and parametric family, indicated in parentheses next to the corresponding BIC value.

Since the parametric family is not known a priori for the presented data set, we applied all three parametric families explored in this study: Weibull, gamma, and lognormal. As shown in Table 7, the majority of packages achieved their best BIC values with the gamma FMM, with the exception of mixR, which yielded a lower BIC for the lognormal FMM. Additionally, regardless of the chosen parametric family, most packages consistently estimated FMMs with five components as the most suitable model. The best-performing gamma FMM, based on the observed BIC values, was estimated using the rebmix package, which also provided the overall best estimate. The top Weibull and lognormal FMMs were estimated using mixR and rebmix, respectively. The best-fitting Weibull, gamma, and lognormal FMMs are visualized in Figure 11. Finally, in terms of computational efficiency, the rebmix package clearly outperformed all others, with mixR being the second fastest, although it was, on average, five times slower than rebmix.

6. Conclusions

This article tackles several open challenges related to the use of FMMs in reliability studies. First, we addressed the shortcomings of the REBMIX&EM parameter estimation algorithm. Second, we demonstrated the advantages of using lognormal, Weibull, and gamma FMMs to approximate system reliability, and derived all associated parameter estimation equations. Our third objective was to promote the use of the R Project for Statistical Computing [49] in the fields of reliability, maintainability, and supportability. To that end, we evaluated six R packages relevant to these domains. All evaluated packages are applicable to reliability modelling, and all but one offer straightforward usage. The mixtools package currently supports only the gamma parametric family. In contrast, the mixdist package requires additional programming effort, as it lacks built-in functionality for estimating initial parameters. To address this, we used the kmeans algorithm [49] to initialize parameter values. We have also provided all accompanying source code to enable full reproducibility and to serve as a starting point for further research.

To demonstrate the improvements to the REBMIX&EM parameter estimation algorithm implemented in rebmix R package the serial system, the high-level redundant system, the low-level redundant system and the linked system were analysed. Four different sample sizes and three parametric families were studied. The data sets were simulated across a wide range of parameters defining the component time-to-failure distributions. A total of 960 data sets were processed with each package. The findings of the article can be summarised as follows:

• Although the true system time-to-failure probability density function (e.g., Equation (26)) differs significantly from the FMM probability density function (e.g., Equation (1)), the latter still serves as an effective and concise alternative.
• The number of components of the FMM is usually lower than the number of components from which the system is composed.
• The lognormal, Weibull and gamma parametric families are all suitable for modelling system reliability. However, it is impossible to determine the best family in advance. Therefore, it is recommended to estimate the FMM parameters for each family and choose the one that yields the lowest information criterion.
• The BIC is the most preferred and widely used information criterion.
• The improved rough component parameter estimation and enhanced Bayes classification of unassigned observations have significantly increased the calculation speed of the REBMIX algorithm.
• The improved REBMIX&EM algorithm has improved the accuracy of estimating the parameters of a FMM, at least for univariate problems. REBMIX is used here to determine the initial parameters for the EM algorithm.
• All calculations of the article can be reproduced via supplementary *.R files in the R programming environment.

Based on the results of the simulation study, the rebmix R package provides very fast and accurate FMM estimates. The mixR package also demonstrated competitive performance. The flexmix and ltmix packages showed similar performance levels and offer extensive feature sets within their respective domains. In contrast, the mixdist package was the least user-friendly, as it requires manual specification of initial FMM parameters. Additionally, the mixtools package currently lacks support for the Weibull and lognormal parametric families.

We also analysed a practical case study involving time-to-failure data from Backblaze hard drives. In this analysis, all packages used in the simulation study were applied for FMM parameter estimation. The best overall FMM, based on BIC, was a gamma FMM with five components, estimated using the rebmix package. Both the rebmix and mixR packages produced the best estimates for the Weibull and lognormal FMMs. As in the simulation study, rebmix achieved the shortest computation time, while mixR again showed strong and competitive performance.

The rebmix package, with the improved REBMIX&EM parameter estimation algorithm presented in this study, offers highly competitive performance, delivering accurate estimates with minimal computation time. However, because the REBMIX algorithm is deterministic, it always produces the same initial parameters for the EM algorithm, unlike random initialisation, which can eventually outperform deterministic approaches given enough time. Packages like flexmix and ltmix, which use random initialisation, can explore the parameter space more broadly and eventually lead to more accurate estimates, or at least equivalent ones, if no better solution exists than that found by deterministic approach. The mixR package uses k-means for initialisation, a middle ground between deterministic and random approaches. While k-means is more efficient than random initialisation and often effective for normal FMMs, it may miss optimal solutions due to its assumption of spherical clusters for non-normal FMMs. Nonetheless, in our study, mixR provided good FMM estimates, confirming the practical value of k-means in this context.

To summarise, with recent improvements, the REBMIX algorithm has become more competitive for non-overlapping and moderately overlapping FMMs. Our future work will focus on enhancing REBMIX’s performance in highly overlapping scenarios, where k-means and random initialisation methods still lead the field [25]. We also plan to investigate kernel density estimation preprocessing within the REBMIX framework more deeply. The current implementation in the rebmix package uses radial basis function kernels, with the bandwidth either specified by the user or selected through an iterative search based on the chosen information criterion, such as BIC. Future research should explore alternative kernel types and develop more robust bandwidth selection strategies. Finally, this study opens several directions for further advancement in statistical analysis. The improved REBMIX&EM algorithm could be extended to support more complex parametric families, and adapted for use with multivariate reliability data, time-varying covariates, or censored observations. These features are commonly encountered in industrial systems.