Search Results (5)

The Expectation-Maximisation (EM) algorithm is widely used for maximum likelihood estimation in incomplete data problems such as mixture modelling, but it often converges slowly, particularly when mixture components overlap substantially. This study presents a comprehensive empirical evaluation of simple EM acceleration schemes for Gaussian mixture models, comparing linear (STEM), quadratic (SQUAREM), and greedy (line search, golden section) methods across 240 simulated mixture configurations spanning three dimensionalities, four component counts, five overlap levels, and four sample sizes. A key contribution is the first systematic comparison of the three acceleration parameter estimates (${\alpha }_1$, ${\alpha }_2$, ${\alpha }_3$) in the mixture modelling context: we show that only ${\alpha }_3$, which is derived as the geometric mean estimate of ${\alpha }_1$ and ${\alpha }_2$, provides genuine acceleration, while ${\alpha }_1$ and ${\alpha }_2$ consistently increase iteration counts by 50–110% relative to ${\alpha }_3$, effectively acting as deceleration. With ${\alpha }_3$, SQUAREM reduces iterations by up to 48% with negligible computational overhead, while greedy methods achieve similar iteration reductions but at 50–110% greater wall-clock time due to repeated log-likelihood evaluations. Crucially, acceleration does not degrade parameter estimation quality under any tested combination of initialisation, overlap, dimensionality, or number of components. We further examine the interaction between acceleration and initialisation, finding that k-means benefits most from acceleration (up to 50% time savings), while the REBMIX (Rough-Enhanced-Bayes MIXture estimation) algorithm benefits least as it already starts near the optimum. Among REBMIX configurations, histogram preprocessing with the outliers mode traversing strategy offers the best trade-off between quality and computational cost. The findings are validated on a real-world Backblaze hard drive failure dataset, confirming the practical utility of EM acceleration. All methods are implemented in the free and open-source R package rebmix, accompanied by full source code. Full article

As system complexity increases, accurately capturing true system reliability becomes increasingly challenging. Rather than relying on exact analytical solutions, it is often more practical to use approximations based on observed time-to-failure data. Finite mixture models provide a flexible framework for approximating arbitrary probability density functions and are well suited for reliability modelling. A critical factor in achieving accurate approximations is the choice of parameter estimation algorithm. The REBMIX&EM algorithm, implemented in the rebmix R package, generally performs well but struggles when components of the finite mixture model overlap. To address this issue, we revisit key steps of the REBMIX algorithm and propose improvements. With these improvements, we derive parameter estimators for finite mixture models based on three parametric families commonly applied in reliability analysis: lognormal, gamma, and Weibull. We conduct a comprehensive simulation study across four system configurations, using lognormal, gamma, and Weibull distributions with varying parameters as system component time-to-failure distributions. Performance is benchmarked against five widely used R packages for finite mixture modelling. The results confirm that our proposal improves both estimation accuracy and computational efficiency, consistently outperforming existing packages. We also demonstrate that finite mixture models can approximate analytical reliability solutions with fewer components than the actual number of system components. Our proposals are also validated using a practical example from Backblaze hard drive data. All improvements are included in the open-source rebmix R package, with complete source code provided to support the broader adoption of the R programming language in reliability analysis. Full article

The problem of characterizing the structural residual life is one of the most challenging issues of the damage tolerance concept currently applied in modern aviation. Considering the complexity of the internal architecture of composite structures widely applied for aircraft components nowadays, as well as the additional complexity related to the appearance of barely visible impact damage, prediction of the structural residual life is a demanding task. In this paper, the authors proposed a method based on detection of structural damage after low-velocity impact loading and its classification with respect to types of acting stress on constituents of composite structures using the developed processing algorithm based on segmentation of 3D X-ray computed tomograms using the rebmix package, real-oriented dual-tree wavelet transform and supporting image processing procedures. The presented algorithm allowed for accurate distinguishing of defined types of damage from X-ray computed tomograms with strong robustness to noise and measurement artifacts. The processing was performed on experimental data obtained from X-ray computed tomography of a composite structure with barely visible impact damage, which allowed better understanding of fracture mechanisms in such conditions. The gained knowledge will allow for a more accurate simulation of structural damage in composite structures, which will provide higher accuracy in predicting structural residual life. Full article

A maximum-likelihood estimation of a multivariate mixture model’s parameters is a difficult problem. One approach is to combine the REBMIX and EM algorithms. However, the REBMIX algorithm requires the use of histogram estimation, which is the most rudimentary approach to an empirical density estimation and has many drawbacks. Nevertheless, because of its simplicity, it is still one of the most commonly used techniques. The main problem is to estimate the optimum histogram-bin width, which is usually set by the number of non-overlapping, regularly spaced bins. For univariate problems it is usually denoted by an integer value; i.e., the number of bins. However, for multivariate problems, in order to obtain a histogram estimation, a regular grid must be formed. Thus, to obtain the optimum histogram estimation, an integer-optimization problem must be solved. The aim is therefore the estimation of optimum histogram binning, alone and in application to the mixture model parameter estimation with the REBMIX&EM strategy. As an estimator, the Knuth rule was used. For the optimization algorithm, a derivative based on the coordinate-descent optimization was composed. These proposals yielded promising results. The optimization algorithm was efficient and the results were accurate. When applied to the multivariate, Gaussian-mixture-model parameter estimation, the results were competitive. All the improvements were implemented in the rebmix R package. Full article

A commonly used tool for estimating the parameters of a mixture model is the Expectation–Maximization (EM) algorithm, which is an iterative procedure that can serve as a maximum-likelihood estimator. The EM algorithm has well-documented drawbacks, such as the need for good initial values and the possibility of being trapped in local optima. Nevertheless, because of its appealing properties, EM plays an important role in estimating the parameters of mixture models. To overcome these initialization problems with EM, in this paper, we propose the Rough-Enhanced-Bayes mixture estimation (REBMIX) algorithm as a more effective initialization algorithm. Three different strategies are derived for dealing with the unknown number of components in the mixture model. These strategies are thoroughly tested on artificial datasets, density–estimation datasets and image–segmentation problems and compared with state-of-the-art initialization methods for the EM. Our proposal shows promising results in terms of clustering and density-estimation performance as well as in terms of computational efficiency. All the improvements are implemented in the rebmix R package. Full article