SUN: Stochastic UNsupervised Learning for Data Noise and Uncertainty Reduction

Abstract

Unsupervised learning methods significantly benefit various practical applications by effectively identifying intrinsic patterns within unlabelled data. However, inherent data noise and uncertainties often compromise model reliability, result interpretability, and the overall effectiveness of unsupervised learning strategies, particularly in complex fields such as biomedical, engineering, and physics research. To address these critical challenges, this study proposes SUN (Stochastic UNsupervised learning), a novel approach that integrates probabilistic unsupervised techniques—specifically Gaussian Mixture Models—into the RUN-ICON unsupervised learning algorithm to achieve optimal clustering, systematically reduce data noise, and quantify inherent uncertainties. The SUN methodology strategically leverages probabilistic modelling for robust classification and detection tasks, explicitly targeting particle dispersion scenarios related to environmental pollution and airborne viral transmission, with implications for minimising public health risks. By combining advanced uncertainty quantification methods and innovative unsupervised denoising techniques, the proposed study aims to provide more reliable and interpretable insights than conventional methods while alleviating issues such as computational complexity and reproducibility that limit traditional mathematical modelling. This research contributes to enhanced trustworthiness and interpretability of unsupervised learning systems, offering a robust methodological framework for handling significant uncertainty in complex real-world data environments.

1. Introduction

Unsupervised learning (UL) holds a central position within the broader field of machine learning due to several intrinsic advantages [1,2,3,4]. Chief among these is its independence from labelled data, a valuable characteristic given the significant resources and manual effort typically required to obtain high-quality annotated datasets [5]. By operating effectively without reliance on labels, UL enhances access to vast amounts of unlabelled, real-world data, thereby overcoming a prevalent bottleneck in supervised learning. Moreover, unsupervised methodologies aid in identifying intrinsic structures and hidden patterns within complex datasets, enabling researchers and practitioners to conduct insightful exploratory analyses and cultivate a deeper, data-driven understanding [6,7].

The practical implications of UL methodologies are extensive and span numerous areas, showcasing their versatility and relevance in contemporary artificial intelligence applications [8,9,10,11]. In clustering applications, techniques such as K-means and hierarchical clustering effectively facilitate market research, customer segmentation, anomaly detection, and the content-based organisation of documents and images [12,13,14,15,16,17]. Dimensionality reduction techniques—including principal component analysis (PCA), manifold learning methods, and autoencoder networks—are essential for simplifying data representations, optimising computational efficiency, visualising complex data relationships, and enhancing subsequent supervised modelling tasks [18,19,20,21]. Moreover, the role of UL is increasingly apparent in anomaly detection, particularly in fields that require the identification of fraudulent activities, security breaches, and quality control deviations [22,23,24]. Advancements in generative modelling, notably Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), diffusion models, and deep generative autoencoders, offer powerful tools for creative generation, data synthesis, and augmentation tasks, which increasingly form the foundation of modern machine-learning frameworks [25,26,27,28,29]. Finally, unsupervised association-based methods significantly impact recommendation and decision support systems, especially in e-commerce and digital service delivery contexts [30,31,32].

Concurrently, self-supervised and contrastive learning paradigms (such as SimCLR, MoCo, and BYOL) have emerged as leading directions, producing high-quality data representations and substantially increasing model accuracy and generalisation in downstream tasks [33,34,35,36]. Transformative developments in sequential and linguistic data representation rely on transformer-based frameworks, demonstrating the prominence of large-scale, unsupervised transformer models such as BERT, GPT, and masked language models [37,38,39]. Simultaneously, progress in graph-based neural architectures, including graph neural networks (GNNs) and graph autoencoders (GAEs), has enhanced unsupervised representation learning and pattern discovery in relational and networked data [40,41,42,43].

Reducing data noise and uncertainty is a focal area across the entire machine learning spectrum. Within generalised machine learning practices, effectively managing data noise enhances model accuracy, improves interpretability, and fosters model robustness and reliability [44,45]. Importantly, noise reduction improves the validity of resulting models and makes actionable insights more readily interpretable by practitioners, thereby supporting informed decisions with greater confidence. In the context of UL in particular, addressing data quality concerns becomes even more significant due to the absence of supporting labelled information; here, unsupervised algorithms depend primarily on intrinsic patterns in the data, which can easily become obscured by excess noise and uncertainty [46,47,48]. Consequently, effectively controlling and reducing data noise and uncertainty is integral to ensuring reliable identification of meaningful structures and thus maintaining confidence in subsequent conclusions.

The current literature identifies numerous state-of-the-art methods explicitly developed to mitigate data noise and manage uncertainty. For data noise reduction, advanced filtering techniques—including median and adaptive filters, wavelet transforms, and non-local means algorithms—remain standard and effective approaches [49,50]. Alongside these traditional techniques, modern deep denoising autoencoders and GAN-based denoising methods (such as Noise2Noise and Noise2Void) increasingly demonstrate superior performance [51,52,53]. Likewise, self-supervised denoising schemes enable adaptation to tasks lacking explicit ground truth, expanding effectiveness and applicability across diverse data types and domains [54]. Approaches specifically designed to quantify, reduce, and manage uncertainty notably include Bayesian methodologies—employing Bayesian neural networks, Gaussian processes, probabilistic graphical models, and approximate inference techniques—which explicitly model and quantify uncertainty in predictions [55,56,57]. Additionally, Monte Carlo dropout and deep ensemble techniques provide reliable estimates of model uncertainty, enabling efficient uncertainty quantification [58,59]. Calibration methods such as temperature scaling, Platt scaling, and label smoothing further refine model confidence estimates, significantly reducing uncertainty and facilitating safer, more robust decision-making in real-world artificial intelligence deployments [60,61,62]. Recent advances in uncertainty-aware clustering have further shaped the landscape. Bayesian Gaussian Mixture Models (GMMs) extend traditional GMMs by placing priors on parameters, enabling robust uncertainty quantification [63]. Variational Deep Clustering methods, such as VaDE [64] and IDEC [65], combine deep learning with probabilistic frameworks to handle complex data distributions. In [66], a comprehensive framework for uncertainty estimation in deep clustering is proposed.

In light of the above, despite the success and potential of UL algorithms, fields such as biomedical engineering and physics applications face heightened uncertainty, primarily due to incomplete scientific understanding and inherent data complexities [67,68]. This challenge underscores the need to explicitly integrate probabilistic methods that quantify uncertainty, thereby improving the interpretability and robustness of results. Therefore, recognising the prominence and rapid advancement of UL, along with the crucial importance of optimising data quality, controlling noise, and reducing uncertainty, the present research aims to comprehensively and critically examine these vital aspects to enhance the current understanding of unsupervised methodologies and their robustness in challenging data environments.

This study addresses the challenges above by integrating UL and probabilistic methodologies, namely the “Reduce UNcertainty and Increase CONfidence” (RUN-ICON) algorithm and Gaussian Mixture Models (GMMs), to mitigate data noise and reduce uncertainty. The resulting algorithm is called the “Stochastic UNsupervised learning” (SUN) algorithm. This topic is significant for detecting and classifying particle dispersion that may be associated with pollution or viral carriers, aiming to minimise public health risks (see, e.g., [69]). Historically, while beneficial, traditional mathematical models require computationally intensive simulations and face challenges regarding reproducibility and validation against experimentally observed data, exacerbating the intrinsic uncertainties in their outputs. The paper is organised as follows: Section 2 presents the two components and implementation of the SUN algorithm. In Section 3, results are presented from the application of the algorithm on four distinct datasets (three two-dimensional and one multimodal), where the optimal clustering is known a priori and its performance is evaluated. Finally, in Section 4, conclusions are drawn, and suggestions for further work are provided.

2. The SUN Algorithm

SUN is a two-stage procedure that combines stability-driven selection of cluster structure with probabilistic assignments. The first stage, RUN-ICON, searches over a range of K values and identifies centres that recur across many K-means++ runs. The second stage fits a Gaussian Mixture Model (GMM) with K components initialised at those centres and returns soft memberships. The full GMM specifications are provided in Appendix A.

2.1. Design Rationale

Cluster analysis often requires a choice of K and can be sensitive to initialisation. RUN-ICON addresses this by favouring settings that are reproducible under repeated random restarts. The GMM then models cluster shape and overlap, yielding assignment probabilities. This separation of roles leads to stable centre selection followed by uncertainty-aware labelling.

2.2. RUN-ICON, Stability Score, and Selection Rule

Despite its broad utility across application areas [70,71,72,73], unsupervised clustering for particle data faces recurring pain points: sensitivity to initialisation and hyperparameter settings, dependence on feature choice and scaling, susceptibility to high dimensionality, and computational burdens at scale [74]. RUN-ICON tackles these issues by casting model selection and centre identification as a stability problem [48]. For each candidate K, we perform many K-means++ fits, pool the resulting centres, and locate dominant centres through the Generalised Cluster Coordinate (GCC) as recurring modes in this centre set under a small matching tolerance $\delta$. We then compute a Clustering Dominance Index $\mathrm{CDI}\left(K\right)$, defined as the proportion of runs that recover the same dominant configuration, and repeat this outer procedure several times to obtain a distribution of $\mathrm{CDI}\left(K\right)$ values. An accompanying uncertainty score $U\left(K\right)$ summarises the spread of these values. The selected K maximises the mean $\mathrm{CDI}\left(K\right)$ subject to a bound on $U\left(K\right)$, with ties broken in favour of lower uncertainty. In practice, this replaces ad hoc elbow rules or single-run SSE heuristics with a reproducibility criterion, prioritises cluster structures that recur under random restarts, and reduces the influence of outliers and background noise. The resulting dominant centres serve as a stable representation of the data structure and as initialisation for the probabilistic stage of SUN. The capabilities of the algorithm have been demonstrated in several previous studies [75,76,77]. The steps of the algorithm for each candidate K are as follows:

• Draw R K-means++ initialisations and fit K-means for each.
• Aggregate the R sets of centres and identify dominant centres by mode seeking under a small distance threshold $\delta$ in the centre space.
• Compute a stability score, $\mathrm{CDI}\left(K\right)$, as the fraction of runs whose fitted centres match the same set of dominant centres within $\delta$.
• Repeat the previous steps M times to obtain a sample of $\mathrm{CDI}\left(K\right)$ values and summarise their spread by an uncertainty measure $U\left(K\right)$, i.e., the difference between upper and lower CDI bounds across outer repeats.

Select $K^{★}$ that maximises the mean $\mathrm{CDI}\left(K\right)$ subject to a cap on $U\left(K\right)$, then break ties by the smallest $U\left(K\right)$. The associated dominant centres are used as the representative solution for $K^{★}$.

2.3. Gaussian Mixture Models

2.3.1. Stochastic Modelling—Mathematical Theory

The GMM is a widely used probabilistic clustering method that can refine and enhance the output of preliminary clustering algorithms [78,79,80,81]. Its integration within UL pipelines often leads to more accurate predictions and clearer insights into complex data. The GMM offers several advantages that make it well-suited in this context. By modelling data as a mixture of Gaussian components, it can represent underlying sub-populations more accurately than simpler approaches [82]. Its ability to accommodate clusters of varying shapes, scales, and orientations enhances performance on heterogeneous real-world data [83]. The GMM also provides probabilistic membership estimates for each data point [84], allowing explicit quantification of uncertainty. Furthermore, its formulation as a weighted sum of Gaussian distributions yields an interpretable representation of latent structure [85]. The GMM can be integrated into hybrid or domain-specific machine learning frameworks [86,87], thereby strengthening predictive accuracy by combining complementary techniques. Finally, because many datasets are inherently multimodal, the GMM is especially effective in modelling multiple modes through distinct Gaussian components [88]. These properties make the GMM a powerful tool for improving UL results, particularly in high-complexity or high-stakes settings.

In the present work, the GMM is employed to refine the cluster centres produced by a preceding clustering stage. While K-means inherently favours spherical, similarly sized clusters [89], the proposed SUN algorithm overcomes this limitation by using GMM to represent each cluster with its own mean and covariance structure. The mathematical formulation is provided in the following section. Importantly, the dominant number of clusters remains stable across iterations, as predicted by RUN-ICON.

2.3.2. Setting up the GMM Framework

The GMM assumes data points are sampled from multiple Gaussian distributions [90]. For a dataset of N points separated into K clusters, we define cluster means ${\mu }_k$ (Equation (A1)), covariance matrices ${\Sigma }_k$ (Equation (A2)), and weights ${\pi }_k=N_k/N$. The Gaussian distribution for cluster k follows the standard formulation [91], with membership probabilities given by Equation (A5). For the complete set of equations, see Appendix A.

2.3.3. Expectation–Maximisation (EM) Algorithm

We now fix $K=K^{★}$ and initialise the K mixture means at the dominant centres from RUN-ICON and covariances and weights from empirical moments. We now proceed to present the EM algorithm, which computes maximum-likelihood estimates for Gaussian distributions in the GMM-based clustering framework. This algorithm iterates between two key steps: the Expectation (E) step and the Maximisation (M) step.

• E-step. In this step, we compute the probability ${\gamma }_{ik}$ that a given data point ${\mathbf{x}}_i$ belongs to cluster k. We evaluate K probabilities for each point corresponding to its membership in any of the K clusters. The highest probability determines the assignment of the point to a cluster. Consequently, a point may be reassigned to a different cluster than it was originally associated with.
• M-step. This step involves reassigning points to clusters and updating cluster parameters. The means, covariance matrices, and weights are recalculated based on the newly assigned points, using Equations (A1), (A2) and (A4), respectively. While the number of clusters K remains unchanged, their centres are adjusted to reflect the updated means.

2.4. Default Settings and Diagnostics

Unless stated otherwise, we use $K\in \{3,\dots ,10\}$, $R=100$ K-means++ runs per K and $M=10$ outer repeats, a matching tolerance $\delta \in [{10}^{-2},{10}^{-1}]$, and an uncertainty cap $U\left(K\right)\le 30\%$. For the EM procedure, we adopt standard convergence criteria unless otherwise stated. Specifically, the covariance matrices are regularised by adding a diagonal term of magnitude ${10}^{-6}$ to ensure numerical stability, and iterations are terminated when the relative change in the log-likelihood falls below ${10}^{-3}$, with a maximum of 100 iterations. In cases where the estimated covariance matrices become poorly conditioned, the diagonal regularisation is increased up to ${10}^{-4}$; if convergence slows, the log-likelihood tolerance is tightened to ${10}^{-4}$ and the iteration limit is raised to 200. We report $\mathrm{CDI}\left(K\right)$ with uncertainty bars, along with GMM responsibilities in the near-boundary and central regions.

2.5. Implementation Steps of the SUN Algorithm

Now, we will outline the implementation steps of the SUN algorithm, which integrates the RUN-ICON method to determine the optimal number of clusters and their corresponding centres. Once the optimal clustering configuration is identified, we apply the GMM to estimate the cluster boundaries. The following steps summarise the process:

• Initialisation with RUN-ICON
  The RUN-ICON algorithm is used to find the dominant clustering, and the algorithm-determined cluster centres are set as the initial means of the GMM. The initial covariance matrices and weights are also calculated through Equations (A2) and (A4), respectively.
• EM algorithm
  Execute the EM algorithm to refine the GMM parameters:
  • E-step: Calculate the probabilities ${\gamma }_{ik}$ for all points to belong to any of the k clusters.
  • M-step: Redistribute points in K clusters based on highest probability scores and recalculate ${\mu }_k$, ${\Sigma }_k$, and ${\pi }_k$.
• Check for convergence
  To check for convergence, the previously and newly calculated means are compared. If they are below the user-set tolerance, the algorithm terminates. Otherwise, we return to Step 2 and repeat.

Table 1 summarises how SUN differs from existing hybrid clustering methods.

Table 1. Comparison with existing hybrid clustering methods.

Method	Deterministic Component	Probabilistic Component	Uncertainty Quantification	Noise Handling
K-means + GMM [79]	Yes	Yes	No	Limited
Bayesian GMM [63]	No	Yes	Yes	Moderate
Spectral + GMM [92]	Yes	Yes	No	Moderate
Deep Clustering [46]	Yes	Partial	No	Good
VADE [64]	Yes	Yes	Partial	Good
SUN (Proposed)	Yes	Yes	Yes	Excellent

The key innovations of SUN include (i) systematic optimal cluster number determination via RUN-ICON, (ii) explicit uncertainty quantification through CDI metrics, and (iii) dual-phase noise reduction combining statistical filtering and probabilistic smoothing. We thus obtain a robust clustering algorithm that leverages RUN-ICON’s ability to identify dominant clusters and the GMM’s flexibility for modelling complex data distributions. The EM algorithm iteratively refines the parameters, resulting in more precise and reliable clustering outcomes. Figure 1 schematically presents the steps of the SUN algorithm. The pseudocode for the algorithm is given in Appendix B. Its capabilities will be demonstrated in the following section, where four different datasets will be tested.

Figure 1. The SUN algorithm.

2.6. Theoretical Convergence Properties and Noise Suppression

2.6.1. Convergence Analysis

The SUN algorithm’s convergence is guaranteed through the following properties:

• RUN-ICON Convergence: The algorithm converges to dominant cluster centres through statistical mode-seeking. Let ${\mathcal{C}}_k^{\left(i\right)}$ denote the cluster centres from the i-th iteration. The dominance frequency converges to a maximum as $N\to \infty$:

  $$f_k={\displaystyle \frac{1}{N}}\sum _{i=1}^N\mathbb{I}[{\mathcal{C}}_k^{\left(i\right)}={\mathcal{C}}_k^{\ast }]\to max$$

  (1)

  where ${\mathcal{C}}_k^{\ast }$ represents the true cluster centres.
• GMM-EM Convergence: The EM algorithm guarantees a monotonic increase in log-likelihood:

  $$\mathcal{L}\left({\mathsf{\Theta }}^{(t+1)}\right)\ge \mathcal{L}\left({\mathsf{\Theta }}^{\left(t\right)}\right)$$

  (2)

  where $\mathsf{\Theta }={\{{\mu }_k,{\Sigma }_k,{\pi }_k\}}_{k=1}^K$ represents the GMM parameters.
• Combined Convergence: The SUN algorithm converges when

  $$\parallel {\mu }_k^{(t+1)}-{\mu }_k^{\left(t\right)}\parallel <ϵ,\forall k\in \{1,\dots ,K\}$$

  (3)

2.6.2. Noise Suppression Mechanism

SUN reduces noise through two complementary mechanisms:

• Statistical Filtering (RUN-ICON): By aggregating multiple K-means++ runs, random initialisation effects are averaged out:

  $$\mathrm{Var}\left[{\mathcal{C}}_k^{\mathrm{dominant}}\right]<\mathrm{Var}\left[{\mathcal{C}}_k^{\mathrm{single}}\right]$$

  (4)
• Probabilistic Smoothing (GMM): The soft assignment probabilities act as a low-pass filter:

  $$P(\mathrm{noise}\mathrm{point}\in \mathrm{true}\mathrm{cluster})={\displaystyle \frac{{\pi }_k\mathcal{N}\left(x\right|{\mu }_k,{\Sigma }_k)}{{\sum }_j{\pi }_j\mathcal{N}\left(x\right|{\mu }_j,{\Sigma }_j)}}<\tau$$

  (5)

  where $\tau$ is a threshold determined by the cluster covariance structure.

3. Results and Discussion

To ensure a comprehensive evaluation of the algorithm, we utilised three two-dimensional and one multimodal datasets, each emphasising specific characteristics relevant to real-world data scenarios:

• The “noisy” dataset (dataset “zelnik2” in [93]) contains randomly generated background noise and was chosen to test the efficiency of the algorithm in situations where noise is present.
• The“Mickey Mouse” dataset was generated by the authors to test the ability of the algorithm to identify objects with distinct shapes.
• The “particles” dataset was generated by the authors to test the algorithm’s ability to identify regions with distinct flow characteristics during particle dispersion.
• The Wine Recognition dataset [94] is a well-established multimodal dataset that enables us to test the robustness of the method under heterogeneous feature spaces, which is essential for applications where the underlying phenomena arise from multiple interacting modes of behaviour.

For the three datasets (i.e., “noisy”, “Mickey Mouse”, Wine Recognition), the optimal number of clusters and optimal configuration were known a priori, whereas for the “particles” dataset, the number of points per cluster and the optimal separation between clusters were not known in advance; nevertheless, its global behaviour conforms to theoretical predictions [95]. The SUN algorithm was utilised to identify optimal clustering configurations and their corresponding centres, and to predict the shapes and positions of clusters. The predicted centres were also used by K-means to estimate cluster shapes and were compared with the SUN-predicted clusters.

All simulations were run on a personal computer with an Intel Core i5 processor and 8 GB RAM. All runs were executed on the CPU using Python (version 3.12.9) with scikit-learn. End-to-end wall-clock times for the SUN pipeline ranged from 6 s to 705 s, depending on the dataset size, the K search range, and the number of RUN-ICON repetitions and EM iterations.

3.1. “Noisy” Dataset

This section presents an analysis of synthetic two-dimensional data consisting of 303 points. The dataset forms two clusters (blobs), with the remaining points scattered and classified as noise. This dataset was created by Zelnik-Manor and Perona [96], who applied self-tuning spectral clustering to segment the data into three clusters: two representing the dense blobs and a third capturing the scattered noise points (Figure 2).

Figure 2. Illustration of the dataset by Zelnik-Manor and Perona [96].

To evaluate the performance of our proposed SUN algorithm, we initially applied the first component, RUN-ICON, to ascertain the optimal number of clusters. The RUN-ICON method identified three clusters as the most suitable partitioning, achieving 66% confidence and 12% uncertainty, which aligns with the ground-truth classification of the dataset. This result is corroborated by the Clustering Dominance Index (CDI) analysis, as illustrated in Figure 3.

Figure 3. RUN-ICON predictions for the optimal number of clusters for the “noisy” dataset. Different numbers of clusters have been considered, and their CDIs are plotted along with their corresponding uncertainties.

Subsequently, we examined the clustering results using various methods. When only the centroids were utilised and K-means clustering was applied, the noise points adjacent to each blob were grouped with the nearest blob, while the third cluster comprised the remaining noise points situated further from the blob centres (Figure 4a). This outcome indicates that K-means, which relies on centroid-based partitioning, struggles to effectively distinguish between dense clusters and scattered noise when no additional statistical modelling is available.

Figure 4. Clustering results for the “noisy” dataset using (a) K-means initialised with RUN-ICON centres and (b) SUN (RUN-ICON + GMM). Colours are kept consistent across panels and correspond to the three clusters: Cluster 1 (purple)—first dense blob; Cluster 2 (green)—second dense blob; Cluster 3 (red)—scattered noise points. Panel (a) shows the hard assignments produced by K-means, while panel (b) presents the probabilistic memberships obtained by SUN, yielding soft boundaries and quantifiable uncertainty.

However, when the second component of the SUN algorithm, the Gaussian Mixture Model (GMM), was utilised, the clustering outcome improved significantly. In this instance, the three clusters comprised the two blobs and the noise as a separate category. The GMM component enabled the algorithm to better capture the data’s underlying distribution, resulting in clearer separation between the structured clusters and the noise points. The corresponding clustering results are illustrated in Figure 4b.

RUN-ICON produced initial cluster membership estimates of 148, 129, and 26 samples. These values showed a significant imbalance compared to the true class distribution of 102, 106, and 95 samples. After applying the GMM algorithm with the RUN-ICON labels as initialisation, the cluster sizes were refined to 102, 106, and 95 samples, matching the true cluster counts exactly (see Table 2). This indicated a 66.3% reduction in error after GMM refinement, demonstrating that the probabilistic second stage fully corrected the initial misallocations in this dataset.

Table 2. True and predicted cluster assignments for RUN-ICON and SUN for the “noisy” dataset.

Cluster	True Count	RUN-ICON Count	SUN (RUN-ICON + GMM) Count
1	102	148	102
2	106	129	106
3	95	26	95

The stochastic nature of the SUN results in Figure 4b is captured through the GMM membership probabilities. For each point ${\mathbf{x}}_i$, instead of a deterministic assignment as in (a), we have the following:

$${\gamma }_{ik}\in [0,1],\sum _{k=1}^K{\gamma }_{ik}=1$$

Points near cluster centres typically have ${max}_k\left({\gamma }_{ik}\right)>0.90$, while boundary points may have ${\gamma }_{ik}$ from 0.5 to less than 1.0, reflecting classification uncertainty.

These findings demonstrate the effectiveness of the SUN algorithm in identifying and segmenting structured clusters from noise. While conventional K-means fails to properly distinguish noise points from dense clusters, integrating the GMM into the SUN framework enhances clustering accuracy by leveraging probabilistic modelling. This highlights the advantage of employing such clustering approaches when dealing with complex data distributions.

3.2. “Mickey Mouse” Dataset

The dataset consists of 6648 points in (x, y) space, which represented the head of Mickey Mouse, i.e, one central circle representing the “face” (3805 points) and two smaller circles, symmetrically attached to the bigger one, representing the “ears” (left “ear”: 1468 points, right “ear”: 1375 points). All data points within the three circles were generated randomly (Figure 5).

Figure 5. Illustration of the “Mickey Mouse” dataset.

It is anticipated that SUN (through RUN-ICON) will be able to predict three dominant clusters, thereby separating the “face” from the “ears”. Indeed, the algorithm predicted the separation into three clusters with very high confidence (99%) and very low uncertainty (2%), as shown in Figure 6. However, when employing the K-means algorithm with the optimal cluster centres provided by RUN-ICON, a portion of the “head” is erroneously assigned to the “ears” clusters (Figure 7a). This misclassification occurs because K-means depends on Euclidean distance to ascertain cluster membership, and certain regions of the “head” may be closer in distance to the centres of the “ears” clusters than to the “head’s” centre.

Figure 6. RUN-ICON predictions for the optimal number of clusters for the “Mickey Mouse” dataset. Different numbers of clusters have been considered, and their CDIs are plotted along with their corresponding uncertainties.

Figure 7. Clustering results for the “Mickey Mouse” dataset using (a) K-means initialised with RUN-ICON centres and (b) SUN (RUN-ICON + GMM). Colours are consistent across panels and correspond to the three anatomical clusters: the “face” (purple), the left “ear” (green), and the right “ear” (yellow). Panel (a) illustrates the hard partitions produced by K-means, which introduce artificial boundaries and misclassifications near the “face”–“ear” intersections. In contrast, panel (b) shows the SUN results, where probabilistic assignments better preserve the natural circular structure and capture the uncertainty in boundary regions.

In contrast, when the GMM component of the SUN algorithm is applied, the majority of the “head” region is correctly assigned to the “head” cluster, with only small areas being misclassified as part of the “ears” (Figure 7b). This improved clustering performance is due to the probabilistic nature of GMMs, which model the data as a mixture of Gaussian distributions, each defined by a mean and a covariance structure. Unlike K-means, which assigns points to the nearest cluster centre based solely on distance, the GMM assigns probabilities to each point belonging to different clusters, considering both the spatial distribution and the shape of the data. In this way, it accounts for the distributional properties of the clusters rather than relying solely on strict proximity. The “head” cluster, which is significantly larger than the “ears” clusters, likely has a broader covariance structure, increasing the likelihood that points in the overlapping region are assigned to the “head” rather than the “ears”. This allows the GMM to better capture the data’s underlying structure while preserving the integrity of natural groupings.

RUN-ICON produced initial cluster membership estimates of 1729, 2462, and 2457 samples for the “head” and left and right “ears”, respectively. These values exhibited noticeable deviations from the true distributions of 3805, 1468, and 1375 samples, particularly underestimating the head region and overestimating the ear regions. After applying the GMM algorithm with the RUN-ICON labels as initialisation, the cluster sizes were refined to 2095, 2279, and 2274 samples, respectively (see Table 3). This corresponds to a 20% improvement in prediction, indicating that the probabilistic second stage improved the points-to-cluster assignments, although the dataset’s complex, non-Gaussian structure limited perfect recovery.

Table 3. True and predicted cluster assignments for RUN-ICON and SUN for the “Mickey Mouse” dataset.

Cluster	True Count	RUN-ICON Count	SUN (RUN-ICON + GMM) Count
“head”	3805	1729	2095
left “ear”	1468	2462	2279
right “ear”	1375	2457	2274

The stochastic nature of the SUN results in Figure 7b is captured through the GMM membership probabilities. For each point ${\mathbf{x}}_i$, instead of a deterministic assignment as in (a), we have the following:

$${\gamma }_{ik}\in [0,1],\sum _{k=1}^K{\gamma }_{ik}=1$$

Points in the interior of each circular region (face and ears) exhibit high confidence with ${max}_k\left({\gamma }_{ik}\right)>0.90$. However, at the intersection regions where the ears connect to the face, membership probabilities ${\gamma }_{ik}$ range from 0.50 to less than 1.0, appropriately reflecting the geometric ambiguity where clusters overlap. This probabilistic framework allows SUN to handle the inherent uncertainty in boundary classification that deterministic methods cannot capture.

3.3. “Particles” Dataset

The dataset comprises 146,400 particles simulated using computational fluid dynamics (CFD) to model particle dispersion from a coughing event in an enclosed room [97]. This setup reflects realistic turbulent flow conditions, relevant to understanding airborne disease transmission [98,99,100]. Figure 8 schematically illustrates the cough, showing particles expelled mainly along a straight trajectory, with some lateral spread.

Figure 8. Visualisation of a coughing person and the resultant spread of particles in multiple directions. The image was generated through Microsoft Bing Image Creator for illustration purposes [101].

Particle diameters range from 19 to 110 µm, and analysis focuses on the early dispersion phase (0.12 s after cough onset). Figure 9 depicts lateral dispersion: the x-axis represents particle diameter, and the y-axis the lateral distance from the mouth (0). The RUN-ICON algorithm within the SUN framework predicts optimal clustering into five groups with 87% confidence and 12% uncertainty (Figure 10).

Figure 9. Illustration of the “particles” dataset.

Figure 10. RUN-ICON predictions for the optimal number of clusters for the “particles” dataset. Different numbers of clusters have been considered, and their CDIs are plotted along with their corresponding uncertainties.

Predicted clusters show that particles near the high-velocity core are separated from those displaced upward or downward by inertial effects, forming three clusters along the main streamline and two above and below it [102]. When K-means is used, small particles are overaggregated across lateral sides (Figure 11a). In contrast, the GMM approach better redistributes particles around the predicted centres, capturing flow-induced segregation and providing a clearer distinction between particles near the core and those displaced laterally consistent with inertial effects and the spatial velocity gradient in the flow [95]. The stochastic nature of the SUN results in Figure 11b is captured through the GMM membership probabilities. For each point ${\mathbf{x}}_i$, instead of a deterministic assignment as in (a), we have the following:

$${\gamma }_{ik}\in [0,1],\sum _{k=1}^K{\gamma }_{ik}=1$$

Figure 11. Clustering of the “particles” dataset using (a) K-means initialised with RUN-ICON centres and (b) SUN (RUN-ICON + GMM). Colours are kept consistent across panels and correspond to the five flow-based clusters: three along the main trajectory (separated by particle size) and two representing upward and downward lateral displacement due to inertial effects. Panel (a) shows the rigid partitions produced by K-means, while panel (b) demonstrates how SUN’s probabilistic assignments better capture the continuous transition between flow regimes and highlight uncertainty near boundary regions.

For particles clearly within a single flow regime, ${max}_k\left({\gamma }_{ik}\right)>0.80$ typically. However, particles at the transitions between flow regions exhibit lower certainties with ${\gamma }_{ik}$ from 0.3 to less than 1.0, reflecting the physical reality that particles near regime boundaries experience competing influences. This soft assignment capability is particularly valuable for understanding the continuous nature of particle dispersion, where hard boundaries imposed by deterministic clustering may not accurately represent the underlying physics.

3.4. Multimodal Wine Recognition Dataset

To evaluate the performance of the SUN algorithm on multimodal and chemically complex data, we applied it to the well-established Wine Recognition dataset [94]. This dataset originates from chemical analyses of wines produced in the same region of Italy, but derived from three distinct cultivars. Each sample is described by thirteen continuous physicochemical measurements, including alcohol content, phenolic compositions, colour intensity, and proline concentration. A total of 178 instances are available, grouped into three classes comprising 59, 48, and 71 samples, respectively. The dataset has been widely used in benchmark studies of chemometric and statistical learning methods, in which RDA, QDA, LDA, and nearest-neighbour classifiers have been assessed. Its well-behaved class structure and moderate dimensionality make it an appropriate and historically validated dataset for testing new algorithms in unsupervised learning and classification [103,104].

The first stage of the SUN algorithm, RUN-ICON, was applied to determine the correct number of clusters and to generate the optimal clustering assignment. The CDI results for the Wine Recognition dataset revealed a strong, unambiguous preference for three clusters. The three-cluster solution achieved a CDI of 98% with an uncertainty of only 3%, whereas all alternative cluster numbers yielded CDI scores below 30%. This finding aligns perfectly with the known class structure of the dataset and is further illustrated in Figure 12, where the three-cluster configuration emerges as overwhelmingly dominant relative to alternatives. RUN-ICON also produced initial cluster membership estimates of 60, 55, and 63 samples for Clusters 1, 2, and 3, respectively.

Figure 12. RUN-ICON predictions for the optimal number of clusters for the Wine Recognition dataset. Different numbers of clusters have been considered, and their CDIs are plotted along with their corresponding uncertainties.

To further refine the distribution of samples across clusters, the GMM algorithm was applied, using the RUN-ICON output as initialisation. After the application of the GMM, the cluster sizes were updated to 63, 50, and 65 samples. To evaluate the quality of the two stages, we compared the predicted cluster sizes with the true class counts. RUN-ICON alone yielded an average absolute relative deviation of 9.18% across the three clusters. After GMM refinement, this error decreased to 6.46%, indicating an improvement of approximately 29.6%. Thus, while RUN-ICON provided an accurate initial estimate of the number of clusters and a reasonable first allocation of points, the addition of the GMM further improved the assignment consistency by redistributing points more faithfully according to the true underlying class distribution.

The overall results are summarised in Table 4, which presents the true cluster sizes alongside those predicted by RUN-ICON alone and those obtained after the full SUN procedure. These findings confirm that the SUN algorithm not only identifies the correct cluster count with high confidence but also benefits from the GMM refinement step, which enhances the internal consistency and accuracy of the final cluster assignments. Given the multimodal composition of the wine chemical attributes and the intrinsic variability among cultivars, this experiment demonstrates that SUN provides a reliable and effective methodology for unsupervised structure identification in real-world high-dimensional datasets.

Table 4. True and predicted cluster assignments for RUN-ICON and SUN for the Wine Recognition dataset.

Cluster	True Count	RUN-ICON Count	SUN (RUN-ICON + GMM) Count
1	59	60	63
2	48	55	50
3	71	63	65

The stochastic nature of the SUN results is captured through the GMM membership probabilities. For each point ${\mathbf{x}}_i$, instead of a deterministic assignment, we have the following:

$${\gamma }_{ik}\in [0,1],\sum _{k=1}^K{\gamma }_{ik}=1$$

Points near cluster centres have ${max}_k\left({\gamma }_{ik}\right)>0.78$, while boundary points may have ${\gamma }_{ik}$ ranging from 0.5 to less than 1.0, reflecting classification uncertainty.

3.5. Comparisons with Other Clustering Methods/Schemes

The results presented for all four cases demonstrate the strengths of the proposed algorithm. While comprehensive benchmarking against alternative clustering schemes (e.g., K-means variants, stand-alone GMM) using standard external validity metrics (Adjusted Rand Index, NMI, Silhouette) can be informative, it is not strictly necessary here because the RUN-ICON methodology has already been systematically evaluated and reported by Christakis and Drikakis [48,74]. In two recent studies the authors introduced RUN-ICON, demonstrated its ability to identify the optimal number of clusters by using the Clustering Dominance Index–CDI and an explicit uncertainty score across repeated K-means++ runs, and validated the method on multiple synthetic and real datasets (including multimodal and particle-dispersion examples), reporting high CDI values, low uncertainty, and practical runtimes for correct cluster recovery. Notably, Silhouette Coefficient (SC) benchmarking has already been reported for related RUN-ICON applications in [74], where SC was used to compare baseline clustering methods, further supporting the use of standard validity metrics in prior evaluations. Furthermore, where probabilistic refinement was appropriate, they showed that RUN-ICON’s dominant centres can be used to initialise a GMM and that EM-based GMM redistribution produces refined, consistent cluster assignments—i.e., RUN-ICON both robustly locates stable centres and integrates effectively with model-based clustering refinements [76]. Taken together, these results (high CDI/low uncertainty on benchmark sets, consistent behaviour across repetitions, and demonstrated integration with GMM refinement) provide direct evidence that RUN-ICON produces reliable clusterings in the kinds of datasets studied here; we therefore adopt RUN-ICON as our primary clustering/initialisation strategy and focus this work on the SUN uncertainty-aware contributions.

3.6. Potential Applications

Beyond particle dispersion analysis and wine recognition, we believe that the SUN framework has broad applicability across diverse domains, such as the following:

• Medical Imaging: Tumour segmentation in MRI/CT scans with uncertainty quantification for surgical planning.
• Financial Markets: Anomaly detection in trading patterns with confidence intervals for risk assessment.
• Climate Science: Classification of weather patterns with probabilistic boundaries for improved forecasting.
• Manufacturing: Quality control clustering with confidence metrics for defect detection.
• Social Networks: Community detection with probabilistic boundaries for influence analysis.
• Genomics: Cell type identification in single-cell RNA sequencing with uncertainty estimates.
• Cybersecurity: Network traffic clustering for intrusion detection with false positive probability estimates.

4. Conclusions

In this work, we have presented a comprehensive study of the “Stochastic UNsupervised learning” (SUN) methodology, which integrates the “Reduce UNcertainty and Increase CONfidence” (RUN-ICON) algorithm for initial cluster identification with Gaussian Mixture Models (GMMs) for refined probabilistic clustering. Our approach has been specifically designed to handle the challenges posed by multimodal and noisy datasets, a scenario commonly encountered in natural processes and real-world applications. The results obtained from extensive experiments on benchmark datasets, including the well-established Wine Recognition dataset, demonstrate that our methodology consistently identifies the true underlying cluster structure with a high degree of accuracy, overcoming limitations observed in traditional clustering approaches.

The primary contribution of this study lies in the synergistic combination of RUN-ICON and the GMM. RUN-ICON efficiently detects initial cluster centroids, providing a robust starting point that significantly reduces the risk of convergence to suboptimal local minima in the subsequent GMM phase. By leveraging these high-quality initial estimates, the GMM is able to iteratively refine the cluster assignments, ensuring that the resulting clusters closely align with the actual data distribution. This two-step procedure not only improves the accuracy of cluster identification but also enhances the interpretability of the results, allowing for more meaningful insights into the underlying structure of complex datasets.

A particularly noteworthy aspect of the SUN methodology is its ability to handle datasets characterised by uneven or imbalanced cluster sizes. Traditional clustering algorithms, such as K-means or a standard GMM with random initialisation, often struggle in these situations, leading to overaggregation or misclassification of minority clusters. Our experiments demonstrate that by initialising the GMM with RUN-ICON centroids, the SUN approach successfully mitigates these biases, producing cluster distributions that match the true class proportions even in challenging scenarios. This property is of critical importance in applications where the accurate identification of small or subtle clusters can significantly impact downstream analysis or decision-making processes.

Furthermore, the SUN methodology exhibits strong performance in the context of multimodal datasets. Real-world data frequently contain complex distributions that cannot be adequately captured by single-mode assumptions, and our results show that the combination of RUN-ICON and the GMM is particularly effective in uncovering these structures. By first identifying approximate cluster centres and then allowing the GMM to model the continuous probabilistic boundaries, the SUN approach captures the intrinsic variability within each cluster, reflecting the true multimodal nature of the data. This capability highlights the method’s generalisability and its potential for application across diverse domains, ranging from chemical analysis to biomedical datasets, where the recognition of subtle multimodal patterns is essential.

In addition to accuracy improvements, our methodology provides practical advantages in terms of computational efficiency and robustness. RUN-ICON’s deterministic nature ensures consistent initialisation, reducing the variability and unpredictability often observed in stochastic clustering methods. This determinism, combined with the probabilistic refinement offered by the GMM, creates a balanced framework that is both reliable and scalable. Consequently, the SUN methodology can be applied to large datasets without significant computational overhead, while maintaining robustness against noise and outliers. These properties render the method particularly suitable for high-dimensional data scenarios, where traditional clustering approaches often fail to provide reliable results.

Another important contribution of this study is the benchmarking and comparison of SUN against established methods. By evaluating cluster quality using metrics such as cluster size distribution alignment, we have provided strong empirical evidence of the superiority of our approach in both balanced and imbalanced datasets. The consistent performance across multiple test cases reinforces the utility of the methodology and validates its design principles. Moreover, the flexibility of the SUN framework allows for the incorporation of alternative clustering algorithms in place of the GMM, potentially extending its applicability to a wider range of data characteristics and problem domains.

From a methodological standpoint, the SUN approach bridges the gap between deterministic and probabilistic clustering paradigms. The initial deterministic clustering ensures a strong foundation by capturing the primary modes of the data, while the subsequent probabilistic modelling accounts for finer nuances and overlapping structures. This dual perspective not only improves clustering fidelity but also provides richer information about the uncertainty and confidence associated with each data point’s cluster assignment. Such information is invaluable in applications where risk assessment or probabilistic reasoning is required, including predictive modelling, quality control, and exploratory data analysis.

Looking forward, the results of this study suggest several promising avenues for further research and development. Integrating feature selection or dimensionality reduction techniques within the SUN framework could enhance its performance on extremely high-dimensional datasets, while exploring adaptive or data-driven mechanisms for determining the optimal number of clusters could reduce reliance on prior knowledge and increase the method’s autonomy. Extending SUN to dynamic or streaming datasets presents an exciting opportunity, allowing the algorithm to continuously update cluster structures in real time and maintain high accuracy in evolving data environments. Investigating scalability on high-performance computing architectures could facilitate its application to large-scale simulations, and hybrid models that incorporate graph-based clustering or deep feature extraction could broaden its applicability across diverse scientific and industrial domains. Further refinement of the probabilistic modelling components may improve interpretability, particularly in scenarios requiring careful assessment of complex uncertainty structures for decision-making. By pursuing these directions, the SUN framework has the potential to advance robust, uncertainty-aware unsupervised learning methodologies, ensuring that AI-driven clustering remains reliable, scalable, and adaptable to the growing demands of real-world data analysis.