High-dimensional engineering datasets often combine class imbalance, noisy structures, and limited ground truth, making unsupervised analysis difficult to evaluate reliably. This study quantifies how three properties—partition stability, minority class recovery, and topological interpretability—are traded off across clustering methods, using a capacity-matched 25-seed comparison
[...] Read more.
High-dimensional engineering datasets often combine class imbalance, noisy structures, and limited ground truth, making unsupervised analysis difficult to evaluate reliably. This study quantifies how three properties—partition stability, minority class recovery, and topological interpretability—are traded off across clustering methods, using a capacity-matched 25-seed comparison on a TCGA-derived RNA expression dataset (10,095 samples, 19 cancer types, 13,634 genes). We compare K-means across cluster counts
, self-organizing maps (SOMs) across lattice sizes from 25 to 625 nodes, consensus K-means, a granularity-matched SOM-Super20 control, and four modern baselines (HDBSCAN, spectral clustering, Gaussian mixtures, and Leiden). At matched prototype budgets, K-means is both more reproducible and substantially better at recovering minority classes than SOMs: at 400 prototypes, K-means achieves pairwise NMI
versus
for the
SOM and recovers the smallest cancers 6–
more effectively (pancreas effective coverage
vs.
).Crucially, the SOM does not close this gap even when given more prototypes (
at 625 nodes), so, under matched capacity, minority recovery is better explained by representational capacity and centroid allocation freedom than by topology preservation. The recovery is not free: increasing
k overfragments the partition and lowers the pairwise ARI stability (
from
to
), while the NMI remains robust (
). The hardest minority, pancreas, is recovered only by high-capacity K-means and by no other method evaluated, including SOMs at any size, consensus K-means, SOM-Super20, HDBSCAN, Gaussian mixtures, spectral clustering, and Leiden. The SOM’s distinct value is therefore not stability or recovery but the interpretable two-dimensional topological visualization that it uniquely provides, including a gradient-organized structure that is reproducible across seeds for kidney (weaker for uterus). No single method optimizes all three properties; the appropriate choice depends on whether a task prioritizes reproducibility, minority recovery, or visual interpretability. Because these conclusions follow from the shape of the data and the allocation behavior of the algorithms rather than from biological semantics, we expect them to transfer to high-dimensional imbalanced engineering data, such as those from fault clustering, condition monitoring, and anomaly detection.
Full article