Prevalence-Insensitive Evaluation of Diagnostic Systems Under Class Imbalance: The Harmonic Mean of Per-Class Sensitivity

Abstract

In safety-critical diagnosis under class imbalance, widely used global metrics—e.g., accuracy, F-score, and MCC—conflate class-conditional behavior with class priors, often yielding overly optimistic assessments of overall performance, thereby hindering stable comparisons across datasets. We formalize the prevalence sensitivity of these indices and advance a prevalence-insensitive evaluation built solely from class-conditional rates (binary TPR/TNR; multiclass per-class sensitivities). We analyze the arithmetic, geometric, and harmonic means as class-symmetric aggregators and provide a geometric characterization of their isocurves. Empirically, on 18 UCI datasets, accuracy exceeded the harmonic mean by ∼3% on average, but the gap widened on skewed sets (e.g., cardiotocography: $\mathrm{Acc}=0.896$ vs. $H=0.803$, $+11.5\%$). On a 6756-sample, 35-class tumor dataset, high overall agreement ($\mathrm{Acc}=0.906$, Cohen’s $\kappa =0.899$) coexisted with dispersed per-class sensitivities ($A=0.795$, $G=0.743$, $H=0.658$) and moderate MCP ($0.564$), evidencing weak minority-class performance despite strong accuracy. These results show that reliance on accuracy or $\kappa$ can overstate performance precisely where it matters most, inflating perceived safety and obscuring rare-class failure modes. Prevalence-insensitive, class-symmetric means—especially the harmonic mean of per-class sensitivity—yield conservative, comparable summaries better aligned with risk-aware evaluation and deployment in critical systems.

1. Introduction

Model evaluation is central to the scientific value of machine learning (ML): the chosen metric defines what counts as reliable performance for a given application and directly shapes model selection and deployment decisions. Because different metrics emphasize different aspects of behavior (e.g., error rate versus sensitivity to class imbalance), they can rank the same models differently on the same data [1]. In safety-critical contexts, this makes the metric choice part of the problem specification itself, since it encodes which kinds of errors the system must avoid and how uneven class performance should be properly penalized [2].

There exists a wide ecosystem of performance indices—accuracy and error rate, class-conditional sensitivity/recall, $\mathrm{F}$-scores [3,4], rank-based and agreement-based measures such as Matthews Correlation Coefficient (MCC) [5] and Cohen’s $\kappa$ [6], the ROC curve [7,8,9], and multiclass generalizations—each with distinct statistical and operational semantics [10,11,12]. This richness is useful but hazardous if metrics are chosen without regard to data and objectives, or when reported scores are internally inconsistent [13].

In many applications the target (positive) class is rare (e.g., fraud, medical screening, fault detection); more broadly, in multiclass settings one or more classes may have very low prevalence—despite being equally important as the common classes—so imbalance manifests as uneven per-class prevalences rather than a single positive–negative asymmetry. Under such class imbalance, metrics that mix class-conditional terms (probabilities conditioned on the true class; e.g., true positive rate) with class priors (unconditional probabilities of each class, i.e., class proportions) can be dominated by the majority class, masking poor minority performance. Foundational studies and surveys document both the ubiquity of imbalance and its effects on learning and evaluation [14,15,16,17]. Consequently, conclusions drawn from prevalence-sensitive metrics can be unstable across datasets that differ only in class proportions.

When the scientific goal is to develop a predictive model that is accurate and reliable, it is preferable to base model selection and validation on prevalence-insensitive criteria that reflect intrinsic class-conditional performance. Using aggregates of per-class sensitivity and threshold-free analyses (e.g., Multiclass Classification Performance (MCP) [18]) decouples model quality from the class prior, yielding estimates that remain stable under base-rate shifts. Operational costs and prevalences can then be incorporated at deployment through decision analysis, without confounding model selection with prevalence effects [19].

Thus, a principled route to prevalence-insensitivity is to construct evaluation indices from the class-conditional true positive rate (sensitivity) and true negative rate (specificity)—and, in the multiclass case, from per-class sensitivities. Because these are conditioned on the true class, they are invariant to changes in prevalence and thus provide a stable basis for summary measures [20].

Aggregating sensitivity (and specificity) via the Pythagorean means (arithmetic, geometric, and harmonic) yields class-symmetric, prevalence-insensitive summaries with different penalizations of imbalance between components. This means-based perspective makes explicit how the choice of mean encodes the desired conservativeness against asymmetric errors, while remaining robust to class-prior changes [21,22].

For binary datasets, the arithmetic mean (balanced accuracy), $\mathrm{A}$, and the geometric mean, $\mathrm{G}$, are widely reported in imbalanced-learning research, whereas the harmonic mean, $\mathrm{H}$, of sensitivity and specificity is comparatively rare despite its appealingly conservative nature. Its use is documented in domain-specific studies (e.g., biomedical diagnostics), yet it remains less common as a headline metric than balanced accuracy or the G-mean in general ML practice [23,24,25]. In multiclass settings, the harmonic mean of per-class true positive rates is even less common, despite its ability to reveal weak per-class performance in critical systems.

In safety-critical diagnosis, the evaluation criterion must encode minimum class-wise detection requirements and remain stable under prevalence shifts; otherwise, aggregate scores can coexist with unacceptable under-detection of specific conditions. Adopting prevalence-insensitive, class-symmetric summaries built from per-class sensitivities—and, in particular, the harmonic mean—aligns model selection with clinical safety by rewarding uniformly adequate coverage rather than compensatory trade-offs. This choice yields estimates that are robust across datasets and supports transparent auditing of class-wise performance.

This paper advocates prevalence-insensitive evaluation in critical domains and, in particular, the use of measures built from per-class true positive rates. It addresses three shortcomings in current practice: (i) the widespread reliance on prevalence-sensitive indices—such as accuracy and F-score—whose values vary with class proportions and can therefore overstate reliability under imbalance; (ii) the absence of a conservative, class-symmetric summary that prevents high overall scores when any class is under-detected; and (iii) the scarcity of multiclass-oriented formulations that remain invariant to base-rate shifts while retaining clear interpretability.

On the UCI benchmarks, we observe a modest yet systematic overestimation by global metrics relative to class-symmetric means. In the multiclass cancer case study, high overall scores coexist with low per-class sensitivities for several tumor types; these global aggregates mask class-specific weaknesses and can lead to missed diagnoses with serious clinical consequences. In summary, these findings support the harmonic mean as a prudent, conservative choice to mitigate overestimation when some classes exhibit weak sensitivity.

Our contributions are: (i) a formal analysis that separates prevalence-sensitive from prevalence-insensitive measures by examining their dependence on class priors; (ii) a systematic treatment that frames the Pythagorean means—arithmetic, geometric, and harmonic—as class-symmetric aggregators of true-positive rates (binary: sensitivity and specificity; multiclass: per-class sensitivities); (iii) practical guidance for safety-critical evaluation, recommending the harmonic mean of per-class sensitivities as a conservative, imbalance-robust summary; and (iv) empirical validation across standard benchmarks and a high-dimensional, 35-class cancer diagnosis task, demonstrating consistent gaps between accuracy-like indices and prevalence-insensitive summaries.

2. Prevalence-Insensitive vs. Prevalence-Sensitive Measures

Let $(X,Y)\sim \mathcal{P}$ with $Y\in \{0,1\}$, where $Y=1$ denotes the positive class and $Y=0$ the negative class. A (deterministic) classifier is a mapping $h:\mathcal{X}\to \{0,1\}$. More generally, a scoring model outputs a function $s:\mathcal{X}\to \mathbb{R}$, which is converted into a classifier by thresholding: for $t\in \mathbb{R}$, the induced classifier is ${\widehat{Y}}_t\left(x\right):=\mathbf{1}\{s\left(x\right)\ge t\}$. Let $\pi =\mathbb{P}(Y=1)$ denote the positive-class prevalence (so $\mathbb{P}(Y=0)=1-\pi$). We write $\mathbf{1}\left\{E\right\}$ for the indicator of an event E, equal to 1 if E holds, and 0 otherwise.

Performance summaries aggregate different types of errors that have distinct operational meaning. Global metrics that average over classes (e.g., overall accuracy) depend on the class prior $\pi$, while class-conditional rates are prevalence-invariant and thus separate the intrinsic discriminative ability of a classifier from the composition of the test population.

For a sample ${\left\{(x_i,y_i)\right\}}_{i=1}^n$ and a fixed classifier $\widehat{Y}$, define the counts

$$\begin{array}{c}\hfill \mathrm{TP}=\sum _{i=1}^n\mathbf{1}\{\widehat{Y}\left(x_i\right)=1,y_i=1\}\mathrm{FP}=\sum _{i=1}^n\mathbf{1}\{\widehat{Y}\left(x_i\right)=1,y_i=0\}\\ \hfill \mathrm{FN}=\sum _{i=1}^n\mathbf{1}\{\widehat{Y}\left(x_i\right)=0,y_i=1\}\mathrm{TN}=\sum _{i=1}^n\mathbf{1}\{\widehat{Y}\left(x_i\right)=0,y_i=0\}\end{array}$$

(1)

Let $\mathrm{P}=\mathrm{TP}+\mathrm{FN}$ and $\mathrm{N}=\mathrm{TN}+\mathrm{FP}$ be the numbers of positive and negative instances, with $n=\mathrm{P}+\mathrm{N}$. The confusion matrix is shown in

Table 1. Confusion Matrix.

	$\widehat{\mathit{Y}}=1$	$\widehat{\mathit{Y}}=0$
$Y=1$	$\mathrm{TP}$	$\mathrm{FN}$
$Y=0$	$\mathrm{FP}$	$\mathrm{TN}$

.

The fundamental class-conditional rates are

$$\begin{array}{c}\hfill \mathrm{TPR}=\mathbb{P}(\widehat{Y}=1\mid Y=1)={\displaystyle \frac{\mathrm{TP}}{\mathrm{P}}}(\mathrm{True}\mathrm{Positive}\mathrm{Rate})\\ \hfill \mathrm{TNR}=\mathbb{P}(\widehat{Y}=0\mid Y=0)={\displaystyle \frac{\mathrm{TN}}{\mathrm{N}}}(\mathrm{True}\mathrm{Negative}\mathrm{Rate})\\ \hfill \mathrm{FPR}=\mathbb{P}(\widehat{Y}=1\mid Y=0)={\displaystyle \frac{\mathrm{FP}}{\mathrm{N}}}(\mathrm{False}\mathrm{Positive}\mathrm{Rate}).\end{array}$$

(2)

These rates quantify, respectively, sensitivity to positives, specificity to negatives, and the spurious activation rate on negatives. Because they are conditional on the true class, $\mathrm{TPR}$ and $\mathrm{TNR}$ are invariant to changes in $\pi$.

In binary classification with class label $Y\in \{0,1\}$, the prevalence $\pi :=\mathbb{P}(Y=1)$ (class prior) can be highly skewed. Class imbalance is ubiquitous in ML and profoundly affects model selection and metric interpretation [14,15,16]. A central distinction is between prevalence-insensitive (class-imbalance-invariant) measures—those that depend only on class-conditional performance—and prevalence-sensitive measures—those whose values change with $\pi$ even if the class-conditional behavior of the classifier remains fixed. Below we structure key measures into these two groups and formalize their dependence.

2.1. Prevalence-Insensitive (Class-Imbalance-Invariant) Measures

Let $\mathrm{TPR}$ (sensitivity) and $\mathrm{TNR}$ (specificity), as defined in Equation (2). These class-conditional rates are invariant to $\pi$ by definition. Several standard summaries inherit this invariance.

2.1.1. Balanced Accuracy (Arithmetic Mean A of TPR and TNR)

$$\mathrm{A}={\displaystyle \frac{\mathrm{TPR}+\mathrm{TNR}}{2}}$$

It aggregates per-class sensitivities additively and equals 1 iff both class-conditional sensitivities are 1. Properties are developed in [26]. By construction, $\mathrm{A}$ is invariant to $\pi$ since it is a function of $(\mathrm{TPR},\mathrm{TNR})$ only.

2.1.2. Youden’s $\mathrm{J}$ (Informedness)

$$\mathrm{J}=\mathrm{TPR}+\mathrm{TNR}-1=2\mathrm{A}-1$$

Introduced for diagnostic testing [27], $\mathrm{J}$ is a linear rescaling of $\mathrm{A}$ and hence also prevalence-invariant. It measures the vertical distance of a ROC point from the no-skill diagonal [9].

2.1.3. $\mathrm{G}$-Mean (Geometric Mean of TPR and TNR)

$$\mathrm{G}=\sqrt{\mathrm{TPR}·\mathrm{TNR}}$$

Widely used in imbalanced learning as a class-symmetric criterion that penalizes uneven performance [15]. As a product of class-conditionals, $\mathrm{G}$ is independent of $\pi$.

2.1.4. $\mathrm{H}$-Mean (Harmonic Mean of TPR and TNR)

$$\mathrm{H}={\displaystyle \frac{2\mathrm{TPR}\mathrm{TNR}}{\mathrm{TPR}+\mathrm{TNR}}}$$

This metric is stricter than $\mathrm{A}$ and $\mathrm{G}$ under dispersion. It has been mainly adopted in biomedical applications [28,29,30]. As a symmetric mean of $(\mathrm{TPR},\mathrm{TNR})$, it is prevalence-invariant.

2.1.5. The ROC Curve

For a score $s\left(X\right)$ thresholded at t, the ROC locus is $\left(\mathrm{FPR}\left(t\right),\mathrm{TPR}\left(t\right)\right)$ with $\mathrm{FPR}=1-\mathrm{TNR}$. Both ROC and its area depend only on the pair of class-conditional score distributions and are therefore invariant to $\pi$ [9].

2.2. Prevalence-Sensitive (Class-Imbalance-Dependent) Measures

The following measures vary with $\pi$ even when $(\mathrm{TPR},\mathrm{TNR})$ are fixed; they mix class-conditional terms with priors or marginals.

2.2.1. Overall Accuracy

The decomposition $\mathrm{Acc}\left(t\right)=\mathbb{P}\left({\widehat{Y}}_t=Y\right)=\pi \mathrm{TPR}\left(t\right)+(1-\pi )\mathrm{TNR}\left(t\right)$ shows that accuracy conflates class prior and conditional performance, whereas $\mathrm{TPR}$ and $\mathrm{TNR}$ isolate sensitivity to positives and negatives, respectively. Thus, for fixed $(\mathrm{TPR},\mathrm{TNR})$, accuracy increases as the prior mass shifts toward the better-performing class [14,15].

2.2.2. Precision

Precision, also named Positive Predictive Value (PPV), is $\mathrm{PPV}=\mathbb{P}(Y=1\mid \widehat{Y}=1)$, which by Bayes’ rule equals

$$\mathrm{PPV}={\displaystyle \frac{\pi \mathrm{TPR}}{\pi \mathrm{TPR}+(1-\pi )(1-\mathrm{TNR})}}$$

explicitly dependent on $\pi$.

2.2.3. F₁-Score

The ${\mathrm{F}}_1$-score [3,4],

$${\mathrm{F}}_1={\displaystyle \frac{2\mathrm{PPV}·\mathrm{TPR}}{\mathrm{PPV}+\mathrm{TPR}}}$$

inherits the PPV dependence.

2.2.4. Precision-Recall Curve

Precision-recall (PR) curves and their area are likewise prevalence-sensitive [11,31]. As PR plots TPR against precision, and the latter depends on $\pi$, the area under the PR curve is sensitive to class imbalance.

2.2.5. Matthews Correlation Coefficient

In binary classification, outcomes are summarized by the $2\times 2$ confusion matrix shown in Table 1, where their elements are calculated in Equation (1). The Matthews correlation coefficient (MCC) [5] is defined as

$$\mathrm{MCC}={\displaystyle \frac{\mathrm{TP}·\mathrm{TN}-\mathrm{FP}·\mathrm{FN}}{\sqrt{(\mathrm{TP}+\mathrm{FP})(\mathrm{TP}+\mathrm{FN})(\mathrm{TN}+\mathrm{FP})(\mathrm{TN}+\mathrm{FN})}}}$$

It equals the Pearson correlation between predicted and true binary labels and takes values in $[-1,1]$ (1 for perfect prediction, $-1$ for total disagreement). $\mathrm{MCC}$ is not invariant to $\pi$: for fixed $(\mathrm{TPR},\mathrm{TNR})$ the numerator and denominator scale differently with positives and negatives, i.e., $(\mathrm{P},\mathrm{N})=(\pi n,(1-\pi \left)n\right)$, so $\mathrm{MCC}$ varies with $\pi$ [1].

2.2.6. Cohen’s $\kappa$

Let $p_o=\mathrm{Acc}$ be the observed agreement and $p_e$ the chance agreement computed from the empirical marginals; then

$$\kappa ={\displaystyle \frac{p_o-p_e}{1-p_e}}$$

In the binary case, if $\pi =Pr(Y=1)$ is the prevalence and $\rho =Pr(\widehat{Y}=1)$ the predicted positive rate, then

$$p_o=\pi \mathrm{TPR}+(1-\pi )\mathrm{TNR}{p}_e=\pi \rho +(1-\pi )(1-\rho )$$

so both $p_o$ and $p_e$ depend on $\pi$ (and on $\rho$). The prevalence paradox is the empirical fact that $\kappa$ can be low—even near zero—despite very high accuracy when classes are rare: as $\pi$ moves away from $1/2$, $p_e$ increases toward $p_o$, shrinking the numerator $p_o-p_e$ [6,32]. Hence $\kappa$ is prevalence-sensitive.

2.3. Summary

In general, prevalence-insensitive measures (TPR, TNR, $\mathrm{A}$-mean, J, $\mathrm{G}$-mean, $\mathrm{H}$-mean, ROC) quantify intrinsic discrimination independently of $\pi$, which is critical for comparing models across datasets with different priors. Prevalence-sensitive measures (accuracy, PPV, F₁, PR, MCC, $\kappa$) remain essential for operational decision-making because they reflect the interaction between model behavior and base rates. However, under imbalance they must be interpreted jointly with class-conditional rates and priors [15,16] or complemented by a class-specific perspective [33].

Prevalence-insensitive evaluation should be the default when reliability across classes matters. Metrics that mix class-conditional terms with class priors (e.g., accuracy and F₁) can change merely because the class proportions change, even if the class-conditional behavior of the classifier does not. In contrast, indices constructed from true positive rates—binary TPR/TNR or, in the multiclass case, the vector of per–class TPRs—are invariant to prevalence by definition. Consequently, any symmetric summary of these components (e.g., the Pythagorean means of per-class TPRs: arithmetic, geometric, and harmonic) inherits prevalence-insensitivity, enabling fair comparisons across datasets and shifts in base rates.

3. The Pythagorean Means

The Pythagorean school, founded by Pythagoras of Samos (6th–5th century BCE), treated number and proportion as the underlying structure of reality. Its program linked arithmetic, geometry, and musical harmony. Within this tradition arose the triad of the so-called Pythagorean means—arithmetic (equality of increments), geometric (proportionality), and harmonic (reciprocity)—which later authors (e.g., Euclid, Nicomachus) formalized and transmitted.

Henceforth, we write A, G, and H for the arithmetic, geometric, and harmonic mean, respectively.

Let $x_1,\dots ,x_n>0$ and let $w_1,\dots ,w_n>0$ be weights with $W={\sum }_{i=1}^n{w}_i$.

$$\mathrm{Arithmetic}\mathrm{mean}:{\mathrm{A}}_w={\displaystyle \frac{1}{W}}\sum _{i=1}^n{w}_i{x}_i$$

$$\mathrm{Geometric}\mathrm{mean}:{\mathrm{G}}_w=\prod _{i=1}^n{x}_i^{w_i/W}$$

$$\mathrm{Harmonic}\mathrm{mean}:{\mathrm{H}}_w={\displaystyle \frac{W}{{\sum }_{i=1}^n{\displaystyle \frac{w_i}{x_i}}}}$$

In the unweighted case ($w_i=1$, $W=n$) these reduce to

$$\mathrm{A}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{x}_i\mathrm{G}={\left(\prod _{i=1}^n{x}_i\right)}^{1/n}\mathrm{H}={\displaystyle \frac{n}{{\sum }_{i=1}^n{\displaystyle \frac{1}{x_i}}}}$$

All three means are symmetric and satisfy the chain of inequalities $H_w\le G_w\le A_w$, with equality throughout if and only if $x_1=\cdots =x_n$.

While the Pythagoreans singled out the triad (A,G,H) for their distinct conceptual roles, later analysis unified them under the power means (a.k.a. generalized means), defined for $x_i>0$ by

$$M_r(x_1,\dots ,x_n)=\left\{\begin{array}{cc}{\left(\frac{1}{n}\sum _{i=1}^n{x}_i^r\right)}^{1/r}\hfill & r\ne 0\hfill \\ {\left(\prod _{i=1}^n{x}_i\right)}^{1/n}\hfill & r=0\hfill \end{array}\right.$$

so that $M_1=$ A, $M_0=$ G, and $M_{-1}=$ H.

In general, the monotonicity property ($r<s\Rightarrow M_r\le M_s$) yields $H\le G\le A$. This perspective clarifies the structural position of the Pythagorean means as canonical representatives of the exponents $-1,0,1$ within a continuous one-parameter family.

Beyond the three classical Pythagorean means, generalized families—such as the Kolmogorov–Nagumo–de Finetti quasi-arithmetic means (Appendix A) and the Gini means (Appendix B)—provide parametric lenses for analyzing monotonicity, convexity, and the comparative behavior of TPR/TNR aggregators. These families subsume the Pythagorean triad as special cases and enable principled trade-offs, while remaining prevalence-insensitive because they can be built exclusively from class-conditional quantities [34,35,36,37,38,39].

4. Mean-Based Evaluation from TPR and TNR

Let $\mathrm{TPR}\in [0,1]$ and $\mathrm{TNR}\in [0,1]$ denote, respectively, the true positive rate (sensitivity) and the true negative rate (specificity) of a binary classifier. Any class-symmetric scalar summary based solely on $(\mathrm{TPR},\mathrm{TNR})$ is invariant to the class prior $\pi =\mathbb{P}(Y=1)$.

4.1. Arithmetic Mean $\left(\mathrm{A}\right)$

$$\mathrm{A}={\displaystyle \frac{\mathrm{TPR}+\mathrm{TNR}}{2}}\in [0,1]$$

It aggregates sensitivity and specificity with equal weight and is widely used as a prevalence-robust alternative to overall accuracy [9,10].

For $w\in [0,1]$,

$${\mathrm{A}}_w=w\mathrm{TPR}+(1-w)\mathrm{TNR}$$

a weighted arithmetic mean that reduces to $\mathrm{A}$ at $w=\frac{1}{2}$. The family $\left\{{\mathrm{A}}_w\right\}$ traces a line segment in the unit square connecting the axes-aligned extremes.

As an evaluation criterion it preserves the probabilistic semantics of class-conditional sensitivities and it is invariant to class-prior skews, thereby mitigating the dominance of the majority class in imbalanced settings. In binary classification it is a strictly proper scoring rule [40] for the pair $(\mathrm{TPR},\mathrm{TNR})$, and it admits a principled uncertainty treatment, deriving exact posteriors and credible intervals under conjugate priors, and showing how to compare classifiers on finite samples without relying on asymptotics [26]. In modern ML practice—particularly in imbalanced binary tasks such as anomaly/fraud detection, medical diagnosis, or cost-sensitive classification—balanced accuracy is routinely reported to reflect symmetric performance across the positive and negative strata, and it is often preferred when one wants an additive aggregation of $\mathrm{TPR}$ and $\mathrm{TNR}$ with transparent interpretation in terms of average per-class sensitivity.

4.2. Geometric Mean $\left(\mathrm{G}\right)$

$$\mathrm{G}=\sqrt{\mathrm{TPR}·\mathrm{TNR}}\in [0,1]$$

A weighted variant uses exponents $(w,1-w)$:

$${\mathrm{G}}_w={\mathrm{TPR}}^w{\mathrm{TNR}}^{1-w}\mathrm{for}w\in [0,1]$$

Both $\mathrm{G}$ and ${\mathrm{G}}_w$ are strictly increasing in each argument on ${(0,1]}^2$ and equal 1 iff $\mathrm{TPR}=\mathrm{TNR}=1$.

The geometric mean G is widely adopted in imbalanced-learning research because it heavily penalizes uneven performance: if either sensitivity or specificity is small, G collapses accordingly; conversely, it rewards classifiers that simultaneously maintain both rates high [10,41]. Comprehensive surveys on imbalanced data explicitly recommend G-mean as a robust, threshold-based indicator that captures the joint behavior of minority- and majority-class sensitivities without being biased by class priors [15].

4.3. Harmonic Mean $\left(\mathrm{H}\right)$

$$\mathrm{H}={\displaystyle \frac{2}{{\displaystyle \frac{1}{\mathrm{TPR}}}+{\displaystyle \frac{1}{\mathrm{TNR}}}}}={\displaystyle \frac{2\mathrm{TPR}\mathrm{TNR}}{\mathrm{TPR}+\mathrm{TNR}}}$$

with $\mathrm{TPR}+\mathrm{TNR}>0$, and define $\mathrm{H}=0$ at $(0,0)$ by continuity. A weighted harmonic mean (emphasizing $\mathrm{TPR}$ versus $\mathrm{TNR}$) is

$${\mathrm{H}}_{\beta }={\displaystyle \frac{(1+{\beta }^2)\mathrm{TPR}\mathrm{TNR}}{{\beta }^2\mathrm{TPR}+\mathrm{TNR}}}\beta >0$$

which reduces to $\mathrm{H}$ at $\beta =1$ and assigns more weight to $\mathrm{TNR}$ as $\beta$ increases (and to $\mathrm{TPR}$ when $\beta <1$). While mathematically most conservative among the three classical means, the harmonic mean of $(\mathrm{TPR},\mathrm{TNR})$ is notably less common in mainstream machine learning reporting than $\mathrm{A}$-mean and $\mathrm{G}$-mean. Contemporary surveys typically favor $\mathrm{A}$ or $\mathrm{G}$ for class-symmetric summaries [10,29], but $\mathrm{H}$ arises in scientific applications where one demands a symmetric and strongly conservative aggregation of $\mathrm{TPR}$ and $\mathrm{TNR}$. In biomedical prediction, $\mathrm{H}$ has been explicitly adopted to evaluate classifiers under class imbalance, precisely because it treats both groups equally and punishes departures in either sensitivity or specificity.

Historically, one of the earliest explicit ML uses appears in medical informatics when developing predictors of diabetic nephropathy under irregular and imbalanced clinical data [23], with subsequent adoption in traffic-safety Bayesian networks [24,42], genome-wide psoriasis susceptibility prediction [25], seizure prediction [28], neurodevelopmental EEG [43], and clinical pharmacovigilance for QTc risk [30], all defining or employing $\mathrm{H}$ as the harmonic mean of sensitivity and specificity to weight both error types equally and to deter degenerate single-class solutions under skew.

Methodologically, $\mathrm{H}$ is a symmetric, reciprocal aggregator that penalizes dispersion between class-conditional accuracies and remains prevalence-insensitive because it is built from within-class rates (TPR and TNR computed conditional on the true class). In multiclass settings, applying the same principle to the per-class sensitivities $\left\{{\mathrm{TPR}}_k\right\}$ yields a stringent scalar summary that typically penalizes dispersion more than the arithmetic or geometric mean; a detailed analysis is provided in Section 6.

5. Geometric Characterization of A, G, and H Isocurves

Let $x=\mathrm{TPR}\in [0,1]$ and $y=\mathrm{TNR}\in [0,1]$. For a fixed level $c\in (0,1)$, the level sets (isocurves) of the three means in the $(x,y)$-plane are

$$\begin{array}{cc}\hfill \mathrm{A}:& {\displaystyle \frac{x+y}{2}}=c⟺x+y=2c,\hfill \\ \hfill \mathrm{G}:& \sqrt{xy}=c⟺xy=c^2,\hfill \\ \hfill \mathrm{H}:& {\displaystyle \frac{2}{x^{-1}+y^{-1}}}=c⟺(2x-c)(2y-c)=c^2.\hfill \end{array}$$

Hence, A has straight isocurves (lines of slope $-1$); $\mathrm{G}$ has rectangular hyperbolas with asymptotes $x=0$ and $y=0$; and $\mathrm{H}$ also yields rectangular hyperbolas but translated, with asymptotes $x={\displaystyle \frac{c}{2}}$ and $y={\displaystyle \frac{c}{2}}$, and center at $({\displaystyle \frac{c}{2}},{\displaystyle \frac{c}{2}})$.

Equivalently, as shown in Figure 1, in the $(F,T)=(\mathrm{FPR},\mathrm{TPR})$ plane—similar to the ROC representation—where $y=\mathrm{TNR}=1-F$, the same level c takes the forms

$$\mathrm{A}:T=F+2c-1\mathrm{G}:T={\displaystyle \frac{c^2}{1-F}}\mathrm{H}:T={\displaystyle \frac{c(1-F)}{2-c-2F}}$$

In these $(F,T)$-coordinates, $\mathrm{A}$ isocurves are again lines (now of slope $+1$); $\mathrm{G}$ isocurves are rectangular hyperbolas with asymptotes $T=0$ and $F=1$; and $\mathrm{H}$ isocurves are rectangular hyperbolas with asymptotes at $F=1-\frac{c}{2}$ and $T=\frac{c}{2}$. Geometrically, $\mathrm{A}$ admits linear trade-offs along straight indifference lines; $\mathrm{G}$ enforces multiplicative balance ($xy=\mathrm{const}$) that pulls level sets toward the axes; and $\mathrm{H}$ is the most stringent, with value-dependent asymptotes that require each coordinate to exceed $c/2$ to attain level c.

6. Harmonic Mean of Per-Class Sensitivity

In the binary case, the harmonic mean of sensitivity and specificity, has been used—albeit sparingly—in ML as a symmetric, prevalence-insensitive trade-off between the true positive rate and the true negative rate. In the multiclass setting, this extends naturally by taking the harmonic mean of the per-class sensitivities. Let ${\mathrm{TPR}}_k=Pr(\widehat{Y}=k\mid Y=k)$ denote the true positive rate of class k for $k=1,\dots ,K$; then

$$\mathrm{H}={\displaystyle \frac{K}{{\sum }_{k=1}^K{\displaystyle \frac{1}{{\mathrm{TPR}}_k}}}}$$

To illustrate how an imbalanced class distribution can distort overall performance while masking weak per-class behavior, consider a four-class problem with supports $\left|A\right|=800$, $\left|B\right|=600$, $\left|C\right|=500$, $\left|D\right|=100$ (total $N=2000$). This yields prevalences $(0.40,0.30,0.25,0.05)$, a long-tailed pattern typical of safety-critical multiclass tasks in which some classes—though equally important—are rare. We will compare the standard metrics with class-symmetric summaries (arithmetic, geometric, and harmonic means of the per-class sensitivities). If misclassifications concentrate in the minority class D, the standard metrics remain dominated by the frequent classes A–C and can appear deceptively high, whereas the means—especially the harmonic mean—drop to reflect the poor recall of D. With this in mind, take the following confusion matrix (rows = true classes, columns = predicted classes):

$$\begin{array}{ccccc}& \widehat{A}& \widehat{B}& \widehat{C}& \widehat{D}\\ A& 800& 0& 0& 0\\ B& 0& 600& 0& 0\\ C& 0& 0& 500& 0\\ D& 40& 24& 20& 16\end{array}$$

For this $4\times 4$ confusion matrix, the classical performance scores are numerically high—accuracy = 0.958, $F_1\approx 0.803$, weighted-$F_1\approx 0.943$, multiclass Matthews Correlation Coefficient $\mathrm{MCC}\approx 0.940$ [44,45], and Cohen’s $\kappa \approx 0.938$. Yet these aggregates conceal a severe weakness: class D has ${\mathrm{TPR}}_D=0.16$ (many false negatives). This happens because averaging and global agreement/correlation summaries allow substantial compensation across classes, so high overall scores can coexist with an underrepresented minority.

Per-class sensitivity (true positive rates) are

$$\mathbf{t}=(t_A,t_B,t_C,t_D)=\left(\frac{800}{800},\frac{600}{600},\frac{500}{500},\frac{16}{100}\right)=(1,1,1,0.16).$$

The three Pythagorean means over $\mathbf{t}$:

$$\mathrm{A}={\displaystyle \frac{1}{4}}\sum _{k=1}^4{t}_k=0.79\mathrm{G}={\left(\prod _{k=1}^4{t}_k\right)}^{1/4}\approx 0.6325\mathrm{H}={\displaystyle \frac{4}{{\sum }_{k=1}^{4}1/t_k}}\approx 0.4324$$

Thus, while $\mathrm{A}$ and $\mathrm{G}$ may remain high, $\mathrm{H}$ can drop sharply—correctly signaling that at least one class has very low sensitivity. This behavior is structural: (i) $\mathrm{H}$ is prevalence-insensitive because it aggregates within-class recalls; (ii) it is strictly more stringent than $\mathrm{G}$ and $\mathrm{A}$ ($\mathrm{H}\le \mathrm{G}\le \mathrm{A}$, with equality only when all ${\mathrm{TPR}}_k$ are equal); (iii) it penalizes dispersion by Schur-concavity (any increase in inter-class inequality lowers $\mathrm{H}$); and (iv) it is quantitatively constrained by underperforming classes (Proposition 1).

Proposition 1 

(Bounds for the harmonic mean of per-class TPRs). Let $r=(r_1,\dots ,r_K)\in {(0,1]}^K$ be the per-class sensitivities and $r_{min}={min}_k{r}_k$ and $r_{max}={max}_k{r}_k$. Then:

1. 

Universal bounds.

$$r_{min}\le \mathrm{H}\le r_{max}$$

with equality on either side if and only if $r_1=\cdots =r_K$.

2. 

Upper bound with m weak classes. Assume at least m classes satisfy $r_k\le \tau$ with $0<\tau \le r_{max}\le 1$, and that $r_k\le r_{max}$ for all classes. Then

$$\mathrm{H}\le {\displaystyle \frac{K}{\frac{m}{\tau }+\frac{K-m}{r_{max}}}}\le r_{max}$$

and the first inequality is attained at the extremal vector $(\underset{m}{\underbrace{\tau ,\dots ,\tau }},\underset{K-m}{\underbrace{r_{max},\dots ,r_{max}}})$.

Proof. 

(1) Since $r_{min}\le r_k\le r_{max}$ and $x\mapsto 1/x$ is decreasing on $(0,\infty )$, we have $\frac{1}{r_{max}}\le \frac{1}{r_k}\le \frac{1}{r_{min}}$. Summing over k yields

$${\displaystyle \frac{K}{r_{max}}}\le \sum _{k=1}^K{\displaystyle \frac{1}{r_k}}\le {\displaystyle \frac{K}{r_{min}}}⟹r_{min}\le \mathrm{H}\le r_{max}$$

Equality on either side forces all $r_k$ to be equal, and conversely.

(2) For the m weak classes, $r_k\le \tau$ implies $\frac{1}{r_k}\ge \frac{1}{\tau }$; for the remaining $K-m$ classes, the global ceiling $r_k\le r_{max}$ implies $\frac{1}{r_k}\ge \frac{1}{r_{max}}$. Therefore

$$\sum _{k=1}^K{\displaystyle \frac{1}{r_k}}\ge {\displaystyle \frac{m}{\tau }}+{\displaystyle \frac{K-m}{r_{max}}}⟹\mathrm{H}={\displaystyle \frac{K}{{\sum }_{k}1/r_k}}\le {\displaystyle \frac{K}{\frac{m}{\tau }+\frac{K-m}{r_{max}}}}$$

To prove that this upper bound is itself $\le r_{max}$, observe

$${\displaystyle \frac{K}{\frac{m}{\tau }+\frac{K-m}{r_{max}}}}\le r_{max}⟺K\le r_{max}\left({\displaystyle \frac{m}{\tau }}+{\displaystyle \frac{K-m}{r_{max}}}\right)⟺m\le {\displaystyle \frac{m{r}_{max}}{\tau }}$$

which holds because $\tau \le r_{max}$. □

Proposition 1 formalizes the nature of the harmonic mean when it aggregates class-conditional true-positive rates. Geometrically, along harmonic-mean isocurves in the binary case, achieving any target level requires both sensitivity and specificity to be sufficiently far from zero; a very low value on one axis cannot be offset by arbitrarily increasing the other. In the multiclass setting, the same principle holds: a subset of classes with low sensitivity constrains the overall harmonic mean, regardless of how well the remaining classes perform. This is precisely the behavior sought in safety-critical evaluation: the score remains conservative unless every class is adequately detected, preventing high aggregate numbers from obscuring poorly served classes.

The upper bound for $\mathrm{H}$ entails two practical consequences: (a) monotonicity: strengthening the weak class or improving the best classes can raise the attainable value of H, whereas adding more weak classes lowers it); (b) meeting a target ${\mathrm{H}}^{★}$: e.g., with $K=35$, $m=1$, $r_{max}=1$, and ${\mathrm{H}}^{★}=0.80$, the critical $\tau \approx 0.103$, which means that a single class with TPR $\le 0.103$ forces $\mathrm{H}<0.80$, regardless of how well the other 34 classes perform.

Figure 2a illustrates Proposition 1 in a stylized four-class scenario (three classes at $\mathrm{TPR}=1$, one varying). The curve $\mathrm{H}\left(r\right)={\displaystyle \frac{4r}{1+3r}}$ coincides with the upper bound (with $K=4$, $m=1$, $r_{max}=1$), while $\mathrm{A}\left(r\right)={\displaystyle \frac{r+3}{4}}$ and $\mathrm{G}\left(r\right)=r^{1/4}$ lie strictly above it for $r\in (0,1)$. The implication is explicit: even when most classes are perfect, a single weak class bounds $\mathrm{H}$ from above and induces a concave, saturating response. Hence, while A and G remain comparatively optimistic, $\mathrm{H}$ exposes—and quantitatively constrains—rare-class underperformance, which is decisive in safety-critical evaluation. Section 7.2 analyzes a multiclass cancer dataset in which a few low-sensitivity tumor types exert a substantial influence on $\mathrm{H}$, producing a pronounced gap with global metrics (e.g., accuracy, $\kappa$) and making the rare-class deficits explicit.

Figure 2b plots Accuracy as a function of the varying class: three classes have perfect sensitivity $\mathrm{TPR}=1$, and the remaining (varying) class has $\mathrm{TPR}=r\in [0,1]$. Denote by ${\pi }_{\mathrm{var}}$ the prevalence of the varying class and by $1-{\pi }_{\mathrm{var}}$ the total prevalence of the three perfect classes. Accuracy depends linearly on r: $\mathrm{Acc}\left(r\right)=(1-{\pi }_{\mathrm{var}})+{\pi }_{\mathrm{var}}r=1-{\pi }_{\mathrm{var}}(1-r)$, so the slope equals ${\pi }_{\mathrm{var}}$ and the intercept at $r=0$ is $1-{\pi }_{\mathrm{var}}$. Thus, for $(25,25,25,25)$ the slope is $0.25$, for $(40,30,20,10)$ it is $0.10$, and for $(80,10,5,5)$ it is $0.05$. As the class becomes rarer, the line flattens and Accuracy becomes largely insensitive to r; even poor sensitivity of the rare class barely changes the overall score. This linearity follows directly from the class-wise decomposition $\mathrm{Acc}={\sum }_k{\pi }_k{\mathrm{TPR}}_k$ and does not depend on how errors are distributed among the other classes. The $\mathrm{H}$ curve (red, identical to panel (a)) shows the opposite behavior: it penalizes small r strongly, making rare-class underperformance visible that Accuracy may mask.

The harmonic mean offers two practical advantages over $\mathrm{A}$ and $\mathrm{G}$ in safety-critical evaluation. First, it is non-compensatory: a single weak class exerts decisive influence, so high scores are impossible unless every class attains adequate sensitivity. Second, it is the most conservative of the Pythagorean means, providing a stable, prevalence-insensitive summary that aligns with worst-class reliability requirements. By contrast, $\mathrm{A}$ and $\mathrm{G}$ are progressively more permissive, allowing strong performance on frequent or easy classes to offset weaknesses elsewhere. Therefore, $\mathrm{H}$ may be not preferable when stakeholders explicitly accept compensatory trade-offs (e.g., screening pipelines where aggregate throughput is prioritized), or when some classes have very scarce/noisy labels, where $\mathrm{H}$ strong penalty near zero can overreact to estimation noise; in such cases $\mathrm{G}$ or $\mathrm{A}$ provide smoother summaries consistent with the application goals. However, if $\mathrm{H}$ is set aside for either reason, the choice should be explicit and well-justified, as it means accepting either compensatory masking of weak classes or reduced protection against rare-class failures—trade-offs with significant consequences in safety-critical settings.

Section 6 formalized the harmonic mean $\mathrm{H}$ as a prevalence-insensitive, class-symmetric aggregator of per-class true-positive rates and established that it is the most conservative of the Pythagorean means; in particular, Proposition 1 proves that even a small set of weak classes places a hard upper bound on $\mathrm{H}$, while $\mathrm{A}$ and $\mathrm{G}$ can remain comparatively high. Building on these theoretical considerations, Section 7 empirically validates the corresponding claims.

7. Experimental Analysis

The experimental analysis is designed to evaluate the benefits of using the Pythagorean means of true positive rates compared to the standard, commonly used classification performance measures, and to analyze the contribution of the particular behavior of the harmonic mean.

As a baseline classifier, the Random Forest (RF) algorithm [46] was chosen because it is an ensemble method rather than a simple model, typically achieves good results (highly competitive), is computationally efficient (built on fast decision trees), and can be considered a black-box model—similar to deep learning approaches, though without large-scale parameterization. Concretely, the settings were: n_estimators = 100 (number of trees); max_depth = None (unbounded); criterion = “gini”; max_features = “sqrt”; bootstrap = True; min_samples_split = 2; min_samples_leaf = 1; class_weight = None; random_state = 0. These are the library defaults; no hyperparameter tuning or resampling was performed. However, the choice of classifier is purely illustrative, i.e., the experiments are model-agnostic, allowing readers to apply any classifier.

All experiments were carried out with stratified 10-fold cross-validation. We evaluated two multiclass performance measures: accuracy and the area under the MCP (Multi-class Classification Performance) curve [18,47]. The MCP curve is computed directly from the class probabilities output by the classifier—analogous to the ROC curve in the binary case—and, unlike accuracy, its area reflects the probabilistic nature of the predictions [48].

7.1. UCI Datasets

The study involves 18 datasets from the UCI Machine Learning Repository [49], varying in the number of samples (150 to 19,020), number of variables (4 to 166), and number of classes (2 to 11) (see

Table 2. Results across datasets from the UCI Repository: dataset name, number of samples (#s), number of variables (#v) and number of class labels (#c), respectively; Acc = Accuracy; A, G, H denote the arithmetic, geometric, and harmonic mean of per-class TPRs; MCP is the area under the MCP curve.

Dataset	#s	#v	#c	Acc	A	G	H	MCP
vertebral_column	310	6	3	0.842	0.792	0.780	0.767	0.702
hepatitis_c	1385	28	4	0.267	0.266	0.262	0.259	0.293
rice_cammeo_osmancik	3810	7	2	0.921	0.919	0.919	0.919	0.848
data_banknote_authentication	1372	4	2	0.993	0.993	0.993	0.993	0.945
musk_v2	6598	166	2	0.979	0.935	0.933	0.931	0.895
magic_gamma_telescope	19,020	10	2	0.882	0.856	0.852	0.847	0.726
cardiotocography_morphologic	2126	23	10	0.896	0.828	0.816	0.803	0.683
divorce	170	54	2	0.977	0.976	0.976	0.976	0.925
australian_credit	690	14	2	0.873	0.871	0.871	0.871	0.701
musk_v1	476	166	2	0.899	0.895	0.895	0.894	0.653
sonar	208	60	2	0.822	0.816	0.812	0.808	0.607
heart	270	13	2	0.833	0.828	0.826	0.824	0.643
iris	150	4	3	0.940	0.940	0.939	0.938	0.905
customer_churn	3150	13	2	0.956	0.902	0.899	0.895	0.883
phishing_websites	11,055	30	2	0.972	0.971	0.971	0.971	0.914
room_occupancy	10,129	16	4	0.998	0.992	0.992	0.992	0.983
landsat_satellite	6435	36	6	0.916	0.891	0.881	0.869	0.780
vowel	990	13	11	0.973	0.973	0.973	0.973	0.605
Mean	-	-	-	0.886	0.869	0.866	0.863	0.761

).

Across the 18 UCI datasets, accuracy exceeds the harmonic mean of per-class sensitivities by a modest margin on average (mean row: $\mathrm{Acc}=0.886$ vs. $\mathrm{H}=0.863$, $\Delta =+0.023$, ≈2.7% relative to $\mathrm{H}$), but the gap is highly dataset-dependent. On several skewed or multiclass problems the difference is pronounced: cardiotocography_morphologic shows $\mathrm{Acc}=0.896$ vs. $\mathrm{H}=0.803$ (absolute $+0.093$, $\approx 11.6\%$ relative), vertebral_column $0.842$ vs. $0.767$ ($+0.075$, $\approx 9.8\%$), customer_churn $0.956$ vs. $0.895$ ($+0.061$, $\approx 6.8\%$), landsat_satellite $0.916$ vs. $0.869$ ($+0.047$, $\approx 5.4\%$), and musk_v2 $0.979$ vs. $0.931$ ($+0.048$, $\approx 5.2\%$). These are precisely the settings where per-class sensitivities are uneven: $\mathrm{A}\ge \mathrm{G}\ge \mathrm{H}$, with $\mathrm{H}$ dropping most when some classes are weak, while $\mathrm{Acc}$ remains buoyed by majority classes. By contrast, on nearly balanced/easy tasks—e.g., rice_cammeo_osmancik, data_banknote_authentication, divorce, phishing_websites, or the classic iris—all three means nearly coincide with accuracy, indicating limited dispersion across per-class recalls.

The last row highlights a striking contrast between class-label summaries and the probability-based MCP: the mean MCP is $0.761$, substantially below $\mathrm{H}$ ($0.863$), $\mathrm{G}$ ($0.866$), and $\mathrm{A}$ ($0.869$). This systematic drop reflects that MCP assesses the full probability vectors, not just the final class labels; consequently it is sensitive to weak margins or miscalibration even when $\mathrm{Acc}$ and $\mathrm{H}$ look high. The effect is pronounced on vowel (11 classes): $\mathrm{Acc}=\mathrm{A}=\mathrm{G}=\mathrm{H}=0.973$ but $\mathrm{MCP}=0.605$, revealing limited probability separation despite excellent label accuracy. Similar patterns appear in musk_v1 ($0.899$ vs. $0.894$ for labels, $\mathrm{MCP}=0.653$), sonar ($0.822$ vs. $0.808$, $\mathrm{MCP}=0.607$), and heart ($0.833$ vs. $0.824$, $\mathrm{MCP}=0.643$). Overall, the mean row ($\mathrm{Acc}/\mathrm{G}/\mathrm{H}\approx 0.86$–$0.89$ vs. $\mathrm{MCP}=0.761$) supports a dual recommendation: use $\mathrm{H}$ to guard against rare-class underperformance in label space, and report MCP to capture probability-level quality. Together they reduce optimism from prevalence-sensitive aggregates and expose weaknesses that accuracy alone can obscure.

7.2. Case Study: Predicting 35 Cancer Types

To illustrate the effect of imbalance in multiclass settings, we adopt the cancer prediction dataset of Nguyen et al. [50]. That study applies machine learning to infer the tumor tissue of origin in advanced-stage metastatic cases—a clinically critical task because a substantial subset of patients lack definitive histopathologic diagnoses and, since therapy choices depend on the primary site, face limited treatment options. The source compendium aggregates multiple molecular assays and initially provides 4131 features designed to discriminate 35 cancer types based on driver/passenger status and simple/complex mutation patterns. It contains 6756 samples drawn from the Hartwig Medical Foundation (metastatic tumors) and the Pan-Cancer Analysis of Whole Genomes consortium (primary tumors). The authors then perform univariate feature selection to retain 463 variables and derive 48 additional regional mutational-density features via nonnegative matrix factorization, yielding the version used here: 6756 samples, 511 predictors, and a 35-class target (biliary, breast, cervix, liver, pancreas, thyroid, etc.).

The multiclass nature of the task makes some binary measures (e.g., ROC) less standardized, since several non-equivalent generalizations exist (one-vs-rest, one-vs-one, macro/micro AUC, volume under the ROC surface). We therefore emphasize prevalence-insensitive, class-conditional summaries and the MCP curve. The class distribution is markedly imbalanced: among 35 tumor types, Breast has 996 samples (14.7%) while Skin_Carcinoma has 25 (0.4%), a ∼40:1 frequency ratio. Such skew can make overall accuracy appear high while concealing weak per-class sensitivities, underscoring the need for metrics that reflect class-wise performance (class-independent vs. class-specific perspective [33]).

On this high-dimensional, many-class dataset, global agreement indices are high ($\mathrm{Acc}=0.906$, Cohen’s $\kappa =0.899$), indicating that most predictions match the true labels and that agreement remains strong even after correcting for chance under a large label space. However, the prevalence-insensitive, aggregated means of the per-class true positive rates reveal substantial heterogeneity across classes: the arithmetic mean is $\mathrm{A}=0.795$, the geometric mean $\mathrm{G}=0.743$, and the harmonic mean $\mathrm{H}=0.658$, with the strict ordering $\mathrm{H}<\mathrm{G}<\mathrm{A}\ll 0.906$. The large gap between $\mathrm{A}$ and $\mathrm{H}$ (about $0.137$) signals marked dispersion in class sensitivities: the harmonic mean is disproportionately penalized by low per-class sensitivities, so even a small subset of under-served classes (with very low TPR) can depress $\mathrm{H}$ while leaving Accuracy and $\kappa$ relatively unaffected if those classes are rare. In fact, the value of $\mathrm{Acc}$ is about 38% higher than the value of $\mathrm{H}$, what reveals that it is extremely important to analyze the class-specific behavior of a system that tries to make predictions on specific tumor classes.

Consistently, the area under the MCP curve is moderate ($\mathrm{MCP}=0.564$), indicating limited operating-point configurations that achieve uniformly high per-class sensitivity; this aligns with the low $\mathrm{H}$, which diagnose a long tail of weak classes. The larger the area under the MCP curve, the lower the uncertainty of the predictive system. The MCP curve for this multiclass dataset (see Figure 6 in [47]) reveals that probabilistic measures are also aligned with $\mathrm{H}$—and they can be even stricter—at highlighting poor per-class behavior of the diagnostic system, which might be dramatic in medical realms.

Therefore, what appeared to be a sound diagnostic system ($\mathrm{Acc}=0.906$) may not be so reliable (H $=0.658$ and AU(MCP) $=0.564$). In sum, while overall agreement is strong, the prevalence-insensitive summaries—especially the harmonic mean—expose uneven class-wise performance that would be obscured by prevalence-sensitive measures, such as Accuracy or Cohen’s $\kappa$.

8. Conclusions

This work clarifies why widely used global indices—accuracy, F₁-score, Cohen’s $\kappa$, MCC—are intrinsically prevalence-sensitive and can therefore distort performance assessment under class imbalance. We formalized this dependence, advocated prevalence-insensitive evaluation built solely from class-conditional rates, and analyzed the arithmetic, geometric, and harmonic means as class-symmetric aggregators, including a geometric account of their isocurves and stringency. Beyond well-known monotonicity $\mathrm{H}\le \mathrm{G}\le \mathrm{A}$, we established bounds that clarify when $\mathrm{H}$ must decrease and by how much: Proposition 1 quantifies the impact of m weak classes and yields actionable constraints. The complementary visual analyses (Figure 2) make these effects explicit: while Accuracy may remain almost flat when the weak class is rare, $\mathrm{H}$ reacts strongly to small per-class sensitivities. Together, these results supply a coherent toolbox for risk-aware evaluation in safety-critical diagnosis.

Empirically, the UCI study shows that the mean gap between Accuracy and $\mathrm{H}$ is modest on average but can be substantial on skewed datasets, confirming that prevalence-sensitive aggregates may overstate performance. The cancer case study further demonstrates that high global agreement can coexist with widely dispersed per-class sensitivities, where $\mathrm{H}$ exposes rare-class deficits that Accuracy, $\kappa$, or MCC may mask. Probability-based MCP complements these label-level summaries by reflecting separation in predictive probabilities, revealing weaknesses even when label metrics look strong.

In general, reliance on global prevalence-sensitive measures can overstate safety precisely where failures are most consequential—rare classes—thereby inflating confidence and masking operational risk. Prevalence-insensitive, class-symmetric means—particularly the harmonic mean of per-class sensitivity—provide conservative, comparable summaries that better track real diagnostic risk.

The theory naturally extends along two axes: (a) families of means (Gini, Kolmogorov–Nagumo–de Finetti, Lehmer, Stolarsky), enabling principled control of sensitivity to dispersion while remaining prevalence-insensitive; (b) majorization tools (Schur-convexity, T-transforms, Karamata) to sharpen guarantees under distributional shifts of $\left\{{\mathrm{TPR}}_k\right\}$. These future work directions aim at evaluation that remains reliable when rare classes matter most.