Next Article in Journal
A Data-Driven Approach of DRG-Based Medical Insurance Payment Policy Formulation in China Based on an Optimization Algorithm
Previous Article in Journal
New Effects and Methods in Brownian Transport
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Distance-Based Relevance Function for Imbalanced Regression

Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea
*
Author to whom correspondence should be addressed.
Stats 2025, 8(3), 53; https://doi.org/10.3390/stats8030053
Submission received: 24 May 2025 / Revised: 25 June 2025 / Accepted: 27 June 2025 / Published: 28 June 2025

Abstract

Imbalanced regression poses a significant challenge in real-world prediction tasks, where rare target values are prone to overfitting during model training. To address this, prior research has employed relevance functions to quantify the rarity of target instances. However, existing functions often struggle to capture the rarity across diverse target distributions. In this study, we introduce a novel Distance-based Relevance Function (DRF) that quantifies the rarity based on the distance between target values, enabling a more accurate and distribution-agnostic assessment of rare data. This general approach allows imbalanced regression techniques to be effectively applied to a broader range of distributions, including bimodal cases. We evaluate the proposed DRF using Mean Squared Error (MSE), relevance-weighted Mean Absolute Error ( MAE ϕ ), and Symmetric Mean Absolute Percentage Error (SMAPE). Empirical studies on synthetic datasets and 18 real-world datasets demonstrate that DRF tends to improve the performance across various machine learning models, including support vector regression, neural networks, XGBoost, and random forests. These findings suggest that DRF offers a promising direction for rare target detection and broadens the applicability of imbalanced regression methods.

1. Introduction

In imbalanced classification—where the minority class is significantly underrepresented—standard learning algorithms often overfit rare examples. To address this, both data-level and algorithm-level solutions have been proposed. A key data-level approach is SMOTE [1], which generates synthetic minority samples via linear interpolation. Combined with majority undersampling, this improves rare-class sensitivity without sacrificing the overall accuracy. In parallel, statistical corrections for rare event bias have been proposed in the modeling stage. For example, King and Zeng [2] highlight how standard logistic regression tends to underestimate the probability of rare events and offer bias correction methods and efficient sampling strategies to improve the predictive accuracy in imbalanced settings.
However, research on imbalanced regression—where the target is continuous—remains limited. Unlike classification, there are no discrete labels to guide resampling. This makes it challenging to define principled rules for identifying which regions of the target distribution warrant augmentation or pruning, rendering resampling in regression inherently more complex.
To address the scarcity of rare targets, prior work introduced relevance functions that assign a normalized rarity score in [0,1] to each continuous target [3]. This stems from the insight that extreme outcomes often hold disproportionate importance in domains like customer analytics, finance, and meteorology. A common approach uses box-plot statistics, labeling values beyond interquartile “whiskers” as highly relevant. However, this tail-focused assumption limits applicability in cases where rare events arise in central or multimodal regions of the distribution.
To move beyond tail-based relevance, we propose the Distance-based Relevance Function (DRF), which quantifies rarity by measuring each target’s proximity to others. DRF highlights sparse regions—whether in the tails, between modes, or elsewhere—without relying on distributional assumptions. This makes it broadly applicable across unimodal, bimodal, and complex distributions. We validate DRF on simulated and 18 real-world datasets using SVR, neural networks, XGBoost, and random forests, evaluating the performance with task-appropriate metrics.
The remainder of this paper is organized as follows. Section 2 reviews prior research on imbalanced regression and introduces the evaluation metrics used throughout the study. Section 3 presents the proposed methodology, including the formulation of DRF. Section 4 provides experimental results and analysis on both simulated and real-world datasets. Finally, Section 5 concludes the paper with a summary of the findings and directions for future work.

2. Preliminaries

2.1. Previous Research

In imbalanced regression, a relevance function maps continuous target values to the [0, 1] range, assigning higher scores to rare observations [3]. This extends the idea of class imbalance to regression, where rare and normal targets parallel the minority and majority classes in classification. The relevance function guides resampling by highlighting critical regions of the target space where improved predictive accuracy is most needed.
The earliest relevance function, proposed by Torgo et al. [3], uses box-plot statistics—median, interquartile range, and whisker endpoints—to define sigmoid curves that assign a relevance of 0 at the median and 1 at the extremes. This tail-focused approach underpins methods such as SMOTER [4], which over-samples high-relevance observations via k-nearest-neighbor interpolation, and SMOGN [5], which further improves the sample quality by injecting Gaussian noise when the nearest neighbors are too distant. However, both methods assume that rare observations reside at the extremes, making them less effective for non-extreme or multimodal distributions.
To address this limitation, kernel-density-based relevance functions have been proposed. For instance, DenseWeight [6] assigns relevance as the inverse of the estimated probability density function, while KSMOTER [7] builds similar functions using kernel density estimation and analyzes the impact of bandwidth selection. These approaches make no assumptions about the location of rare values, allowing for broader applicability. However, they depend heavily on kernel choices and shape parameters, making them sensitive to data quality and hyperparameter tuning.
Recent advances in imbalanced regression have introduced more flexible methods for handling rare target values. Geometric SMOTE [8] generates synthetic examples within localized geometric regions, while deep learning techniques like Label Distribution Smoothing (LDS) and Feature Distribution Smoothing (FDS) improve prediction by smoothing target and feature distributions during training [9]. Gonzalez-Abril et al. [10] extend this work by exploring preprocessing strategies such as fuzzy logic and meta-learning, and Branco et al. [11] propose relevance-aware metrics for evaluating rare-target performance in multi-target regression. Despite these advances, many methods rely on user-defined parameters or complex estimation procedures, which can reduce the robustness. This motivates the development of adaptive, distribution-agnostic approaches to relevance estimation and resampling.
To address these shortcomings, we introduce DRF, a fully parameter-free function that measures rarity exclusively via pairwise distances among target values. Our approach is inherently robust to noise and kernel choice, and it applies to any target distribution—even those with non-extreme or multimodal rare regions.

2.2. Performance Measures

Choosing appropriate performance metrics is crucial for imbalanced regression, where the focus is on accurately predicting rare target values. Unlike classification’s binary labels, regression outputs are continuous, so correctness must be defined by an error tolerance. Some approaches treat predictions within a fixed error bound as correct and compute an F1-style score [12], but the choice of that tolerance is arbitrary and can dramatically alter the outcome. Consequently, such threshold-based metrics may be unreliable for evaluating imbalanced regression.
We assess the overall predictive performance using the Mean Squared Error (MSE) and highlight the accuracy on rare targets with a relevance-weighted Mean Absolute Error (MAE ϕ ), which weights each error by the corresponding data point’s rarity [13]. To ensure scale invariance, we also report Symmetric Mean Absolute Percentage Error (SMAPE), which measures percentage deviation without the upward bias of standard MAPE. The corresponding formulas are given below:
MSE = 1 N i = 1 N ( y i y ^ i ) 2
MAE ϕ = 1 N i = 1 N ϕ ( y i ) × | y i y ^ i |
SMAPE = 100 % N i = 1 N | y i y ^ i | ( | y i | + | y ^ i | ) / 2 ,
where, N is the number of observations, y i the true value, and y ^ i the prediction of the ith observation.

3. Proposed Method

3.1. Distance-Based Relevance Function

We propose the Distance-based Relevance Function (DRF) that quantifies rarity solely from the one-dimensional distribution of target values. Let { y 1 , , y n } be our observed targets (after standardization). For each observation y, we first compute a raw score that reflects both the number of neighbors and their proximity:
s ( y ) = i = 1 n I ( | y y i | < D ) | y y i | + δ ,
where D = 1 (one standard deviation) defines the search radius to ignore distant points, and δ = 1 is a small stabilization constant preventing division-by-zero when | y y i | 0 .
Because s ( y ) grows with local density, rarer (more isolated) points produce smaller scores. We therefore invert and normalize these scores to yield relevance values in [ 0 , 1 ] :
ϕ D ( y ) = max j s ( y j ) s ( y ) max j s ( y j ) min j s ( y j ) .
Finally, we apply a smooth cubic Hermite interpolation to the discrete pairs { ( y , ϕ D ( y ) ) } , producing a continuous relevance function over the entire target range.
Figure 1 illustrates how DRF adapts to a skewed histogram of targets without any reliance on box-plot extremes. For completeness, the pseudocode for computing DRF is included as Algorithm A1 in Appendix A.

3.2. DRF-SMOGN

We integrate DRF into the SMOGN framework to form DRF-SMOGN. First, we compute the distance-based relevance score ϕ D ( y ) for each instance and partition the data into rare and normal bins via a relevance threshold t R . We then apply random under-sampling of the normal bin at u% and random over-sampling of the rare bin at o%.
For each rare bin B, we randomly sample seed instances and, for each seed x, identify its k nearest neighbors using the Heterogeneous Euclidean–Overlap Metric (HEOM) [14], which applies Euclidean distance to normalized numerical features and overlap distance to nominal features.
We compute a safety threshold:
maxD = 1 2 median distances ( x , B ) .
To generate each synthetic instance, we pick one neighbor x neigh . If dist ( x , x neigh ) < maxD , we perform SMOTER-style linear interpolation; otherwise, we add Gaussian noise with standard deviation min ( maxD , 0.02 ) to x. Over-sampled predictors are paired with target values computed as a distance-weighted average of the two originals.
For clarity, Algorithm A2 in Appendix B provides a summary of the full DRF-SMOGN procedure, covering the steps from relevance-based binning to the conditional use of SMOTER or noise generation, followed by the final target assignment.
The DRF-SMOGN pipeline makes no distributional assumptions or user-tuned shape parameters, yet retains SMOGN’s adaptive interpolation vs. noise balance—now driven by a fully parameter-free relevance metric. Because DRF imposes no constraints on the placement of rare observations, it applies seamlessly to any target distribution—including bimodal cases—and can even generate multiple rare or normal bins, marking a significant advance over prior relevance functions.

4. Experimental Study

4.1. Data Description

4.1.1. Simulation Data

Simulation datasets were generated to evaluate DRF-SMOGN on bimodal target distributions, reflecting scenarios where rare observations may appear anywhere in the target space.
Each simulation dataset consists of a training set, intentionally made bimodal by removing values near the median, and a test set that follows the original unimodal distribution (Figure 2). Both sets originate from the same nonlinear relationship involving six predictor variables. To induce bimodality, we define an interval around the median of the target values and delete each observation within this interval with probability
p ( y ) = 1 exp α × | y median ( Y ) | ,
where α is a rarity constant. As α decreases, the deletion probability near the median increases, removing more central observations and creating a more pronounced bimodal gap. This mechanism ensures that rare regions (away from the median) become denser relative to the center.
We report the average training and test set sizes over 100 replications for each rarity constant α . The test set size is fixed at 300 samples across all settings. As α increases, fewer deletions occur near the median, resulting in larger training sets. Specifically, the average training sizes corresponding to α = 0.025 , 0.050 , 0.100 , and 0.200 are 1161, 1350, 1663, and 2055, respectively.
Figure 3 shows both the histogram of each training set and the corresponding DRF relevance curve. As α increases, the gap around the median narrows and DRF smoothly captures the emerging dense and sparse regions, demonstrating its ability to adapt to varying rarity patterns.

4.1.2. Real Data

To evaluate DRF on real data, we selected 18 publicly available benchmark datasets, consistent with prior imbalanced-regression studies focusing on extreme-tail imbalance [4]. These datasets are available at https://www.dcc.fc.up.pt/~ltorgo/EPIA2013/. Table 1 summarizes each dataset’s characteristics: total sample size N, total number of predictors p total , counts of nominal ( p nom ) and numerical ( p num ) predictors, number of rare observations n Rare , and rarity percentage (% Rare), where rarity is defined by ϕ D ( y ) 0.8 .
Although these real-world targets are unimodal, their skewness varies widely—some are heavily skewed, placing rare values only in one tail. Because DRF does not assume that rarity lies at the extremes, it naturally adapts to these different shapes.

4.2. Method

We present our DRF–SMOGN approach, which combines DRF with the SMOGN over-sampling procedure.

4.2.1. Relevance Threshold

DRF requires no extra hyperparameters beyond the relevance threshold t R , a user-specified cutoff also used in prior works [3]. For our simulation studies, we set
t R = 0.6 , α { 0.025 , 0.05 } , 0.4 , α { 0.10 , 0.20 } ,
since higher α produces fewer central deletions and thus lower raw relevance scores. Across all 18 real-world datasets, we adopt a relevance threshold of t R = 0.8 , in line with prior studies [5,6].

4.2.2. Sampling via SMOGN

Once each observation is assigned ϕ D ( y ) , we partition the data into rare ( ϕ D ( y ) t R ) and normal ( ϕ D ( y ) < t R ) bins. Normal bins undergo random under-sampling at rate p u , while rare bins are over-sampled at rate p o , exactly as in SMOGN [5]. Key SMOGN hyperparameters include the number of neighbors k and the Gaussian perturbation constant pert . To ensure fair comparison, we adopt the original SMOGN settings k = 5 and p e r t = 0.02 . Other settings such as the over-sampling and under-sampling rates were kept the same as well.

4.2.3. Experimental Setup

We compare three sampling strategies—none (original data), SMOGN, and DRF–SMOGN—using four regressors:
  • Support Vector Regressor (SVR).
  • Neural network (NNET).
  • XGBoost Regressor (XGB).
  • Random Forest Regressor (RF).
All models are tuned on the original (unsampled) data via grid search; the parameter ranges are shown in Table 2. Simulation datasets are replicated 100 times, and each real-world dataset is assessed over 100 random train–test splits (80:20).

4.2.4. Evaluation Metrics

We evaluate the overall model fit using Mean Squared Error (MSE) and Symmetric Mean Absolute Percentage Error (SMAPE), and emphasize performance on rare observations via relevance-weighted MAE ( MAE ϕ ) [13], which assigns greater weight to higher-relevance points. By systematically comparing sampling strategies, regression models, and evaluation metrics, we isolate the influence of DRF on predicting rare versus normal targets.

4.3. Results

4.3.1. Simulation Data Results

We assessed four simulated datasets—each with rarity constant α { 0.025 , 0.05 , 0.10 , 0.20 } and 100 replicates—using three sampling strategies (Original, SMOGN, DRF-SMOGN) alongside four regressors (SVR, NNET, XGB, RF). Table 3 reports the mean values of MSE, MAE ϕ , and SMAPE across 100 runs. Standard deviations for MSE and MAE ϕ were approximately 1% of the corresponding mean values, and those for SMAPE were around 0.5%. Overall, DRF-SMOGN achieved the best results across all metrics and datasets. As α increases (rarity near the median decreases), DRF-SMOGN’s advantage over the Original and SMOGN diminishes, reflecting the growing balance of the data.
Figure 4 visualizes the distribution of 100 MSE values for SVR and NNET across the four datasets using boxplots, highlighting DRF-SMOGN’s consistent gains, particularly in the highly imbalanced cases ( α 0.10 ).
These results confirm that DRF-SMOGN reliably enhances the performance on bimodal distributions, while SMOGN alone struggles when rare observations lie near the median. As the rarity gap narrows ( α increases), the benefit of DRF diminishes, aligning with expectations for more balanced data.

4.3.2. Real Data Results

We evaluated 18 benchmark datasets under three sampling strategies—Original, SMOGN, and DRF–SMOGN—using SVR, NNET, XGB, and RF. Each experiment was repeated 100 times with different random seeds and an 80:20 train–test split. Apart from omitting SMAPE, all other settings (hyperparameters, grid search, evaluation metrics) mirror the simulation study.
Table 4 present the mean results for MSE and MAE ϕ . As the standard deviations were within a few percent of the means, they were omitted. Under MSE, the Original approach most often yields the lowest error, with DRF–SMOGN a close second and SMOGN frequently failing to improve—and sometimes degrading—the overall performance. The differences among the methods based on SMAPE results exhibited a similar pattern to those based on MSE, and were therefore omitted for brevity.
When focusing on rare-target accuracy via MAE ϕ , DRF–SMOGN outperforms both Original and SMOGN in the majority of cases, demonstrating its superior ability to predict high-relevance observations. Where the Original approach is marginally better, the gap is small; SMOGN occasionally edges out DRF–SMOGN but only inconsistently and by narrow margins.
Among the four learners, SVR and NNET generally perform best, although NNET’s results can degrade on datasets with many nominal features—highlighting SVR’s robustness across heterogeneous real-world tasks. In summary, DRF–SMOGN maintains the Original dataset’s overall accuracy while substantially improving rare-data predictions, whereas SMOGN’s unimodal assumptions hamper its effectiveness on more complex distributions.
Figure 5 presents boxplots of both MSE and MAE ϕ for the three selected datasets. We focus on SVR’s results, as its trends mirror those of other learners and it achieves the most stable performance overall. In the ‘bank’ dataset, SMOGN’s MSE errors are substantially higher than those of the Original data, whereas DRF-SMOGN closely matches the Original performance. The MAE ϕ plots likewise show DRF-SMOGN maintaining parity with the Original, while SMOGN remains suboptimal. Although ‘accel’ and ‘bank’ exhibit slight MAE ϕ improvements under DRF-SMOGN, the ‘fuel’ dataset does not, highlighting how distributional characteristics can influence sampling efficacy.
When evaluating on real datasets, two factors in particular can affect the performance:
  • Nominal features: Unlike our simulations, some real datasets include up to 13 categorical predictors. HEOM handles these by treating each mismatched category as distance 1, which can overwhelm the Euclidean contributions from numerical features when categories dominate. Since DRF-SMOGN weights synthetic targets by inverse distance, overly coarse categorical distances may distort both neighbor selection and target interpolation.
  • Gaussian noise sampling: SMOGN’s safety threshold rule falls back on adding Gaussian noise when neighbors are too far apart—a useful idea in principle, but one that depends on a user-defined perturbation constant. Without clear guidelines for setting this constant across diverse datasets, noise levels can vary unpredictably, undermining the stability. The systematic calibration of this parameter—or the adoption of alternative noise models—could yield more consistent results.
Overall, DRF-SMOGN tends to offer more stable and reliable improvements over SMOGN—both on bimodal (simulation) and unimodal (real) data—while often preserving the Original data’s accuracy. Using standard metrics such as MSE and MAE ϕ allows for a balanced evaluation of performance across the entire distribution.

5. Conclusions

This paper introduces the Distance-based Relevance Function (DRF), a parameter-free approach to imbalanced regression that quantifies target rarity via pairwise distances among standardized values. Unlike box-plot or density-based methods, DRF requires no distributional assumptions and applies broadly to both unimodal and bimodal targets. Integrated into SMOGN, DRF-SMOGN often improves the performance on both synthetic and real-world datasets.
Overall, DRF enables resampling across diverse target distributions without additional tuning. Our evaluation framework highlights both its benefits and the limitations of existing methods—particularly the instability of Gaussian-noise sampling in SMOGN with many nominal features. Future work may explore advanced distance metrics for mixed-type data and refined noise or interpolation strategies to further enhance sample quality.

Author Contributions

Conceptualization, D.D.I. and H.K.; methodology, D.D.I.; software, D.D.I.; validation, D.D.I. and H.K.; data curation, D.D.I.; writing—original draft preparation, D.D.I.; writing—review and editing, H.K.; visualization, D.D.I.; supervision, H.K.; funding acquisition, H.K. All authors have read and agreed to the published version of the manuscript.

Funding

Hyunjoong Kim’s work was supported by the MSIT (Ministry of Science and ICT), Republic of Korea, under the ICAN (ICT Challenge and Advanced Network of HRD) support program (IITP-2023-00259934) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation) and by the National Research Foundation of Korea (NRF) grant funded by the Korean government (No. RS-2016-NR017145).

Data Availability Statement

The data are public, and can be downloaded from the relevant site.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A. Algorithm A1

Algorithm A1 Distance-based Relevance Function ϕ D
Input: Y—sorted list of n target values (standardized)
            D—search radius (e.g., 1 × std )
             δ —stabilization constant (e.g., 1)
Output: ϕ D —relevance scores in [ 0 , 1 ]
  initialize array raw_scores of length n
  for j = 1 to n do
         s 0
        for  i = 1 to n do
               d | Y j Y i |
              if  0 < d < D  then
                    s s + 1 d + δ
              end if
        end for
        raw_scores[j] s
  end for
   s min min ( raw _ scores )
   s max max ( raw _ scores )
  for j = 1 to n do
         ϕ D ( Y j ) s max raw _ scores [ j ] / s max s min
  end for
  return { ϕ D ( Y 1 ) , , ϕ D ( Y n ) }

Appendix B. Algorithm A2

Algorithm A2 DRF-SMOGN
Require: Dataset D with continuous targets Y
 1:
Relevance threshold t R
 2:
Under-sampling rate p u , Over-sampling rate p o
 3:
Number of neighbors k, Distance metric (HEOM) dist
Output: Modified dataset D new
 4:
Sort D by ascending Y.
 5:
Compute ϕ D ( y ) for each ( x , y ) D using Algorithm 1.
 6:
Partition D into:
  • Normal bins B N = { ( x , y ) ϕ D ( y ) < t R }
  • Rare bins B R = { ( x , y ) ϕ D ( y ) t R }
 7:
D new B R
 8:
for all  B B N do                     ▹ Under-sampling
 9:
    Sample p u × | B | points uniformly at random from B and add to D new .
10:
end for
11:
for all  B B R do                     ▹ Over-sampling
12:
     n p o × | B |
13:
    for all  ( x , y ) B  do
14:
        Find k nearest neighbors of x in B using dist.
15:
        Let maxD 1 2 median ( { dist ( x , x ) : x B } ) .
16:
        for  i = 1 to n do
17:
           Randomly select a neighbor x neigh .
18:
           if dist ( x , x neigh ) < maxD  then
19:
               Generate x new by SMOTER interpolation between x and x neigh .
20:
           else
21:
               Generate x new by adding Gaussian noise to x with σ = min ( maxD , 0.02 ) .
22:
           end if
23:
           Assign y new as the distance-weighted average of y and y neigh .
24:
           Add ( x new , y new ) to D new .
25:
        end for
26:
    end for
27:
end for
28:
return  D new

References

  1. Chawla, N.; Bowyer, K.; Hall, L.; Kegelmeyer, W. SMOTE: Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res. 2002, 16, 321–357. [Google Scholar] [CrossRef]
  2. King, G.; Zeng, L. Logistic regression in rare events data. Political Anal. 2001, 9, 137–163. [Google Scholar] [CrossRef]
  3. Torgo, L.; Ribeiro, R.P. Utility-based regression. In Proceedings of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, 17–21 September 2007; pp. 597–604. [Google Scholar]
  4. Torgo, L.; Ribeiro, R.P.; Pfahringer, B.; Branco, P. SMOTE for regression. In Proceedings of the EPIA 2013: Progress in Artificial Intelligence, Azores, Portugal, 9–12 September 2013; pp. 378–389. [Google Scholar]
  5. Branco, P.; Torgo, L.; Ribeiro, R.P. SMOGN: A pre-processing approach for imbalanced regression. In Proceedings of the First International Workshop on Learning with Imbalanced Domains: Theory and Applications, Skopje, Macedonia, 22 September 2017; Volume 74, pp. 36–50. [Google Scholar]
  6. Steininger, M.; Kobs, K.; Davidson, P.; Krause, A.; Hotho, A. Density-based weighting for imbalanced regression. Mach. Learn. 2021, 110, 2187–2211. [Google Scholar] [CrossRef]
  7. Son, J. KSMOTER: KDE SMOTE for Imbalanced Regression. Master’s Thesis, Yonsei University, Seoul, Republic of Korea, 2023. [Google Scholar]
  8. Camacho, L.; Douzas, G.; Bacao, F. Geometric SMOTE for regression. Expert Syst. Appl. 2022, 193, 116387. [Google Scholar] [CrossRef]
  9. Yang, Y.; Zha, K.; Chen, Y.; Wang, H.; Katabi, D. Delving into deep imbalanced regression. In Proceedings of the 38th International Conference on Machine Learning, Virtual, 18–24 July 2021; pp. 11842–11851. [Google Scholar]
  10. Gonzalez-Abril, L.; Guerrero-Gonzalez, A.; Torres, J.; Ortega, J.A. A review on imbalanced data preprocessing for supervised learning: Evolutionary fuzzy systems and beyond. Appl. Sci. 2020, 10, 4014. [Google Scholar]
  11. Branco, P.; Torgo, L.; Ribeiro, R.P. Relevance-based evaluation metrics for multi-target regression. Mach. Learn. 2017, 106, 1779–1800. [Google Scholar]
  12. Torgo, L.; Ribeiro, R.P. Precision and recall for regression. In Proceedings of the DS 2009: Discovery Science, Porto, Portugal, 3–5 October 2009; pp. 332–346. [Google Scholar]
  13. Song, X.Y.; Dao, N.; Branco, P. DistSMOGN: Distributed SMOGN for imbalanced regression problems. In Proceedings of the Fourth International Workshop on Learning with Imbalanced Domains: Theory and Applications, Grenoble, France, 23 September 2022; Volume 183, pp. 38–52. [Google Scholar]
  14. Wilson, D.L.; Martinez, T.R. Improved Heterogeneous Distance Functions. In Proceedings of the Fourteenth International Conference on Machine Learning (ICML). Morgan Kaufmann, Nashville, Tennessee, 8–12 July 1997; pp. 656–666. [Google Scholar]
Figure 1. Histogram of target values, corresponding DRF-derived relevance scores for each observation, and the resulting relevance function via cubic Hermite interpolation.
Figure 1. Histogram of target values, corresponding DRF-derived relevance scores for each observation, and the resulting relevance function via cubic Hermite interpolation.
Stats 08 00053 g001
Figure 2. Histograms of simulation training (bimodal) and test (unimodal) datasets.
Figure 2. Histograms of simulation training (bimodal) and test (unimodal) datasets.
Stats 08 00053 g002
Figure 3. Histograms and DRF relevance score curves for training data across different α values. Denser regions receive lower raw scores and thus lower relevance, while isolated regions near the median gap yield higher relevance.
Figure 3. Histograms and DRF relevance score curves for training data across different α values. Denser regions receive lower raw scores and thus lower relevance, while isolated regions near the median gap yield higher relevance.
Stats 08 00053 g003
Figure 4. MSE comparison for (a) SVR and (b) NNET on simulation datasets.
Figure 4. MSE comparison for (a) SVR and (b) NNET on simulation datasets.
Stats 08 00053 g004
Figure 5. Boxplots of MSE and MAE ϕ over 100 runs on real datasets (a) accel, (b) bank, and (c) fuel using Support Vector Regressor.
Figure 5. Boxplots of MSE and MAE ϕ over 100 runs on real datasets (a) accel, (b) bank, and (c) fuel using Support Vector Regressor.
Stats 08 00053 g005
Table 1. Characteristics of real datasets: sample size (N), predictor counts ( p total ), number of nominal ( p nom ) and numerical ( p num ) predictors, rare-observation count ( n Rare ), and rarity percentage (% Rare, ϕ D ( y ) 0.8 ).
Table 1. Characteristics of real datasets: sample size (N), predictor counts ( p total ), number of nominal ( p nom ) and numerical ( p num ) predictors, rare-observation count ( n Rare ), and rarity percentage (% Rare, ϕ D ( y ) 0.8 ).
DatasetN p total p nom p num n Rare % Rare
a119811384623.2
a219811383517.7
a319811383417.2
a419811382713.6
a519811383015.2
a619811383316.7
a719811382713.6
abal417781767916.3
accel1732153121377.9
avail1802167921712.0
bank449990966214.7
bost506130137615.0
cpu8192130137559.2
dAil712950577310.8
dElev951760699010.4
fuel176438122624814.1
heat7400128480210.8
maxt18023313201297.2
Table 2. Regressor hyperparameter grid for SVR, NNET, XGB, and RF.
Table 2. Regressor hyperparameter grid for SVR, NNET, XGB, and RF.
LearnerParameter Variants
SVR C { 10 , 150 , 300 } , γ { 0.01 , 0.001 }
NNEThidden_layer_sizes { 1 , 5 , 10 , 50 } , activation { identity , logistic , tan h , relu } ,
solver { lbfgs , sgd , adam } , alpha { 5 × 10 5 , 5 × 10 4 , 5 × 10 3 }
XGBlearning_rate { 0.1 , 0.5 } , n_estimators { 500 , 1000 } , max_depth { 3 , 5 , 7 }
RFn_estimators { 100 , 200 , 500 , 700 , 1500 } , max_depth { 3 , 5 , 7 }
Table 3. Simulation data performance for Original, SMOGN, and DRF-SMOGN across MSE, MAE ϕ , and SMAPE. The best results (i.e., lowest values) for each metric and dataset are marked with an asterisk.
Table 3. Simulation data performance for Original, SMOGN, and DRF-SMOGN across MSE, MAE ϕ , and SMAPE. The best results (i.e., lowest values) for each metric and dataset are marked with an asterisk.
DatasetAlgorithmMSE MAE ϕ SMAPE
Orig.SMOGNDRFOrig.SMOGNDRFOrig.SMOGNDRF
Data 1 ( α = 0.025 ) SVR40.3740.3736.87 *5.145.144.95 *88.2888.2886.55 *
NNET37.1137.1235.94 *5.415.415.26 *84.0584.0083.08 *
XGB53.0453.0451.39 *6.426.426.29 *89.7489.8789.34 *
RF43.4943.47 *44.476.236.236.2386.1686.12 *86.79
Data 2 ( α = 0.050 ) SVR40.2540.6537.02 *5.105.124.91 *87.8387.9586.16 *
NNET33.8534.0933.12 *5.055.064.93 *82.4182.5181.95 *
XGB50.4150.2650.17 *6.066.055.98 *88.9689.0288.90 *
RF38.1838.1839.365.645.645.6584.1784.14 *84.79
Data 3 ( α = 0.100 ) SVR35.0435.4732.35 *4.934.954.73 *86.6286.7984.83 *
NNET31.1131.5928.59 *4.764.794.65 *81.4281.6581.08 *
XGB45.1345.1743.80 *5.655.645.53 *81.4281.6581.08 *
RF33.45 *33.5834.105.105.115.06 *82.60 *82.7383.15
Data 4 ( α = 0.200 ) SVR32.65 *42.7735.925.00 *5.475.1085.84 *88.8586.78
NNET30.74 *35.4131.064.56 *4.874.6380.49 *83.3381.78
XGB42.56 *44.7144.985.43 *5.545.5180.49 *83.3381.78
RF30.72 *33.1232.774.864.944.84 *82.01 *83.0782.79
Table 4. MSE and MAE ϕ evaluations on 18 real datasets. Each value represents a mean over multiple runs. Standard deviations, which were within a few percent of the means, are omitted for clarity. The best (i.e., lowest) results for each metric and dataset are marked with an asterisk.
Table 4. MSE and MAE ϕ evaluations on 18 real datasets. Each value represents a mean over multiple runs. Standard deviations, which were within a few percent of the means, are omitted for clarity. The best (i.e., lowest) results for each metric and dataset are marked with an asterisk.
DatasetAlgorithmMSEMAE ϕ
OriginalSMOGNDRF-SMOGNOriginalSMOGNDRF-SMOGN
a1SVR336.3428.2286.7 *21.216.7 *16.9
NNET556.1 *581.2563.930.0 *31.030.4
XGB323.6 *454.0443.217.7 *19.818.9
RF264.2 *424.0350.915.9 *17.416.0
a2SVR116.1139.6111.3 *13.010.2 *11.8
NNET140.6 *150.1140.915.4 *16.215.5
XGB137.8 *164.7160.712.312.012.0 *
RF108.7 *150.2133.410.710.610.5 *
a3SVR50.655.743.1 *9.97.57.4 *
NNET50.151.248.4 *9.59.49.3 *
XGB56.4 *83.973.27.3 *7.57.5
RF47.8 *75.763.07.17.06.6 *
a4SVR21.6 *24.522.55.24.94.6 *
NNET20.821.217.8 *4.64.24.1 *
XGB32.7 *37.937.94.54.44.3
RF21.2 *28.023.24.44.84.2 *
a5SVR51.767.850.2 *8.68.28.1 *
NNET62.9 *70.563.010.310.710.2 *
XGB55.9 *82.377.08.2 *8.98.6
RF51.2 *82.064.07.88.07.4 *
a6SVR150.6152.8136.6 *16.816.215.2 *
NNET138.2143.2128.4 *16.516.615.8 *
XGB147.7 *211.0189.014.414.914.4 *
RF132.3 *196.2151.013.212.512.4 *
a7SVR28.128.926.6 *8.27.87.7 *
NNET26.1 *35.927.57.47.27.1 *
XGB34.4 *50.443.76.2 *7.26.4
RF25.9 *40.433.16.26.15.6 *
abalSVR4.95.34.8 *2.32.22.2 *
NNET5.2 *6.36.22.22.0 *2.3
XGB5.5 *6.25.82.22.1 *2.2
RF4.8 *5.74.82.12.0 *2.1
accelSVR0.9 *1.30.90.80.90.8 *
NNET1.3 *3.41.71.0 *1.51.1
XGB0.6 *1.10.70.5 *0.80.6
RF0.8 *1.30.80.7 *0.90.7
availSVR194.81174.7186.8 *11.0 *33.611.4
NNET334.7 *1449.11459.616.4 *31.035.2
XGB22.4 *368.232.32.0 *13.52.6
RF55.8 *560.157.35.6 *20.15.6
bankSVR0.05 *0.830.110.12 *0.580.13
NNET0.05 *0.830.120.12 *0.570.14
XGB0.05 *0.830.120.08 *0.540.08 *
RF0.06 *0.830.120.080.520.07 *
bostSVR17.3 *18.518.23.3 *3.43.4
NNET127.5 *184.1208.210.2 *12.013.0
XGB10.5 *16.012.32.7 *3.22.8
RF11.1 *15.212.12.8 *3.22.9
cpuSVR65.9 *118.884.86.6 *7.57.5
NNET1227.2825.9 *2586.527.9 *28.740.6
XGB12.5 *14.314.52.7 *2.92.9
RF82.083.172.8 *3.03.13.8 *
dAilSVR0.32 *0.450.32 *1.551.911.47 *
NNET0.32 *0.450.331.431.781.35 *
XGB0.360.480.35 *1.512.011.47 *
RF0.31 *0.460.321.532.011.48 *
dElevSVR0.370.560.36 *1.161.401.15 *
NNET0.37 *0.550.381.191.441.10 *
XGB0.45 *0.510.461.161.481.14 *
RF0.38 *0.500.391.151.461.14 *
fuelSVR0.23 *0.290.250.35 *0.400.36
NNET0.45 *0.780.480.45 *0.620.49
XGB0.19 *0.350.240.33 *0.460.36
RF0.28 *0.460.310.43 *0.570.45
heatSVR165.9144.2 *166.512.911.1 *12.7
NNET183.0 *421.0527.212.9 *17.623.4
XGB11.5 *39.017.93.1 *5.63.8
RF50.5 *78.453.97.0 *8.67.2
maxtSVR283.9 *1784.8369.014.2 *37.216.7
NNET3518.2 *4958.37282.453.6 *64.390.0
XGB31.4 *759.045.82.2 *20.22.9
RF45.3 *1204.361.74.5 *29.45.1
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

In, D.D.; Kim, H. Distance-Based Relevance Function for Imbalanced Regression. Stats 2025, 8, 53. https://doi.org/10.3390/stats8030053

AMA Style

In DD, Kim H. Distance-Based Relevance Function for Imbalanced Regression. Stats. 2025; 8(3):53. https://doi.org/10.3390/stats8030053

Chicago/Turabian Style

In, Daniel Daeyoung, and Hyunjoong Kim. 2025. "Distance-Based Relevance Function for Imbalanced Regression" Stats 8, no. 3: 53. https://doi.org/10.3390/stats8030053

APA Style

In, D. D., & Kim, H. (2025). Distance-Based Relevance Function for Imbalanced Regression. Stats, 8(3), 53. https://doi.org/10.3390/stats8030053

Article Metrics

Back to TopTop