TS-SMOTE: An Improved SMOTE Method Based on Symmetric Triangle Scoring Mechanism for Solving Class-Imbalanced Problems

Abstract

The imbalanced classification problem is a key research in machine learning as the relevant algorithms tend to focus on the features and patterns of the majority class instead of insufficient learning of the minority class, resulting in unsatisfactory performance of machine learning. Scholars have attempted to solve this problem and proposed many ideas at the data and algorithm levels. The SMOTE (Synthetic Minority Over-sampling Technique) method is an effective approach at the data level. In this paper, we propose an oversampling method based on SMOTE and symmetric regular triangles scoring mechanism. This method uses symmetrical triangles to flatten the plane, and then establishes a suitable scoring mechanism to select the minority samples that participate in the synthesis. After selecting the minority samples, it conducts multiple linear interpolations according to the established rules to generate new minority samples. In the experimental section, we select 30 imbalanced datasets to test their performance of the proposed method and some classical oversampling methods under different indicators. In order to demonstrate the performance of these oversampling methods with classifiers, we select three different classifiers and test their performance. The experimental results show that the TS-SMOTE method has the best performance.

1. Introduction

In recent years, machine learning has become increasingly popular [1,2] and has gradually become key research. In real life, as one of the important areas of machine learning, the classification problem [3,4] can be felt in various aspects. In classification problems, we often encounter binary classification problems [5,6], and many binary classification problems in life are class imbalance problems, for example, medical diagnosis, financial fraud detection, natural disaster prediction, facial recognition, speech recognition and so on. However, when dealing with classification problems in real-world applications, it is inevitable that we encounter significant imbalances in the proportion between classes. The case is also known as imbalanced classification and has drawn considerable attention and research efforts in the industry. For example, in the employment field, there is the problem of gender minority class imbalance [7]. In certain specific industries or occupations, there is a serious imbalance in the gender ratio. The number of cases of a certain gender is relatively small, while the number of cases of the other gender is extremely large. This will lead to the traditional neural network not paying enough attention to the minority samples, thus overly learning the data of the majority samples, resulting in poor learning performance of the network. Typically, the classes that possess a smaller number of samples are referred to as minority classes. On the other hand, those classes that have a larger quantity of samples are known as majority classes. The imbalanced ratio (IR) [8] is calculated by dividing the number of majority samples by that of minority samples. We measure the degree of class imbalance through the imbalance ratio. The larger the ratio, the more imbalanced the data is. For an imbalanced dataset, the samples belonging to the class with a relatively small sample number are called minority samples. The case that is the opposite of the minority samples is called the majority samples. When dealing with the problem of class imbalance, it is undeniable that due to the significant difference in the number of majority and minority samples in the collected data, traditional classifiers have no effective means to identify minority class samples, resulting in insignificant classification results. Therefore, in order to solve this problem, many scholars have overcome difficulties and have proposed a large number of methods from the aspects of data and algorithms [9,10] to improve the recognition accuracy of classifiers. At the data level, there are two main methods: undersampling [11,12] and oversampling [13,14,15]. Undersampling refers to reducing the number of majority samples when dealing with imbalanced data to make it equal to the number of minority samples. In contrast, oversampling performs the opposite, increasing the number of minority samples to make it the same as the number of majority samples. Obviously, the method of undersampling, which reduces the number of majority samples, may lead to the deletion of some important information and features, thus affecting the model’s learning and understanding of majority class data and making it prone to the problem of overfitting. Oversampling, by contrast, preserves more information and represents a more scientific approach. Currently, the most widely used oversampling method is the Synthetic Minority Over-sampling Technique (SMOTE) [16,17,18,19].

The operation of SMOTE is as follows: First, select a minority sample. Then, randomly pick a minority sample from the k-nearest neighbors of this selected sample. After linear interpolation is employed between these two samples, then generate a new minority sample (see Figure 1A). Repeat these processes continuously until the number of samples of the majority class and the minority class is balanced or meets the set proportion. The samples synthesized by this method are generally located in the region of the minority class. It can effectively expand the minority samples in the feature space, assist the network model in learning more abundant feature information, and improve the classification performance of the model for minority samples.

Subsequently, numerous scholars carried out optimization work on the SMOTE algorithm. With the booming development of the fields of deep learning and neural networks, an increasing number of SMOTE-based models have started to integrate with these cutting-edge technologies. Cost-sensitive learning focuses on algorithm-level optimization, adjusting the model’s learning focus by setting different misclassification costs for different classes. For instance, the cost of misclassifying minority class samples is set much higher than that of the majority class, prompting the model to pay more attention to minority samples during training. This method does not require altering the original data distribution and is highly suitable for scenarios where samples are scarce and generating new samples is difficult, such as precious case data in medical diagnosis. Anomaly-detection-based classifiers adopt a different approach, treating minority class samples as “anomalies” and achieving classification by modeling the distribution of majority class samples. This method has certain applicability in extremely imbalanced scenarios, that is, when the proportion of minority class samples is extremely low (usually less than 1%). AutoSMOTE [20] proposes a method for achieving automated imbalanced learning based on deep hierarchical reinforcement learning. This method is capable of automatically selecting appropriate sampling and ensemble strategies to enhance the classification performance on imbalanced data, thereby realizing an automated imbalanced learning process. In addition, some scholars have combined generative adaptive networks (GANs) with oversampling techniques to develop new SMOTE methods. The latest research achievements include methods such as SMOTified-GAN [21] and GAN-SMOTE [22]. In addition, there are many other methods based on boosting or ensemble learning techniques [23], such as SMOTEBoost [24] and Ensemble- SMOTE [25]. The concept of ensemble learning holds significant importance in cost-sensitive learning. A prime illustration of this is AdaCost [26], which is founded on AdaBoost. It is crucial to note that ensemble learning does not inherently define an algorithm’s unique concept; Rather, it serves as a versatile approach that can be integrated with nearly all algorithms. Moreover, undersampling presents a notable drawback as it discards a substantial amount of valuable data, potentially leading to the loss of important information. On the other hand, generative adaptive networks (GANs) come with their own set of challenges. They are relatively more intricate in their implementation, which can limit their reliability. In contrast, oversampling methods based on the SMOTE tend to offer the most promising results. SMOTE-based approaches strike a balance between preserving data integrity and enhancing the performance of models dealing with imbalanced datasets, making them a preferred choice in many practical applications.

Although SMOTE is an effective method, it still has some problems. Firstly, the selection of k-nearest neighbors in SMOTE is blind [27]. The lower limit of the parameter k is 1, but there is no fixed upper limit value for different datasets.

In the SMOTE algorithm, k is usually set to 5, which is a common practice. This has a certain degree of rationality, but there are also corresponding limitations. For some datasets with complex data distributions, multiple sub-clusters, or significant changes in data density, k = 5 may not be applicable. Especially when the dataset is at small scale or the number of minority samples is negligible, SMOTE can give rise to sample overlap because of repeated sampling. This situation will further magnify the noise problem. For example, in a sparse sub-cluster of data, five nearest neighbors may not be sufficient to fully represent the characteristics of that region. On the other hand, in a data-dense area, five nearest neighbors may be excessive, resulting in synthetic samples that cannot accurately reflect the local structure [28].

In specific experiments, due to the different characteristics of various datasets, a fixed k value of 5 obviously cannot meet the requirements of different datasets. Therefore, we need to continuously adjust the value of k and ensure that the k value is optimized to yield the best experimental results. For example, if unreasonable synthetic samples are generated near the class boundaries, it may make it difficult for the classifier to accurately distinguish between different classes, thus leading to a decline in classification performance. As a result, the boundary between the majority samples and the minority samples becomes increasingly blurred [29]. Another crucial problem is that it does not consider the distribution patterns of both the samples and their neighboring points. Instead, it merely performs an average sampling operation on the points of the minority class. This approach renders it highly susceptible to causing overfitting [30].

Considering all the above problems, we present three common issues that arise during the synthesis of new samples by SMOTE, and will explain them with the aid of images. In the first situation, for a certain noisy point x, if the selected k-nearest neighbor point $x^{\prime }$ is within the region of minority samples (see Figure 1B), then the newly synthesized sample $x_{new}$ is highly likely to be situated near the boundary, which is unfavorable for the subsequent learning. In the second case, both x and $x^{\prime }$ are noisy points. Then, the line segment connecting them lies within the boundary of the majority samples. This will cause the newly generated sample $x_{new}$ to also be a noisy point (see Figure 1C), resulting in a poor generation effect. In the third case, although both x and $x^{\prime }$ are located in the minority region, due to the excessively close distance between these two points, the newly generated sample $x_{new}$ almost coincides with the original two samples. Even though this sample is indeed synthesized in the minority region (see Figure 1D), it does not provide much help for our subsequent analysis. Moreover, it may lead to overfitting during the learning process.

Based on the above-mentioned deficiencies of the SMOTE method, we propose an improved SMOTE method (TS-CMOTE) based on regular triangle scoring mechanism. Firstly, the PCA dimensionality reduction technique is used to reduce multidimensional samples to a two-dimensional space. Then, according to the obtained new data, an appropriate side length of the regular symmetric triangle is calculated, and they are closely connected in the two-dimensional space in the form of tessellation. After that, we establish a regular triangle scoring mechanism to dynamically select samples. Finally, the multidimensional samples corresponding to the selected two-dimensional samples are found and used to participate in the synthesis of minority class samples. By calculating and limiting the range of the side length of the regular triangle, the TS-SMOTE method can effectively reduce the possibility of synthesizing new noise samples. In addition, instead of performing linear interpolation between two samples as in SMOTE, TS-SMOTE selects multiple samples simultaneously and synthesizes a new sample within the closed region of these samples. This expands the sample selection and can enrich the information content of the new samples.

The symmetric triangle scoring mechanism has an outstanding value. Our method fully takes into account the impact brought by the distribution of the majority samples. By considering different categories and combining it with our method, we can generate minority samples reasonably. In particular, because our method can effectively reduce the generation of noise points, the generated points at the boundaries are also more reasonable.

We conducted experiments using 30 imbalanced datasets from the website https://sci2s.ugr.es/keel/imbalanced.php, accessed on 4 March 2025. The experimental results show that compared with the original SMOTE method and the improved SMOTE methods, the network model trained by the TS-SMOTE method has better classification performance. In addition, to prevent the uncertainty of the classifier, we tested the performance of each oversampling method on three commonly used classifiers: multilayer perceptron (MLP) [31], support vector machine (SVM) [32], and Adaptive Boosting (AdaBoost) [33]. By comparing the experimental results, it can be seen that TS-SMOTE can be effectively applied to solve the class imbalance problem.

The remaining content of this paper is arranged as follows. In Section 2, some related work will be briefly introduced first. Section 3 will provide a detailed introduction to the TS-SMOTE method. In Section 4, data experiments and corresponding analyses will be conducted. Some conclusions will be drawn in Section 5.

2. Related Works

Regarding the decision-making methods for handling imbalanced classification problems, we have introduced many methods previously. However, since our research mainly focuses on SMOTE and the improved SMOTE, we will no longer cover other non-SMOTE methods. Instead, we will concentrate our efforts on the discussion of SMOTE and the improved SMOTE.

The Synthetic Minority Over-sampling Technique (SMOTE) is a method for oversampling minority classes. It was proposed by Nitesh V. Chawla and other researchers in 2002 [16]. This method is an oversampling algorithm specifically designed for imbalanced datasets.

The specific method is as follows:

• Determine the minority class sample set $S_{minority}$. For each sample $x_i\in S_{minority}$ in it, calculate and find its k nearest neighbors through methods such as the Euclidean distance.
• For each minority class sample $x_i$, randomly select a nearest neighbor sample $x_j$ from its k nearest neighbors.
• Synthesize a new sample through the formula $x_{new}=x_i+\lambda \times (x_j-x_i)$, where $\lambda \in [0,1]$ is a random number.
• Repeat the above steps to generate a sufficient number of minority class samples, so as to balance the class distribution of the dataset and solve the problem of data imbalance.

In the previous chapter, we pointed out some problems existing in SMOTE, such as blurred boundaries, sample overlap, noise expansion, overfitting, and the blindness of k-nearest neighbors, among others (see Figure 1). Subsequent scholars have continuously proposed new improved methods of SMOTE. Borderline-SMOTE [34] is an improved algorithm of SMOTE, proposed by Han et al., and it is used to deal with imbalanced datasets. The traditional SMOTE oversamples all minority class samples, which is likely to lead to data overlap and overfitting. Borderline-SMOTE focuses on the boundary samples of the minority class. First, the minority class samples are classified: for a sample x, its k nearest neighbors are found. If the number of majority class samples among them exceeds half, x is a dangerous sample; if the number of majority class samples is 0, x is a safe sample; and the rest are uncertain samples. This algorithm only oversamples the dangerous samples. For a dangerous sample x, a sample y is selected from the minority class samples among its k nearest neighbors, and a new sample is generated according to the formula $x_{new}=x+\lambda \times (y-x)$ ($\lambda \in [0,1]$ is a random number). In this way, the new samples are closer to the classification boundary and can better reflect the class distribution relationship. When dealing with imbalanced data, compared with SMOTE, it can reduce overfitting and improve the classification performance.

The blindness of the SMOTE algorithm has been a long-standing issue. Some algorithms place more emphasis on how to select and generate points. MWMOTE (Majority Weighted Minority Oversampling Technique) [35] is used to solve the problem of data imbalance. MWMOTE can effectively adjust the data distribution, making the generated minority class samples more representative. The models trained on imbalanced datasets can obtain better classification performance and generalization ability. DTO-SMOTE [36] is an improved data oversampling method. It combines density and topological structure information to process minority class samples. Therefore, it can more reasonably increase the number of minority class samples, improve the data imbalance situation, and enhance the performance of models trained on imbalanced data. Safe-Level SMOTE [37] is a technique used to deal with data class imbalance. It generates samples based on the information within the safe region, which can better improve the imbalance and enhance the model’s ability to recognize the minority classes. SMOTE-Tomek Links [38] is an algorithm that combines SMOTE and Tomek Links, and it is used to deal with the problem of class imbalance in data. This algorithm first uses SMOTE to oversample the minority classes to increase the number of their samples, and then removes those sample pairs that may be confusing at the class boundaries (usually removing the majority class samples) through Tomek Links, so as to clean the data, reduce noise interference, optimize the distribution of the dataset, and improve the classification performance of the model for the minority classes. SASMOTE (Self-Inspected Adaptive SMOTE) [39] is an algorithm used to address the problem of highly imbalanced data classification. This algorithm uses an adaptive nearest neighbor selection algorithm to identify “visible” nearest neighbors for generating minority class samples, improving sample quality. At the same time, it introduces a self-inspection uncertainty elimination method to filter out low-quality synthetic samples that are difficult to distinguish from the majority class. SMOTE-ENN [40] is an algorithm used to address the problem of data imbalance. It combines SMOTE and ENN (Edited Nearest Neighbors). SMOTE generates new samples by interpolating between the neighbors of minority class samples to expand the number of minority class samples. ENN, based on the nearest neighbor rule, removes noisy samples from the dataset, that is, those samples whose class is different from the majority of their neighbors. SMOTE-ENN first uses SMOTE to oversample the minority class to increase its sample size, and then applies ENN to clean the oversampled data, removing noisy and misclassified samples, optimizing the dataset, and enhancing the classification performance of the model on imbalanced data. KNNOR (K-Nearest Neighbor Oversampling Approach) [13], that is, the K-nearest neighbor oversampling method, is an oversampling technique for dealing with imbalanced datasets. Based on the SMOTE algorithm, it determines the key and safe enhancement regions of minority class samples through a three-step process and generates synthetic data points. When generating artificial points, it takes into account the relative density of the entire dataset, enabling more reliable oversampling of the minority class and having stronger robustness to noise.

Simultaneously, due to the overfitting problem resulting from randomness, a greater number of algorithms focus on researching the distribution of samples. ADASYN (Adaptive Synthetic Sampling Approach) [41] is an adaptive synthetic sampling method and an oversampling technique for dealing with imbalanced datasets. It adaptively determines the number of synthetic samples to be generated for each minority class sample based on the local density of the minority class samples. In this way, the distribution of minority class samples can be made more reasonable, effectively alleviating the problem of data imbalance, enabling the learning algorithm to better learn the features of minority class samples during training, and improving the classification performance of the model on imbalanced data. Gaussian-SMOTE [42] is an algorithm for handling the problem of data imbalance. It combines the Gaussian distribution with SMOTE. Based on SMOTE, it introduces the Gaussian distribution to generate new samples. Gaussian-SMOTE utilizes the characteristics of the Gaussian distribution to make the newly generated samples more diverse and reasonable. This algorithm first determines the k-nearest neighbors of minority class samples, and then generates new samples between the samples and their neighbors according to the Gaussian distribution, expanding the number of minority class samples, alleviating data imbalance, and effectively improving the classification performance of the model on imbalanced data. Geometric SMOTE (Geometric Synthetic Minority Over-sampling Technique) [43] is an improved method of the traditional SMOTE. Based on geometric principles, it takes into account the spatial distribution patterns of minority class samples. When generating new samples, instead of simply performing linear interpolation between minority class samples and their nearest neighbors, it determines a more reasonable position for sample generation according to the geometric structure. For example, geometric figures formed by sample points (such as convex hulls, etc.) are used to restrict the generation area, making the new samples more consistent with the inherent geometric characteristics of the minority class data. This can more effectively increase the number of minority class samples, improve the distribution of imbalanced datasets, and enhance the model’s ability to learn and classify minority class samples.

The original purpose of most of the algorithms described above is to solve the balance between classes. However, the uneven distribution of similar samples in the feature space will also have an impact on the classification task, which is referred to as intra-class imbalance. ADASYN [41], namely, the Adaptive Synthetic Sampling Approach, is used to deal with imbalanced datasets. It improves the data distribution by generating samples of the minority class so as to enhance the performance of the model. Some scholars have also combined the clustering algorithm with SMOTE to reduce the intra-class imbalance. K-means SMOTE [44] is a method that combines the K-means clustering algorithm and SMOTE. SMOTE is an oversampling algorithm. By analyzing the feature space of minority class samples, it synthesizes new samples around them to alleviate the problem of data imbalance. However, it does not take into account the sample distribution density. K-means is a commonly used clustering algorithm that can divide a dataset into K clusters, making the samples within a cluster highly similar and the differences between clusters large. K-means SMOTE first uses K-means to cluster the minority class samples to obtain different clusters. Then, SMOTE is applied to each cluster separately to generate new minority class samples within the clusters. In this way, the generated samples are more in line with the distribution characteristics of the samples within each cluster, and it can more reasonably expand the number of minority class samples, avoiding the problem of unreasonable sample distribution that may occur when SMOTE is simply applied. When dealing with imbalanced datasets, it can improve the model’s ability to recognize the minority class and enhance the overall performance of the model.

Other non-SMOTE algorithms have also been further developed. GB-SMOTE [45] is used to solve the problem of imbalanced data classification. Firstly, it uses the slack variables of Support Vector Machine (SVM) to divide the minority class samples into a misclassification set, a margin set, and a correctly classified set. When selecting samples, samples are, respectively, selected from the margin set and the correctly classified set, and the selection is carried out according to the weights determined by the distances between the samples and the decision hyperplane. In the sample generation stage, the occurrence frequency of sample pairs and the distances in the feature space are calculated. The distances are evenly divided into corresponding sub-segments, and the midpoints are taken as new samples. NBG [46] is used to solve the problem of imbalanced data classification. It combines the NPSMOTE oversampling algorithm, the BALO feature selection algorithm, and the GVM classification algorithm. NPSMOTE generates effective positive class samples by removing noisy samples, assigning weights, and limiting distances. BALO can adaptively search the feature space and select important features. GVM has strong generalization ability. Experiments show that when dealing with the classification of imbalanced small-sample datasets, the NBG algorithm has significant advantages compared with a variety of existing algorithms and can effectively improve the recognition rate of the minority class. In practical applications, the performance of non-SMOTE algorithms is inferior to that of the variants of SMOTE.

3. Specific Methods of the Improved Symmetric Triangle Scoring Mechanism

3.1. Detailed Introduction of Methods

In the previous sections, for different oversampling methods, we explained their basic ideas. SMOTE is the most fundamental algorithm among them. However, SMOTE still has many drawbacks. For example, the selection of new samples for SMOTE is randomly selected between the line of two minority samples with equal possibility. This will lead to the generation of many noise samples. In addition, SMOTE has problems with samples overlapping and overgeneralization, these issues will continue to expand the scope of the original noise. Regarding the above-mentioned issues with SMOTE, many scholars have proposed improvement methods based on SMOTE. The MWMOTE provided us with some ideas for our work. We mainly borrowed the scoring weights and selection probabilities from MWMOTE to distinguish different samples. For linear interpolation in Borderline-SMOTE, we discard this simple method, choosing synthesize samples in the geometric area. In summary, T-SMOTE is an oversampling method based on the regular triangle scoring mechanism, used for multiple dynamic interpolation. The detailed introduction is as follows.

• Step 1. Multidimensional data is projected onto a 2D plane via PCA.

PCA (principal component analysis [47], as a commonly used dimensionality reduction technique, plays an important role in reducing dimensions from imbalanced multidimensional datasets to two-dimensional data.

In this tessellation problem, we choose triangles to cover a plane. For a regular triangle, there are three surrounding regular triangles with sides connected and nine regular triangles angles connected. When scoring, we will select different coefficients to calculate the score of this regular triangle area.

Nevertheless, discovering a regular triangular region that yields consistent effects across various high-dimensional spaces is arduous. Thus, we plan to make use of PCA to obtain the first two-dimensional features of the data. In an effort to minimize the losses resulting from PCA, the processed data is only utilized for the purpose of sample selection. From this perspective, PCA can mitigate noise to a certain extent as well. Finally, the creation of new samples remains in the original dimension.

• Step 2. Pave the entire two-dimensional plane with regular triangles.

In the two-dimensional plane, we choose regular triangles and pave the entire two-dimensional plane according to the method of tessellation. Now we will place the dimensionality reduction data obtained in step 1 into their respective triangles. However, at this time, we have not determined the side lengths of the triangle, so we need to determine some parameters to determine the side length of the triangle. Since the distance in the data points is easy to measure, we use the Euclidean distance [48] as the measure of distance. For each regular triangle, we assign it a field G. On this two-dimensional plane, we record the entire sample set as $X=\{x_1,x_2,\dots ,x_n\}$, where n is the number of samples in the entire dataset, and we record the minority samples as $X^{min}=\{x_1^{min},x_2^{min},\dots ,x_m^{min}\}$, where m is the number of minority samples in the entire dataset. We define the distance between two points a and b as $distance(\mathbf{a},\mathbf{b})$; the coordinate of a is $(a_1,a_2)$, The coordinate of b is $(b_1,b_2)$.

$$distance(\mathbf{a},\mathbf{b})=\sqrt{{(a_1-b_1)}^2+{(a_2-b_2)}^2}.$$

(1)

Thus far, we define the following four distances for measuring the length of the sides of a triangle.

$$\begin{array}{cc}\hfill {\overline{d}}^{min}& ={\displaystyle \frac{{\sum }_{i=1}^m{\sum }_{j=1}^m\mathrm{distance}(x_i^{min},x_j^{min})}{m·(m-1)}},j\ne i\mathrm{and}j,i=1,2,\dots ,m,\hfill \\ \hfill {\overline{d}}_{min}^{min}& ={\displaystyle \frac{{\sum }_{i=1}^{m}min\left(\mathrm{distance}(x_i^{min},x_j^{min})\right)}{m}},j\ne i\mathrm{and}j,i=1,2,\dots ,m,\hfill \\ \hfill \overline{d}& ={\displaystyle \frac{{\sum }_{i=1}^n{\sum }_{j=1}^n\mathrm{distance}(x_i,x_j)}{n·(n-1)}},j\ne i\mathrm{and}j,i=1,2,\dots ,n,\hfill \\ \hfill {\overline{d}}_{min}& ={\displaystyle \frac{{\sum }_{i=1}^{n}min\left(\mathrm{distance}(x_i,x_j)\right)}{n}},j\ne i\mathrm{and}j,i=1,2,\dots ,n.\hfill \end{array}$$

(2)

${\overline{d}}^{min}$ is used to calculate the average distance between pairs of minority samples, ${\overline{d}}_{min}^{min}$ represents the mean of the minimum distances from every minority sample to the other minority samples, $\overline{d}$ indicates the average of the distances that each sample has to other samples, and ${\overline{d}}_{min}$ represents the average value of the minimum distances from each individual sample to the rest of the samples. The side length $d_G$ of field G, at this point in time, ought to conform to the subsequent requirements:

$$d_G=3a\ast {\overline{d}}_{min}.$$

(3)

In this case, a represents the adaptation factor for the side length and complies with the subsequent requirements.

$${\displaystyle \frac{{\overline{d}}^{min}}{\overline{d}}}\le a\le {\displaystyle \frac{{\overline{d}}_{min}^{min}}{{\overline{d}}_{min}}}.$$

(4)

Through the above equation, we can obtain a relatively suitable size for the side length of a triangle. For the side length of a triangle, if the side length is too large, it will result in all points being within certain regions. If the side length is too small, there will be too few points in the field, losing the meaning of dividing the field.

Note: For rare cases where the sample may fall on the edge of a triangle, the sample is considered as noise and does not participate in the synthesis.

• Step 3. Rules for naming and assimilation.

• Rules for naming.

Based on the above rules, we can determine the side length of a triangle. The following steps require us to establish some rules to determine the type of field for triangles, which can serve as a prerequisite for our scoring mechanism. Our rule is based on the number of minority samples and majority samples for classification.

For a regular triangle, it has a total of 12 adjacent triangles. Three adjacent triangles are connected by their edges, while the remaining nine triangles are connected by their vertices.

As shown in Figure 2, Figure 2a shows the situation of a regular triangle and its 12 adjacent regular triangles, which make up a symmetrical shape.

Figure 2b shows the first situation, where three triangles are connected to the sides of a regular triangle, with the red triangle representing the initial triangle and the blue triangle representing the three triangles connected to its sides. Obviously, these three blue triangles are also symmetrical to each other.

Figure 2c shows nine triangles connected by vertices, where the green triangles are what we call triangles connected by vertices, which is the second situation. It is easy to see that these nine green triangles are also symmetrical about their centers (red triangle).

After introducing the above rules, we explain the meanings of the following symbols:

• T: Each regular triangles is recorded as a field T.
• $\mathrm{NGB}-T$: The remaining 12 regular triangles adjacent to field T are called neighbors of field T and recorded as $\mathrm{NGB}-T$. They include three triangles with connected sides and nine triangles with adjacent vertices.
• $\mathrm{NGB}-T-1$: Triangles connected to a regular triangle by their sides.
• $\mathrm{NGB}-T-2$: Triangles connected to a regular triangle by their vertices.
• $T_{min}$: A triangle containing only minority samples.
• $T_{maj}$: A triangle containing only majority samples.
• $T_{empty}$: A triangle that does not contain any samples.
• $T_{deb}$: A triangle that contains both majority and minority samples, which requires further debate.

• Rules for Assimilation.

For an empty field $T_{empty}$, when all of its neighboring fields $\mathrm{NGB}-T_{empty}$ belong to $T_G$, where $T_G$ denotes any type of field except for $T_{deb}$, if $T_{empty}$ is not an element of $T_G$, an assimilation process is initiated to make $T_{empty}$ an element of $T_G$. The assimilation rules for $T_{min}$ and $T_{maj}$ are identical to those stated above (see Figure 3). Nevertheless, for the controversial field $T_{deb}$, if all of its neighboring fields $\mathrm{NGB}-T_{deb}$ are elements of $T_D$ (where $D=T_{min}$ or $D=T_{maj}$), there exists noise in $T_{deb}$, and the noisy samples are either majority samples or minority samples within $T_{deb}$. Ultimately, $T_{deb}$ in $T_D$ is achieved via the assimilation operation.

• Step 4. Obtain marking mechanism.

After following the above steps, we will conduct the corresponding marking mechanism. Based on our previous steps, we have a premise: only $T_{min}$ and $T_{deb}$ that have not been assimilated can be selected for sampling, and the fields have been assimilated merely to take part in the scoring and weighting processes. Subsequently, we assign different scores based on the types of fields selected. For a regular triangle, our dense tiling method will have 12 triangles around it. We set a selected field as $T_{one}$, as his neighbor $NGB-T_{one}$. We have 12 choices. Let us consider these neighbors and $T_{one}$ as $T_m$, then $m=1,2,3,4,5,6,7,8,9,10,11,12,13$. Below are the specific rules.

(1)

The overall score is 18 points, and the score assigned to $T_m$ is represented as $\mathrm{mark}\left(T_m\right)$.

(2)

If $T_m\in T_{min}$ and they are connected by side, called $T_{min}-s$, then $\mathrm{mark}\left(T_m\right)=1.9$.

(3)

If $T_m\in T_{min}$ and they are connected by vertex, called $T_{min}-v$, then $\mathrm{mark}\left(T_m\right)=0.45$.

(4)

If $T_m\in T_{deb}$ and they are connected by side, called $T_{deb}-s$, then $\mathrm{mark}\left(T_m\right)=0.6$.

(5)

If $T_m\in T_{deb}$ and they are connected by vertex, called $T_{deb}-v$, then $\mathrm{mark}\left(T_m\right)=0.15$.

(6)

For itself, it is calculated according to the highest score.

(7)

If $T_m\in \{T_{empty},T_{maj}\}$ then $\mathrm{mark}\left(T_m\right)=0$.

Therefore, according to the scoring rules, the total score of the $T_m$ is

$$\mathrm{Point}\left(T_{\mathrm{one}}\right)=\sum _{m=1}^{13}\mathrm{Point}\left(T_m\right).$$

(5)

If the total score is less than 3 points, then this field does not participate in sampling, expressed as follows:

$$\mathrm{Point}\left(T_{\mathrm{one}}\right)=0.$$

(6)

This means that the field will not participate in the rating and will be simplified as an empty field.

Then, the probability formula of $T_m$ is expressed as

$${\displaystyle \frac{\mathrm{point}\left(T_{\mathrm{one}}\right)\times b^{\mathrm{point}\left(T_{\mathrm{one}}\right)}}{13\times b^{13}}},$$

(7)

where b is the boundary-weight factor; taking $b<1(b>1)$ can strengthen (weaken) the selection frequency of boundary samples. In order to avoid the situation that our sample population is not just boundary samples, the value of b in this paper is 1.

After normalization, the selection probability of $T_{one}$ is

$$P_{\mathrm{select}}={\displaystyle \frac{p}{\sum p}}={\displaystyle \frac{\frac{\mathrm{point}\left(T_{\mathrm{one}}\right)\times b^{\mathrm{Point}\left(T_{\mathrm{one}}\right)}}{13\times b^{13}}}{{\sum }_{i=1}^K\frac{\mathrm{point}\left(T_{\mathrm{one}}^i\right)\times b^{\mathrm{Point}\left(T_{\mathrm{one}}^i\right)}}{13\times b^{13}}}},$$

(8)

where K represents the total quantity of all $T_{one}$ fields. Figure 4 provides an example of a specific scoring mechanism.

• Step 5. Sampling and synthesizing new samples.

We need to randomly select a field as $T_{one}$ from the unadulterated $T_{min}$ and $T_{deb}$ based on the selection probability formula. Afterwards, we will observe neighbors of $T_{one}$: $NGB-T_{one}$, check valid $T_{min}$ and $T_{deb}$ in $NGB-T_{one}$. If they exist, we record the sum of the quantities of $T_{min}$ and $T_{deb}$ as m. If the quantity is zero, we discard it.

As shown in Figure 5, this is our rough diagram, first selecting samples and then generating new samples. After selecting $T_{one}$, we randomly select a minority sample from it. Similarly, we select a minority sample from each $NGB-T_{one}$ that meets the criteria. Let us peel off these points. We hope that the sum of the generated minority samples and the original minority samples will balance with the majority class samples. Next, let us peel off these points. We select $m+1$ sample, and let them be $a_0$, $a_1$, $a_2$, …, $a_m$. In the line connecting $a_0$ and $a_1$, we generate $A_1$ using linear interpolation method. Then, we continue to generate $A_2$ using linear interpolation in the line connecting $A_1$ and $a_2$. Similarly, we will generate the final sample $A_m$ on the line connecting $A_{m-1}$ and $a_m$. Letting r be a random number between 0 and 1, $K=1,2,\dots ,m$, we have the following synthesis formula:

$$A_K=A_{K-1}+r(a_K-A_{K-1}).$$

(9)

As shown in Figure 5, the points generated are ultimately equivalent to conducting random sampling. This sampling occurs within the closed area formed by the selected points. Overall, different partitioning methods have different selection probabilities, and the final synthesis situation is also different.

Note: To ensure that the data information is not distorted, we will not directly interpolate linearly in two-dimensional space. The specific approach is as follows. First, we select the reduced dimensional samples based on scores and probabilities; secondly, we find the high-dimensional samples corresponding to these samples before dimensionality reduction; finally, linear interpolation is applied to high-dimensional samples. In this way, we can ensure that the information of multidimensional samples is not lost.

• TS-SMOTE algorithm

To sum up, the TS-SMOTE algorithm (Algorithm 1) process is as follows:

Algorithm 1 TS-SMOTE

1: Input: Total sample set S, minority sample set $S^{min}$, total number of samples N, total number of minority samples m.  2: Output: Synthetic balanced dataset.  3: if $m==(N-m)$ then  4:     Print “The original sample dataset is balanced.”  5:     Break  6: else  7:     Step.1: Multidimensional data is projected onto a 2D plane via PCA  8:     for $S_i,S_j^{min}$ do  9:         By PCA, $X_i\Leftarrow S_i$, $X_j^{min}\Leftarrow S_j^{min}$, where $i=1,2,...,N$, $j=1,2,...,m$; 10:     end for 11:     Step.2: Pave the entire two-dimensional plane with regular triangles 12:     $a\Leftarrow X_i,X_j^{min}$ following the relevant Formulas (1) and (3); 13:     $d_G\Leftarrow a,X_i,X_j^{min}$ following the relevant Formula (2); 14:     Obtain all the fields, $G_t\Leftarrow d_G$, where $t=1,2,...,n$, and n is the number of fields; 15:     Step.3: Rules for Naming and Assimilation 16:     for $i=1:n$ do 17:         if $X_i\notin T_t$ then 18:            remove $X_i$; 19:         end if 20:     end for 21:     for $t=1:n$ do 22:         Name $T_t$; 23:         Assimilate $T_t$; 24:         Collect all $T_{one}$; 25:     end for 26:     Step.4: Obtain marking mechanism 27:     for $T_{one}$ do 28:         $Point\left(T_{one}\right)\Leftarrow T_{one}$ following the scoring rules and the associated Formulas (4) and (5); 29:         $P_{select}\Leftarrow Point\left(T_{one}\right)$ according to Formulas (6) and (7); 30:     end for 31:     Step.5: Sampling and synthesis 32:     while $m\ne N-m$ do 33:         Choose one randomly $T_{one}\Leftarrow P_{select}$; 34:         Obtain the sampling neighborhood $NGB-T_{one}\Leftarrow T_{one}$; 35:         $M\Leftarrow$ the total number of $T_{one}$ and $NGB-T_{one}$; 36:         Randomly choose $X_M^{min}\Leftarrow T_{one},NGB-T_{one}$; 37:         $S_M^{min}\Leftarrow X_M^{min}$; 38:         $NewSample\Leftarrow S_M^{min}$ following the Formula (8); 39:         $m\Leftarrow m+1$; 40:     end while 41:     Print dataset $\left\{NewSample\right\}$, $\{S,NewSample\}$; 42: end if

3.2. Discussion of Parameters

3.2.1. Discussion on Factor a

In the previous discussion, we examined the effects of factor a being either too large or too small and confined the edge adaptation factor a to a relatively appropriate range. Then, within this known range, we will discuss the optimal value of a from the perspectives of performance and time complexity.

Let us consider a dataset containing n samples with an imbalance ratio of r. The number of minority samples can then be expressed as $n/(r+1)$.

Given that each regular triangle has a side length of a, the space can accommodate i triangles horizontally and j triangles vertically. Thus, we can approximate the required number of fields to construct as $m=i\ast j$.

The original SMOTE algorithm requires iterating through all minority-class samples and searching for their k-nearest neighbors among all samples, resulting in a time complexity of $O(n\ast n/(r+1))=O\left(n^2\right)$.

Our new method differs from the original SMOTE approach primarily in three aspects: application of PCA dimensionality reduction, selection criteria for data points, and strategy for generating new samples. Next, we will analyze the time complexity of each component. First, we observe that traditional PCA requires full eigendecomposition of an $n\times n$ matrix, resulting in $O\left(n^3\right)$ time complexity. Based on principal component analysis theory, the dimensionality reduction process requires us to project the dataset onto the first q principal components. This involves computing the top q eigenvalues and eigenvectors, yielding an overall time complexity of $O(q·n^2)$ [49]. In our preliminary work, we specifically reduce the data to two dimensions ($q=2$). Since two is negligible compared to n, the effective time complexity of PCA in our case becomes $O\left(n^2\right)$. Next, we analyze the time complexity for the triangular tessellation space. Based on our previous work, we have determined the length i and width j of the triangular tessellation space. We generate approximately $m=i\times j$ regular triangles, resulting in a time complexity of $O(i\times j\times m)=O\left(m^2\right)$. Subsequently, according to our defined rules, each triangular region requires four operations: assimilation, scoring, probability calculation, and synthesis. Since these operations involve traversing both the current region and its 12 adjacent triangular regions, the time complexity becomes $O(13\times m)=O\left(m\right)$.

Finally, when we have a grasp of the sample and the divided regions, we need to match them one by one. Since the number of samples is n and the number of regions is approximately m, considering the matching between samples and regions, it is necessary to traverse the samples and regions. We can obtain a time complexity of $O(n\ast m)$.

In summary, the overall time complexity of our algorithm is dominated by the biggest component, which can be expressed as $O(max(n^2,m^2,n\times m))$. We should select the smallest possible value of m with $m<n$ to ensure the optimal performance of the algorithm. Notably, the smaller the value of m is, the larger the side length of the triangle needs to be, which in turn indicates that the value of factor a should also increase accordingly. At this point, the difference in time complexity between our algorithm and SMOTE is not pronounced.

Based on the above analysis, when the performance remains consistent, a larger value of factor a corresponds to a lower time complexity. Therefore, in the experimental section of Section 4, we choose the maximum value of a for our experiments.

3.2.2. Discussion on Factor b

We define b as the boundary weight factor because it significantly influences the proportion of new sample generation at the boundary. The scores of boundary regions are relatively lower compared to other regions. Moreover, all regions participating in the synthesis of new samples have scores no less than 3. In the probability calculation Formula (8), the scores of these regions serve as the exponent of b. Therefore, when the value of b is less than 1, the smaller b is, and the higher the probability of selecting from the lower-scoring boundary regions. In this case, we can ensure that more samples are generated near the boundary. This strategy is similar to the concept of borderline-SMOTE and helps to obtain clearer boundaries. When the value of b is greater than 1, the larger b is, the lower the probability of selecting boundary regions, thus ensuring that more generated samples are distributed away from the boundary. In this paper, b is set to 1. Since we aim for a more universal approach and do not overly focus on the generation of boundary samples, we expect to achieve a compromise.

4. Specific Settings of the Experiment and Comparative Analysis

After establishing the above rules, we need to conduct experiments to analyze the effectiveness of this method. We compare with other methods to demonstrate the advantages of TS-SMOTE. We first introduce the experimental setup, which mainly includes the dataset, classifiers, and comparison parameters. In the following chapters, we will present the experimental results and the analysis.

4.1. Related Methods and Experimental Settings

The dataset for this experiment mainly comes from the imbalanced dataset in the KEEL dataset. We selected 30 imbalanced datasets among them.

Table 1. The imbalanced dataset required for the experiment.

Datesets	Samples	Minority Samples	Majority Samples	IR
glass1	214	76	138	1.82
wisconsin	683	238	444	1.86
pima	768	268	500	1.87
glass0	214	70	144	2.06
yeast1	1484	429	1055	2.46
haberman	306	81	225	2.78
vehicle3	846	212	634	2.99
vehicle0	846	199	647	3.25
ecoli1	336	77	259	3.36
new-thyroid2	215	35	180	5.14
ecoli2	336	52	284	5.46
segment0	2308	329	1979	6.02
yeast3	1484	163	1321	8.10
yeast-2_vs_4	514	51	463	9.08
yeast-0-2-5-7-9_vs_3-6-8	1004	99	905	9.14
yeast-0-5-6-7-9_vs_4	528	51	477	9.35
vowel0	988	90	898	9.98
yeast-1_vs_7	459	30	429	14.30
ecoli4	336	20	316	15.80
page-blocks-1-3_vs_4	472	28	444	15.86
dermatology-6	358	20	338	16.9
yeast-1-4-5-8_vs_7	693	30	663	22.10
yeast4	1484	51	1433	28.1
winequality-red-4	1599	53	1546	29.17
yeast-1-2-8-9_vs_7	947	30	917	30.57
yeast5	1484	44	1440	32.73
yeast6	1484	35	1449	41.4
poker-8-9_vs_5	1485	25	1460	58.4
poker-8-9_vs_6	2075	25	2050	82
poker-8_vs_6	1477	17	1460	85.88

shows the specific information of these datasets.

For SMOTE, there are many improved methods. Among these methods, we choose ADASYN, G-SMOTE, MWMOTE, GAUSSIAN-SMOTE, DTO-SMOTE, KNNOR-SMOTE, and SMOTE to conduct experimental comparisons. We simplify the class-imbalanced problem by using synthesized minority samples transformed into a conventional binary classification problem. This can make the experiment more convenient and facilitate our further analysis. In the experiments, multi-layer perceptron (MLP) [50] networks, support vector machines (SVM) [51], and Adaptive Boosting (AdaBoost) [33] are used to assess the rationality of the synthesized samples under diverse methods.

For TS-SMOTE and other methods, we need some standards in the experiment to measure their performance and help us make better comparisons. Here is the confusion matrix (shown in Figure 6) for binary classification, along with the formula representations of $Accuracy,Precision,andRecall$ [52]:

$$\begin{array}{cc}\hfill \mathrm{Accuracy}& ={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}},\hfill \\ \hfill \mathrm{Precision}& ={\displaystyle \frac{TP}{TP+FP}},\hfill \\ \hfill \mathrm{Recall}& ={\displaystyle \frac{TP}{TP+FN}}.\hfill \end{array}$$

(10)

However, among the three evaluation indicators stated earlier, accuracy predominantly emphasizes the majority samples. In contrast, precision and recall place greater emphasis on the performance aspects of the minority samples. Therefore, in order to evaluate the ability of comprehensive classification, we need other indicators to measure [53].

Here are the three additional metrics in detail.

G-mean (geometric mean) is a comprehensive metric that takes into account both the precision and recall of a classifier. It is often used to evaluate the performance of binary classification models with imbalanced positive and negative samples. A larger G-mean indicates better model performance.

$$G-mean=\sqrt{{\displaystyle \frac{TP}{TP+FN}}\ast {\displaystyle \frac{TN}{TN+FP}}}.$$

(11)

The $F1-measure$ is an indicator used to evaluate classification problems. In many multi-classification machine learning problems, the $F1-measure$ is often chosen as the final evaluation metric. It is the harmonic mean of precision and recall, with a value range from 0 to 1, where 1 is the maximum and 0 is the minimum.

$$F1-measure={\displaystyle \frac{2\ast \mathrm{Precision}}{\mathrm{Precision}+\mathrm{Recall}}}.$$

(12)

AUC stands for area under the curve. Here, the curve denotes the ROC curve, and AUC represents the area beneath the ROC curve. The area ranges from 0.1 to 1. As a number, AUC (area under the curve) can intuitively evaluate the advantages and disadvantages of the classifier. The larger the value, the better the performance of the classifier.

To solve various class imbalance problems, we use the five-fold cross-validation method. That is, the dataset is randomly divided into five parts with equal or similar numbers; one part is selected as the test sample each time, and the other four parts serve as the training sample. By transforming the part of the test set, five results are obtained. The process was then repeated 20 times to obtain 100 sets of results. Finally, the average of these 100 groups of results was calculated for subsequent comparison. Using average value for comparison can avoid the extreme results in a single experiment and make the comparison conclusion more convincing.

4.2. Experimental Results and Comparative Analysis

We set up three sections to describe the our experiment. In part one, we analyze the results of our experiment. The effect of this part is to show the advantages of the TS-SMOTE. In part two, we want to make the data more intuitive, so we list the images of data visualization and related experimental data. In part three, we use some statistical methods to test the significant effect of our method.

4.2.1. Comparison and Analysis of Various Metric Evaluations

After we finished our TS-SMOTE method, we chose seven different oversamling methods for comparison. They are, respectively, ADASYN, BORDERLINE, MWMOTE, GAUSSIAN-SMOTE, DTO-SMOTE, KNNOR-SMOTE, and SMOTE. In the table, we use A, B, M, G, D, K, and S as substitutes. The TS-SMOTE method uses T instead. Through these eight methods, we obtain the oversampled data. These datasets have been processed and are now balanced datasets. Next, our focus will be on detecting which method holds the advantage. To avoid potential anomalies caused by a single classifier, we selected three classifiers: MLP, SVM, and AdaBoost. We choose 30 different datasets, and the related results are shown in

Table 2. The scores of each dataset across different metrics when using MLP as a classifier (The bold numbers representing the corresponding methods is the best).

Datasets		Estimators								IR
		A	B	M	G	D	K	S	T
glass1	F1-score	0.7962	0.7969	0.5584	0.5786	0.5425	0.5485	0.6309	0.7994	1.82
	G-mean	0.7853	0.7816	0.2905	0.3182	0.4503	0.4444	0.7050	0.7924	1.82
	AUC	0.8439	0.8538	0.4016	0.4058	0.5265	0.5183	0.8008	0.8708	1.82
wisconsin	F1-score	0.9791	0.9727	0.9793	0.9671	0.9740	0.9752	0.9554	0.9741	1.86
	G-mean	0.9786	0.9720	0.9788	0.9669	0.9738	0.9749	0.9690	0.9739	1.86
	AUC	0.9894	0.9885	0.9918	0.9945	0.9947	0.9944	0.9953	0.9953	1.86
pima	F1-score	0.7800	0.7949	0.6934	0.6950	0.7095	0.7177	0.6665	0.8002	1.87
	G-mean	0.7770	0.7857	0.6748	0.6763	0.6848	0.6965	0.7409	0.8019	1.87
	AUC	0.8410	0.8478	0.7539	0.7571	0.7750	0.7839	0.8208	0.8843	1.87
glass0	F1-score	0.8591	0.8000	0.7466	0.7388	0.7744	0.7753	0.7201	0.8513	2.06
	G-mean	0.8416	0.7928	0.6809	0.6811	0.7042	0.7028	0.7949	0.8430	2.06
	AUC	0.9078	0.8693	0.8083	0.8106	0.8217	0.8203	0.8597	0.9183	2.06
yeast1	F1-score	0.7765	0.7461	0.7379	0.7116	0.7476	0.7471	0.5981	0.8262	2.46
	G-mean	0.7525	0.7408	0.7077	0.7092	0.7376	0.7359	0.7195	0.8321	2.46
	AUC	0.8292	0.8171	0.7899	0.7930	0.8154	0.8172	0.7929	0.9143	2.46
haberman	F1-score	0.6750	0.6972	0.6434	0.6070	0.6266	0.6114	0.4723	0.7797	2.78
	G-mean	0.6676	0.6988	0.6272	0.6402	0.6569	0.6464	0.6255	0.7826	2.78
	AUC	0.7493	0.7597	0.6858	0.7097	0.7389	0.7490	0.6949	0.8447	2.78
vehicle3	F1-score	0.8911	0.8984	0.5827	0.6056	0.6285	0.6128	0.6732	0.8873	2.99
	G-mean	0.8848	0.8933	0.5689	0.5719	0.5597	0.5749	0.7916	0.8872	2.99
	AUC	0.9400	0.9551	0.7365	0.7550	0.7660	0.7656	0.8952	0.9621	2.99
vehicle0	F1-score	0.9885	0.9888	0.9364	0.9178	0.9347	0.9352	0.9550	0.9863	3.25
	G-mean	0.9885	0.9886	0.9334	0.9123	0.9326	0.9319	0.9787	0.9862	3.25
	AUC	0.9981	0.9984	0.9799	0.9811	0.9851	0.9855	0.9978	0.9988	3.25
ecoli1	F1-score	0.8999	0.9114	0.9063	0.8732	0.8953	0.8919	0.7864	0.9238	3.36
	G-mean	0.8952	0.9114	0.8977	0.8704	0.8911	0.8879	0.8862	0.9223	3.36
	AUC	0.9558	0.9595	0.9413	0.9516	0.9633	0.9647	0.9546	0.9758	3.36
new-thyroid2	F1-score	0.9935	0.9937	0.9973	0.9911	0.9973	0.9973	0.9746	0.9961	5.14
	G-mean	0.9933	0.9936	0.9972	0.9911	0.9972	0.9972	0.9923	0.9961	5.14
	AUC	0.9998	1.0000	0.9999	0.9992	0.9997	0.9996	0.9996	0.9997	5.14
ecoli2	F1-score	0.9667	0.9875	0.8671	0.9130	0.9403	0.9365	0.8480	0.9605	5.46
	G-mean	0.9654	0.9883	0.8669	0.9113	0.9384	0.9345	0.9194	0.9605	5.46
	AUC	0.9841	0.9963	0.9497	0.9525	0.9753	0.9717	0.9671	0.9883	5.46
segment0	F1-score	0.9988	0.9989	0.9986	0.9974	0.9986	0.9986	0.9922	0.9982	6.02
	G-mean	0.9988	0.9989	0.9986	0.9974	0.9986	0.9986	0.9951	0.9982	6.02
	AUC	0.9998	0.9999	0.9998	0.9997	0.9998	0.9998	0.9997	0.9994	6.02
yeast3	F1-score	0.9665	0.9678	0.9431	0.9065	0.9528	0.9513	0.7544	0.9682	8.10
	G-mean	0.9656	0.9670	0.9415	0.9070	0.9521	0.9506	0.8938	0.9684	8.10
	AUC	0.9828	0.9894	0.9765	0.9700	0.9846	0.9848	0.9652	0.9948	8.10
yeast-2_vs_4	F1-score	0.9787	0.9773	0.8711	0.9205	0.9168	0.9159	0.7294	0.9742	9.08
	G-mean	0.9781	0.9797	0.8729	0.9167	0.9172	0.9160	0.8662	0.9742	9.08
	AUC	0.9890	0.9914	0.9579	0.9621	0.9679	0.9671	0.9506	0.9949	9.08
yeast-0-2-5-7-9_vs_3-6-8	F1-score	0.9741	0.9783	0.8218	0.9089	0.9063	0.9086	0.7648	0.9792	9.14
	G-mean	0.9730	0.9777	0.8187	0.9117	0.9071	0.9097	0.8822	0.9793	9.14
	AUC	0.9887	0.9935	0.9046	0.9565	0.9519	0.9560	0.9337	0.9925	9.14
yeast-0-5-6-7-9_vs_4	F1-score	0.9526	0.9562	0.8114	0.9490	0.8224	0.8227	0.5053	0.9494	9.35
	G-mean	0.9505	0.9546	0.8061	0.9498	0.8219	0.8249	0.7432	0.9499	9.35
	AUC	0.9808	0.9810	0.9002	0.9785	0.9123	0.9091	0.8635	0.9821	9.35
vowel0	F1-score	0.9990	0.9986	0.9984	0.9977	0.9983	0.9984	0.9893	0.9992	9.98
	G-mean	0.9990	0.9986	0.9984	0.9977	0.9983	0.9984	0.9989	0.9992	9.98
	AUC	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	9.98
yeast-1_vs_7	F1-score	0.9342	0.9621	0.8262	0.7903	0.8040	0.8220	0.3021	0.9612	14.3
	G-mean	0.9288	0.9614	0.8136	0.7886	0.7995	0.8163	0.6579	0.9615	14.3
	AUC	0.9754	0.9857	0.8803	0.8722	0.8926	0.8998	0.7690	0.9796	14.3
ecoli4	F1-score	0.9874	0.9765	0.9856	0.9741	0.9856	0.9871	0.7984	0.9845	15.8
	G-mean	0.9870	0.9811	0.9851	0.9738	0.9854	0.9870	0.9003	0.9843	15.8
	AUC	0.9955	0.9985	0.9980	0.9964	0.9990	0.9993	0.9863	0.9963	15.8
page-blocks-1-3_vs_4	F1-score	0.9970	0.9965	0.9959	0.9764	0.9943	0.9950	0.9107	0.9935	15.86
	G-mean	0.9969	0.9965	0.9958	0.9761	0.9942	0.9949	0.9918	0.9935	15.86
	AUC	0.9973	0.9971	0.9970	0.9806	0.9959	0.9961	0.9982	0.9978	15.86
dermatology-6	F1-score	1.0000	1.0000	1.0000	0.9997	1.0000	1.0000	1.0000	1.0000	16.9
	G-mean	1.0000	1.0000	1.0000	0.9997	1.0000	1.0000	1.0000	1.0000	16.9
	AUC	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	16.9
yeast-1-4-5-8_vs_7	F1-score	0.9409	0.9490	0.7908	0.7248	0.7853	0.7905	0.1399	0.9739	22.1
	G-mean	0.9362	0.9467	0.7745	0.7103	0.7675	0.7707	0.5011	0.9742	22.1
	AUC	0.9769	0.9824	0.8677	0.8068	0.8598	0.8523	0.6412	0.9854	22.1
yeast4	F1-score	0.9747	0.9797	0.8799	0.8553	0.8857	0.8905	0.3594	0.9812	28.1
	G-mean	0.9738	0.9793	0.8766	0.8558	0.8833	0.8885	0.7187	0.9813	28.1
	AUC	0.9874	0.9907	0.9502	0.9280	0.9514	0.9554	0.8627	0.9948	28.1
winequality-red-4	F1-score	0.9757	0.9771	0.8702	0.8628	0.8586	0.8726	0.1336	0.9760	29.17
	G-mean	0.9747	0.9768	0.8654	0.8620	0.8522	0.8677	0.4250	0.9762	29.17
	AUC	0.9906	0.9887	0.9336	0.9350	0.9286	0.9365	0.6719	0.9890	29.17
yeast-1-2-8-9_vs_7	F1-score	0.9588	0.9637	0.7935	0.7716	0.7860	0.7907	0.1529	0.9837	30.57
	G-mean	0.9571	0.9633	0.7771	0.7727	0.7796	0.7852	0.5071	0.9838	30.57
	AUC	0.9862	0.9920	0.8623	0.8552	0.8790	0.8821	0.7009	0.9907	30.57
yeast5	F1-score	0.9922	0.9920	0.9814	0.9642	0.9819	0.9824	0.6945	0.9886	32.73
	G-mean	0.9922	0.9919	0.9808	0.9630	0.9813	0.9818	0.9118	0.9887	32.73
	AUC	0.9964	0.9972	0.9903	0.9868	0.9925	0.9923	0.9874	0.9995	32.73
yeast6	F1-score	0.9861	0.9883	0.9396	0.9102	0.9366	0.9373	0.4551	0.9910	41.4
	G-mean	0.9857	0.9882	0.9370	0.9098	0.9356	0.9363	0.7965	0.9910	41.4
	AUC	0.9939	0.9958	0.9798	0.9670	0.9845	0.9844	0.9226	0.9982	41.4
poker-8-9_vs_5	F1-score	0.9931	0.9913	0.9920	0.9190	0.9927	0.9913	0.2277	0.9928	58.4
	G-mean	0.9930	0.9912	0.9919	0.9182	0.9926	0.9911	0.5858	0.9929	58.4
	AUC	0.9992	0.9979	0.9990	0.9771	0.9993	0.9990	0.7983	0.9967	58.4
poker-8-9_vs_6	F1-score	0.9999	1.0000	1.0000	1.0000	1.0000	0.9999	1.0000	1.0000	82
	G-mean	0.9999	1.0000	1.0000	1.0000	1.0000	0.9999	1.0000	1.0000	82
	AUC	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	82
poker-8_vs_6	F1-score	1.0000	1.0000	1.0000	1.0000	0.9999	1.0000	0.9938	1.0000	85.88
	G-mean	1.0000	1.0000	1.0000	1.0000	0.9999	1.0000	0.9963	1.0000	85.88
	AUC	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	85.88

,

Table 3. When using SVM as a classifier, the scores of each dataset across different metrics.

Datasets		Estimators								IR
		A	B	M	G	D	K	S	T
glass1	F1-score	0.7705	0.7639	0.4011	0.4027	0.4031	0.4041	0.626	0.7918	1.82
	G-mean	0.7303	0.7046	0.094	0.0912	0.0845	0.0866	0.6985	0.7709	1.82
	AUC	0.8176	0.8293	0.3623	0.3961	0.4334	0.4271	0.7819	0.8519	1.82
wisconsin	F1-score	0.9772	0.9703	0.9778	0.9734	0.977	0.9783	0.9575	0.9775	1.86
	G-mean	0.9766	0.9691	0.9769	0.973	0.9764	0.9778	0.9723	0.9771	1.86
	AUC	0.9852	0.9765	0.9843	0.9891	0.986	0.9869	0.9864	0.991	1.86
pima	F1-score	0.7712	0.7963	0.6935	0.7038	0.7114	0.717	0.6657	0.8042	1.87
	G-mean	0.7647	0.7759	0.6885	0.7085	0.7208	0.7251	0.7402	0.8063	1.87
	AUC	0.8338	0.8339	0.7717	0.8051	0.8152	0.8192	0.8248	0.894	1.87
glass0	F1-score	0.8334	0.8085	0.584	0.5887	0.5883	0.5877	0.7157	0.8478	2.06
	G-mean	0.7875	0.7895	0.3491	0.3634	0.3621	0.3602	0.7913	0.8284	2.06
	AUC	0.8677	0.8635	0.452	0.4408	0.4493	0.4476	0.8523	0.8831	2.06
yeast1	F1-score	0.7586	0.7461	0.7477	0.7179	0.7366	0.7395	0.5859	0.744	2.46
	G-mean	0.7156	0.7134	0.6968	0.7134	0.7236	0.7253	0.7091	0.7582	2.46
	AUC	0.795	0.8117	0.7798	0.8004	0.811	0.8145	0.7857	0.8321	2.46
haberman	F1-score	0.6473	0.6483	0.5939	0.5362	0.4895	0.4803	0.4323	0.744	2.78
	G-mean	0.6502	0.6672	0.437	0.6001	0.5637	0.5571	0.589	0.7582	2.78
	AUC	0.7331	0.7372	0.4458	0.7164	0.7166	0.7184	0.68	0.8321	2.78
vehicle3	F1-score	0.8606	0.8679	0.6265	0.6445	0.645	0.6458	0.6537	0.8347	2.99
	G-mean	0.835	0.8474	0.6486	0.661	0.6642	0.664	0.7951	0.8356	2.99
	AUC	0.8976	0.9168	0.7244	0.7351	0.7388	0.7396	0.8713	0.9185	2.99
vehicle0	F1-score	0.9742	0.9736	0.8515	0.8414	0.8419	0.8426	0.9248	0.9679	3.25
	G-mean	0.9737	0.9725	0.8066	0.7884	0.7893	0.7905	0.9699	0.9677	3.25
	AUC	0.998	0.9948	0.9479	0.9552	0.9619	0.9624	0.9942	0.9972	3.25
ecoli1	F1-score	0.9106	0.9057	0.9171	0.9099	0.9096	0.9133	0.7707	0.9188	3.36
	G-mean	0.9046	0.9048	0.9087	0.9024	0.9031	0.9072	0.8824	0.9185	3.36
	AUC	0.952	0.9511	0.9569	0.9504	0.9676	0.9679	0.9469	0.976	3.36
new-thyroid2	F1-score	0.9945	0.9945	0.9138	0.8922	0.8652	0.8649	0.9671	0.9882	5.14
	G-mean	0.9944	0.9944	0.9082	0.8979	0.8737	0.8735	0.9872	0.9882	5.14
	AUC	1	1	0.9786	0.9913	0.9817	0.9916	0.9994	0.9993	5.14
ecoli2	F1-score	0.9153	0.9814	0.9127	0.9328	0.9588	0.9595	0.8794	0.9618	5.46
	G-mean	0.9166	0.9827	0.9108	0.9335	0.9588	0.9597	0.9414	0.9622	5.46
	AUC	0.9781	0.9891	0.9776	0.9716	0.9864	0.986	0.9597	0.9893	5.46
segment0	F1-score	0.9943	0.9964	0.948	0.9851	0.9882	0.9871	0.9892	0.9968	6.02
	G-mean	0.9943	0.9964	0.9435	0.9851	0.9881	0.9871	0.9911	0.9968	6.02
	AUC	0.9998	0.9999	0.9986	0.9977	0.9989	0.9988	0.9998	0.9999	6.02
yeast3	F1-score	0.9471	0.937	0.9416	0.9086	0.9559	0.956	0.7352	0.9577	8.1
	G-mean	0.9447	0.9335	0.9389	0.9095	0.9548	0.955	0.8988	0.9576	8.1
	AUC	0.9801	0.9792	0.9764	0.9722	0.9851	0.9852	0.9688	0.9861	8.1
yeast-2_vs_4	F1-score	0.966	0.9663	0.9476	0.9328	0.9211	0.931	0.7452	0.9768	9.08
	G-mean	0.9643	0.9693	0.9429	0.9279	0.9229	0.932	0.882	0.9771	9.08
	AUC	0.9918	0.9871	0.9842	0.9698	0.9885	0.9895	0.9696	0.9961	9.08
yeast-0-2-5-7-9_vs_3-6-8	F1-score	0.8978	0.9078	0.8118	0.9127	0.9211	0.924	0.7904	0.9656	9.14
	G-mean	0.8926	0.9005	0.7969	0.9155	0.9229	0.9261	0.8825	0.9661	9.14
	AUC	0.9602	0.9758	0.8975	0.9454	0.9885	0.9559	0.941	0.9907	9.14
yeast-0-5-6-7-9_vs_4	F1-score	0.9062	0.9051	0.8388	0.8078	0.8336	0.8506	0.4727	0.934	9.35
	G-mean	0.9008	0.9046	0.8321	0.8123	0.8389	0.8543	0.7433	0.9355	9.35
	AUC	0.9584	0.9693	0.9138	0.9015	0.9264	0.9298	0.8778	0.9828	9.35
vowel0	F1-score	0.9991	0.9997	0.9841	0.9787	0.9819	0.9819	0.9968	0.9994	9.98
	G-mean	0.9991	0.9997	0.9841	0.978	0.9813	0.9814	0.9997	0.9994	9.98
	AUC	1	1	0.9975	0.9991	0.9992	0.9994	1	1	9.98
yeast-1_vs_7	F1-score	0.8699	0.9354	0.8445	0.8201	0.8329	0.8511	0.275	0.9486	14.3
	G-mean	0.8594	0.9344	0.8259	0.817	0.8264	0.8418	0.6656	0.9489	14.3
	AUC	0.9398	0.9747	0.9116	0.8975	0.9179	0.9254	0.7682	0.9777	14.3
ecoli4	F1-score	0.9875	0.9765	0.9871	0.9854	0.9878	0.9903	0.7748	0.99	15.8
	G-mean	0.9872	0.9811	0.9867	0.985	0.9878	0.9901	0.8831	0.99	15.8
	AUC	0.9982	0.9961	0.9991	0.9974	0.9995	0.9997	0.9909	0.9994	15.8
page-blocks-1-3_vs_4	F1-score	0.9824	0.9976	0.9201	0.7186	0.7595	0.7869	0.8741	0.9882	15.86
	G-mean	0.9824	0.9976	0.9199	0.747	0.7813	0.8036	0.968	0.9882	15.86
	AUC	0.9994	1	0.9675	0.9064	0.9281	0.9169	0.9981	0.9995	15.86
dermatology-6	F1-score	1	1	0.9956	0.9845	0.9876	0.9839	1	0.9981	16.9
	G-mean	1	1	0.9955	0.9842	0.9873	0.9837	1	0.9981	16.9
	AUC	1	1	1	0.9986	0.9995	0.9995	1	1	16.9
yeast-1-4-5-8_vs_7	F1-score	0.8697	0.8939	0.7847	0.7495	0.7766	0.7656	0.1377	0.9742	22.1
	G-mean	0.8573	0.8863	0.7556	0.7471	0.7446	0.7321	0.5672	0.9745	22.1
	AUC	0.9339	0.9556	0.8611	0.8243	0.8501	0.8448	0.6461	0.9808	22.1
yeast4	F1-score	0.9286	0.9532	0.884	0.8739	0.8769	0.8955	0.2942	0.9786	28.1
	G-mean	0.9235	0.9528	0.8789	0.8741	0.879	0.895	0.7475	0.9789	28.1
	AUC	0.9753	0.9857	0.9485	0.9365	0.9524	0.9577	0.8831	0.9932	28.1
winequality-red-4	F1-score	0.9173	0.9472	0.6572	0.6032	0.6261	0.6241	0.1719	0.9772	29.17
	G-mean	0.9115	0.9462	0.6777	0.6387	0.6462	0.6481	0.614	0.9774	29.17
	AUC	0.9636	0.9793	0.7517	0.7123	0.7147	0.7145	0.7163	0.987	29.17
yeast-1-2-8-9_vs_7	F1-score	0.8548	0.9284	0.8037	0.7655	0.8016	0.7974	0.1156	0.9823	30.57
	G-mean	0.8463	0.9273	0.7715	0.7773	0.7878	0.7854	0.5731	0.9825	30.57
	AUC	0.9426	0.9782	0.857	0.8638	0.8804	0.8818	0.6894	0.9879	30.57
yeast5	F1-score	0.985	0.9852	0.9741	0.9647	0.9749	0.9748	0.6332	0.9885	32.73
	G-mean	0.9847	0.9849	0.973	0.963	0.9739	0.9737	0.9191	0.9885	32.73
	AUC	0.9937	0.9935	0.9886	0.9879	0.9908	0.9906	0.9854	0.9991	32.73
yeast6	F1-score	0.9494	0.9806	0.9046	0.9154	0.9247	0.9266	0.394	0.989	41.4
	G-mean	0.9476	0.9802	0.9022	0.9156	0.9253	0.9271	0.815	0.9891	41.4
	AUC	0.9863	0.9937	0.9689	0.9666	0.9799	0.9802	0.9242	0.9976	41.4
poker-8-9_vs_5	F1-score	0.9714	0.97	0.9633	0.909	0.9656	0.9632	0.12	0.9939	58.4
	G-mean	0.97	0.9688	0.9614	0.9069	0.9641	0.9612	0.5999	0.9939	58.4
	AUC	0.9957	0.9952	0.9914	0.9683	0.9941	0.9909	0.7749	0.9946	58.4
poker-8-9_vs_6	F1-score	1	0.9993	0.9186	0.8697	0.9408	0.9331	0.821	0.9947	82
	G-mean	1	0.9993	0.9102	0.877	0.9383	0.9273	0.8449	0.9947	82
	AUC	1	0.9999	0.9881	0.9677	0.9898	0.9904	0.9805	0.9998	82
poker-8_vs_6	F1-score	1	0.9997	0.9297	0.8997	0.9382	0.9292	0.7803	0.997	85.88
	G-mean	1	0.9997	0.9212	0.9037	0.9316	0.9205	0.8093	0.997	85.88
	AUC	1	1	0.9953	0.9791	0.9967	0.995	0.9854	0.9997	85.88

and

Table 4. When using AdaBoost as a classifier, the scores of each dataset across different metrics.

Datasets		Estimators								IR
		A	B	M	G	D	K	S	T
glass1	F1-score	0.7914	0.8175	0.7906	0.805	0.8216	0.7987	0.6742	0.8128	1.82
	G-mean	0.7897	0.8144	0.7858	0.8053	0.8153	0.7943	0.7407	0.8101	1.82
	AUC	0.8376	0.8465	0.8396	0.8662	0.8574	0.8453	0.7975	0.8722	1.82
wisconsin	F1-score	0.9695	0.9637	0.9727	0.969	0.9697	0.9702	0.943	0.9671	1.86
	G-mean	0.9695	0.9632	0.9726	0.969	0.9696	0.97	0.9575	0.9671	1.86
	AUC	0.9897	0.984	0.9907	0.9948	0.9937	0.9939	0.991	0.9928	1.86
pima	F1-score	0.7426	0.746	0.7581	0.7707	0.7742	0.7773	0.6539	0.8003	1.87
	G-mean	0.7459	0.7358	0.7547	0.7747	0.7721	0.7757	0.7301	0.8007	1.87
	AUC	0.8197	0.7994	0.8279	0.8633	0.8488	0.8492	0.8106	0.884	1.87
glass0	F1-score	0.8259	0.8764	0.8115	0.8299	0.8525	0.8669	0.7101	0.8471	2.06
	G-mean	0.8128	0.8738	0.8012	0.8282	0.8445	0.8622	0.7818	0.8452	2.06
	AUC	0.8725	0.8952	0.8656	0.9038	0.9046	0.9087	0.847	0.909	2.06
yeast1	F1-score	0.7694	0.7395	0.7571	0.8301	0.7693	0.7721	0.5913	0.8297	2.46
	G-mean	0.7509	0.7363	0.7427	0.8356	0.7641	0.7668	0.7126	0.8354	2.46
	AUC	0.8267	0.8221	0.8226	0.9176	0.8444	0.8439	0.7891	0.9157	2.46
haberman	F1-score	0.7091	0.6288	0.7126	0.7532	0.7063	0.6993	0.3879	0.788	2.78
	G-mean	0.7028	0.6385	0.7083	0.7621	0.7077	0.7034	0.5505	0.794	2.78
	AUC	0.7713	0.6829	0.7724	0.8265	0.7581	0.7648	0.6057	0.8612	2.78
vehicle3	F1-score	0.8312	0.8429	0.8287	0.8117	0.824	0.8233	0.5901	0.8375	2.99
	G-mean	0.83	0.8371	0.8241	0.8129	0.8211	0.8209	0.7286	0.8417	2.99
	AUC	0.9034	0.9078	0.8998	0.9047	0.9008	0.9001	0.8348	0.9283	2.99
vehicle0	F1-score	0.9791	0.9791	0.9771	0.9743	0.9769	0.9799	0.9171	0.9709	3.25
	G-mean	0.9791	0.9787	0.9766	0.974	0.9765	0.9796	0.9577	0.9707	3.25
	AUC	0.9959	0.9954	0.9955	0.9934	0.9959	0.9966	0.9871	0.9938	3.25
ecoli1	F1-score	0.8855	0.915	0.8991	0.9237	0.9153	0.9103	0.7575	0.9282	3.36
	G-mean	0.8845	0.9177	0.8949	0.9227	0.9136	0.9087	0.8591	0.9272	3.36
	AUC	0.9382	0.9509	0.9444	0.9731	0.9565	0.9548	0.9218	0.9741	3.36
new-thyroid2	F1-score	0.994	0.9904	0.9928	0.9842	0.9909	0.9906	0.9286	0.981	5.14
	G-mean	0.9938	0.9902	0.9927	0.984	0.9907	0.9905	0.9521	0.9809	5.14
	AUC	0.9997	0.9996	0.9997	0.999	0.9998	0.9997	0.9971	0.999	5.14
ecoli2	F1-score	0.9363	0.9686	0.929	0.9636	0.9576	0.9492	0.7802	0.9618	5.46
	G-mean	0.9357	0.9702	0.9271	0.9639	0.9575	0.9492	0.874	0.962	5.46
	AUC	0.9723	0.9738	0.967	0.9906	0.9819	0.9802	0.9337	0.9867	5.46
segment0	F1-score	0.9982	0.9983	0.9982	0.9977	0.9982	0.9985	0.9884	0.9975	6.02
	G-mean	0.9982	0.9983	0.9982	0.9977	0.9982	0.9985	0.9927	0.9975	6.02
	AUC	1	1	0.9999	0.9999	0.9998	0.9999	0.9995	0.9998	6.02
yeast3	F1-score	0.953	0.9468	0.9504	0.9697	0.9586	0.9585	0.7709	0.971	8.1
	G-mean	0.9523	0.9456	0.9498	0.9696	0.9583	0.9582	0.9028	0.9711	8.1
	AUC	0.9807	0.9828	0.9817	0.996	0.9873	0.9873	0.9684	0.9925	8.1
yeast-2_vs_4	F1-score	0.9683	0.9655	0.9699	0.9452	0.9699	0.967	0.7365	0.9744	9.08
	G-mean	0.9679	0.9682	0.9692	0.9454	0.9694	0.9667	0.8511	0.9745	9.08
	AUC	0.9867	0.9743	0.9855	0.9872	0.9874	0.9888	0.9097	0.9893	9.08
yeast-0-2-5-7-9_vs_3-6-8	F1-score	0.9224	0.9362	0.9339	0.9772	0.9421	0.9317	0.7188	0.9781	9.14
	G-mean	0.9218	0.9355	0.9319	0.9774	0.9427	0.9326	0.8746	0.9782	9.14
	AUC	0.9712	0.9753	0.9701	0.9934	0.9827	0.9821	0.9324	0.992	9.14
yeast-0-5-6-7-9_vs_4	F1-score	0.8899	0.9263	0.9037	0.949	0.9093	0.9025	0.4829	0.9475	9.35
	G-mean	0.8892	0.9244	0.901	0.9498	0.9083	0.902	0.7392	0.9482	9.35
	AUC	0.949	0.9634	0.9509	0.9785	0.9589	0.9586	0.8072	0.9754	9.35
vowel0	F1-score	0.9964	0.9966	0.9959	0.9964	0.9963	0.9971	0.9628	0.9908	9.98
	G-mean	0.9964	0.9966	0.9959	0.9964	0.9963	0.997	0.984	0.9909	9.98
	AUC	0.9996	0.9996	0.9998	0.9997	0.9997	0.9998	0.9982	0.9976	9.98
yeast-1_vs_7	F1-score	0.886	0.9333	0.8976	0.9557	0.8966	0.8993	0.294	0.9546	14.3
	G-mean	0.8823	0.9295	0.8946	0.9561	0.8944	0.897	0.6198	0.9548	14.3
	AUC	0.9459	0.9609	0.9488	0.9823	0.9523	0.9538	0.7581	0.9795	14.3
ecoli4	F1-score	0.9873	0.9926	0.9888	0.9864	0.9882	0.9885	0.7837	0.9838	15.8
	G-mean	0.9872	0.9925	0.9887	0.9863	0.9881	0.9884	0.8826	0.9838	15.8
	AUC	0.9984	0.9975	0.999	0.9992	0.9992	0.9992	0.9878	0.9992	15.8
page-blocks-1-3_vs_4	F1-score	0.9981	0.9969	0.9979	0.9911	0.9989	0.9984	0.9551	0.9964	15.86
	G-mean	0.9981	0.997	0.9979	0.9911	0.9989	0.9984	0.972	0.9964	15.86
	AUC	1	0.9988	0.9989	0.9982	0.9989	0.9989	0.9989	0.9998	15.86
dermatology-6	F1-score	0.9972	0.9984	0.9979	0.9964	0.9982	0.9985	0.9378	0.9985	16.9
	G-mean	0.9972	0.9984	0.9979	0.9964	0.9982	0.9985	0.9634	0.9985	16.9
	AUC	0.9985	0.9985	0.9985	0.9994	0.9985	0.9985	0.9985	0.9985	16.9
yeast-1-4-5-8_vs_7	F1-score	0.8747	0.9243	0.883	0.9708	0.8827	0.8811	0.1419	0.9702	22.1
	G-mean	0.8694	0.9222	0.8792	0.9712	0.8787	0.8765	0.4893	0.9705	22.1
	AUC	0.9419	0.969	0.9494	0.9865	0.9485	0.9433	0.6543	0.9857	22.1
yeast4	F1-score	0.932	0.9625	0.9322	0.9787	0.9407	0.932	0.3462	0.9789	28.1
	G-mean	0.9313	0.962	0.9312	0.9788	0.9401	0.9318	0.7409	0.9791	28.1
	AUC	0.9724	0.9844	0.9717	0.9937	0.977	0.976	0.8392	0.9923	28.1
winequality-red-4	F1-score	0.8446	0.9531	0.8633	0.881	0.848	0.8449	0.1227	0.9675	29.17
	G-mean	0.8419	0.9525	0.8617	0.8839	0.8454	0.8434	0.538	0.9676	29.17
	AUC	0.9191	0.9827	0.9331	0.9549	0.9193	0.9197	0.6305	0.9863	29.17
yeast-1-2-8-9_vs_7	F1-score	0.8895	0.9484	0.8945	0.9813	0.9025	0.8969	0.1496	0.9803	30.57
	G-mean	0.8863	0.9481	0.8908	0.9814	0.9009	0.8949	0.5398	0.9804	30.57
	AUC	0.9534	0.9827	0.9561	0.9907	0.9635	0.9588	0.695	0.9895	30.57
yeast5	F1-score	0.9863	0.9881	0.9863	0.9892	0.989	0.9883	0.6745	0.9902	32.73
	G-mean	0.9862	0.9879	0.9861	0.9892	0.9889	0.9882	0.8938	0.9902	32.73
	AUC	0.9937	0.9947	0.9939	0.9996	0.9955	0.9954	0.9835	0.9994	32.73
yeast6	F1-score	0.9578	0.9842	0.9586	0.9881	0.9595	0.9623	0.3768	0.9894	41.4
	G-mean	0.9573	0.984	0.9579	0.9881	0.9592	0.9621	0.7675	0.9894	41.4
	AUC	0.9852	0.9934	0.9855	0.9982	0.9889	0.9887	0.8756	0.9975	41.4
poker-8-9_vs_5	F1-score	0.9229	0.851	0.9434	0.9932	0.9467	0.9363	0.0251	0.9939	58.4
	G-mean	0.919	0.8431	0.9425	0.9932	0.9466	0.9354	0.3727	0.9939	58.4
	AUC	0.9714	0.902	0.985	0.9931	0.9885	0.9848	0.471	0.994	58.4
poker-8-9_vs_6	F1-score	0.9004	0.7933	0.9461	0.9896	0.9584	0.9421	0.0279	0.9913	82
	G-mean	0.8985	0.7934	0.9465	0.9897	0.9587	0.9427	0.3535	0.9913	82
	AUC	0.9624	0.8755	0.9838	0.9889	0.9875	0.9858	0.4155	0.9912	82
poker-8_vs_6	F1-score	0.9247	0.9145	0.9659	0.9931	0.9669	0.9635	0.0361	0.9933	85.88
	G-mean	0.9234	0.9108	0.9657	0.9932	0.9668	0.9631	0.4112	0.9934	85.88
	AUC	0.9816	0.9649	0.9922	0.9944	0.9928	0.9917	0.5509	0.9951	85.88

.

In the previous section, we selected three evaluation metrics: F1-score, G-mean, and AUC. In the 30 datasets under 3 classifiers, we conducted 90 sets of experiments.

Under the three classifiers (MLP, SVM, and AdaBoost), with 90 rankings per classifier (3 metrics * 30 datasets), our TS-SMOTE method achieved first place 53 times for MLP, 60 times for SVM, and 44 times for AdaBoost. The results confirm TS-SMOTE’s unequivocal superiority. The data demonstrates that under the SVM classifier, our TS-SMOTE method achieved the strongest performance compared to the other two classifiers.

To provide a clearer visualization of the rankings, we calculated the average rank of each method across different classifiers under various evaluation metrics, as presented in

Table 5. Average rankings of different metrics across classifiers.

Classifier	MLP			SVM			AdaBoost
Estimators	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC
ADASYN	2.47	2.53	2.93	2.77	3.10	2.90	5.27	5.37	5.40
BORDERLINE	2.30	2.53	2.93	2.70	3.10	2.90	3.97	3.37	4.87
MWMOTE	4.47	5.00	5.53	5.23	5.83	6.13	4.60	4.80	5.00
GAUSSIAN-SMOTE	6.20	6.47	5.73	6.50	6.43	6.83	3.43	3.40	3.00
DTO-SMOTE	4.80	4.47	4.03	5.27	5.20	4.77	3.67	3.70	3.00
KNNOR-SMOTE	4.40	4.43	4.00	4.90	4.87	4.57	4.00	3.90	3.00
SMOTE	7.13	6.37	5.53	6.73	6.07	6.00	8.00	8.00	8.00
TS-SMOTE	2.20	2.10	1.87	1.73	1.63	1.43	2.77	2.70	2.00

. Across all three classifiers (MLP, SVM, AdaBoost) and three evaluation metrics (F1-score, G-mean, AUC), TS-SMOTE consistently ranked first in all nine comparative assessments. TS-SMOTE secured first place in all three classifiers’ aggregated rankings (shown in

Table 6. Average rankings across different classifiers.

Classifier	ADASYN	BORDERLINE	MWMOTE	GAUSSIAN	DTO-SMOTE	KNNOR-SMOTE	SMOTE	TS-SMOTE
MLP	2.64	2.30	4.90	6.13	4.43	4.28	6.34	2.06
SVM	2.92	2.68	5.73	6.59	5.08	4.78	6.27	1.60
AdaBoost	5.34	4.27	4.80	3.04	3.61	3.86	7.82	2.58

), demonstrating consistent dominance across MLP, SVM, and AdaBoost. Furthermore, we provide violin-box plots to comprehensively visualize the comparative performance of all methods. First, let us briefly introduce the violin-box plot. Violin plots reveal the probability density of evaluation scores, and box plots superimposed on violins quantify quartile statistics. Among them, the solid lines from top to bottom represent the maximum, upper quartile, median, lower quartile, and minimum values, respectively. We also computed the mean values of TS-SMOTE (shown in

Table 7. Average values of different metrics across classifiers.

Classifier	MLP			SVM			AdaBoost
Estimators	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC
ADASYN	0.9405	0.9373	0.9666	0.9147	0.9072	0.9492	0.9088	0.9066	0.9479
BORDERLINE	0.9414	0.9400	0.9642	0.9245	0.9195	0.9556	0.9159	0.9149	0.9453
MWMOTE	0.8716	0.8553	0.9079	0.8420	0.8114	0.8793	0.9146	0.9122	0.9503
GAUSSIAN-SMOTE	0.8676	0.8553	0.9094	0.8243	0.8099	0.8848	0.9382	0.9389	0.9689
DTO-SMOTE	0.8792	0.8698	0.9220	0.8374	0.8201	0.8976	0.9203	0.9190	0.9556
KNNOR-SMOTE	0.8801	0.8715	0.9228	0.8395	0.8218	0.8969	0.9175	0.9166	0.9549
SMOTE	0.6728	0.8165	0.8943	0.6433	0.8160	0.8927	0.5822	0.7578	0.8330
TS-SMOTE	0.9493	0.9492	0.9748	0.9404	0.9402	0.9679	0.9457	0.9462	0.9724

), demonstrating that TS-SMOTE achieves the highest average scores across all groups. The violin plot in Figure 7 reveals that TS-SMOTE exhibits superior stability, with its performance metrics consistently clustering in the higher value range, thereby confirming its exceptional robustness. This represents the scenario under the MLP classifier. Figure 8 and Figure 9 illustrate the performance under SVM and AdaBoost, showing similar trends to MLP. In summary, TS-SMOTE demonstrates high performance and robust stability. Regarding median rankings, although TS-SMOTE’s G-mean performance under the MLP classifier is slightly inferior to BORDERLINE (by only 0.008), its average ranking is higher than BORDERLINE. Therefore, we can conclude that TS-SMOTE outperforms BORDERLINE in this scenario. A similar situation is observed for AUC performance under AdaBoost, where the difference between TS-SMOTE and GAUSSIAN-SMOTE is even smaller (only 0.003). Moreover, TS-SMOTE achieves a better (lower) average ranking than GAUSSIAN-SMOTE, further confirming its superior performance in this case.

Additionally, we calculated the percentage difference in mean values between TS-SMOTE and other methods (shown in

Table 8. Percentage differences between TS-SMOTE and other methods.

Classifier	MLP			SVM			AdaBoost
Estimators	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC
ADASYN	0.93%	1.25%	0.85%	2.77%	3.58%	1.95%	3.98%	4.27%	2.55%
BORDERLINE	0.84%	0.97%	1.09%	1.70%	2.23%	1.28%	3.20%	3.36%	2.83%
MWMOTE	8.54%	10.41%	7.11%	11.04%	14.70%	9.59%	3.35%	3.65%	2.30%
GAUSSIAN	9.00%	10.40%	6.94%	13.16%	14.89%	8.97%	0.80%	0.77%	0.36%
DTO-SMOTE	7.67%	8.73%	5.57%	11.59%	13.65%	7.53%	2.73%	2.91%	1.74%
KNNOR-SMOTE	7.57%	8.53%	5.48%	11.33%	13.44%	7.61%	3.03%	3.18%	1.81%
SMOTE	34.09%	15.03%	8.61%	37.51%	14.14%	8.08%	47.59%	22.11%	15.44%

). The percentage difference serves to compare values of the same type, helping us understand their relative differences. The larger the difference, the more significant the disparity. Since TS-SMOTE’s mean values exceed all other methods, all computed differences are greater than 0, demonstrating that our method is optimal.

4.2.2. Data Visualization of Some Characteristic Datasets

After completing the data-level analysis, it is necessary to demonstrate the generation effects of TS-SMOTE. We selected three sample datasets (haberman, winequality-red-4, and yeast3) to show the generation results.

ADASYN adjusts the number of generated samples dynamically, according to the “learning difficulty” of the minority class samples. BORDERLINE-SMOTE generates samples concentrated in the boundary region rather than being uniformly distributed. MWMOTE identifies and filters noisy samples through the k-nearest neighbors (k-NN) method to avoid generating invalid data. The core characteristic of GAUSSIAN-SMOTE is that it enhances the diversity of the generated samples and the rationality of their distribution through the Gaussian distribution. DTO-SMOTE is suitable for scenarios of imbalanced data with high noise levels and blurred boundaries. KNNOR-SMOTE analyzes the local neighborhood (k-nearest neighbors) of minority class samples and decides whether it is necessary to generate new samples dynamically. SMOTE generates new data among minority class samples through linear interpolation. Finally, TS-SMOTE filters out the noise points and outliers, and then conducts multiple linear interpolations in the selected area to generate new samples.

TS-SMOTE can filter out the noise samples effectively, and generate new samples that are more concentrated. For example, as presented in Figure 10, we can observe that under the dataset yeast3, the samples generated by TS-SMOTE are more concentrated compared to BORDERLINE-SMOTE. The essential reason is that BORDERLINE-SMOTE uses a clustering method to generate new samples. In contrast, TS-SMOTE employs multiple linear interpolations and combines them with a scoring mechanism. This approach effectively ensures that new samples can be generated in an orderly manner within the regions between the minority samples, thereby further guaranteeing the effectiveness of the newly generated samples.

Particularly evident in Figure 10 and Figure 11, when the samples are dispersed in a primary region along with some outliers, traditional oversampling techniques such as SMOTE, Borderline-SMOTE, ADASYN, Guassian-SMOTE, and DTO-SMOTE tend to produce a significant amount of noise in the area between the main region and the outliers. Subsequently, as illustrated in Figure 10, Figure 11 and Figure 12, Borderline-SMOTE, MWMOTE, and KNNOR-SMOTE are limited to generating new samples within a relatively small geometric area. This limitation makes them highly susceptible to overfitting issues. Moreover, Borderline-SMOTE specifically generates samples only around the boundary, which results in the loss of a substantial amount of information regarding the original data distribution.

In stark contrast to these conventional methods, our proposed approach, TS-SMOTE, effectively prevents the generation of samples either between the outliers and the main regions or between separate distinct regions. Through practical applications and experiments, it has been convincingly demonstrated that TS-SMOTE is capable of efficiently addressing the problem of imbalanced classification in scenarios involving multiple regions and outliers, offering a more reliable and effective solution compared to the existing techniques.

4.2.3. Friedman Test and Wilcoxon Signed Rank Test

The Friedman test is a non-parametric statistical test that is widely used to compare multiple classifiers over multiple datasets. The role of the Friedman test is to determine whether there are significant differences in the performance of multiple classifiers across different datasets. For L classifiers and Q datasets, the Friedman test statistic ${\chi }_F^2$ is calculated as follows:

$${\chi }_F^2={\displaystyle \frac{12Q}{L(L+1)}}\sum _{i=1}^L{(R_i-{\displaystyle \frac{L+1}{2}})}^2,$$

(13)

where $R_i$ is the average rank of the i-th classifier, $1\le i\le L$. The null hypothesis $H_0$ is that all classifiers have identical performance distributions. Under $H_0$, ${\chi }_F^2$ follows a chi-squared distribution with ($L-1$) degrees of freedom. A small p-value (e.g., $p<0.05$) allows us to reject $H_0$, indicating significant differences among classifiers.

The Wilcoxon signed-rank test is a non-parametric test for paired data. Its role is to compare the performance of two related samples, in this case, the performance of TS-SMOTE and another method. For each dataset, let $d_j=x_{j,1}-x_{j,2}$ be the difference between the performance of TS-SMOTE ($x_{j,1}$) and another method ($x_{j,2}$). The test statistic W is computed as follows:

$$W=\sum _{j:d_j>0}{r}_j,$$

(14)

where $r_j$ is the rank of $|d_j|$. Under the null hypothesis that $d_j$ has a symmetric distribution around zero, W follows a Wilcoxon distribution. A p-value $<0.05$ indicates that TS-SMOTE’s performance differs significantly from the compared method.

The two tables (

Table 9. Results for Friedman test of different estimators.

Classifier	MLP			SVM			AdaBoost
Estimators	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC	F1-Score	G-Mean	AUC
p-value	2.968 × ${10}^{-23}$	5.305 × ${10}^{-21}$	6.116 × ${10}^{-18}$	6.467 × ${10}^{-23}$	9.582 × ${10}^{-21}$	9.465 × ${10}^{-24}$	6.864 × ${10}^{-17}$	1.808 × ${10}^{-17}$	1.197 × ${10}^{-20}$

and

Table 10. Results for the Wilcoxon signed rank test.

Classifier	ADASYN	BORDERLINE	MWMOTE	GAUSSIAN	DTO-SMOTE	KNNOR-SMOTE	SMOTE
MLP	8.691 × ${10}^{-6}$	9.346 × ${10}^{-4}$	2.041 × ${10}^{-12}$	4.654 × ${10}^{-15}$	1.310 × ${10}^{-12}$	1.540 × ${10}^{-12}$	6.963 × ${10}^{-15}$
SVM	7.957 × ${10}^{-11}$	1.683 × ${10}^{-8}$	4.169 × ${10}^{-16}$	1.743 × ${10}^{-16}$	1.802 × ${10}^{-16}$	3.293 × ${10}^{-16}$	8.920 × ${10}^{-16}$
AdaBoost	1.934 × ${10}^{-12}$	6.107 × ${10}^{-9}$	5.350 × ${10}^{-12}$	2.586 × ${10}^{-2}$	1.068 × ${10}^{-9}$	3.023 × ${10}^{-10}$	2.730 × ${10}^{-16}$

) below show the results of the Friedman test [54] and the Wilcoxon signed-rank test [55]. From the results of these two tests, we can observe that both tests yield statistically significant results ($p<0.05$), leading us to reject the null hypothesis. This indicates a significant difference between TS-SMOTE and the other methods, demonstrating the uniqueness of TS-SMOTE.

Therefore, by combining statistical testing with experimental analysis, we compared TS-SMOTE against the seven other methods across multiple dimensions. The comprehensive results confirm that TS-SMOTE achieves the best overall performance.

5. Conclusions

This paper proposes a novel SMOTE method based on the symmetric triangle scoring mechanism. This improved method first reduces the dimensionality of the basic data, divides regions according to the properties of symmetric triangles (regular triangles), and establishes a corresponding scoring mechanism. Then, samples are dynamically selected based on different regions, and new samples are generated through multiple linear interpolation. We tested with 30 imbalanced datasets and compared with 7 other methods. Three classifiers were adopted, and three metrics were selected for the comparative experiments. We integrated the obtained data, and verified through the Friedman test and the Wilcoxon signed rank test that TS-SMOTE has the best comprehensive performance and can effectively handle the classification problems of imbalanced datasets.

The symmetric triangle scoring mechanism emerges as a valuable tool for imbalanced classification. TS-SMOTE fully considers the impact of the distribution of majority samples on the synthetic points of minority samples and obtains more trustworthy results in imbalanced classification tasks.

There are many directions for future expansion in this paper. For example, only the case of symmetric triangles has been considered in this paper, and it can also be extended to three-dimensional space based on symmetry. Meanwhile, only binary classification problems were considered in this paper, and multi-classification problems are also an area to be explored. Therefore, in the future, we will study more methods based on symmetry.