Absolute Value Inequality SVM for the PU Learning Problem
Abstract
:1. Introduction
 (1)
 Our approach employs TEDA for the extraction of reliable negative examples from the unlabeled dataset. Unlike alternative methods for negative example selection, TEDA operates without predefined assumptions, relying solely on the spatial distribution of the data. Furthermore, this method is not only computationally straightforward but also remains effective even with a limited number of positive examples.
 (2)
 Our approach is the first to use the convex absolute value inequality technique to solve PU problems. This technique enables the successive linearization algorithm to resolve the optimization model, thus reducing the computational costs.
 (3)
 Our approach adopts the hyperparameter optimization method HORD to set the optimal values of the parameters in the algorithm, thus reducing the amount of resources spent in manually adjusting the parameters.
2. Related Work
3. Preliminaries
3.1. Convex Absolute Value Inequality with SVM
3.2. Typicality and Eccentricity Data Analytics
3.3. HORD Algorithm
4. Proposed Approach
4.1. Reliable Negative Samples
Algorithm 1 Generation of reliable negative samples 

4.2. AVISVM Formulation for the Resulting SemiSupervised Problem
Algorithm 2 Successive linearization algorithm for (20) 

4.3. Performance Metric
4.4. Parameter Tuning
Algorithm 3 Parameter tuning by HORD 

4.5. AVISVM Algorithm
Algorithm 4 AVISVM algorithm for the PU learning problem 

5. Numerical Experiments
5.1. Algorithms for Comparison
 LUHC [9]: It exploits both the geometrical and the discriminant properties of the examples; thus, it can improve the classification performance. This algorithm follows the oneclass classification scheme.
 B${l}_{1}$NPSVM [14]: It replaces the ${L}_{2}$ norm in NPSVM with an ${L}_{1}$ regularization term to address the PU learning problem, achieving satisfactory results in both classification and feature selection aspects. This algorithm follows the biased support vector method.
 kNNSVM [20]: It sorts the unlabeled examples based on the sum of their distances to the knearest positive examples; it then selects the examples at the largest distance as reliable negative examples. The iterative SVM is used to train the classifier. This algorithm is one of the representative algorithms using the twostep strategy.
5.2. Experimental Results
5.2.1. Illustration on the Iris Dataset
5.2.2. Results on Other UCI Datasets
6. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
 Stolfi, P.; Mastropietro, A.; Pasculli, G.; Tieri, P.; Vergni, D. NIAPU: Networkinformed adaptive positiveunlabeled learning for disease gene identification. Bioinformatics 2023, 39, btac848. [Google Scholar] [CrossRef] [PubMed]
 Fung, G.P.C.; Yu, J.X.; Lu, H.; Yu, P.S. Text classification without negative examples revisit. IEEE Trans. Knowl. Data Eng. 2005, 18, 6–20. [Google Scholar] [CrossRef]
 Li, F.Y.; Dong, S.Y.; Leier, A.; Han, M.Y.; Guo, X.D.; Xu, J.; Wang, X.Y.; Pan, S.R.; Jia, C.Z.; Zhang, Y.; et al. Text classification without negative examples revisit Positiveunlabeled learning in bioinformatics and computational biology: A brief review. Brief. Bioinform. 2022, 23, 1–13. [Google Scholar]
 de Souza, M.C.; Nogueira, B.M.; Rossi, R.G.; Marcacini, R.M.; Dos Santos, B.N.; Rezende, S.O. A networkbased positive and unlabeled learning approach for fake news detection. Mach. Learn. 2022, 111, 3549–3592. [Google Scholar] [CrossRef] [PubMed]
 Jaskie, K.; Martin, J.; Spanias, A. PV Fault Detection Using Positive Unlabeled Learning. Appl. Sci. 2021, 11, 5599. [Google Scholar] [CrossRef]
 Yang, W.T.; Chen, C.T.; Sang, C.Y.; Huang, S.H. Reinforced pulearning with hybrid negative sampling strategies for recommendation. ACM Trans. Intell. Syst. Technol. 2023, 14, 1–25. [Google Scholar] [CrossRef]
 Manevitz, L.M.; Yousef, M. Oneclass SVMs for document classification. J. Mach. Learn. Res. 2001, 2, 139–154. [Google Scholar]
 Schölkopf, B.; Platt, J.C.; ShaweTaylor, J.; Smola, A.J.; Williamson, R.C. Estimating the support of a highdimensional distribution. Neural Comput. 2001, 13, 1443–1471. [Google Scholar] [CrossRef] [PubMed]
 Shao, Y.H.; Chen, W.J.; Liu, L.M.; Deng, N.Y. Laplacian unithyperplane learning from positive and unlabeled examples. Inf. Sci. 2015, 314, 152–168. [Google Scholar] [CrossRef]
 Zhou, K.; Xue, G.R.; Yang, Q.; Yu, Y. Learning with positive and unlabeled examples using topicsensitive PLSA. IEEE Trans. Knowl. Data Eng. 2009, 22, 46–58. [Google Scholar] [CrossRef]
 Liu, B.; Dai, Y.; Li, X.; Lee, W.S.; Yu, P.S. Building text classifiers using positive and unlabeled examples. In Proceedings of the Third IEEE International Conference on Data Mining, Melbourne, FL, USA, 19–22 November 2003; pp. 179–186. [Google Scholar]
 Ke, T.; Li, M.; Zhang, L.; Lv, H.; Ge, X. Construct a biased SVM classifier based on Chebyshev distance for PU learning. J. Intell. Fuzzy Syst. 2020, 39, 3749–3767. [Google Scholar] [CrossRef]
 Zhang, Y.; Ju, X.; Tian, Y. Nonparallel hyperplane support vector machine for pu learning. In Proceedings of the 2014 10th International Conference on Natural Computation (ICNC), Xiamen, China, 19–21 August 2014; pp. 703–708. [Google Scholar]
 Bai, F.; Yuan, Y. L1norm Nonparallel Support Vector Machine for PU Learning. In Proceedings of the 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP), Shanghai, China, 19–21 November 2018; pp. 1–5. [Google Scholar]
 Luo, C.; Zhao, P.; Chen, C.; Qiao, B.; Du, C.; Zhang, H.Y.; Wu, W.; Cai, H.W.; He, B.; Rajmohan, S.; et al. Pulns: Positiveunlabeled learning with effective negative sample selector. In Proceedings of the AAAI Conference on Artificial Intelligence, Virtually, 2–9 February 2021; pp. 8784–8792. [Google Scholar]
 Liu, B.; Lee, W.S.; Yu, P.S.; Li, X. Partially supervised classification of text documents. ICML 2002, 2, 387–394. [Google Scholar]
 Yu, H.; Han, J.; Chang, K.C.C. PEBL: Positive example based learning for web page classification using SVM. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Edmonton, AB, Canada, 23–26 July 2002; pp. 239–248. [Google Scholar]
 Li, X.; Liu, B. Learning to classify texts using positive and unlabeled data. In Proceedings of the 18th International Joint Conference on Artificial Intelligence, Acapulco, Mexico, 9–15 August 2003; pp. 587–592. [Google Scholar]
 Liu, B.; Liu, Q.; Xiao, Y. A new method for positive and unlabeled learning with privileged information. Appl. Intell. 2022, 52, 2465–2479. [Google Scholar] [CrossRef]
 Zhang, B.; Zuo, W. Reliable Negative Extracting Based on kNN for Learning from Positive and Unlabeled Examples. J. Comput. 2009, 4, 94–101. [Google Scholar] [CrossRef]
 Mangasarian, O.L. Unsupervised classification via convex absolute value inequalities. Optimization 2015, 64, 81–86. [Google Scholar] [CrossRef]
 Angelov, P. Anomaly detection based on eccentricity analysis. In Proceedings of the 2014 IEEE Symposium on Evolving and Autonomous Learning Systems (EALS), Orlando, FL, USA, 9–12 December 2014. [Google Scholar]
 Ilievski, I.; Akhtar, T.; Feng, J.; Shoemaker, C. Efficient hyperparameter optimization for deep learning algorithms using deterministic rbf surrogates. In Proceedings of the ThirtyFirst AAAI Conference on Artificial Intelligence, San Francisco, CA, USA, 4–9 February 2017. [Google Scholar]
 Tian, Y.; Qi, Z.; Ju, X.; Shi, Y.; Liu, X. Nonparallel support vector machines for pattern classification. IEEE Trans. Cybern. 2013, 44, 1067–1079. [Google Scholar] [CrossRef] [PubMed]
 Tian, Y.; Ju, X.; Qi, Z. Efficient sparse nonparallel support vector machines for classification. Neural Comput. Appl. 2014, 24, 1089–1099. [Google Scholar] [CrossRef]
 Jung, C.; Shen, Y.; Jiao, L. Learning to rank with ensemble ranking SVM. Neural Process. Lett. 2015, 42, 703–714. [Google Scholar] [CrossRef]
 Angelov, P. Outside the box: An alternative data analytics framework. J. Autom. Mob. Robot. Intell. Syst. 2014, 8, 29–35. [Google Scholar] [CrossRef]
 Saw, J.G.; Yang, M.C.; Mo, T.C. Chebyshev inequality with estimated mean and variance. Am. Stat. 1984, 38, 130–132. [Google Scholar] [CrossRef]
 Mangasarian, O.L. Absolute value equation solution via concave minimization. Optim. Lett. 2007, 1, 3–8. [Google Scholar] [CrossRef]
 Lee, W.S.; Liu, B. Learning with positive and unlabeled examples using weighted logistic regression. In Proceedings of the Twentieth International Conference on Machine Learning, Washington, DC, USA, 21–24 August 2003; pp. 448–455. [Google Scholar]
 UCI Machine Learning Repository. Available online: http://archive.ics.uci.edu/ml (accessed on 6 March 2024).
Dataset  # Examples  # Features  # Positive  # Negative 

Sonar  208  60  97  111 
Hearts  270  13  120  150 
Haberman  306  3  225  81 
BUPA  345  6  145  200 
Australian  690  14  383  307 
German  1000  24  700  300 
Banknote  1372  4  610  762 
Spambase  4601  57  1803  2788 
Twonorm  7400  20  3703  3697 
HTRU2  17898  8  1639  16259 
Dataset  LUHC  B${\mathit{l}}_{1}$NPSVM  KNNSVM  AVISVM 

$\mathit{Acc}$ (%)  $\mathit{Acc}$ (%)  $\mathit{Acc}$ (%)  $\mathit{Acc}$ (%)  
$\mathit{F}$Score  $\mathit{F}$Score  $\mathit{F}$Score  $\mathit{F}$Score  
Sonar  53.23  57.10  58.71  51.32 
0.512  0.607  0.555  0.567  
Hearts  62.47  60.49  65.43  74.44 
0.695  0.741  0.696  0.791  
Haberman  73.15  71.15  70.14  72.39 
0.584  0.757  0.809  0.832  
BUPA  45.77  47.12  44.33  43.27 
0.585  0.592  0.593  0.561  
Australian  55.56  67.21  72.22  83.09 
0.773  0.723  0.778  0.822  
German  65.67  70.10  68.24  70.67 
0.723  0.700  0.828  0.819  
Banknote  92.23  87.62  93.06  93.71 
0.921  0.900  0.913  0.933  
Spambase  52.10  50.36  49.71  54.64 
0.492  0.567  0.479  0.634  
Twonorm  97.84  90.14  92.18  97.39 
0.978  0.921  0.932  0.975  
HTRU2  54.64  45.54  52.10  43.96 
0.165  0.189  0.207  0.243 
Dataset  LUHC  B${\mathit{l}}_{1}$NPSVM  KNNSVM  AVISVM 

$\mathit{Acc}$ (%)  $\mathit{Acc}$ (%)  $\mathit{Acc}$ (%)  $\mathit{Acc}$ (%)  
$\mathit{F}$Score  $\mathit{F}$Score  $\mathit{F}$Score  $\mathit{F}$Score  
Sonar  54.84  57.58  64.35  54.11 
0.550  0.534  0.574  0.580  
Hearts  70.37  61.73  56.79  76.79 
0.699  0.762  0.724  0.802  
Haberman  76.09  73.80  65.22  73.91 
0.630  0.802  0.844  0.846  
BUPA  43.27  43.17  44.71  43.37 
0.595  0.602  0.596  0.569  
Australian  62.80  69.64  78.82  83.57 
0.754  0.821  0.829  0.843  
German  68.73  65.33  73.08  71.33 
0.738  0.728  0.828  0.823  
Banknote  94.66  88.92  94.64  96.87 
0.924  0.923  0.937  0.975  
Spambase  60.97  52.88  54.84  61.45 
0.493  0.585  0.669  0.674  
Twonorm  97.61  92.24  92.65  97.52 
0.978  0.961  0.940  0.977  
HTRU2  55.11  51.06  57.80  59.04 
0.267  0.255  0.289  0.304 
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. 
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Yuan, Y.; Bai, F. Absolute Value Inequality SVM for the PU Learning Problem. Mathematics 2024, 12, 1454. https://doi.org/10.3390/math12101454
Yuan Y, Bai F. Absolute Value Inequality SVM for the PU Learning Problem. Mathematics. 2024; 12(10):1454. https://doi.org/10.3390/math12101454
Chicago/Turabian StyleYuan, Yongjia, and Fusheng Bai. 2024. "Absolute Value Inequality SVM for the PU Learning Problem" Mathematics 12, no. 10: 1454. https://doi.org/10.3390/math12101454