Next Article in Journal
Do Randomized Algorithms Improve the Efficiency of Minimal Learning Machine?
Previous Article in Journal
COVID-19 Symptoms Detection Based on NasNetMobile with Explainable AI Using Various Imaging Modalities
Open AccessArticle

Towards Knowledge Uncertainty Estimation for Open Set Recognition

1
Associação Fraunhofer Portugal Research, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
2
Laboratório de Instrumentação, Engenharia Biomédica e Física da Radiação (LIBPhys-UNL), Departamento de Física, Faculdade de Ciências e Tecnologia (FCT), Universidade Nova de Lisboa, 2829-516 Caparica, Portugal
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Mach. Learn. Knowl. Extr. 2020, 2(4), 505-532; https://doi.org/10.3390/make2040028
Received: 16 September 2020 / Revised: 23 October 2020 / Accepted: 26 October 2020 / Published: 30 October 2020
Uncertainty is ubiquitous and happens in every single prediction of Machine Learning models. The ability to estimate and quantify the uncertainty of individual predictions is arguably relevant, all the more in safety-critical applications. Real-world recognition poses multiple challenges since a model’s knowledge about physical phenomenon is not complete, and observations are incomplete by definition. However, Machine Learning algorithms often assume that train and test data distributions are the same and that all testing classes are present during training. A more realistic scenario is the Open Set Recognition, where unknown classes can be submitted to an algorithm during testing. In this paper, we propose a Knowledge Uncertainty Estimation (KUE) method to quantify knowledge uncertainty and reject out-of-distribution inputs. Additionally, we quantify and distinguish aleatoric and epistemic uncertainty with the classical information-theoretical measures of entropy by means of ensemble techniques. We performed experiments on four datasets with different data modalities and compared our results with distance-based classifiers, SVM-based approaches and ensemble techniques using entropy measures. Overall, the effectiveness of KUE in distinguishing in- and out-distribution inputs obtained better results in most cases and was at least comparable in others. Furthermore, a classification with rejection option based on a proposed combination strategy between different measures of uncertainty is an application of uncertainty with proven results. View Full-Text
Keywords: uncertainty; machine learning; open set recognition; entropy; out-of-distribution uncertainty; machine learning; open set recognition; entropy; out-of-distribution
Show Figures

Figure 1

MDPI and ACS Style

Pires, C.; Barandas, M.; Fernandes, L.; Folgado, D.; Gamboa, H. Towards Knowledge Uncertainty Estimation for Open Set Recognition. Mach. Learn. Knowl. Extr. 2020, 2, 505-532.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

1
Back to TopTop