Next Article in Journal
Artificial Intelligence to Enhance Aerodynamic Shape Optimisation of the Aegis UAV
Previous Article in Journal
A Near Real-Time Automatic Speaker Recognition Architecture for Voice-Based User Interface
Open AccessFeature PaperReview

Evaluation of Regression Models: Model Assessment, Model Selection and Generalization Error

1
Predictive Society and Data Analytics Lab, Faculty of Information Technolgy and Communication Sciences, Tampere University, 33100 Tampere, Finland
2
Institute of Biosciences and Medical Technology, 33520 Tampere, Finland
3
Institute for Intelligent Production, Faculty for Management, University of Applied Sciences Upper Austria, Steyr Campus, 4400 Steyr, Austria
4
Department of Mechatronics and Biomedical Computer Science, University for Health Sciences, Medical Informatics and Technology, 6060 Hall in Tirol, Austria
5
College of Computer and Control Engineering, Nankai University, Tianjin 300071, China
*
Author to whom correspondence should be addressed.
Mach. Learn. Knowl. Extr. 2019, 1(1), 521-551; https://doi.org/10.3390/make1010032
Received: 9 February 2019 / Revised: 13 March 2019 / Accepted: 18 March 2019 / Published: 22 March 2019
(This article belongs to the Section Learning)
When performing a regression or classification analysis, one needs to specify a statistical model. This model should avoid the overfitting and underfitting of data, and achieve a low generalization error that characterizes its prediction performance. In order to identify such a model, one needs to decide which model to select from candidate model families based on performance evaluations. In this paper, we review the theoretical framework of model selection and model assessment, including error-complexity curves, the bias-variance tradeoff, and learning curves for evaluating statistical models. We discuss criterion-based, step-wise selection procedures and resampling methods for model selection, whereas cross-validation provides the most simple and generic means for computationally estimating all required entities. To make the theoretical concepts transparent, we present worked examples for linear regression models. However, our conceptual presentation is extensible to more general models, as well as classification problems. View Full-Text
Keywords: machine learning; statistics; model selection; model assessment; regression models; high-dimensional data; data science; bias-variance tradeoff; generalization error machine learning; statistics; model selection; model assessment; regression models; high-dimensional data; data science; bias-variance tradeoff; generalization error
Show Figures

Figure 1

MDPI and ACS Style

Emmert-Streib, F.; Dehmer, M. Evaluation of Regression Models: Model Assessment, Model Selection and Generalization Error. Mach. Learn. Knowl. Extr. 2019, 1, 521-551.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

1
Back to TopTop