Next Article in Journal
Virtual UV Fluorescence Microscopy from Hematoxylin and Eosin Staining of Liver Images Using Deep Learning Convolutional Neural Network
Previous Article in Journal
Rearfoot, Midfoot, and Forefoot Motion in Naturally Forefoot and Rearfoot Strike Runners during Treadmill Running
Article

Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network

1
Faculty of Science, University of Split, R. Boskovica 33, Split 21 000, Croatia
2
Faculty of Electrical Engineering, Mechanical Engineering and Naval Architecture, University of Split, R. Boskovica 32, Split 21 000, Croatia
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(21), 7817; https://doi.org/10.3390/app10217817
Received: 22 September 2020 / Revised: 23 October 2020 / Accepted: 31 October 2020 / Published: 4 November 2020
(This article belongs to the Section Computing and Artificial Intelligence)
The main goal of any classification or regression task is to obtain a model that will generalize well on new, previously unseen data. Due to the recent rise of deep learning and many state-of-the-art results obtained with deep models, deep learning architectures have become one of the most used model architectures nowadays. To generalize well, a deep model needs to learn the training data well without overfitting. The latter implies a correlation of deep model optimization and regularization with generalization performance. In this work, we explore the effect of the used optimization algorithm and regularization techniques on the final generalization performance of the model with convolutional neural network (CNN) architecture widely used in the field of computer vision. We give a detailed overview of optimization and regularization techniques with a comparative analysis of their performance with three CNNs on the CIFAR-10 and Fashion-MNIST image datasets. View Full-Text
Keywords: neural networks; optimization; regularization; overfitting; model generalization; image processing neural networks; optimization; regularization; overfitting; model generalization; image processing
Show Figures

Figure 1

MDPI and ACS Style

Marin, I.; Kuzmanic Skelin, A.; Grujic, T. Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network. Appl. Sci. 2020, 10, 7817. https://doi.org/10.3390/app10217817

AMA Style

Marin I, Kuzmanic Skelin A, Grujic T. Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network. Applied Sciences. 2020; 10(21):7817. https://doi.org/10.3390/app10217817

Chicago/Turabian Style

Marin, Ivana, Ana Kuzmanic Skelin, and Tamara Grujic. 2020. "Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network" Applied Sciences 10, no. 21: 7817. https://doi.org/10.3390/app10217817

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop