Next Article in Journal
Performance Analysis of a Hybrid Electric Vehicle with Multiple Converter Configuration
Next Article in Special Issue
Detecting Green Mold Pathogens on Lemons Using Hyperspectral Images
Previous Article in Journal
Two-Dimensional Impact Modelling with Three Degrees of Freedom and Its Application in the Dynamics of Planing Hulls
Previous Article in Special Issue
Image Completion with Hybrid Interpolation in Tensor Representation
Open AccessArticle

An Effective Optimization Method for Machine Learning Based on ADAM

1
Division of Creative Integrated General Studies, Daegu University College, Kyungsan 38453, Korea
2
Department of Mathematics, College of Natural Sciences, Chungnam National University, Daejeon 34134, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(3), 1073; https://doi.org/10.3390/app10031073
Received: 11 December 2019 / Revised: 18 January 2020 / Accepted: 25 January 2020 / Published: 5 February 2020
(This article belongs to the Special Issue Intelligent Processing on Image and Optical Information)
A machine is taught by finding the minimum value of the cost function which is induced by learning data. Unfortunately, as the amount of learning increases, the non-liner activation function in the artificial neural network (ANN), the complexity of the artificial intelligence structures, and the cost function’s non-convex complexity all increase. We know that a non-convex function has local minimums, and that the first derivative of the cost function is zero at a local minimum. Therefore, the methods based on a gradient descent optimization do not undergo further change when they fall to a local minimum because they are based on the first derivative of the cost function. This paper introduces a novel optimization method to make machine learning more efficient. In other words, we construct an effective optimization method for non-convex cost function. The proposed method solves the problem of falling into a local minimum by adding the cost function in the parameter update rule of the ADAM method. We prove the convergence of the sequences generated from the proposed method and the superiority of the proposed method by numerical comparison with gradient descent (GD, ADAM, and AdaMax). View Full-Text
Keywords: numerical optimization; ADAM; machine learning; stochastic gradient methods numerical optimization; ADAM; machine learning; stochastic gradient methods
Show Figures

Figure 1

MDPI and ACS Style

Yi, D.; Ahn, J.; Ji, S. An Effective Optimization Method for Machine Learning Based on ADAM. Appl. Sci. 2020, 10, 1073.

AMA Style

Yi D, Ahn J, Ji S. An Effective Optimization Method for Machine Learning Based on ADAM. Applied Sciences. 2020; 10(3):1073.

Chicago/Turabian Style

Yi, Dokkyun; Ahn, Jaehyun; Ji, Sangmin. 2020. "An Effective Optimization Method for Machine Learning Based on ADAM" Appl. Sci. 10, no. 3: 1073.

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop