Next Article in Journal
Common Switch Fault Diagnosis for Two-Stage DC-DC Converters Used in Energy Harvesting Applications
Previous Article in Journal
Correction: Casula, G.A. A Design Rule to Reduce the Human Body Effect on Wearable PIFA Antennas. Electronics 2019, 8, 244
Article Menu
Issue 3 (March) cover image

Export Article

Open AccessFeature PaperReview

A State-of-the-Art Survey on Deep Learning Theory and Architectures

1
Department of Electrical and Computer Engineering, University of Dayton, Dayton, OH 45469, USA
2
Department of Earth and Atmospheric Sciences, Saint Louis University, Saint Louis, MO 63108, USA
3
Comcast Labs, Washington, DC 20005, USA
4
Lawrence Livermore National Laboratory (LLNL), Livermore, CA 94550, USA
*
Author to whom correspondence should be addressed.
Electronics 2019, 8(3), 292; https://doi.org/10.3390/electronics8030292
Received: 17 January 2019 / Accepted: 31 January 2019 / Published: 5 March 2019
(This article belongs to the Section Computer Science & Engineering)

Abstract

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models. View Full-Text
Keywords: deep learning; convolutional neural network (CNN); recurrent neural network (RNN); auto-encoder (AE); restricted Boltzmann machine (RBM); deep belief network (DBN); generative adversarial network (GAN); deep reinforcement learning (DRL); transfer learning deep learning; convolutional neural network (CNN); recurrent neural network (RNN); auto-encoder (AE); restricted Boltzmann machine (RBM); deep belief network (DBN); generative adversarial network (GAN); deep reinforcement learning (DRL); transfer learning
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Alom, M.Z.; Taha, T.M.; Yakopcic, C.; Westberg, S.; Sidike, P.; Nasrin, M.S.; Hasan, M.; Van Essen, B.C.; Awwal, A.A.S.; Asari, V.K. A State-of-the-Art Survey on Deep Learning Theory and Architectures. Electronics 2019, 8, 292.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Electronics EISSN 2079-9292 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top