Next Article in Journal
Vector Generation of Quantum Contextual Sets in Even Dimensional Hilbert Spaces
Previous Article in Journal
Rolling Element Bearing Fault Diagnosis by Combining Adaptive Local Iterative Filtering, Modified Fuzzy Entropy and Support Vector Machine
Article Menu

Export Article

Open AccessArticle
Entropy 2018, 20(12), 927; https://doi.org/10.3390/e20120927

DBN Structure Design Algorithm for Different Datasets Based on Information Entropy and Reconstruction Error

National Digital Switching System Engineering and Technological Research Center (NDSC), Zhengzhou 450000, Henan, China
*
Author to whom correspondence should be addressed.
Received: 23 October 2018 / Revised: 1 December 2018 / Accepted: 2 December 2018 / Published: 4 December 2018
Full-Text   |   PDF [1935 KB, uploaded 4 December 2018]   |  

Abstract

Deep belief networks (DBNs) of deep learning technology have been successfully used in many fields. However, the structure of a DBN is difficult to design for different datasets. Hence, a DBN structure design algorithm based on information entropy and reconstruction error is proposed. Unlike previous algorithms, we innovatively combine network depth and node number and optimizes them simultaneously. First, the mathematical model of the structural design problem is established, and the boundary constraint for node number based on information entropy is derived by introducing the idea of information compression. Moreover, the optimization objective of the network performance based on reconstruction error is proposed by deriving the fact that network energy is proportional to reconstruction error. Finally, the improved simulated annealing (ISA) algorithm is used to adjust the DBN network layers and nodes simultaneously. Experiments were carried out on three public datasets (MNIST, Cifar-10 and Cifar-100). The results show that the proposed algorithm can design its proper structure to different datasets, yielding a trained DBN which has the lowest reconstruction error and prediction error rate. The proposed algorithm is shown to have the best performance compared with other algorithms and can be used to assist the setting of DBN structural parameters for different datasets. View Full-Text
Keywords: deep learning; DBN; artificial intelligence; structure design; information entropy; reconstruction error; improved simulated annealing algorithm deep learning; DBN; artificial intelligence; structure design; information entropy; reconstruction error; improved simulated annealing algorithm
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Jiang, J.; Zhang, J.; Zhang, L.; Ran, X.; Jiang, J.; Wu, Y. DBN Structure Design Algorithm for Different Datasets Based on Information Entropy and Reconstruction Error. Entropy 2018, 20, 927.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top