Compressed Learning of Deep Neural Networks for OpenCL-Capable Embedded Systems
Computer Science, Hanyang University ERICA, Ansan 15588, Korea
Computer Science and Engineering, Hanyang University ERICA, Ansan 15588, Korea
Author to whom correspondence should be addressed.
Received: 21 March 2019 / Revised: 14 April 2019 / Accepted: 16 April 2019 / Published: 23 April 2019
PDF [695 KB, uploaded 23 April 2019]
Deep neural networks (DNNs) have been quite successful in solving many complex learning problems. However, DNNs tend to have a large number of learning parameters, leading to a large memory and computation requirement. In this paper, we propose a model compression framework for efficient training and inference of deep neural networks on embedded systems. Our framework provides data structures and kernels for OpenCL-based parallel forward and backward computation in a compressed form. In particular, our method learns sparse representations of parameters using
-based sparse coding while training, storing them in compressed sparse matrices. Unlike the previous works, our method does not require a pre-trained model as an input and therefore can be more versatile for different application environments. Even though the use of
-based sparse coding for model compression is not new, we show that it can be far more effective than previously reported when we use proximal point algorithms and the technique of debiasing. Our experiments show that our method can produce minimal learning models suitable for small embedded devices.
This is an open access article distributed under the Creative Commons Attribution License
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
Share & Cite This Article
MDPI and ACS Style
Lee, S.; Lee, J. Compressed Learning of Deep Neural Networks for OpenCL-Capable Embedded Systems. Appl. Sci. 2019, 9, 1669.
Lee S, Lee J. Compressed Learning of Deep Neural Networks for OpenCL-Capable Embedded Systems. Applied Sciences. 2019; 9(8):1669.
Lee, Sangkyun; Lee, Jeonghyun. 2019. "Compressed Learning of Deep Neural Networks for OpenCL-Capable Embedded Systems." Appl. Sci. 9, no. 8: 1669.
Show more citation formats
Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
[Return to top]
For more information on the journal statistics, click here
Multiple requests from the same IP address are counted as one view.