Layer-Wise Compressive Training for Convolutional Neural Networks
AbstractConvolutional Neural Networks (CNNs) are brain-inspired computational models designed to recognize patterns. Recent advances demonstrate that CNNs are able to achieve, and often exceed, human capabilities in many application domains. Made of several millions of parameters, even the simplest CNN shows large model size. This characteristic is a serious concern for the deployment on resource-constrained embedded-systems, where compression stages are needed to meet the stringent hardware constraints. In this paper, we introduce a novel accuracy-driven compressive training algorithm. It consists of a two-stage flow: first, layers are sorted by means of heuristic rules according to their significance; second, a modified stochastic gradient descent optimization is applied on less significant layers such that their representation is collapsed into a constrained subspace. Experimental results demonstrate that our approach achieves remarkable compression rates with low accuracy loss (<1%). View Full-Text
Share & Cite This Article
Grimaldi, M.; Tenace, V.; Calimera, A. Layer-Wise Compressive Training for Convolutional Neural Networks. Future Internet 2019, 11, 7.
Grimaldi M, Tenace V, Calimera A. Layer-Wise Compressive Training for Convolutional Neural Networks. Future Internet. 2019; 11(1):7.Chicago/Turabian Style
Grimaldi, Matteo; Tenace, Valerio; Calimera, Andrea. 2019. "Layer-Wise Compressive Training for Convolutional Neural Networks." Future Internet 11, no. 1: 7.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.