Next Article in Journal
Lightweight Compound Scaling Network for Nasopharyngeal Carcinoma Segmentation from MR Images
Previous Article in Journal
Self-Sufficient Sensor Node Embedding 2D Visible Light Positioning through a Solar Cell Module
 
 
Article

Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques

1
Doctoral Program in Computer Science, University of Tsukuba, 1-1-1 Tennodai, Tsukuba 305-8573, Ibaraki, Japan
2
JST, SPRING, 4-1-8 Honcho, Kawaguchi 332-0012, Saitama, Japan
3
Faculty of Engineering, Information and Systems, University of Tsukuba, 1-1-1 Tennodai, Tsukuba 305-8573, Ibaraki, Japan
4
JST, PRESTO, 4-1-8 Honcho, Kawaguchi 332-0012, Saitama, Japan
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(15), 5874; https://doi.org/10.3390/s22155874
Received: 15 July 2022 / Revised: 2 August 2022 / Accepted: 3 August 2022 / Published: 5 August 2022
(This article belongs to the Section Physical Sensors)
Network pruning techniques have been widely used for compressing computational and memory intensive deep learning models through removing redundant components of the model. According to the pruning granularity, network pruning can be categorized into structured and unstructured methods. The structured pruning removes the large components in a model such as channels or layers, which might reduce the accuracy. The unstructured pruning directly removes mainly the parameters in a model as well as the redundant channels or layers, which might result in an inadequate pruning. To address the limitations of the pruning methods, this paper proposes a heuristic method for minimizing model size. This paper implements an algorithm to combine both the structured and the unstructured pruning methods while maintaining the target accuracy that is configured by its application. We use network slimming for the structured pruning method and deep compression for the unstructured one. Our method achieves a higher compression ratio than the case when the individual pruning method is applied. To show the effectiveness of our proposed method, this paper evaluates our proposed method with actual state-of-the-art CNN models of VGGNet, ResNet and DenseNet under the CIFAR-10 dataset. This paper discusses the performance of the proposed method with the cases of individual usage of the structured and unstructured pruning methods and then proves that our method achieves better performance with higher compression ratio. In the best case of the VGGNet, our method results in a 13× reduction ratio in the model size, and also gives a 15× reduction ratio regarding the pruning time compared with the brute-force search method. View Full-Text
Keywords: deep learning; convolutional neural network; network pruning; object detection deep learning; convolutional neural network; network pruning; object detection
Show Figures

Figure 1

MDPI and ACS Style

Tian, D.; Yamagiwa, S.; Wada, K. Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques. Sensors 2022, 22, 5874. https://doi.org/10.3390/s22155874

AMA Style

Tian D, Yamagiwa S, Wada K. Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques. Sensors. 2022; 22(15):5874. https://doi.org/10.3390/s22155874

Chicago/Turabian Style

Tian, Danhe, Shinichi Yamagiwa, and Koichi Wada. 2022. "Heuristic Method for Minimizing Model Size of CNN by Combining Multiple Pruning Techniques" Sensors 22, no. 15: 5874. https://doi.org/10.3390/s22155874

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop