# A Novel Filter-Level Deep Convolutional Neural Network Pruning Method Based on Deep Reinforcement Learning

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- A DDPG-based automatic DCNN pruning method is proposed, which prunes the redundant filters in the network to compress the network.
- The DDPG algorithm optimizes the pruning rate of each layer. A tailored reward function considering the change of both network accuracy and complexity before and after layer-wise pruning is developed for DDPG training.
- A Taylor-expansion-based filter ranking criterion is considered for filter pruning in the FPRL method, which proves to be much more efficient than the widely used minimum-weight-based ranking criterion.
- To illustrate the efficiency of the proposed method in DCNN pruning, extensive experiments have been conducted with several classical DCNNs including VGGNet and ResNet on CIFAR10 and CIFAR100 datasets. The results demonstrate that the FPRL can achieve more than 10× the parameter compression and 3× the FLOPs reduction while maintaining accuracy similar to the initial network.

## 2. Related Work

#### 2.1. Pruning

#### 2.2. Architecture Search

#### 2.3. Different Filter Selection Criteria

## 3. Filter-Level Pruning Based on RL

#### 3.1. DCNN Pruning as a Markov Decision Process

#### 3.1.1. State Representation

#### 3.1.2. Action Representation

#### 3.1.3. State Transition and Reward Function

#### 3.2. Optimization of Pruning Rate with DDPG

#### 3.3. Filter Selection and Pruning

_{1}denotes the remainder of Taylor first-order expansion. The Taylor-expansion-based filter sorting criterion in (13) can be obtained by substituting (12) into (11) and ignoring the remainder.

- Obtain the number of filters to be pruned in the current layer by multiplying the number of filters in the current layer and the pruning rate.
- Sort filters according to the Taylor-expansion-based criterion.
- Prune less important filters as determined by steps 1 and 2.

#### 3.4. The FPRL Algorithm

- Observe the network state. This step obtains the characteristics of the current network layer, the compression rate, and the accuracy of the network as the state information of DDPG.
- Network pruning. This step prunes less important filters given the filter pruning rate.
- Fine-tuning. The pruned network is fine-tuned for several epochs to restore accuracy.
- DDPG update. Use the rewards to update the actor and critic networks in DDPG.

Algorithm 1. The FPRL method | |

1: | Stage 1: Train DDPG agent |

2: | Initialize DDPG model |

3: | episodes$\leftarrow $ 0 |

4: | Whileepisodes$\le $max_episodesdo |

5: | network $\leftarrow $ load original network |

6: | For each layer t in the network do |

7: | Observe network state ${s}_{t}$; |

8: | Select action ${a}_{t}$ according to the current policy in DDPG; |

9: | Determine the number of filters to be pruned given ${a}_{t}$; |

10: | Sort the filters of layer t based on Taylor expansion; |

11: | Prune the less important filters of layer t; |

12: | Fine-tune network; |

13: | Calculate reward ${r}_{t}$ and new state ${s}_{t+1}$; |

14: | Update actor network and critic network in DDPG; |

15: | episodes $\leftarrow $ episodes + 1 |

16: | Stage 2: Prune and retrain network |

17: | network $\leftarrow $ load original network |

18: | Prune network using the optimized pruning rate from well-trained DDPG based on Taylor expansion |

19: | Retrain pruned network |

## 4. Experiments and Analysis

#### 4.1. Datasets and DCNNs

#### 4.2. Experiment Setting

#### 4.2.1. Hyperparameters Setting for VGGNet/ResNet

^{−4}is used. The learning rate is initialized to 0.1 and will be reduced by half every 25 epochs. The performance of VGGNet and ResNet on different datasets are shown in Table 1, including the number of parameters (Params), FLOPs, and accuracy (Acc.).

#### 4.2.2. Hyperparameters Setting for DDPG

#### 4.2.3. Network Fine-Tuning

#### 4.3. Pruning Performance Analysis on Taylor-Expansion-Based Filter Sorting Criterion

#### 4.4. Pruning Performance Comparison

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM
**2017**, 60, 84–90. [Google Scholar] [CrossRef] [Green Version] - Guo, C.; Fan, B.; Zhang, Q.; Xiang, S.; Pan, C. AugFPN: Improving Multi-Scale Feature Learning for Object Detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; IEEE: Seattle, WA, USA, 2020; pp. 12592–12601. [Google Scholar] [CrossRef]
- Lin, M.; Ji, R.; Zhang, Y.; Zhang, B.; Wu, Y.; Tian, Y. Channel Pruning via Automatic Structure Search. arXiv
**2020**, arXiv:2001.08565. [Google Scholar] [CrossRef] - Liu, Z.; Mu, H.; Zhang, X.; Guo, Z.; Yang, X.; Cheng, K.T.; Sun, J. MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning. arXiv
**2019**, arXiv:1903.10258. [Google Scholar] [CrossRef] [Green Version] - Mousavi, S.S.; Schukat, M.; Howley, E. Deep Reinforcement Learning: An Overview. In Proceedings of the SAI Intelligent Systems Conference (IntelliSys) 2016, London, UK, 21–22 September 2016; Bi, Y., Kapoor, S., Bhatia, R., Eds.; Springer International Publishing: Cham, Switzerland, 2018; Volume 16, pp. 426–440. [Google Scholar] [CrossRef] [Green Version]
- Hinton, G.; Vinyals, O.; Dean, J. Distilling the Knowledge in a Neural Network. Comput. Sci.
**2015**, 14, 38–39. [Google Scholar] [CrossRef] - Son, S.; Nah, S.; Lee, K.M. Clustering Convolutional Kernels to Compress Deep Neural Networks. In Proceedings of the Computer Vision—ECCV 2018, Munich, Germany, 8–14 September 2018; Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y., Eds.; Springer International Publishing: Cham, Switzerland, 2018; Volume 11212, pp. 225–240. [Google Scholar] [CrossRef]
- Zhang, X.; Zou, J.; He, K.; Sun, J. Accelerating Very Deep Convolutional Networks for Classification and Detection. IEEE Trans. Pattern Anal. Mach. Intell.
**2015**, 38, 1943–1955. [Google Scholar] [CrossRef] [Green Version] - Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going Deeper with Convolutions. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; IEEE: Boston, MA, USA, 2015; pp. 1–9. [Google Scholar] [CrossRef] [Green Version]
- Wang, J.; Yu, S.; Yuan, Z.; Yue, J.; Yuan, Z.; Liu, R.; Wang, Y.; Yang, H.; Li, X.; Liu, Y. PACA: A Pattern Pruning Algorithm and Channel-Fused High PE Utilization Accelerator for CNNs. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
**2022**, 41, 5043–5056. [Google Scholar] [CrossRef] - Wu, C.; Cui, Y.; Ji, C.; Kuo, T.-W.; Xue, C.J. Pruning Deep Reinforcement Learning for Dual User Experience and Storage Lifetime Improvement on Mobile Devices. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
**2020**, 39, 3993–4005. [Google Scholar] [CrossRef] - Huang, K.; Li, B.; Chen, S.; Claesen, L.; Xi, W.; Chen, J.; Jiang, X.; Liu, Z.; Xiong, D.; Yan, X. Structured Term Pruning for Computational Efficient Neural Networks Inference. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
**2022**. [Google Scholar] [CrossRef] - Han, S.; Mao, H.; Dally, W.J. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. arXiv
**2016**, arXiv:1510.00149. [Google Scholar] [CrossRef] - Yang, T.-J.; Chen, Y.-H.; Sze, V. Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; IEEE: Honolulu, HI, USA, 2017; pp. 6071–6079. [Google Scholar] [CrossRef] [Green Version]
- Li, H.; Kadav, A.; Durdanovic, I.; Samet, H.; Graf, H.P. Pruning Filters for Efficient ConvNets. arXiv
**2017**, arXiv:1608.08710. [Google Scholar] [CrossRef] - He, Y.; Kang, G.; Dong, X.; Fu, Y.; Yang, Y. Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks. arXiv
**2018**, arXiv:1808.06866. [Google Scholar] [CrossRef] - Liu, Z.; Li, J.; Shen, Z.; Huang, G.; Yan, S.; Zhang, C. Learning Efficient Convolutional Networks through Network Slimming. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; IEEE: Venice, Italy, 2017; pp. 2755–2763. [Google Scholar] [CrossRef] [Green Version]
- Huang, K.; Chen, S.; Li, B.; Claesen, L.; Yao, H.; Chen, J.; Jiang, X.; Liu, Z.; Xiong, D. Acceleration-Aware Fine-Grained Channel Pruning for Deep Neural Networks via Residual Gating. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.
**2022**, 41, 1902–1915. [Google Scholar] [CrossRef] - Hu, Y.; Sun, S.; Li, J.; Wang, X.; Gu, Q. A novel channel pruning method for deep neural network compression. arXiv
**2018**, arXiv:1805.11394. [Google Scholar] [CrossRef] - He, Y.; Zhang, X.; Sun, J. Channel Pruning for Accelerating Very Deep Neural Networks. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; IEEE: Venice, Italy, 2017; pp. 1398–1406. [Google Scholar] [CrossRef]
- Li, J.; Louri, A. AdaPrune: An Accelerator-Aware Pruning Technique for Sustainable CNN Accelerators. IEEE Trans. Sustain. Comput.
**2022**, 7, 47–60. [Google Scholar] [CrossRef] - Zoph, B.; Le, Q.V. Neural Architecture Search with Reinforcement Learning. arXiv
**2017**, arXiv:1611.01578. [Google Scholar] [CrossRef] - Sun, Y.; Yen, G.G.; Yi, Z. Evolving Unsupervised Deep Neural Networks for Learning Meaningful Representations. IEEE Trans. Evol. Computat.
**2019**, 23, 89–103. [Google Scholar] [CrossRef] - Sun, Y.; Xue, B.; Zhang, M.; Yen, G.G. Completely Automated CNN Architecture Design Based on Blocks. IEEE Trans. Neural Netw. Learn. Syst.
**2020**, 31, 1242–1254. [Google Scholar] [CrossRef] - Sun, Y.; Xue, B.; Zhang, M.; Yen, G.G. Evolving Deep Convolutional Neural Networks for Image Classification. IEEE Trans. Evol. Comput.
**2020**, 24, 394–407. [Google Scholar] [CrossRef] [Green Version] - Baker, B.; Gupta, O.; Naik, N.; Raskar, R. Designing Neural Network Architectures using Reinforcement Learning. arXiv
**2017**, arXiv:1611.02167. [Google Scholar] [CrossRef] - He, Y.; Lin, J.; Liu, Z.; Wang, H.; Li, L.-J.; Han, S. AMC: AutoML for Model Compression and Acceleration on Mobile Devices. In Proceedings of the IEEE European Conference on Computer Vision, Munich, Germany, 8–14 September 2018; pp. 784–800. [Google Scholar] [CrossRef] [Green Version]
- Hu, H.; Peng, R.; Tai, Y.-W.; Tang, C.-K. Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures. arXiv
**2016**, arXiv:1607.03250. [Google Scholar] [CrossRef] - Molchanov, P.; Tyree, S.; Karras, T.; Aila, T.; Kautz, J. Pruning Convolutional Neural Networks for Resource Efficient Inference. In Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France, 24–26 April 2017; pp. 1–17. [Google Scholar] [CrossRef]
- Silver, D.; Lever, G. Deterministic Policy Gradient Algorithms. In Proceedings of the 31st International Conference on International Conference on Machine Learning (PMLR), Beijing, China, 22–24 June 2014; pp. 387–395. [Google Scholar]
- Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature
**2015**, 518, 529–533. [Google Scholar] [CrossRef] [PubMed] - Lillicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep reinforcement learning. arXiv
**2019**, arXiv:1509.02971. [Google Scholar] [CrossRef] - Krizhevsky, A. Learning Multiple Layers of Features from Tiny Images; Technical Report; University of Toronto: Toronto, ON, Canada, 2009. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv
**2015**. [Google Scholar] [CrossRef] - He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar] [CrossRef]
- Ashok, A.; Rhinehart, N.; Beainy, F.; Kitani, K.M. N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning. arXiv
**2017**, arXiv:1709.06030. [Google Scholar] [CrossRef]

Network | Params | FLOPs | Acc |
---|---|---|---|

CIFAR10/CIFAR100 | CIFAR10/CIFAR100 | CIFAR10/CIFAR100 | |

VGG-16 | 14.05M/14.09M | 299.48M/299.53M | 93.67%/71.10% |

VGG-19 | 19.11M/19.16M | 380.56M/380.60M | 92.58%/69.71% |

ResNet-18 | 10.66M/10.70M | 530.86M530.91M | 95.28%/74.71% |

ResNet-34 | 20.30M/20.34M | 1107.64M/1107.69M | 94.46%/74.51% |

CIFAR10 | |||||
---|---|---|---|---|---|

Network | Acc | $\Delta \mathbf{A}\mathbf{c}\mathbf{c}$ | Params.(×) | FLOPs.(×) | |

VGG-16 | Baseline | 93.67% | - | - | - |

Minimum weight | 93.24% | −0.43% | 11.42× | 3.53× | |

Taylor expansion | 93.29% | −0.38% | 12.11× | 3.60× | |

VGG-19 | Baseline | 92.58% | - | - | - |

Minimum weight | 92.26% | −0.32% | 21.32× | 5.28× | |

Taylor expansion | 92.31% | −0.27% | 22.94× | 5.76× | |

ResNet-18 | Baseline | 95.28% | - | - | - |

Minimum weight | 94.51% | −0.77% | 11.67× | 3.27× | |

Taylor expansion | 94.76% | −0.52% | 11.43× | 3.43× | |

ResNet-34 | Baseline | 94.46% | - | - | - |

Minimum weight | 93.92% | −0.54% | 13.30× | 3.67× | |

Taylor expansion | 94.07% | −0.39% | 14.10× | 3.69× | |

CIFAR100 | |||||

VGG-16 | Baseline | 71.10% | - | - | - |

Minimum weight | 69.12% | −1.98% | 11.35× | 3.59× | |

Taylor expansion | 69.15% | −1.95% | 12.10× | 3.78× | |

VGG-19 | Base | 69.71% | - | - | - |

Minimum weight | 67.76% | −1.95% | 17.27× | 4.10× | |

Taylor expansion | 67.95% | −1.76% | 17.67× | 4.15× | |

ResNet-18 | Baseline | 74.71% | - | - | - |

Minimum weight | 72.44% | −2.27% | 9.20× | 3.37× | |

Taylor expansion | 72.67% | −2.04% | 10.93× | 3.55× | |

ResNet-34 | Baseline | 74.51% | - | - | - |

Minimum weight | 72.11% | −2.30% | 15.34× | 3.95× | |

Taylor expansion | 72.24% | −2.27% | 16.18× | 4.02× |

Architecture | Dataset | Method | Acc (Base) | Acc (Pruned) | $\Delta \mathbf{A}\mathbf{c}\mathbf{c}$ | Params.(×) |
---|---|---|---|---|---|---|

VGG-19 | CIFAR10 | N2N | 91.97% | 91.64% | −0.33% | 20.53× |

FPRL | 92.58% | 92.31% | −0.27% | 22.94× | ||

ResNet-18 | CIFAR10 | N2N | 92.01% | 91.81% | 0.18% | 11.10× |

FPRL | 95.28% | 94.76% | −0.52% | 11.43× | ||

CIFAR100 | N2N | 72.22% | 68.01% | −4.21% | 4.64× | |

FPRL | 74.71% | 72.67% | −2.04% | 10.93× | ||

ResNet-34 | CIFAR10 | N2N | 92.05% | 92.35% | 0.30% | 10.2× |

FPRL | 94.46% | 94.07% | −0.39% | 14.10× | ||

CIFAR100 | N2N | 72.86% | 70.11% | −2.75% | 5.02× | |

FPRL | 74.51% | 72.24% | −2.27% | 16.18× |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Feng, Y.; Huang, C.; Wang, L.; Luo, X.; Li, Q.
A Novel Filter-Level Deep Convolutional Neural Network Pruning Method Based on Deep Reinforcement Learning. *Appl. Sci.* **2022**, *12*, 11414.
https://doi.org/10.3390/app122211414

**AMA Style**

Feng Y, Huang C, Wang L, Luo X, Li Q.
A Novel Filter-Level Deep Convolutional Neural Network Pruning Method Based on Deep Reinforcement Learning. *Applied Sciences*. 2022; 12(22):11414.
https://doi.org/10.3390/app122211414

**Chicago/Turabian Style**

Feng, Yihao, Chao Huang, Long Wang, Xiong Luo, and Qingwen Li.
2022. "A Novel Filter-Level Deep Convolutional Neural Network Pruning Method Based on Deep Reinforcement Learning" *Applied Sciences* 12, no. 22: 11414.
https://doi.org/10.3390/app122211414