A Multi-Core Object Detection Coprocessor for Multi-Scale/Type Classification Applicable to IoT Devices
Abstract
:1. Introduction
1.1. Related Work
1.2. Contribution
1.3. Structure
2. VLSI-Oriented Hardware Algorithm for Feature Extraction
2.1. Block-Level Feature Extraction and Normalization Circuitry
2.2. Scalable-Block-Size Based Sliding-Detection-Window (SBSSDW) Mechanism
2.3. Multi-Scale Detection with Multi-Core Implementation
3. Experiment Results and Discussion
3.1. Hardware Implementation and Performance Analysis
3.2. Discussion and Comparison
4. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Redmon, J.; Farhadi, A. Yolov3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Aziz, L.; Salam, S.B.H.; Ayub, S. Exploring deep learning-based architecture, strategies, applications and current trends in generic object detection: A comprehensive review. IEEE Access 2020, 8, 170461–170495. [Google Scholar] [CrossRef]
- Haque, M.F.; Lim, H.; Kang, D. Object Detection Based on VGG with ResNet Network. In Proceedings of the 2019 International Conference on Electronics, Information, and Communication (ICEIC), Auckland, New Zealand, 22–25 January 2019; pp. 1–3. [Google Scholar]
- Ren, J.; Chen, X.; Liu, J.; Sun, W.; Pang, J.; Yan, Q.; Tai, Y.; Xu, L. Accurate single stage detector using recurrent rolling convolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 5420–5428. [Google Scholar]
- Nayebi, A.; Bear, D.; Kubilius, J.; Kar, K.; Ganguli, S.; Sussillo, D.; DiCarlo, J.; Yamins, D. Task-driven convolutional recurrent models of the visual system. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18), Montréal, QC, Canada, 3–8 December 2018; pp. 5290–5301. [Google Scholar]
- Fukagai, T.; Maeda, K.; Tanabe, S.; Shirahata, K.; Tomita, Y.; Ike, A.; Nakagawa, A. Speed-Up of Object Detection Neural Network with GPU. In Proceedings of the 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 7–10 October 2018; pp. 301–305. [Google Scholar]
- Nakahara, H. A Lightweight YOLOv2. In Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA), Monterey, CA, USA, 25–27 February 2018. [Google Scholar]
- Arunmozhi, A.; Park, J. Comparison of HOG, LBP and Haar-Like Features for On-Road Vehicle Detection. In Proceedings of the 2018 IEEE International Conference on Electro/Information Technology (EIT), Rochester, MI, USA, 3–5 May 2018. [Google Scholar]
- Shobha, B.S.; Deepu, R. A Review on Video Based Vehicle Detection, Recognition and Tracking. In Proceedings of the 2018 3rd International Conference on Computational Systems and Information Technology for Sustainable Solutions (CSITSS), Bengaluru, India, 20–22 December 2018; pp. 183–186. [Google Scholar]
- Kawamoto, R.; Taichi, M.; Kabuto, M.; Watanabe, D.; Izumi, S.; Yoshimoto, M.; Kawaguchi, H. A 1.15-TOPS 6.57-TOPS/W DNN Processor for Multi-Scale Object Detection. In Proceedings of the 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genova, Italy, 31 August–2 September 2020; pp. 203–207. [Google Scholar]
- Kawamoto, R.; Taichi, M.; Kabuto, M.; Watanabe, D.; Izumi, S.; Yoshimoto, M.; Kawaguchi, H.; Matsukawa, G.; Goto, T.; Kojima, M. A 1.15-TOPS 6.57-TOPS/W Neural Network Processor for Multi-Scale Object Detection With Reduced Convolutional Operations. IEEE J. Sel. Top. Signal Process. 2020, 14, 634–645. [Google Scholar]
- Rashid, M.M.; Kamruzzaman, J.; Wasimi, S. Sliding Window-based Regularly Frequent Patterns Mining Over Sensor Data Streams. In Proceedings of the 2019 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), Melbourne, Australia, 9–11 December 2019; pp. 1–6. [Google Scholar]
- Ghaffari, S.; Soleimani, P.; Li, K.F.; Capson, D.W. Analysis and Comparison of FPGA-Based Histogram of Oriented Gradients Implementations. IEEE Access 2020, 8, 79920–79934. [Google Scholar] [CrossRef]
- Song, P.; Zhu, Y.; Zhang, Z.; Zhang, J. Subsampling-based HOG for Multi-scale real-time Pedestrian Detection. In Proceedings of the 2019 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), Bangkok, Thailand, 18–20 November 2019; pp. 24–29. [Google Scholar]
- An, F.; Xu, P.; Xiao, Z.; Wang, C. FPGA-based object detection processor with HOG feature and SVM classifier. In Proceedings of the 2019 32nd IEEE International System-on-Chip Conference (SOCC), Singapore, 3–6 September 2019; pp. 187–190. [Google Scholar]
- Ma, X.; Najjar, W.A.; Roy-Chowdhury, A.K. Evaluation and Acceleration of High-Throughput Fixed-Point Object Detection on FPGAs. IEEE Trans. Circuits Syst. Video Technol. 2015, 25, 1051–1062. [Google Scholar]
- Wang, M.; Zhang, Z. FPGA implementation of HOG based multi-scale pedestrian detection. In Proceedings of the 2018 IEEE International Conference on Applied System Invention (ICASI), Chiba, Japan, 13–17 April 2018; pp. 1099–1102. [Google Scholar]
- Bilal, M.; Khan, A.; Khan, M.U.K.; Kyung, C. A Low-Complexity Pedestrian Detection Framework for Smart Video Surveillance Systems, in IEEE Transactions on Circuits and Systems for Video Technology. IEEE Trans. Circuits Syst. Video Technol. 2017, 27, 2260–2273. [Google Scholar] [CrossRef]
- An, F.; Zhang, X.; Luo, A.; Chen, L.; Mattausch, H.J. A Hardware Architecture for Cell-Based Feature-Extraction and Classification Using Dual-Feature Space. IEEE Trans. Circuits Syst. Video Technol. 2018, 28, 3086–3098. [Google Scholar] [CrossRef]
- Wang, L.; Lin, Y.; Jiang, W. Implementation of Fast and High-precision Division Algorithm on FPGA. Comput. Eng. 2011, 37, 240–242. [Google Scholar]
- Stratix IV GX FPGA DevelopmLent Kit. March 2014. Available online: https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_sivgx_fpga_dev_kit.pdf (accessed on 21 October 2020).
- STC-MC_MB83PCL_Spec_Manual. July 2017. Available online: https://aegis-elec.com/mwdownloads/download/link/id/2289/ (accessed on 21 October 2020).
- Geiger, A.; Lenz, P.; Urtasun, R. Are we ready for autonomous driving? The kitti vision benchmark suite. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 16–21 June 2012. [Google Scholar]
Sobel Filter | HOG Feature | Normalization | SVM | Total | |
---|---|---|---|---|---|
Combinational ALUTs | 442 | 549 | 4678 | 3017 | 8686 |
Dedicated Logic Registers | 616 | 587 | 1694 | 5319 | 8216 |
Block Memory Bits | 18,432 | 49,152 | 39,616 | 174,080 | 281,280 |
Max Working Frequency | 356.63 MHz | 323.31 MHz | 162.81 MHz | 136.43 MHz | |
Actual Working Frequency | 65 MHz | 65 MHz | 65 MHz | 130 MHz | |
Power Consumption | 2.84 mW | 3.51 mW | 9.47 mW | 65.17 mW | 80.98 mW |
[16] | [17] | [18] | [15] | This Study | |
---|---|---|---|---|---|
Combinational ALUTs | 85,837 | 87,306 | 6551 | 7652 | 8686 |
Dedicated Logic Registers | 406,978 | 77,726 | 4375 | 4503 | 8216 |
Block Memory Bits (Mbit) | 2.55 | 2.97 | NA | 0.13 | 0.28 |
Max Working Frequency (MHz) | 135 | 108.19 | 50 | * NA | 130 |
Frame rate (fps) | 68.18 | 60 | 25 | 60 | 60 |
Resolution | 640 × 480 | 640 × 480 | 640 × 480 | 1920 × 1080 | 1024 × 768 |
FPGA platform | Virtex-6 | Cyclone IV | Stratix IV |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xu, P.; Xiao, Z.; Wang, X.; Chen, L.; Wang, C.; An, F. A Multi-Core Object Detection Coprocessor for Multi-Scale/Type Classification Applicable to IoT Devices. Sensors 2020, 20, 6239. https://doi.org/10.3390/s20216239
Xu P, Xiao Z, Wang X, Chen L, Wang C, An F. A Multi-Core Object Detection Coprocessor for Multi-Scale/Type Classification Applicable to IoT Devices. Sensors. 2020; 20(21):6239. https://doi.org/10.3390/s20216239
Chicago/Turabian StyleXu, Peng, Zhihua Xiao, Xianglong Wang, Lei Chen, Chao Wang, and Fengwei An. 2020. "A Multi-Core Object Detection Coprocessor for Multi-Scale/Type Classification Applicable to IoT Devices" Sensors 20, no. 21: 6239. https://doi.org/10.3390/s20216239
APA StyleXu, P., Xiao, Z., Wang, X., Chen, L., Wang, C., & An, F. (2020). A Multi-Core Object Detection Coprocessor for Multi-Scale/Type Classification Applicable to IoT Devices. Sensors, 20(21), 6239. https://doi.org/10.3390/s20216239