Early Bruise Detection in Apple Based on an Improved Faster RCNN Model
Abstract
1. Introduction
2. Materials and Methods
2.1. Experimental Materials
2.2. Imaging Systems
2.3. Dataset
2.4. Model and Evaluation
2.5. Improved Faster RCNN
2.5.1. FPN
2.5.2. NAM Normalization-Based Attention Module
2.5.3. Complete-IoU
3. Results and Discussion
3.1. Validation Based on Hybrid Datasets
3.2. Comparison of Model Results
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Du, Z.J.; Zeng, X.Q.; Li, X.X.; Ding, X.M.; Cao, J.K.; Jiang, W.B. Recent advances in imaging techniques for bruise detection in fruits and vegetables. Trends Food Sci. Technol. 2020, 99, 133–141. [Google Scholar] [CrossRef]
- Milajerdi, A.; Ebrahimi-Daryani, N.; Dieleman, L.A.; Larijani, B.; Esmaillzadeh, A. Association of Dietary Fiber, Fruit, and Vegetable Consumption with Risk of Inflammatory Bowel Disease: A Systematic Review and Meta-Analysis. Adv. Nutr. 2021, 12, 735–743. [Google Scholar] [CrossRef]
- Opara, U.L.; Pathare, P.B. Bruise damage measurement and analysis of fresh horticultural produce-A review. Postharvest Biol. Technol. 2014, 91, 9–24. [Google Scholar] [CrossRef]
- Komarnicki, P.; Stopa, R.; Szyjewicz, D.; Kuta, A.; Klimza, T. Influence of Contact Surface Type on the Mechanical Damages of Apples Under Impact Loads. Food Bioprocess Technol. 2017, 10, 1479–1494. [Google Scholar] [CrossRef]
- Wani, N.A.; Mishra, U. An integrated circular economic model with controllable carbon emission and deterioration from an apple orchard. J. Clean. Prod. 2022, 374, 133962. [Google Scholar] [CrossRef]
- Springael, J.; Paternoster, A.; Braet, J. Reducing postharvest losses of apples: Optimal transport routing (while minimizing total costs). Comput. Electron. Agric. 2018, 146, 136–144. [Google Scholar] [CrossRef]
- Arango, J.D.; Staar, B.; Baig, A.M.; Freitag, M. Quality control of apples by means of convolutional neural networks—Comparison of bruise detection by color images and near-infrared images. Procedia CIRP 2021, 99, 290–294. [Google Scholar] [CrossRef]
- Leemans, V.; Destain, M.F. A real-time grading method of apples based on features extracted from defects. J. Food Eng. 2004, 61, 83–89. [Google Scholar] [CrossRef]
- Moscetti, R.; Haff, R.P.; Monarca, D.; Cecchini, M.; Massantini, R. Near-infrared spectroscopy for detection of hailstorm damage on olive fruit. Postharvest Biol. Technol. 2016, 120, 204–212. [Google Scholar] [CrossRef]
- Defraeye, T.; Lehmann, V.; Gross, D.; Holat, C.; Herremans, E.; Verboven, P.; Verlinden, B.E.; Nicolai, B.M. Application of MRI for tissue characterisation of ‘Braeburn’ apple. Postharvest Biol. Technol. 2013, 75, 96–105. [Google Scholar] [CrossRef]
- Diels, E.; van Dael, M.; Keresztes, J.; Vanmaercke, S.; Verboven, P.; Nicolai, B.; Saeys, W.; Ramon, H.; Smeets, B. Assessment of bruise volumes in apples using X-ray computed tomography. Postharvest Biol. Technol. 2017, 128, 24–32. [Google Scholar] [CrossRef]
- Mahanti, N.K.; Pandiselvam, R.; Kothakota, A.; Ishwarya, S.P.; Chakraborty, S.K.; Kumar, M.; Cozzolino, D. Emerging non-destructive imaging techniques for fruit damage detection: Image processing and analysis. Trends Food Sci. Technol. 2021, 120, 418–438. [Google Scholar] [CrossRef]
- Azadbakht, M.; Torshizi, M.V.; Mahmoodi, M.J. The use of CT scan imaging technique to determine pear bruise level due to external loads. Food Sci. Nutr. 2019, 7, 273–280. [Google Scholar] [CrossRef] [PubMed]
- Baranowski, P.; Mazurek, W.; Pastuszka-Wozniak, J. Supervised classification of bruised apples with respect to the time after bruising on the basis of hyperspectral imaging data. Postharvest Biol. Technol. 2013, 86, 249–258. [Google Scholar] [CrossRef]
- Pandiselvam, R.; Mayookha, V.P.; Kothakota, A.; Ramesh, S.V.; Thirumdas, R.; Juvvi, P. Biospeckle laser technique-A novel non-destructive approach for food quality and safety detection. Trends Food Sci. Technol. 2020, 97, 1–13. [Google Scholar] [CrossRef]
- Liu, N.L.; Chen, X.; Liu, Y.; Ding, C.Z.; Tan, Z.J. Deep learning approach for early detection of sub-surface bruises in fruits using single snapshot spatial frequency domain imaging. J. Food Meas. Charact. 2022, 16, 3888–3896. [Google Scholar] [CrossRef]
- Unay, D. Deep learning based automatic grading of bi-colored apples using multispectral images. Multimed. Tools Appl. 2022, 81, 38237–38252. [Google Scholar] [CrossRef]
- Hekim, M.; Comert, O.; Adem, K. A hybrid model based on the convolutional neural network model and artificial bee colony or particle swarm optimization-based iterative thresholding for detection of bruised apples. Turk. J. Electr. Eng. Comput. Sci. 2020, 28, 61–79. [Google Scholar] [CrossRef]
- Zhu, X.L.; Li, G.H. Rapid detection and visualization of slight bruise on apples using hyperspectral imaging. Int. J. Food Prop. 2019, 22, 1709–1719. [Google Scholar] [CrossRef]
- Luo, W.; Zhang, H.L.; Liu, X.M. Hyperspectral/multispectral reflectance imaging combining with watershed segmentation algorithm for detection of early bruises on apples with different peel colors. Food Anal. Methods 2019, 12, 1218–1228. [Google Scholar] [CrossRef]
- Pan, X.Y.; Sun, L.J.; Li, Y.S.; Che, W.K.; Ji, Y.M.; Li, J.L.; Li, J.; Xie, X.; Xu, Y.T. Non-destructive classification of apple bruising time based on visible and near-infrared hyperspectral imaging. J. Sci. Food Agric. 2019, 99, 1709–1718. [Google Scholar] [CrossRef] [PubMed]
- Ferrari, C.; Foca, G.; Calvini, R.; Ulrici, A. Fast exploration and classification of large hyperspectral image datasets for early bruise detection on apples. Chemom. Intell. Lab. Syst. 2015, 146, 108–119. [Google Scholar] [CrossRef]
- Gai, Z.D.; Sun, L.J.; Bai, H.Y.; Li, X.X.; Wang, J.Y.; Bai, S.N. Convolutional neural network for apple bruise detection based on hyperspectral. Spectrochim. Acta Part A Mol. Biomol. Spectrosc. 2022, 279, 121432. [Google Scholar] [CrossRef] [PubMed]
- Tian, X.; Liu, X.F.; He, X.; Zhang, C.; Li, J.B.; Huang, W. Detection of early bruises on apples using hyperspectral reflectance imaging coupled with optimal wavelengths selection and improved watershed segmentation algorithm. J. Sci. Food Agric. 2023, 103, 6689–6705. [Google Scholar] [CrossRef] [PubMed]
- Pang, Q.; Huang, W.Q.; Fan, S.X.; Zhou, Q.; Wang, Z.L.; Tian, X. Detection of early bruises on apples using hyperspectral imaging combining with YOLOv3 deep learning algorithm. J. Process Eng. 2022, 45, 13952. [Google Scholar] [CrossRef]
- Fan, S.X.; Liang, X.T.; Huang, W.Q.; Zhang, V.J.; Pang, Q.; He, X.; Li, L.J.; Zhang, C. Real-time defects detection for apple sorting using NIR cameras with pruning-based YOLOV4 network. Comput. Electron. Agric. 2022, 193, 106715. [Google Scholar] [CrossRef]
- Yuan, Y.H.; Yang, Z.R.; Liu, H.B.; Wang, H.B.; Li, J.H.; Zhao, L.L. Detection of early bruise in apple using near-infrared camera imaging technology combined with deep learning. Infrared Phys. Technol. 2022, 127, 104442. [Google Scholar] [CrossRef]
- Yang, Z.R.; Yuan, Y.H.; Zheng, J.H.; Wang, H.B.; Li, J.H.; Zhao, L.L. Early apply bruise recognition based on near-infrared imaging and grayscale gradient images. J. Food Meas. Charact. 2023, 17, 2841–2849. [Google Scholar] [CrossRef]
- Keresztes, J.C.; Diels, E.; Goodarzi, M.; Nguyen-Do-Trong, N.; Goos, P.; Nicolai, B.; Saeys, W. Glare based apple sorting and iterative algorithm for bruise region detection using shortwave infrared hyperspectral imaging. Postharvest Biol. Technol. 2017, 130, 103–115. [Google Scholar] [CrossRef]
- Siedliska, A.; Baranowski, P.; Mazurek, W. Classification models of bruise and cultivar detection on the basis of hyperspectral imaging data. Comput. Electron. Agric. 2014, 106, 66–74. [Google Scholar] [CrossRef]
- Li, X.; Liu, Y.D.; Yan, Y.J.; Wang, G.T. Detection of early bruises in honey peaches using shortwave infrared hyperspectral imaging. Spectroscopy 2022, 37, 33–41,48. [Google Scholar] [CrossRef]
- Han, Z.Y.; Li, B.; Wang, Q.; Yang, A.K.; Liu, Y.D. Detection storage time of mild bruise’s loquats using hyperspectral imaging. J. Spectrosc. 2022, 2022, 9989002. [Google Scholar] [CrossRef]
- Zhou, X.; Ampatzidis, Y.; Lee, W.S.; Zhou, C.L.; Agehara, S.; Schueller, J.K. Deep learning-based postharvest strawberry bruise detection under UV and incandescent light. J. Chemom. 2022, 202, 107389. [Google Scholar]
- Lopez-Maestresalas, A.; Keresztes, J.C.; Goodarzi, M.; Arazuri, S.; Jaren, C.; Saeys, W. Non-destructive detection of blackspot in potatoes by Vis-NIR and SWIR hyperspectral imaging. Food Contral 2016, 70, 229–241. [Google Scholar] [CrossRef]
- Zhu, Q.B.; Guan, J.Y.; Huang, M.; Lu, R.F.; Mendoza, F. Predicting bruise susceptibility of ‘Golden Delicious’ apples using hyperspectral scattering technique. Postharvest Biol. Technol. 2016, 114, 86–94. [Google Scholar] [CrossRef]
- Xi, R.; Hou, J.; Lou, W. Potato Bud detection with improved Faster R-CNN. Trans. ASABE 2020, 63, 557–569. [Google Scholar] [CrossRef]
- Li, Y.C.; Zhou, S.L.; Chen, H. Attention-based fusion factor in FPN for object detection. Appl. Intell. 2022, 52, 15547–15556. [Google Scholar] [CrossRef]
- Wu, A.; Zhu, J.H.; Ren, T.Y. Detection of apple defect using laser-induced light backscattering imaging and convolutional neural network. Comput. Electr. Eng. 2020, 81, 106454. [Google Scholar] [CrossRef]
- Liu, Y.C.; Shao, Z.R.; Teng, Y.Y. NAM: Normalization-based Attention Module. arXiv 2021, arXiv:2111.12419. [Google Scholar]
- Zhang, T.W.; Zhang, X.L. Squeeze-and-Excitation Laplacian Pyramid Network with Dual-Polarization Feature Fusion for Ship Classification in SAR Images. IEEE Geosci. Remote Sens. Lett. 2022, 19, 4019905. [Google Scholar] [CrossRef]
- Wang, W.; Tan, X.A.; Zhang, P.; Wang, X. A CBAM Based Multiscale Transformer Fusion Approach for Remote Sensing Image Change Detection. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2022, 15, 6817–6825. [Google Scholar] [CrossRef]
- Zheng, Z.H.; Wang, P.; Ren, D.W.; Liu, W.; Ye, R.G.; Hu, Q.H.; Zuo, W.M. Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation. IEEE Trans. Cybern. 2021, 52, 8574–8586. [Google Scholar] [CrossRef] [PubMed]
- Anzidei, M.; Di Martino, M.; Sacconi, B.; Saba, L.; Boni, F.; Zaccagna, F.; Geiger, D.; Kirchin, M.A.; Napoli, A.; Bezzi, M. Evaluation of image quality, radiation dose and diagnostic performance of dual-energy CT datasets in patients with hepatocellular carcinoma. Clin. Radiol. 2015, 70, 966–973. [Google Scholar] [CrossRef] [PubMed]
- Zhang, X.; Zhao, C.; Luo, H.Z.; Zhao, W.Q.; Zhong, S.; Tang, L.; Peng, J.Y.; Fan, J.P. Automatic learning for object detection. Neurocomputing 2022, 484, 260–272. [Google Scholar] [CrossRef]
- Yao, J.; Qi, J.M.; Zhang, J.; Shao, H.M.; Yang, J.; Li, X. A Real-Time Detection Algorithm for Kiwifruit Defects Based on YOLOv5. Electronics 2021, 10, 1711. [Google Scholar] [CrossRef]
Category | Bruise | Calyx | Stem | ||
---|---|---|---|---|---|
Small | Medium | Large | |||
number | 298 | 285 | 244 | 248 | 270 |
Model | Test Set Type | APbruise | F1 |
---|---|---|---|
Model 1 | dataset A 1 | 77.89% | 0.714 |
dataset B 2 | 68.38% | 0.667 | |
dataset A + B | 72.18% | 0.689 | |
Model 2 | dataset A | 81.04% | 0.750 |
dataset B | 75.34% | 0.702 | |
dataset A + B | 78.28% | 0.729 | |
Model 3 | dataset A | 86.75% | 0.786 |
dataset B | 80.63% | 0.741 | |
dataset A + B | 83.28% | 0.762 |
Network | APbruise | APcalyx | APstem | mAP | F1 | ||
---|---|---|---|---|---|---|---|
Small | Medium | Large | |||||
Faster RCNN [36] | 90.6% | 88.5% | 91.2% | 93.2% | 93.1% | 91.3% | 0.833 |
YOLOv4P [26] | 88.7% | 86.3% | 86.4% | 90.9% | 89.9% | 88.4% | 0.824 |
YOLOv5s [27] | 91.5% | 89.2% | 91.8% | 94.5% | 94.5% | 92.3% | 0.849 |
The improved Faster RCNN | 96.4% | 93.4% | 97.6% | 99.9% | 99.6% | 97.4% | 0.870 |
Model | APbruise | APcalyx | APstem | mAP | F1 | ||
---|---|---|---|---|---|---|---|
Small | Medium | Large | |||||
Model Ⅰ | 90.6% | 88.5% | 91.2% | 93.2% | 93.1% | 91.3% | 0.833 |
Model Ⅱ | 94.2% | 90.9% | 94.8% | 97.1% | 96.9% | 94.8% | 0.856 |
Model Ⅲ | 92.8% | 90.0% | 93.2% | 96.2% | 95.8% | 93.6% | 0.851 |
Model Ⅳ | 91.4% | 89.6% | 92.5% | 94.4% | 94.2% | 92.4% | 0.843 |
Model Ⅴ | 95.8% | 92.6% | 97.1% | 99.5% | 99.4% | 96.9% | 0.865 |
Model Ⅵ | 93.7% | 91.4% | 95.6% | 97.9% | 97.2% | 95.1% | 0.852 |
Model Ⅶ | 94.9% | 91.8% | 96.5% | 98.2% | 97.9% | 95.9% | 0.859 |
Model Ⅷ | 96.4% | 93.4% | 97.6% | 99.9% | 99.6% | 97.4% | 0.870 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hou, J.; Che, Y.; Fang, Y.; Bai, H.; Sun, L. Early Bruise Detection in Apple Based on an Improved Faster RCNN Model. Horticulturae 2024, 10, 100. https://doi.org/10.3390/horticulturae10010100
Hou J, Che Y, Fang Y, Bai H, Sun L. Early Bruise Detection in Apple Based on an Improved Faster RCNN Model. Horticulturae. 2024; 10(1):100. https://doi.org/10.3390/horticulturae10010100
Chicago/Turabian StyleHou, Jingli, Yuhang Che, Yanru Fang, Hongyi Bai, and Laijun Sun. 2024. "Early Bruise Detection in Apple Based on an Improved Faster RCNN Model" Horticulturae 10, no. 1: 100. https://doi.org/10.3390/horticulturae10010100
APA StyleHou, J., Che, Y., Fang, Y., Bai, H., & Sun, L. (2024). Early Bruise Detection in Apple Based on an Improved Faster RCNN Model. Horticulturae, 10(1), 100. https://doi.org/10.3390/horticulturae10010100