A Hybrid Deep Learning Architecture for Apple Foliar Disease Detection
Abstract
:1. Introduction
- A novel DL architecture, called CTPlantNet, which integrates two CNNs as the first block and one vision transformer as the second block, is proposed for the recognition of plant diseases, thus enhancing the performance of deep learning-based plant-anomaly-detection tasks.
- The recent deep CNNs (SEResNext and EfficientNet-V2S) and the Swin-Large vision transformer were adapted for classifying plant disease images, leveraging the advantages of the transfer learning technique.
- The performances of the proposed models were assessed against each other and compared with state-of-the-art models using two open-access plant pathology datasets (Plant Pathology 2020-FGVC-7 and Plant Pathology 2021-FGVC-8), applying a cross-validation strategy.
- The testing results show that the proposed DL models, including the CNNs and the vision transformer, performed better than the state-of-the-art (SOTA) works, where the proposed CTPlantNet architecture slightly increases the performance of the three models (SEResNext, EfficientNet-V2S, and Swin-Large) when trained and tested separately.
- The introduced DL models demonstrate their ability to detect the related challenges faced by the traditional optical inspection methods, such as the similarities between the features of some diseases, thus resulting in an early and swift detection of the pathologies in apple leaves and assisting farmers and orchard managers in safeguarding their crops and producing healthy and high-quality fruits.
2. Related Work
3. Materials and Methods
3.1. Datasets
- Plant Pathology 2020-FGVC-7 [2,21] is a publicly available collection of apple leaf images with different angles, illumination, noise, and backgrounds captured and labeled manually by specialists. The main objective of releasing this dataset was to provide a resource for researchers and developers to create models for automatic image classification in the field of apple leaf pathology. This dataset comprises 3651 apple leaf images distributed across four categories: 865 healthy leaf images, 1200 with scab, 1399 with rust, and 187 showcasing multiple diseases. While each image predominantly displays a singular disease, the “multiple disease” category includes images where each leaf bears more than one ailment. All images are rendered in high resolution and RGB color mode.
- Plant Pathology 2021-FGVC-8 [22] is a large publicly available collection of images of plant leaves, which are labeled with different disease categories. The dataset was provided in the 2021 Fine-Grained Visual Categorization (FGVC) challenge, specifically for plant pathology detection. This is an extended version of Plant Pathology 2020-FGVC-7 with more images and more diseases. This dataset contains a total of 18,632 images classified into six different classes (6225 healthy images, 4826 images with scab, 3434 images with rust, 3181 images with powdery mildew, 2010 images with frog eye spot, and 956 images with multiple diseases).
3.2. Proposed Approach
3.2.1. Convolutional Neural Network Models
- SEResNeXt-50 is a CNN model that incorporates a Squeeze-and-Excitation (SE) block with the ResNeXt architecture. The SE block in the SEResNeXt model employs a channelwise mechanism to capture channel dependencies, allowing for the adaptive recalibration of feature maps. This mechanism utilizes global average pooling to extract spatial information from the feature maps, followed by the application of a fully connected layer to recalibrate the feature maps based on the learned importance of each channel. SEResNext-50 effectively enhances the feature representation by leveraging the SE block. In addition to the SE block, SEResNext-50 takes advantage of the ResNeXt architecture [29], which is an extension of the widely used ResNet architecture. ResNeXt introduces the concept of “cardinality” as a new dimension with depth and width, allowing the network to better exploit the potential of parallel paths. Combining SE blocks with the ResNeXt architecture systematically enhances performance over different depths at an extremely minimal rise in computing complexity [30]. The SEResNeXt-50 model is capable of providing highly accurate predictions in a wide range of classification tasks, including image recognition and object detection.
- EfficientNet-V2S is a cutting-edge CNN model developed by Google [31]. This model is a member of the EfficientNet family, which is widely known for its superior efficiency and performance. The EfficientNet-V2S model is particularly noteworthy for its ability to achieve high accuracy with relatively few parameters. This is achieved through the use of novel techniques, such as compound scaling, which optimizes the architecture and scalability coefficients of the network, in addition to the Fused-MBConv blocks, which combine and employ two types of convolutions (pointwise and depthwise) into a single layer, resulting in lower computational complexity and speeding up the training of the EfficientNet-V2S model. As a result of the incorporated techniques in EfficientNet-V2S, the model shows strong potential in image classification tasks, achieving high performance while maintaining a relatively reduced size [31].
3.2.2. Vision Transformer Model
- Swin-Large is a transformer-based DL model consisting of 197 million trainable parameters trained on the ImageNet22k dataset for image classification tasks. Its name refers to the “shifted window” method employed to calculate the hierarchical representation of the model. This method enhances the model’s efficiency by restricting the computation of self-attention to non-overlapping local windows, simultaneously permitting connections across these windows. This approach enables the Swin-Large model to capture both local and global contextual information in images while keeping computational complexity manageable. In addition, the Swin transformer architecture incorporates a learnable relative position bias to account for the spatial relationships between image patches, further improving its performance in computer vision tasks [33].
3.3. Implementation
3.4. Evaluation Metrics
- Accuracy (ACC): describes the number of correct predictions over all predictions considering the true/false negative/positive predictions (TP, TN, FP, FN).
- Precision (PRE): quantifies the ratio of accurately classified positive instances among all instances classified as positive.
- F1-score (F1): quantifies the harmonic mean of precision and recall, providing a balance between the two metrics.
- Area the under curve (AUC): widely used in classification problems. This metric measures the performance of the proposed pipeline by highlighting the difference between its good and bad predictions. It is computed by measuring the area under the Receiver Operating Characteristic (ROC) curve [39].
4. Experimental Results
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
ML | machine learning |
DL | deep learning |
CNN | convolutional neural network |
FAO | Food and Agriculture Organization |
GPU | Graphical Processing Unit |
SVM | Support Vector Machine |
KNN | K-Nearest Neighbors |
LR | Logistic Regression |
RF | Random Forest |
NB | Naïve Bayes |
SMOTE | Synthetic Minority Over-Sampling |
CIDA | Cornell Initiative for Digital Agriculture |
SE | Squeeze-and-Excitation |
GAP | global average pooling |
FCL | fully connected layer |
DP | Dropout |
LRU | LeakyReLU |
GD | Gaussian Dropout |
SGD | Stochastic Gradient Descent |
LR | learning rate |
ACC | accuracy |
PRE | precision |
AUC | area under the curve |
TP | true positive |
TN | true negative |
FP | false positive |
FN | false negative |
NSERC | Natural Sciences and Engineering Research Council of Canada |
References
- FAOSTAT. Food and Agriculture Organization of the United Nations. Available online: https://www.fao.org/faostat/en/##data/QCL (accessed on 7 March 2023).
- Thapa, R.; Zhang, K.; Snavely, N.; Belongie, S.; Khan, A. The Plant Pathology Challenge 2020 data set to classify foliar disease of apples. Appl. Plant Sci. 2020, 8, e11390. [Google Scholar] [CrossRef] [PubMed]
- Azgomi, H.; Haredasht, F.R.; Safari Motlagh, M.R. Diagnosis of some apple fruit diseases by using image processing and artificial neural network. Food Control 2023, 145, 109484. [Google Scholar] [CrossRef]
- Harvey, C.A.; Rakotobe, Z.L.; Rao, N.S.; Dave, R.; Razafimahatratra, H.; Rabarijohn, R.H.; Rajaofara, H.; MacKinnon, J.L. Extreme vulnerability of smallholder farmers to agricultural risks and climate change in Madagascar. Philos. Trans. R. Soc. B Biol. Sci. 2014, 369, 20130089. [Google Scholar] [CrossRef] [PubMed]
- Jayawardena, K.; Perera, W.; Rupasinghe, T. Deep learning based classification of apple cedar rust using convolutional neural network. In Proceedings of the 4th International Conference on Computer Science and Information Technology, Samsun, Turkey, 11–15 September 2019; pp. 1–6. [Google Scholar]
- Agriculture Victoria (Victoria State Government). Pome Fruits. Available online: https://agriculture.vic.gov.au/biosecurity/plant-diseases/fruit-and-nut-diseases/pome-fruits (accessed on 7 March 2023).
- Legendre, R.; Basinger, N.T.; van Iersel, M.W. Low-Cost Chlorophyll Fluorescence Imaging for Stress Detection. Sensors 2021, 21, 2055. [Google Scholar] [CrossRef] [PubMed]
- Gudkov, S.V.; Matveeva, T.A.; Sarimov, R.M.; Simakin, A.V.; Stepanova, E.V.; Moskovskiy, M.N.; Dorokhov, A.S.; Izmailov, A.Y. Optical Methods for the Detection of Plant Pathogens and Diseases (Review). AgriEngineering 2023, 5, 1789–1812. [Google Scholar] [CrossRef]
- Pham, T.C.; Nguyen, V.D.; Le, C.H.; Packianather, M.; Hoang, V.D. Artificial intelligence-based solutions for coffee leaf disease classification. IOP Conf. Ser. Earth Environ. Sci. 2023, 1278, 012004. [Google Scholar] [CrossRef]
- Kirola, M.; Singh, N.; Joshi, K.; Chaudhary, S.; Gupta, A. Plants Diseases Prediction Framework: A Image-Based System Using Deep Learning. In Proceedings of the 2022 IEEE World Conference on Applied Intelligence and Computing (AIC), Sonbhadra, India, 17–19 June 2022; pp. 307–313. [Google Scholar]
- Ahmed, I.; Yadav, D.P.K. A systematic analysis of machine learning and deep learning based approaches for identifying and diagnosing plant diseases. Sustain. Oper. Comput. 2023, 4, 96–104. [Google Scholar] [CrossRef]
- Kawasaki, Y.; Uga, H.; Kagiwada, S.; Iyatomi, H. Basic Study of Automated Diagnosis of Viral Plant Diseases Using Convolutional Neural Networks. In Proceedings of the Advances in Visual Computing, Las Vegas, NV, USA, 14–16 December 2015; pp. 638–645. [Google Scholar]
- Ma, J.; Du, K.; Zheng, F.; Zhang, L.; Gong, Z.; Sun, Z. A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network. Comput. Electron. Agric. 2018, 154, 18–24. [Google Scholar] [CrossRef]
- Chen, H.C.; Widodo, A.M.; Wisnujati, A.; Rahaman, M.; Lin, J.C.W.; Chen, L.; Weng, C.E. AlexNet Convolutional Neural Network for Disease Detection and Classification of Tomato Leaf. Electronics 2022, 11, 951. [Google Scholar] [CrossRef]
- Sulistyowati, T.; Purwanto, P.; Alzami, F.; Pramunendar, R.A. VGG16 Deep Learning Architecture Using Imbalance Data Methods for The Detection of Apple Leaf Diseases. Monet. J. Keuang. Dan Perbank. 2023, 11, 41–53. [Google Scholar] [CrossRef]
- Yadav, A.; Thakur, U.; Saxena, R.; Pal, V.; Bhateja, V.; Lin, J.C.W. AFD-Net: Apple Foliar Disease multi classification using deep learning on plant pathology dataset. Plant Soil 2022, 477, 595–611. [Google Scholar] [CrossRef]
- Alsayed, A.; Alsabei, A.; Arif, M. Classification of apple tree leaves diseases using deep learning methods. Int. J. Comput. Sci. Netw. Secur. 2021, 21, 324–330. [Google Scholar]
- Bansal, P.; Kumar, R.; Kumar, S. Disease detection in apple leaves using deep convolutional neural network. Agriculture 2021, 11, 617. [Google Scholar] [CrossRef]
- Subetha, T.; Khilar, R.; Christo, M.S. A comparative analysis on plant pathology classification using deep learning architecture—Resnet and VGG19. Mater. Today Proc. 2021; in press. [Google Scholar]
- Kejriwal, S.; Patadia, D.; Sawant, V. Apple Leaves Diseases Detection Using Deep Convolutional Neural Networks and Transfer Learning. In Computer Vision and Machine Learning in Agriculture; Uddin, M.S., Bansal, J.C., Eds.; Springer: Singapore, 2022; Volume 2, pp. 207–227. [Google Scholar]
- Thapa, R.; Zhang, K.; Snavely, N.; Belongie, S.; Khan, A. Plant Pathology 2020-FGVC7. Available online: https://www.kaggle.com/competitions/plant-pathology-2020-fgvc7/ (accessed on 7 March 2023).
- Fruit Pathology, S.D. Plant Pathology 2021-FGVC8. 2021. Available online: https://www.kaggle.com/competitions/plant-pathology-2021-fgvc8 (accessed on 27 March 2024).
- Nasser, A.A.; Akhloufi, M.A. CTPlantNet: A Hybrid CNN-Transformer Architecture for Plant Disease Classification. In Proceedings of the 2022 International Conference on Microelectronics (ICM), Casablanca, Morocco, 4–7 December 2022; pp. 156–159. [Google Scholar] [CrossRef]
- Ghali, R.; Akhloufi, M.A.; Mseddi, W.S. Deep learning and transformer approaches for UAV-based wildfire detection and segmentation. Sensors 2022, 22, 1977. [Google Scholar] [CrossRef] [PubMed]
- Nasser, A.A.; Akhloufi, M. Chest Diseases Classification Using CXR and Deep Ensemble Learning. In Proceedings of the 19th International Conference on Content-based Multimedia Indexing, Graz, Austria, 14–16 September 2022; pp. 116–120. [Google Scholar]
- Nasser, A.A.; Akhloufi, M. Classification of CXR Chest Diseases by Ensembling Deep Learning Models. In Proceedings of the 23rd International Conference on Information Reuse and Integration for Data Science (IRI), San Diego, CA, USA, 9–11 August 2022; pp. 250–255. [Google Scholar]
- Lee, D.I.; Lee, J.H.; Jang, S.H.; Oh, S.J.; Doo, I.C. Crop Disease Diagnosis with Deep Learning-Based Image Captioning and Object Detection. Appl. Sci. 2023, 13, 3148. [Google Scholar] [CrossRef]
- Albawi, S.; Mohammed, T.A.; Al-Zawi, S. Understanding of a convolutional neural network. In Proceedings of the International Conference on Engineering and Technology (ICET), Antalya, Turkey, 21–23 August 2017; pp. 1–6. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Tan, M.; Le, Q. EfficientNetV2: Smaller Models and Faster Training. In Proceedings of the 38th International Conference on Machine Learning, Virtual, 18–24 July 2021; Volume 139, pp. 10096–10106. [Google Scholar]
- Khan, S.; Naseer, M.; Hayat, M.; Zamir, S.W.; Khan, F.S.; Shah, M. Transformers in vision: A survey. ACM Comput. Surv. (CSUR) 2022, 54, 1–41. [Google Scholar] [CrossRef]
- Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 10–17 October 2021; pp. 10012–10022. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- Lu Lu, Y.S.; Yanhui Su, G.E.K. Dying ReLU and Initialization: Theory and Numerical Examples. arXiv 2019, arXiv:1903.06733. [Google Scholar] [CrossRef]
- Wang, S.; Manning, C. Fast dropout training. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA, 16–21 June 2013; Volume 28, pp. 118–126. [Google Scholar]
- Seetala, K.; Birdsong, W.; Reddy, Y.B. Image classification using tensorflow. In Proceedings of the 16th International Conference on Information Technology-New Generations (ITNG 2019), Las Vegas, NV, USA, 1–3 April 2019; pp. 485–488. [Google Scholar]
- Loshchilov, I.; Hutter, F. SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv 2016, arXiv:1608.03983. [Google Scholar]
- Hoo, Z.H.; Candlish, J.; Teare, D. What is an ROC Curve? Emerg. Med. J. 2017, 34, 357–359. [Google Scholar] [CrossRef] [PubMed]
Models | ACC (%) | AUC (%) | PRE (%) |
---|---|---|---|
SEResNeXt50 | 97.68 | 99.89 | 98.12 |
EfficientNetV2-S | 97.88 | 99.94 | 98.40 |
Swin-Large | 98.70 | 99.58 | 99.04 |
Alsayed et al. [17] | 94.70 | - | - |
Sulistyowati et al. [15] | 92.94 | - | - |
Yadav et al. [16] | 98.70 | - | - |
Bansal et al. [18] | 96.25 | - | - |
Subetha et al. [19] | 87.70 | - | - |
CTPlantNet | 98.28 | 99.82 | 98.67 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ait Nasser, A.; Akhloufi, M.A. A Hybrid Deep Learning Architecture for Apple Foliar Disease Detection. Computers 2024, 13, 116. https://doi.org/10.3390/computers13050116
Ait Nasser A, Akhloufi MA. A Hybrid Deep Learning Architecture for Apple Foliar Disease Detection. Computers. 2024; 13(5):116. https://doi.org/10.3390/computers13050116
Chicago/Turabian StyleAit Nasser, Adnane, and Moulay A. Akhloufi. 2024. "A Hybrid Deep Learning Architecture for Apple Foliar Disease Detection" Computers 13, no. 5: 116. https://doi.org/10.3390/computers13050116
APA StyleAit Nasser, A., & Akhloufi, M. A. (2024). A Hybrid Deep Learning Architecture for Apple Foliar Disease Detection. Computers, 13(5), 116. https://doi.org/10.3390/computers13050116