A Convolutional Neural Networks Based Method for Anthracnose Infected Walnut Tree Leaves Identification
2.1. Process Description
- Load the dataset: The images are loaded recursively based on their pathnames. Depending on the prefix of their filename, a label value of ‘0’ or ‘1’ is appended to each loaded image, explained in detail in Section 2.3.
- Color conversion: Two functions for color conversion were implemented in this study: a conversion from the BGR to the RGB color-space, where the order of the image’s color channels changes to the most common (RGB) format, and a conversion from BGR to the grayscale color-space, where the images lose all the color information and maintain only the brightness and saturation for each pixel.
- Feature extraction (only for grayscale): For the grayscale images, there is additionally applied a feature extraction method, the Fast Fourier Transform (FFT). This method helps bring out the edge features of the leaves, thus potentially improving the performance of the model compared to a straightforward approach of grayscale images.
- Background segmentation (optional): This method removes the background from images with the use of thresholding, dilation and erosion, as commonly used methods in computer vision.
- Data normalization: The dataset values are normalized within the (0, 1) range in order to ensure that the loss function, which is usually not convex, finds the global minimum as easy as possible. Reducing the range of input values also assists the convergence of the backpropagation algorithm.
- Dataset shuffling: The dataset is being shuffled by Python’s “random.shuffle” method, which is an order-shuffling algorithm based on a random number generator. After the application, the order of the images, which originally was alphabetical, is now mixed throughout the dataset.
- Dataset splitting: The dataset is being split in the following fashion: First, 15% of the total dataset is held out and set as the testing set, then the remaining dataset is split again in the 80/20 fashion, where the small portion is used as the validation set. The rest is used for training the algorithm. The training and validation sets are used for the training of the algorithm, while the testing dataset is used only after the classifier is trained, in order to make predictions.
2.1.2. Learning Process
- Convolutional Neural Network (CNN) architecture definition: The architecture of the CNN is defined based on experimentation and trials. The number, type and order of layers is the important aspect that can lead to effective network architecture.
- Functions definition: Various functions that are used throughout the network are being defined through bibliographic research and trials. We carefully choose the activation function, which is the function that defines the output of the layer, and the loss function, which is the function that is used for the optimization of the network’s weights.
- Training: This is the process where the algorithm tries to create a function that describes the desired relation, based on the training data. It then makes predictions based on this function and moves to the next step.
- Validation: Following the previous step, the function that the algorithm created is being evaluated against the validation dataset. The error in the predictions is being calculated, and the algorithm tries to find a way to minimize this error. This part defines the “learning” of the process.
- Finalizing the model: The training-validation process repeats itself until the algorithm cannot improve itself anymore. When the training loss and the validation loss have become almost the same, lowest-possible value, and before the validation loss starts to increase, we stop the procedure and finalize our model.
- Test the CNN based on trained data: The performance of the finalized model is measured by its validation accuracy; i.e., the accuracy it could achieve based on the data that it has been given to train and evaluate. This is indicative of the model’s performance, but the real performance is only evident when predictions on unknown data take place.
- Make predictions on the test data: We use the testing data, which is completely unknown to the model, to make predictions on each image’s class. For each image, the classifier provides a predicted class, which is stored alongside its true class.
- Calculate performance metrics: This is the final step where the predicted classes are being cross-checked over the true classes of the testing images. Then, the testing accuracy, precision, recall and f1-score of the model are calculated. A confusion matrix is also used in order to visually evaluate the performance of the model, and also check if it is biased over any class.
2.2. Data Acquisition
2.3. Data Preparation
2.4. Data Split
2.5. Performance Metrics
2.6. Feature Extraction Preprocessing with Fast Fourier Transform
2.7. Convolutional Neural Network Architecture
2.8. Visualization of Convolutions
2.8.2. Activation Maps for RGB
2.8.3. Activation Maps for Grayscale
2.8.4. Filter Effect on Images
4. Discussion and Conclusions
- The proposed CNN method exhibits outstanding performance when RGB analysis is performed for the examined images of this case study. This can be emanated from the fact that the results produced from the application of the CNN architecture are based on distinct features that appear specifically on the anthracnose-infected leaves, compared to the healthy leaves, i.e. brown spots and areas.
- The fast Fourier transform method seems to be of major significance in feature extraction in the case of the grayscale images, because it accentuates the abrupt changes and edges of the leaves. This denotes that the infected leaves have more edgy features than the healthy ones.
- The proposed CNN architecture exhibits better, or similar, performance to well-known CNN architectures (i.e., DenseNet121, VGG16, ResNet50 and InceptionV3) which have been efficiently used as benchmarks in image processing problems for the past years.
- Overall, for the purpose of image analysis and classification, the CNN methodology is proved to be proper for complex image classification tasks, such as the one under hand here, when a large number of images is considered, outweighing the popular CNN architectures for image analysis, such as DenseNet121, VGG16, ResNet50 and InceptionV3.
Conflicts of Interest
Appendix A. Proposed CNN Architecture
Appendix B. Model Performance
Appendix B.1. Grayscale without Background Removal
Appendix B.2. Fast Fourier without Background Removal
Appendix B.3. RGB without Background Removal
Appendix B.4. Grayscale with Background Removal
Appendix B.5. Fast Fourier with Background Removal
Appendix B.6. RGB with Background Removal
- Moshou, D.; Bravo, C.; Oberti, R.; West, J.S.; Ramon, H.; Vougioukas, S.; Bochtis, D. Intelligent multi-sensor system for the detection and treatment of fungal diseases in arable crops. Biosyst. Eng. 2011, 108, 311–321. [Google Scholar] [CrossRef]
- Patrício, D.I.; Rieder, R. Computer vision and artificial intelligence in precision agriculture for grain crops: A systematic review. Comput. Electron. Agric. 2018, 153, 69–81. [Google Scholar] [CrossRef][Green Version]
- Chou, Y.C.; Kuo, C.J.; Chen, T.T.; Horng, G.J.; Pai, M.Y.; Wu, M.E.; Lin, Y.C.; Hung, M.H.; Su, W.T.; Chen, Y.C.; et al. Deep-learning-based defective bean inspection with GAN-structured automated labeled data augmentation in coffee industry. Appl. Sci. 2019, 9, 4166. [Google Scholar] [CrossRef][Green Version]
- Liakos, K.G.; Busato, P.; Moshou, D.; Pearson, S.; Bochtis, D. Machine learning in agriculture: A review. Sensors 2018, 18, 2674. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Pantazi, X.-E.; Moshou, D.; Bochtis, D. Intelligent Data Mining and Fusion Systems in Agriculture; Academic Press: Cambridge, MA, USA; Elsevier: Amsterdam, The Netherlands, 2020. [Google Scholar]
- Pantazi, X.E.; Moshou, D.; Tamouridou, A.A. Automated leaf disease detection in different crop species through image features analysis and One Class Classifiers. Comput. Electron. Agric. 2019, 156, 96–104. [Google Scholar] [CrossRef]
- Ramya, V.; Lydia, M.A. Leaf Disease Detection and Classification using Neural Networks. Int. J. Adv. Res. Comput. Commun. Eng. 2016, 5, 207–210. [Google Scholar]
- Muthukannan, K.; Latha, P.; Selvi, R.P.; Nisha, P. Classification of diseased plant leaves using neural network algorithms. ARPN J. Eng. Appl. Sci. 2015, 10, 1913–1919. [Google Scholar]
- Picon, A.; Alvarez-Gila, A.; Seitz, M.; Ortiz-Barredo, A.; Echazarra, J.; Johannes, A. Deep convolutional neural networks for mobile capture device-based crop disease classification in the wild. Comput. Electron. Agric. 2019, 161, 280–290. [Google Scholar] [CrossRef]
- Hanson, A.M.G.J.; Joel, M.G.; Joy, A.; Francis, J. Plant Leaf Disease Detection using Deep Learning and Convolutional Neural Network. Int. J. Eng. Sci. Comput. 2017, 7, 2324. [Google Scholar]
- Sladojevic, S.; Arsenovic, M.; Anderla, A.; Culibrk, D.; Stefanovic, D. Deep Neural Networks Based Recognition of Plant Diseases by Leaf Image Classification. Comput. Intell. Neurosci. 2016, 2016, 11. [Google Scholar] [CrossRef][Green Version]
- Ferentinos, K.P. Deep learning models for plant disease detection and diagnosis. Comput. Electron. Agric. 2018, 145, 311–318. [Google Scholar] [CrossRef]
- Mohanty, S.P.; Hughes, D.P.; Salathé, M. Using Deep Learning for Image-Based Plant Disease Detection. Front. Plant Sci. 2016, 7, 1419. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Barbedo, J.G.A. Impact of dataset size and variety on the effectiveness of deep learning and transfer learning for plant disease classification. Comput. Electron. Agric. 2018, 153, 46–53. [Google Scholar] [CrossRef]
- Zhang, S.; Huang, W.; Zhang, C. Three-channel convolutional neural networks for vegetable leaf disease recognition. Cogn. Syst. Res. 2019, 53, 31–41. [Google Scholar] [CrossRef]
- Raza, S.E.A.; Prince, G.; Clarkson, J.P.; Rajpoot, N.M. Automatic detection of diseased tomato plants using thermal and stereo visible light images. PLoS ONE 2015, 10, e0123262. [Google Scholar] [CrossRef] [PubMed]
- Ma, J.; Du, K.; Zheng, F.; Zhang, L.; Gong, Z.; Sun, Z. A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network. Comput. Electron. Agric. 2018, 154, 18–24. [Google Scholar] [CrossRef]
- Liu, B.; Zhang, Y.; He, D.J.; Li, Y. Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry 2018, 10, 11. [Google Scholar] [CrossRef][Green Version]
- Wang, G.; Sun, Y.; Wang, J. Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning. Comput. Intell. Neurosci. 2017, 2017, 8. [Google Scholar] [CrossRef][Green Version]
- Anand, R.; Veni, S.; Aravinth, J. An application of image processing techniques for detection of diseases on brinjal leaves using k-means clustering method. In Proceedings of the 2016 International Conference on Recent Trends in Information Technology, Bengaluru, India, 12 November 2016. [Google Scholar]
- Amara, J.; Bouaziz, B.; Algergawy, A. A Deep Learning-Based Approach for Banana Leaf Diseases Classification. BTW 2017. Available online: https://pdfs.semanticscholar.org/adae/9446cb66eaa6645dca78fd81b21d43aebdda.pdf (accessed on 23 December 2019).
- Wang, H.; Li, G.; Ma, Z.; Li, X. Application of neural networks to image recognition of plant diseases. In Proceedings of the 2012 International Conference on Systems and Informatics, Yantai, China, 19–20 May 2012. [Google Scholar]
- Fuentes, A.; Yoon, S.; Kim, S.C.; Park, D.S. A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors 2017, 17, 2022. [Google Scholar] [CrossRef][Green Version]
- Hasan, M.; Ullah, S.; Khan, M.J.; Khurshid, K. Comparative analysis of SVM, ann and cnn for classifying vegetation species using hyperspectral thermal infrared data. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences—ISPRS Archives, Enschede, The Netherlands, 10–14 June 2019. [Google Scholar]
- Arsenovic, M.; Karanovic, M.; Sladojevic, S.; Anderla, A.; Stefanovic, D. Solving Current Limitations of Deep Learning Based Approaches for Plant Disease Detection. Symmetry 2019, 11, 939. [Google Scholar] [CrossRef][Green Version]
- Stuchi, J.A.; Angeloni, M.A.; Pereira, R.F.; Boccato, L.; Folego, G.; Prado, P.V.; Attux, R.R. Improving image classification with frequency domain layers for feature extraction. In Proceedings of the IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), Tokyo, Japan, 25–28 September 2017; pp. 1–6. [Google Scholar]
- Leversha, G.; Bachman, G.; Narici, L.; Beckenstein, E. Fourier and Wavelet Analysis; Springer: Berlin/Heidelberg, Germany, 2001. [Google Scholar]
- Kingma, D.P.; Ba, J.L. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015—Conference Track Proceedings, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Chollet, F. Keras. 2015. Available online: https://github.com/fchollet/keras (accessed on 23 December 2019).
- Jupyter.org. Available online: https://jupyter.org/ (accessed on 23 December 2019).
- Abadi, M.; Agarwal, A.; Barham, P.; Brevdo, E.; Chen, Z.; Citro, C.; Corrado, G.S.; Davis, A.; Dean, J.; Devin, M.; et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. arXiv 2016, arXiv:1603.04467. [Google Scholar]
- Anagnostis, A.; Asiminari, G.; Dolias, G.; Arvanitis, C.; Papageorgiou, E.; Myresiotis, C.; Bochtis, D. Machine Learning Algorithms Comparison for Image Classification on Anthracnose Infected Walnut Tree Canopies. In Proceedings of the XXXVIII CIOSTA & CIGR V International Conference, Rhodes, Greece, 24–26 June 2019; p. 6. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; Fei-Fei, L. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009. [Google Scholar]
- Huang, G.; Liu, Z.; van der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017. [Google Scholar]
- Simonyan, K.; Zisserman, A. VGG-16. arXiv 2014, arXiv:409.1556. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016. [Google Scholar]
|% of total dataset||68%||17%||15%|
|# of images||3053||764||674|
|Accuracy||ratio of correctly predicted observation to the total observations (preferred in balanced datasets)||(TP + TN/TP + FP + FN + TN)|
|Precision||ratio of correctly predicted positive observations to the total predicted positive observations||(TP/TP + FP)|
|Recall||ratio of correctly predicted positive observations to all observations in actual class||(TP/TP + FN)|
|F1 score||is the weighted average of Precision and Recall (preferred in unbalanced datasets)||[2 · Recall · Precision]/[Recall + Precision]|
|Layer (Type)||Output Shape||Param #|
|conv2d_1 (Conv2D)||(None, 254, 254, 32)||896|
|max_pooling2d_1 (MaxPooling2)||(None, 127, 127, 32)||0|
|conv2d_2 (Conv2D)||(None, 125, 125, 64)||18,496|
|max_pooling2d_2 (MaxPooling2)||(None, 62, 62, 64)||0|
|conv2d_3 (Conv2D)||(None, 60, 60, 128)||73,856|
|max_pooling2d_3 (MaxPooling2)||(None, 30, 30, 128)||0|
|conv2d_4 (Conv2D)||(None, 28, 28, 256)||295,168|
|max_pooling2d_4 (MaxPooling2)||(None, 14, 14, 256)||0|
|conv2d_5 (Conv2D)||(None, 12, 12, 512)||1,180,160|
|max_pooling2d_5 (MaxPooling2)||(None, 6, 6, 512)||0|
|flatten_1 (Flatten)||(None, 18,432)||0|
|dropout_1 (Dropout)||(None, 18,432)||0|
|dense_1 (Dense)||(None, 512)||9,437,696|
|dense_2 (Dense)||(None, 1)||513|
|Total params: 11,006,785|
|Trainable params: 11,006,785|
|Non-trainable params: 0|
|Layer||Filter Dimensions||Number of Filters|
|Conv2d_1||3 × 3||32|
|Conv2d_2||3 × 3||64|
|Conv2d_3||3 × 3||128|
|Conv2d_4||3 × 3||256|
|Conv2d_5||3 × 3||512|
|Decision Trees||Random Forest||Ada-Boost||Support Vector Machines||Artificial Neural Networks|
|VGG16||DenseNet121||ResNet50||Inception V3||Proposed CNN|
|Execution time (s)||1391.31||1866.893||1290.42||1712.213||989.185|
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Anagnostis, A.; Asiminari, G.; Papageorgiou, E.; Bochtis, D. A Convolutional Neural Networks Based Method for Anthracnose Infected Walnut Tree Leaves Identification. Appl. Sci. 2020, 10, 469. https://doi.org/10.3390/app10020469
Anagnostis A, Asiminari G, Papageorgiou E, Bochtis D. A Convolutional Neural Networks Based Method for Anthracnose Infected Walnut Tree Leaves Identification. Applied Sciences. 2020; 10(2):469. https://doi.org/10.3390/app10020469Chicago/Turabian Style
Anagnostis, Athanasios, Gavriela Asiminari, Elpiniki Papageorgiou, and Dionysis Bochtis. 2020. "A Convolutional Neural Networks Based Method for Anthracnose Infected Walnut Tree Leaves Identification" Applied Sciences 10, no. 2: 469. https://doi.org/10.3390/app10020469