Classification of Pear Leaf Diseases Based on Ensemble Convolutional Neural Networks

Abstract

Over the last few years, the impact of climate change has increased rapidly. It is influencing all steps of plant production and forcing farmers to change and adapt their crop management practices using new technologies based on data analytics. This study aims to classify plant diseases based on images collected directly in the field using deep learning. To this end, an ensemble learning paradigm is investigated to build a robust network in order to predict four different pear leaf diseases. Several convolutional neural network architectures, named EfficientNetB0, InceptionV3, MobileNetV2 and VGG19, were compared and ensembled to improve the predictive performance by adopting the bagging strategy and weighted averaging. Quantitative experiments were conducted to evaluate the model on the DiaMOS Plant dataset, a self-collected dataset in the field. Data augmentation was adopted to improve the generalization of the model. The results, evaluated with a range of metrics, including accuracy, recall, precison and f1-score, showed that the proposed ensemble convolutional neural network outperformed the single convolutional neural network in classifying diseases in real field-condition with variation in brightness, disease similarity, complex background, and multiple leaves.

1. Introduction

Over the last few years, the impact of climate change has increased rapidly. It is influencing all steps of plant production and forcing farmers to change and adapt their crop management practices using new technologies based on data analytics. Plant disease prediction has become a research hotspot as biotic and abiotic stresses represent the limiting factors of agriculture productivity.

Plant disease prediction is a complex and interconnected problem requiring different technical skills [1]. The traditional approach sees the involvement of a specialist in the sector, who through a careful analysis of the foliar surface, makes a diagnosis. However, not all farmers in underdeveloped countries can afford advice, as it is expensive. Therefore, the use of technology becomes an indispensable tool for recognizing the onset of diseases from the appearance of the first symptoms. To this end, scientific and technological progress is concentrating studies on the development of decision support systems (DSS), capable of assisting farmers in adopting sustainable practices, environmentally, socially and economically. The core of these systems lies in the integration of predictive models that extract useful information from a large amount of agricultural data. In particular, on the basis of the different factors that interact with the epidemiology of plant diseases, it is possible to integrate models of different types [2]: (i) forecasting models based on weather data; (ii) forecasting models based on image processing; (iii) forecasting models based on different types of data from several heterogeneous sources.

Several state-of-the-art studies [3] have addressed the disease diagnosis through image processing, applying conventional machine learning techniques [4]. Subsequently, the progress of deep learning techniques in image classification activities, in particular convolutional neural network (CNN) architectures, has also generated interest in the field of digital agriculture. The convolutional neural networks are characterized by a broad horizon of analysis. Indeed, the study of Brahimi et al. [5] comparing different algorithms, such as Support Vector Machines, Random Forest and GoogleNet, demonstrated the efficiency of deep learning compared to machine learning. Multiple models have been adopted for different tasks: plant and disease classification [6], pest recognition [7], insect counting [8], weed detection [9], yield prediction [10], fruit classification [11], seed classification [12] etc. Sladojevic et al. [13] studied a convolutional neural network to classify 13 different types of plant diseases from leaf images. The dataset was built by the authors downloading the pictures from the Internet. The experimental results achieved an overall accuracy of the trained model of 96.3%. Lu et al. [14] proposed a CNNs-based model to recognize 10 rice diseases, achieving an accuracy of 95.48%. Liu et al. [15] explored the application of deep learning model in classifying four apple leaf disease. The authors proposed a novel model based on AlexNet and GoogleNet’s Inception structure, reaching satisfactory results with an accuracy of 97.62%. Ferentinos [16] adopted AlexNet and VGG to classify several plant diseases, where most of the images were taken in laboratory conditions. Too et al. [17] performed a study to predict different diseases, performing fine-tuning technique. The CNNs architectures were trained on PlantVillage dataset, where DenseNet obtained a test accuracy score of 99.75%. Similarly, Waheed et al. [18] optimized the DenseNet architecture to recognize three corn leaf diseases. Ramcharan et al. [19] used a smartphone-based CNN model to identify cassava plant diseases with an accuracy rate of 80.6%. Javierto et al. [20] trained a YOLOv3-MobileNetv2 model for detecting diseases in robusta coffee leaves into four classes: Cercospora, miner, phoma, and rust. Hassan et al. [21] compared different convolutional neural networks (InceptionV3, InceptionResnetV2, MobileNetV2, EfficientNetB0) for the detection of plant diseases using PlantVillage dataset with healthy- and diseased-leaf images of plants.

However, although deep learning represents a frontier of artificial intelligence capable of automating predictive analysis, its application in digital agriculture, is still an area of exploration that needs further development to solve various problems related to crop disease prediction. Deep learning algorithms are considered “black box”, as it is difficult to understand and explain what factors lead to the final result. This aspect, in disease prediction, is of paramount importance in understanding the relationships that contribute to disease onset. They also have the drawback of computational complexity. This presents a major challenge for the deployment of such models on mobile devices with limited resources [22]. Finally, training these algorithms requires a large amount of data. In crop disease prediction, there are few datasets open to the scientific community. A comprehensive review is available in [23]. The accuracy and reliability of the models is strongly influenced by the representativeness and completeness of the dataset used in training the algorithm. At the state of the art, many datasets have been constructed in the laboratory, in which the leaf is portrayed against a homogeneous background under controlled light conditions. An appplication, should be able to recognize the leaf symptom directly in the field. As demonstrated in [24], performance deteriorates if the same model is trained with a dataset collected directly in the field. As demonstrated in [24], performance degrades if the same model is trained with a dataset collected directly in the field.

The main contribution of this study is the implementation of a more robust classifier (i.e., greater generalization ability), using a field-collected dataset and ensembling multiple CNNs to achieve enhanced predictive performance. To the best of our knowledge, the first investigation for the identification of pear leaf diseases. To identify the base learners, we investigated and compared different architectures such as EfficientNetB0, InceptionV3, MobileNetV2, and VGG19 on a dataset built in field conditions, where images were captured with different cameras, in different ranges, lighting conditions, angles and phenological phase of the disease. The performance of the proposed ensemble convolutional neural network (ECNN) was evaluated with a range of metrics, including accuracy, recall, precison and f1-score.

The paper is structured as follows. Section 2 describes the dataset, the experimental approach and the setup adopted to conduct the study. Section 4 illustrates the experimental results and presents a discussion. We conclude the paper in Section 5.

2. Material and Methods

2.1. Dataset

A field dataset was collected to diagnose pear tree diseases, called DiaMOS Plant dataset [23] (see

Table 1. Dataset description.

DiaMOS Plant Dataset [23]
Plant	Pear
Cultivar	Septoria Piricola
Type of data	RGB Images
ROI (Region of Interest) captured	leaf, fruit
Total size	3505 images (3006 leaves images + 499 fruit images)
Data Accessibility	https://doi.org/10.5281/zenodo.5557313 Date: 16 January 2023
Application	The images are suitable for different machine and
	deep learning tasks such as images detection
	and classification.

). The images were collected using various devices.

Table 2. Devices configurations.

	Smartphone Camera	DSRL Camera
Image size	2976 × 3968	3456 × 5184
Model device	Honor 6×	Canon EOS 60D
Focal length	3.83 mm	50 mm
Focal ratio	f/2.2	f/4.5
Color space	RGB	RGB

reports the set-up of each device. The symptoms were captured from the adaxial (upper) leaf side under real conditions. In order to maximize the intrinsic and extrinsic factors that allow for more representative datasets to be built, the images were taken at different time intervals, with variations in lighting, disease similarity, zoom, angles, complex background, and multiple leaves. Furthermore, the images were collected for an entire season, from February to July, in order to collect the greatest number of samples representing the various developmental stages of the disease, especially concerning the initial phase. A total of 3006 images were collected, including healthy leaves and diseased leaves, affected by one or more of the following symptoms: leaf spot, leaf curl, and slug. The

Table 3. Dataset details.

Leaf Disease	Size
Healthy	43
Spot	884
Curl	54
Slug	2025
Total	3006

provides information on 4 classes of diseases, including the number of pictures in each class. Figure 1 shows some samples of each class taken under real conditions.

2.2. Experimental Approach

Figure 2 shows the workflow of the proposed study. The design system is divided into consecutive steps illustrated in the following paragraphs. Based on the dataset collected in the pear orchand, four convolutional neural networks were compared in order to find the best two classifiers. A pre-processing and data augmentation techniques were applied to reinforce the model generalization. After training the networks in the classification of the four symptoms, the best two base learners were selected, and a bootstrap aggregation was adopted using the the weighted average for the final prediction.

2.3. Convolutional Neural Network Architecture

We adopted four notable convolutional neural networks architectures for the purpose of our study, including Inception [25], MobileNet [26] and EfficientNet [27], and VGG19 [28].

VGG19. This is a convolutional neural network designed by Visual Geometric Group (VGG) at the University of Oxford [28] in the year of 2014. The model achieved 92.7% top-5 test accuracy in ImageNet, which is a database of over 14 million images belonging to 1000 object categories. The novelty of VGG architectures was its simplicity in using a deeper layer with smaller filters [1], in order to classify images. The first version, VGG16 comprises 16 layers, while the second version, named VGG19, consist of 16 layers. Both architectures take as input an image of size 224 × 224 with three color channels.

Inception. This is a network developed by [25] in 2014, designed to improve image classification performance while maintaining optimal use of computational resources. A disadvantage of convolutional networks is the computational cost, dictated by the number of parameters for each layer. This architecture implements several expedients to efficiently manage computational resources in terms of costs as well as the number of parameters, introducing the inception modules (see Figure 3). The model is based on an oriented acyclic graph, in which the input is processed by several parallel convolutional branches whose outputs are then merged in a single tensor.

MobileNet. Ideated by [26], similar to the Inception network, MobileNet is a convolutional architecture created to achieve high results by reducing the computational complexity required by convolutional layers. To this end, the model uses depthwise separable convolutions (see Figure 4) to build deep and light networks. Thanks to this arrangement, the small size of the model allows easy integration into mobile and embedded devices.

EfficientNet. This is a MobileNet-inspired network released in 2019 by [27]. The authors, in order to improve accuracy and maintain low computational costs, studied network scaling and proposed a method known as compound scaling. Traditional architectures perform this operation with an arbitrary scaling constant in any dimension. In contrast EfficientNetB0, implements network scaling using a constant ratio in all dimensions: widh/depth/resolution.

2.4. Transfer Learning

Transfer learning is a technique used in machine learning to improve the performance of the models despite a relative lack of data. It consists in employing the knowledge learned and developed in one domain in another domain, adjusting the weights of the pre-trained models to the new target task.

2.5. Data Augmentation

Data augmentation is a method designed to mitigate overfitting, where the size of the dataset is enriched by generating new samples from the training set by performing multiple and different transformations. In this study, we applied the standard methods: rotation, shearing, zooming, cropping, flipping and color variation.

2.6. Proposed Ensemble Convolutional Neural Network

Ensemble learning is a machine learning paradigm where different models, called base learners, are combined to solve the same problem. There are several approaches for ensemble modeling, such as Bagging (bootstrapped aggregated ensemble), Boosting, and Stacking. In this study, we adopted the Bagging strategy, which consist of training single classifiers and combine the final prediction. The ensemble convolutional neural network (ECNN) is designed with four pre-trained architectures such as EfficieNetB0, MobileNetV2, IncpetionV3, and VGG19.

Let $N=\{EfficientNetB0,MobileNetV2,InceptionV2,VGG19\}$ is the set of the pre-trained CNNs architectures. Each network $n\in N$ was trained on field-collected images of 4 different symptoms in the dataset $(M{}_i,S{}_i)$, where M is the number of pictures, each resized to 224 × 224 × 3, and normalized in the [0, 1] interval, and S is the relative symptoms, that are the labels of images, $S=\{healthy,leafspot,leafcurl,slug\}$

The final prediction is calculated with a weighted average, where at each model $n\in N$ is assigned a fixed weight that is multiplied by the prediction made by the classifier and used in the average prediction calculation.

The weighted average method is given by the following equation:

$$p^{\prime }=\frac{1}{n}\sum _{i=1}^n{\alpha }_i\overrightarrow{y}$$

(1)

where $\alpha$ is the weight values that multiply with the weight vector $\overrightarrow{y}$ and n is the number of ensemble CNN.

Algorithm 1 summarizes the detailed procedures.

Algorithm 1: The detailed illustration of the algorithm.

Input   : Leaf Images $(M,S)$ using dataset D    Output: Class prediction $s\in S$ 1 Step 1: D is divided into training set ($D_{train}$) (60%), validation set ($D_{validation}$)(20%), test set ($D_{test}$) (20%) 2 Step 2: Pre-processing: 3         The input images are resized to 224 × 224 × 3 4         The input images are normalized 5         The data augmentation techniques are applied 6 Step 3: Training 7     foreach $\forall n\in N$ 8         l = 0.001 9         for epochs = 1 to 100 do 10             Update the parameters of the model n 11             foreach mini batch ($M{}_i,S{}_i$) ∈ ($M_{train},S_{train}$) do 12                if the test accuracy is not improving for 10 epochs then 13                    l = l × 0.2 14                end 15             end 16         end 17      end 18 Step 4: 19     foreach $d\in D_{test}$ do 20         ensemble the output of all networks $n\in N$ 21    end

2.7. Evaluation Metrics

The evaluation metrics adopted to evaluate the study are shown in the following equations, including Accuracy, Precision, Recall and F1 score. In the description of these metrics, we used the following definitions: False Positives (FP): diseased leaves that were misclassified as healthy; False Negatives (FN): healthy leaves that were misclassified as diseased; True Positive (TP): diseased leaves that were correctly classified as diseased; True Negative (TN): healthy leaves that were correctly classified as healthy.

• Accuracy is defined as $(TP+TN)/(TP+TN+FP+FN)$;
• Precision is defined as $TP/(TP+FP)$;
• Recall is defined as $TP/(TP+FN)$;
• F1 score is defined as $2\ast \frac{(Precision\ast Recall)}{(Precision+Recall)}$;

3. Experimental Setup

The experiment was conducted adopting a Python deep learning framework called Keras, executed on a server equipped with a 3.000 GHz Intel(R) Xeon(R) Gold. The dataset built and described in the Section 2.1, was split applying the ShuffleSplit strategy provided by the scikit-learn 0.23.2 library with a ratio of 6:2:2 for the training, validation and test set, respectively. In order to prevent overfitting situation, the training set was expanded with a data augmentation strategy, including horizontal mirroring, vertical mirroring, rotation (90° counterclockwise rotation, 180° rotation, 90° clockwise rotation), and random color variation. The data augmentation technique is applied in real time using Keras library, where every single image (with bach size of 32) is augmented at the start of every epoch according with the aforementioned settings. As feature extraction a transfer learning technique was performed by adapting CNN networks trained using the ImageNet dataset [29]. Regarding the hyper-parameters settings, two optimizer functions were tested Adam and RMSprop, a Cross-Entropy loss function and a momentum of 0.9 were used. We released the code written as a modular and reusable toolbox, called LeafBox, to facilitate the reproduction of the results obtained (https://leafbox.francescamalloci.com, accessed on 16 January 2023).

4. Results and Discussion

This section reports the results obtained to classify diseased leaves.

Table 4. Accuracy obtained on the training set, validation set and test set.

	Accuracy (%)
CNN	Optimizer	Train	Validation	Test
EfficientNetB0	RMSprop	81.13	82.82	83.38
	Adam	89.02	86.33	86.05
InceptionV3	RMSprop	81.96	79.66	82.72
	Adam	84.44	80.29	83.39
MobileNetV2	RMSprop	85.38	81.12	83.06
	Adam	87.70	83.83	84.05
VGG19	RMSprop	72.42	71.68	73.75
	Adam	76.66	76.53	75.75

shows the scores obtained on training, validation, and test using RMSprop and Adam optimizers. It can be observed that the Adam optimizer achieved higher accuracy when training with EfficientNetB0, MobileNetV2 and InceptionV3. The RMSprop optimizer achieved highest results when training with VGG19. Moreover, similar performances are recorded by EfficientNetB0, MobileNetV2 and InceptionV3, where the accuracy was 83.38%, 82.72% and 83.06%, respectively.

In

Table 5. Precision, recall and F1-score obtained on the training set, validation set and test set.

CNN	Optimizer	Precision (%)	Recall (%)	F1-Score (%)
EfficientNetB0	RMSprop	81.14	83.38	82.23
	Adam	84.42	86.04	85.03
InceptionV3	RMSprop	80.21	82.72	81.45
	Adam	81.14	83.38	82.23
MobileNetV2	RMSprop	81.35	83.05	82.07
	Adam	82.37	84.05	83.06
VGG19	RMSprop	70.47	73.75	71.76
	Adam	72.71	75.74	74.05

, the precision, recall and F1-score are reported. The scores do not differ significantly, but confirm the performance trend. Indeed, the F1-score scores on the test set confirm a good harmonic average, EfficientNetB0 of 85.03%, InceptionV3 82.23%, MobileNetV2 83.06%, and VGG19 74.05%.

The results obtained differ from the state of the art, as a considerable part used different datasets and parameters, different biotic and abiotic stresses. Most of these have addressed the problem with a dataset built under controlled conditions, called laboratory, reaching high accuracy of around 95–98%. However, the results obtained corroborate with studies that used a field dataset. The study in [24] demonstrated how model performance degrades from a laboratory dataset to a field dataset. Similarly, the work in [30] compared different architectures for the classification of six tomato diseases photographed directly in the field. Among these, VGG16 achieved an accuracy of 76.20%, and Incpetion an accuracy of 85.36%.

The best three convolutional neural networks were selected based on the results in Table 4. From these, EfficientNetB0, InceptionV3, and MobileNetV2 were considered to build the ensemble architecture. The results are presented in

Table 6. Performance of the ensemble CNNs.

	Test Accuracy—Weighted Average
Ensemble CNNS	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)
EfficientNetB0 + InceptionV3	91.14	89.84	90.02	89.93
EfficientNetB0 + MobileNetV2	86.21	84.13	85.51	84.82
InceptionV3 + MobileNetV2	85.35	83.02	85.14	84.08

.

The scores obtained reveal that the proposed methods can improve the overall performance of classification, where the highest accuracy of 91.14% was achieved by combining EfficientNetB0 with InceptionV3. The respective precision was 89.84(%),the recall 90.02(%), and the F1-score 89.93(%). By combining EfficientNetB0 with MobileNetV2 there were slight improvements, where accuracy was 86.21(%), the precision 84.13(%), the recall 85.51(%), the F1-score 84.82(%). Similarly, InceptionV3 with MobileNetV2 achieved an accuracy of 85.35(%), the precision 83.02(%), the recall 85.14(%), the F1-score 84.08(%).

The ensemble learning relies on the fact that several different good classifiers are able to learn different aspects of the “truth” of the data. MobileNet uses depthwise separable convolutions, EfficientNet uses similar convolutions named spatial separable convolutions, while Incpetion uses inception modules. The results confirm that the best result is given by two more different models. The proposed ensemble CNN (ECNN) has outperformed all the single pre-trained convolutional neural networks in Table 4.

Our results are compared to the most closely related literature, as the experiments are conducted with different conditions, as the dataset built in laboratory. Among these, the best known is the Plant Village dataset.

Vallabhajosyula et al. [31] proposed an ensemble neural network trained on Plant Village dataset, comparing ResNet, Dense Net, InceptionV3, and NasNet Mobile. The proposed DENN achieved an accuracy of 99%. Exploring the same dataset Sutaji and Yıldız [32], proposed a LEMOXINET network, composed by MobileNetV2 and Xception, obtained an accuracy of 99.52%. A study conducted by Kaur et al. [33] studied a comparison between Random Forest and ensemble aftificial neural network (ANN), Support Vector Machine (SVM), K- nearest neighbour (KNN), logistic regression and naïve bayes classifier, to predict 15 classes from PlantVillage dataset. An accuracy of 92.13% has been observed in Bell Pepper (2 classes), 95.66% in Potato (3 classes) and 90.23% in Tomato (10 classes). Consequently, the authors of Chen et al. [34], adopted a stacking ensemble model, called Es-MbNet, trained on Plant Village dataset extended by images collected by the authors on field-cultivation, reaching an accuracy of 98.96% on the validation set.

From the results obtained, it can be seen that ensemble learning is a promising technique in the classification of pear leaf diseases in complex experimental conditions (variations in lighting, disease similarity, zoom, angles, complex background, and multiple leaves), where the representativeness of the sample reflects more the target area. In particular, the combination of the two EfficientNetB0+IncpetionV2 models, compared to the single model, led to a better accuracy of 91.14%, against 85% for EfficientNetB0. To our knowledge, there are no other works analyzing the same foliar symptoms as our study. However, from the more closely related literature that used a field dataset, it can be seen that the use of two models allows to achieve better performances that are close to 90%. An accuracy that differs from the high scores obtained on the PlantVillage dataset, but as demonstrated by recent studies, the models find it more difficult to classify the disease directly in the field. Furthermore, the proposed CNN model, even if trained on a field dataset with real-time data augmentation technique, suffer to the overfitting problem caused by the unbalancing of the classes.

5. Conclusions

In this paper, we studied an application of ensemble learning to improve the performance of plant disease classification in a field-collected dataset, named DiaMOS PLant dataset. To design the ensemble models based on CNNs, we compared four state-of-the-art CNNs architectures: EfficientNetB0, MobileNetV2, InceptionV3 and VGG19. The classifiers were trained with a transfer learning technique using the data augmentation techniques to increase the amount of samples.

The models were evaluated on a dataset that we collected directly in the fields of a pear orchard, labeling four diseases. To obtain a representative dataset and train a more robust model to real-world application conditions, images were taken at different time intervals, with variations in lighting, disease similarity, zoom, angles, complex background, and multiple leaves. Furthermore, the images were collected for an entire season, from February to July, in order to collect the greatest number of samples representing the various developmental stages of the disease, especially concerning the initial phase.

The three best models were selected to build ensemble architectures, where the final prediction was obtained by applying weighted average methods. The results showed that the best three single trained convolutional neural network obtained are EfficientNetB0, MobileNetV2 and InceptionV3. The ensemble architecture have improved overall performance, where combinations of two different architectures bring better accuracy. Indeed, EfficientNetB0 + InceptionV3 achieved an accuracy of 91.14%.

In conclusion, the proposed study provides an application of the ensemble learning in the classification of leaf diseases, adopted for the first time for the identification of pear diseases. The performance of the ECNN model was evaluated with field-collected images, with promising results. As future lines of research, we are going to apply different techniques to further increase the robustness of directly trained models with field datasets, such as extending the current dataset, in order to increase its representativeness with more biotic stresses, as well as applying further augmentation techniques (e.g., GAN augmentation) and hyper-parameter optimization.