An Algorithm for Severity Estimation of Plant Leaf Diseases by the Use of Colour Threshold Image Segmentation and Fuzzy Logic Inference: A Proposed Algorithm to Update a “Leaf Doctor” Application

Abstract

This paper explains a proposed algorithm for severity estimation of plant leaf diseases by using maize leaf diseased samples. In the literature, a number of researchers have addressed the problem of plant leaf disease severity estimation, but a few, such as Sannakki et al., have used fuzzy logic to determine the severity estimations of the plant leaf diseases. The present paper aims to update the current algorithm used in the “Leaf Doctor” application that is used to estimate the severities of the plant leaf diseases by introducing the benefits of fuzzy logic decision making rules. This method will contribute to precision agriculture technology as it introduces an algorithm that may be embedded in smartphone devices and used in applications, such as a “Leaf Doctor” application. The applications designed based on the algorithm proposed in this study will help users who are inexperienced and not plant pathologists understand the level of the estimated disease severity. The use of fuzzy logic inference rules along with image segmentation determines the novelty of this approach in comparison with the available methods in the literature.

1. Introduction and Related Work

Plant leaf diseases may pose a serious threat to Global food security if correct control measures are not seriously considered. The application of smart agricultural techniques, such as machine learning models, is progressively becoming an option for most of the commercial agricultural farmers. Plant leaf disease-detecting systems were modelled using different machine learning algorithms with outstanding accuracy results by researchers such as Sladojevic, Mohanty, and Amara [1,2,3]. In the literature, most studies, of which the philosophy is the severity estimation of the plant leaf diseases, utilize manual laboratory-based methods. Manual laboratory-based methods require skilled plant pathologists. Direct methods for manual laboratory-based plant leaf diseases were reviewed in a study by Fang and Ramasamy [4]. A few researchers, such as Sannakki et al., have pioneered the severity estimation of plant leaf diseases by the use of computer vision [5]. In their study, they used image processing and fuzzy logic to estimate the severity of the plant leaf diseases. They used the K-means clustering method to carry out segmentation of the diseased plant leaves. By means of fuzzy logic inference, they were able to estimate the severities of the detected plant leaf diseases by grading them using a MATLAB framework [5]. Wang, Sun, and Wang addressed an issue of plant disease severity estimation by using deep learning [6]. Their study utilized a procedure that was completely automatic, as it avoided the labour-intensive engineering features and threshold-based segmentation. They used the Plant Village image data set to estimate the severities of fine-grained diseases, such as apple rot. In their study, they used a series of deep convolutional neural networks, and the best performing model was the VGG16 model with an accuracy of 90.4% on the held-out test set.

Sun, Wei, Zhang, and Yang investigated the results of comparing tomato disease severities using image segmentation with visual estimates using a category scale for genetic analysis of resistance [7]. In their study, the image processing and analysis was performed by using image analysis software, called ASSESS V2.2. They used hue, saturation intensity, colour space, and the median filter in the colour panel. Another study compared, visual estimates with image analysis measurements, was conducted by Jarroudi et al. to determine Septoria leaf blotch severities in winter wheat [8]. In their study, they used a software called ASSESS V2.0 to perform digital image processing. They chose a classic panel and hue, saturation, and intensity-colour space to differentiate the rest of the leaf from the blue background. Murakami et al. worked with colourful autumn samples of sugar maple (Acer saccharum Marsh) leaves, to develop and test their method of digital image analysis that used Scion Image or NIH image public domain software and quantified leaf colour [9]. In their work, they provided step-by-step instructions for using the Scion software to measure the percentages of green and red in the leaves, colours of particular importance for the assessment of plant health. Comparisons of results from digital analyses of 326 scanned images of the leaves and concurrent spectrophotometric measures of chlorophyll a, chlorophyll b, and anthocyanin verified that image analysis provided a reliable quantitative measure of leaf colour and the relative concentrations of the underlying plant pigments [9]. Wijekoon, Goodwin, and Hsiang used the method that was proposed by Murakami et al. to quantify fungal infection of the plant leaves by digital image analysis [10].

A study of an automatic method to detect and measure leaf disease symptoms using digital image processing was conducted by Barbedo [11]. The Barbedo’s method was designed to be completely automatic, eliminating the possibility of human error with reduced time taken to measure the disease severity [11]. In his method, the RGB images were converted to L*a*b format. The L*a*b format was chosen because the channel A effectively highlighted different types of the symptoms [11]. The channel was used from the point of RGB to L*a*b conversion to the end of the algorithm. All pixels of the channel A that were lying outside the masks were then converted to zero. The segmentation of the diseased lesions on sugar cane by use of rectangle segmentation methods was proposed by Patil and Bodhe when they compared the segmented area to the leaf area [12]. The purpose of their approach was to eliminate the usage of pesticides, such that they were only applied according to the total calculated disease severity. Other methods of image segmentation by use of KNN (K-Nearest Neighbours) segmentation were used by researchers such as Pallottino et al. [13]. In their work, Pallottinno et al. used the KNN algorithm for the implementation of the old tractor-tillage process that recognized the weed from the plants by means of a fitted camera for data acquisition [13].

The method that was proposed by Owomugisha and Mwebaze utilized a Linear SVC classifier to classify the plant leaf diseases according to levels [14]. The levels were classified as Healthy class, Level 2 severity, level 3 severity, and level 4 severity. Bock, Cook, Parker, and Gottwald used ASSESS V1.0 for its advantage of being able to perform a colour threshold in the image of the diseased area [15]. In their work, they used the raters to estimate the severities of foliar citrus canker symptoms at a predetermined range of the percentage of infections.

Pethybridge and Nelson designed an interactive, iterative smartphone application called “Leaf Doctor” that is used to distinguish the diseased from healthy plant tissues and calculate the percentage of disease severity [16]. To operate the “Leaf Doctor application”, the user touches the application’s display screen to select up to eight different colours that represent healthy tissues. The user then moves a threshold slider until only the symptomatic tissues have been transformed into a blue hue. The pixilated image is then analysed to calculate the diseased percentage [16].

In this study, we carefully detail how the Otsu threshold colour segmentation method was used to approximate the areas of the diseased ROIs (Regions of interest) and the entire leaf in order to calculate the POI (Percentage of infections). The POI of the leaf area was calculated by taking the ratio of the $D{L}_A$ (Diseased leaf area) to a $T{L}_A$ (Total leaf area). Fuzzy logic techniques were used to take the calculated POI as an input to the fuzzy logic inference system and computed it for the estimation of the plant leaf disease severities. Our proposed algorithm is recommended for implementation in the future updates of the “Leaf Doctor” application. This paper is organized as follows. In Section 2 we explain the Tools, Methodology, and the theoretical principles of ideology. Our results are detailed and discussed in Section 3. Section 4 discusses the equations involved in the study, as well as the results. Section 5 draws up a conclusion based on the objectives and the results of the study.

2. Tools and Methods

2.1. Tools

The ratio of $D{L}_A$ to $T{L}_A$ resulted in the POI, which was utilized as the input to the fuzzy inference system to be computed for the estimation of plant leaf disease severity. Fiji image J was used as a software package to perform the threshold colour segmentation of the maize image samples that were tested for the estimation of disease severities. LabVIEW is the systems engineering software that was used in this study to design and simulate the fuzzy logic inference rules that governed the severity estimations of the maize leaf diseases.

2.2. Methodology

Figure 1 shows the process block diagram for severity estimations of the maize image diseases that were tested in this study.

Figure 1. Process block diagram for severity estimation of the tested maize image diseases.

After retrieving the image to be tested, the next stage was to select all the diseased ROIs by means of Otsu threshold colour segmentation method in the Fiji ImageJ package to approximate the area of the diseased leaf. Image histogram classic panels and the settings for the image colour threshold segmentation are shown in Figure 2. To accomplish the threshold segmentation of the diseased leaf regions, we varied the Hue and Saturation in colour panel space until the symptomatic tissues were covered with black, as shown in Figure 3b. We then approximated the total area of the leaf image by also using the Otsu threshold colour segmentation method in the Fiji ImageJ package. This was performed by varying the colour space panel until the entire leaf was covered in black, as shown in Figure 4. Finally, we calculated the POI and used it as the input to the fuzzy inference system, which was modelled in LabVIEW to compute the fuzzy membership functions that estimated the maize leaf disease severities. Image segmentation by threshold colour is segmentation that is conducted based on the differentiation of colour in the image. Threshold colour segmentation was used as a grouping process that enabled the image pixels to be separated according to their colour intensities. The tested diseased maize images were segmented using a histogram classic panel of the Fiji ImageJ by choosing the appropriate threshold values for Hue and Saturation, as shown in Figure 2.

Figure 2. Image histogram classic panels and the settings for the threshold colour segmentation method to filter the diseased ROIs, as shown in Figure 3b.

Figure 3. (a) The original image to be segmented using the Otsu threshold colour segmentation method; (b) the diseased ROIs filtered using the Otsu threshold colour segmentation method; (c) the diseased ROIs are selected using a select option in the threshold colour window in order to approximate the $D{L}_A$ (Diseased leaf area); (d) the approximated $D{L}_A$ (Diseased leaf area) is displayed in the results window.

Figure 4. The entire leaf filtered using the Otsu threshold colour segmentation.

2.3. Procedure of Image Segmentation by Threshold Colour

Step1: We collected the diseased maize image samples from the database that were saved after they were collected from the maize field. The collection of the maize data was done by means of Google’s pixel 3 smartphone from the maize field. The maize data collection was manually conducted for every 10 square meters of the maize field. The tested images were RGB colour images.

Step2: Next, we selected the diseased ROIs in the image by using the threshold colour segmentation method in a classic panel of the Fiji ImageJ software. With the Otsu threshold method selected, we then varied the HSB (Hue, Saturation, and Brightness) settings using black as the threshold colour until the ROIs of the diseased portions were selected. The segmented area of the diseased ROIs was approximated and the result reflected under “Area”, as shown in Figure 3d. The segmented area represented the diseased leaf area, abbreviated as $D{L}_A \left(in pixels\right)$ in Equation (1).

$$POI= \frac{D{L}_A}{T{L}_A} X 100$$

(1)

Figure 2 shows the image histogram windows and the threshold colour segmentation settings that were selected for this study.

Table 1 summarizes the available options to perform the threshold colour segmentation method. The red ticks show the options that were selected for this study.

Table 1. Tabular summary of threshold colour segmentation settings.

Threshold Method	Threshold Colour	Colour Space
Default	Red	HSB (Hue, Saturation, Brightness)
Inter modes	White	RGB (Red, Green, Blue)
Huang	Black	Lab
IsoData	Black and White	YUV (Luminance and the Chroma)
Li
MaxEntropy
Mean
Minimum
Otsu

Step 3: We calculated the POI by using the approximated $D{L}_A$ and $T{L}_A$, as shown in Equation (1). However, it can be seen that in Equation (1), the approximated total leaf area $T{L}_A \left(in pixels\right)$ is not yet known. To calculate the approximated $T{L}_A$, we also used the Otsu threshold colour segmentation to segment the original image, shown in Figure 3a. The outcomes were as shown in Figure 4, Figure 5 and Figure 6.

Figure 5. The entire leaf selected using a select option in the threshold colour window in order to approximate the $T{L}_A$ (Total leaf area in pixels).

Figure 6. The approximated $T{L}_A$ (Total leaf area) is displayed in the results window.

Following the approximated area values of $D{L}_A$ and $T{L}_A$, it can be seen that by the use of Equation (1), $POI= \frac{13697}{65536} X 100$ = 20.89%.

The maize sample images to be tested were selected from the database of the images that were collected from the maize field, as previously explained. The tested data set consisted of 12 Northern Corn Leaf Blight (Exserohilum), 12 Gray Leaf Spot (Cerospora), and 12 Common Rust (Puccinia sorghi) images of maize diseases. A total of 36 diseased test images was used to conduct the proposed study. For best results, the background of the tested images needed to be of any colour selected in the range of 0–255 of the gray scale.

2.4. Model of Fuzzy Logic Inference System

Fuzzy logic is an elegant approach to vagueness and uncertainty that was proposed by Lofti Zadeh in 1965. It incorporates the use of heuristic linguistic rules to solve problems that cannot be modelled by conventional techniques, such as the decisions taken by an experienced operator [17]. The percentage of the infected leaf area was used as the input to the fuzzy inference system that is proposed in this study. The fuzzified output consisted of the diseased leaf severity rating scales. Table 2 summarizes the fuzzification of both the input and output for the fuzzy inference system proposed in this study.

Table 2. Summary of leaf disease sample tests and the resulting severities.

Number of Sample Test Images	Input POI (Percentage of Infections)	Input POI Membership Functions	Output Rating Scale of Disease Severity	Output Membership Grade of Disease Severity
10	1–20%	Low POI	1–10	Low severity
12	20–50%	Medium POI	10–20	Medium severity
14	50–100%	High POI	20–30	High severity

The proposed fuzzy logic inference system to estimate plant leaf diseases was modelled in LabVIEW. Figure 7 shows the input and output membership functions of the fuzzy logic inference system. The system was modelled with 3 fuzzy logic decision rules that associated the POI with their corresponding severity grades of the maize diseases. The degree of membership ($\mu$) determined how much of the input was involved with its membership function. The fuzzy logic rules were implemented as follows:

Figure 7. Input membership functions for POI and output membership functions for the disease severity grades.

• Rule 1: If ‘Percentage of infected leaf area (POI)′ IS ‘Low POI’ THEN ‘Grade of disease severity (OUTPUT)’ IS ‘Low severity’.
• Rule 2: If ‘Percentage of infected leaf area (POI)′ IS ‘Medium POI’ THEN ‘Grade of disease severity (OUTPUT)’ IS ‘Medium severity’.
• Rule 3: If ‘Percentage of infected leaf area (POI)′ IS ‘High POI’ THEN ‘Grade of disease severity (OUTPUT)’ IS ‘High severity’.

The fuzzified control action of the inference system needed to be converted to a crisp output. The fuzzified output of the fuzzy logic inference system was converted to a crisp output by use of the Equation (2), which is a defuzzifier formula. This defuzzifier uses the centre of gravity to compute the crisp output of the fuzzy logic inference system.

$$z_{COG}= \frac{\int {\mu }_{\overline{c}\left(z\right).z.\partial z}}{\int {\mu }_{\overline{c}}.z.\partial z}$$

(2)

In this regard, it is clear that Fuzzy logic is a rule-based system that can rely on the practical experience of an operator, which is particularly useful for capturing experienced operator knowledge [17]. The fuzzy logic membership function scale ratings used in this study to estimate the severities of the maize diseases were chosen based on the advice of the experienced pathologists at ARC (Agricultural Research Council), South Africa.

3. Results

Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13 show the 3-D surface results and fuzzy logic inference-system performance tests when rules 1 to 3 were invoked.

Figure 8. The 3-D surface results when rule 1 was invoked.

Figure 9. Fuzzy logic inference test system to realize the feasibility of rule 1.

Figure 10. The 3-D surface results when rule 2 was invoked.

Figure 11. Fuzzy logic inference test system to realize the feasibility of rule 2.

Figure 12. The 3-D surface results when rule 3 was invoked.

Figure 13. Fuzzy logic inference test system to realize the feasibility of rule 3.

Table 3 summarizes the fuzzy logic-system performance results for severity estimations of the maize diseases, as shown in Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13.

Table 3. A summary of the fuzzy logic-system performance results for the severity estimation of the maize diseases, as shown in Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13.

Input Degree of Membership	Input POI	Associated Input POI Membership Functions	Output Scale of Disease Severity	Output Membership Grade of Disease Severity	Rules Invoked
0.99	10.529%	Low POI	5.49	Low severity	Rule 1
0.19	23.76%	Medium POI	15	Medium severity	Rule 2
0.99	75.11%	High POI	24.9	High severity	Rule3

To test the fuzzy logic inference system, LabVIEW was used in this study. Figure 8, Figure 9, Figure 10, Figure 11, Figure 12 and Figure 13 show the simulation results of the fuzzy logic inference system that was designed using 3 fuzzy logic based rules. The input to the fuzzy inference system was the percentage of infections, POI. In Table 3, it can be seen that the disease severities changed according to the calculated percentage of infections, POI. A POI of 10.59% with 0.99 degrees of membership invoked rule 1 and estimated the maize leaf to have a low disease severity. It can also be seen in Table 3 that a POI of 23.76% with 0.19 degrees of membership invoked rule 2 and estimated the maize leaf to have a medium disease severity. Finally, it is shown in Table 3 that a POI of 75.11% with 0.99 degrees of membership invoked rule 3 and estimated the maize leaf to have a high disease severity.

4. Discussion

The number of test images to maize diseases, as shown in Table 2, consisted of Northern Corn Leaf Blight (Exserohilum), Gray Leaf Spot (Cerospora), and Common Rust (Puccinia sorghi) maize diseases. It can be seen in Table 2 that 10 test images were used to estimate low disease severity, 11 test images to estimate medium disease severity, and 14 test images to estimate a high disease severity. Fiji ImageJ was used for threshold colour segmentation. Both the diseased leaf area and the total leaf area were used to calculate the percentage of infections (POI), as shown in Equation (1). The results of the approximated $D{L}_A$ and $T{L}_A$, as shown in Figure 3d and Figure 6, respectively, were used to calculate the POI equal to 20.89%. By use of the fuzzy inference system proposed in this study, a POI of 20.89% represented a maize disease with a medium estimated disease severity. A method that was used by Sannakki et al. [5], utilized K-means clustering for image segmentation and fuzzy logic for the severity estimation of the plant leaf diseases. However, in their method, they did not clearly explain the membership functions in terms of the degree of membership. In computer vision, the recognition of objects, mostly depends on identifying particular shapes in an image. Clearly, using K-means clustering for segmentation, however good it is, has a disadvantage of having to find the K value by means of trial and error. In the literature, plenty of evidence has shown studies that use different versions of the ASSESS software to perform image analysis for the estimation of plant leaf disease severities [7,8,15]. However, none of these methods used fuzzy logic to estimate the severities of the plant leaf diseases.

Pallottinno et al. used the KNN algorithm for the implementation of the old tractor-tillage process that recognized the weed from the plants by means of a fitted camera for the data acquisition [13]. The main weakness in their study is that they made no attempt to compensate for disease severity estimation in the useful plants. A combination of our proposed method with theirs would have resulted in a system capable of weed detection and removal, and fuzzified plant disease severity estimation. Owomugisha and Mwebaze proposed a method that utilized a Linear SVC (Support Vector Classifier) classifier to classify the plant leaf diseases according to levels [14]. The levels were classified as Healthy class, Level 2 severity, level 3 severity, and level 4 severity. What Owomugisha and Mwebaze did not explain in their study was how they graded the plant diseases, according to their corresponding levels [8]. In our view, their findings are implausible, as they did not explain in their study how the severity levels were used to estimate the degree of the plant leaf diseases within a certain level. In our proposed method, the percentage of infection’s strength was determined by the degree of membership in its membership function. By use of the designed fuzzy logic decision rules in the fuzzy logic inference system, Equation (2) was used to decide on the level of the disease severity, which could be “Low severity”, “Medium severity”, or “High severity”. Our proposed method is reliable and not expensive for the following reasons:

• The Fiji ImageJ package is an open source software package that is meant for image processing and segmentation.
• Data acquisition is not done by means of expensive cameras that are computationally demanding.
• Threshold colour segmentation is conducted based on the differentiation of colour on the image, and the areas of the segmented regions of interest can be approximated in the results window of the Fiji ImageJ.

The integration of the Otsu threshold colour segmentation and fuzzy logic inference rules that were conducted in this study is a novel way of estimating the plant leaf disease severities for implementation in the future updates of a “Leaf Doctor” application. Figure 14, Figure 15, Figure 16 and Figure 17 show the steps to be followed when using the “Leaf Doctor” application. Pethybridge and Nelson designed an interactive, iterative smartphone application called “Leaf Doctor” that is used to distinguish diseased from healthy plant tissues and calculate the percentage of disease severity [16]. To operate the “Leaf Doctor application”, the user touches the application’s display screen to select up to eight different colours that represent healthy tissues. The user then moves a threshold slider until only the symptomatic tissues have been transformed into a blue hue. The pixilated image is then analysed to calculate the diseased percentage [16]. Our proposed method suggests a novel algorithm for future updates of the “Leaf Doctor” application. It can be seen in Figure 17 that the results of the tested maize leaf give 54.79% of the diseased leaf area and 45.21% of the healthy area. However, the information on the severity of the diseased leaf area of 54.79% might not be clear to an inexperienced user of the application, who also might not be a qualified or experienced pathologist. Our method represents an innovative alternative to the one currently used in the “Leaf Doctor” application, as it uses fuzzy logic to estimate the percentage severity of the diseased leaf.

Figure 14. Step 1 for loading the tested image to the “Leaf Doctor” application.

Figure 15. Step 2 shows the loaded image ready for testing.

Figure 16. Step 3 shows two selected colours that represented the healthy regions and the threshold slider was varied to cover the diseased tissues with a blue hue.

Figure 17. Step 4 shows the estimated results by clicking the tick on the upper right corner of Figure 15.

Figure 14 shows step 1 by using the “Leaf Doctor” application to analyse the maize image for possible disease. By clicking “load image from gallery”, the maize image to be analysed was selected among other images available in the smart phone’s gallery.

Figure 15 shows the maize image to be analysed with a slider that was used to threshold the image.

By selecting the colours that represented the healthy regions in the image of Figure 15, a selection of up to eight selections could be made. However, it can be seen in Figure 16 that a selection of two colours was made to represent the healthy regions.

By varying the threshold slider, the diseased leaf tissues were covered in blue. The results were obtained by clicking a tick on the upper right corner of Figure 16. Figure 17 shows how the results were displayed for the healthy regions and diseased regions. As mentioned previously, an inexperienced user, who might not be a pathologist, may not understand the interpretation of the disease severity if it is only displayed in percentage. As shown in Figure 7, it can be seen that our proposed method has a capability of rating the percentage of infections as “Low severity”, “Medium severity”, or “High severity”. For instance, if our proposed method would have been used to interpret the results shown in Figure 17, the diseased leaf area of 54.79% would have been estimated as “High severity”. As mentioned previously, it has to be remembered that the fuzzy logic scale ratings that were used in this study were chosen based on advice that was given by the experienced pathologists at ARC (Agricultural Research Council), South Africa.

Table 4 shows a comparison of the displayed results using a current “Leaf Doctor” method, with a recommended method using our proposed method with fuzzy logic decisions.

Table 4. A comparison table to interpret the results of current “Leaf Doctor” method and our proposed method.

Current Display of Results in “Leaf Doctor” without Fuzzy Logic	Display of Results in “Leaf Doctor” as Recommended by Our Proposed Method with Fuzzy Logic
Healthy: 45.21% Diseased: 54.79%	Healthy: 45.21% Diseased: 54.79%, High severity

5. Conclusions

The estimated disease severities of the maize leaves were successfully measured by the use of the Otsu threshold colour segmentation and fuzzy logic decision rules. The method proposed in this study was not time consuming and is recommended for smart phone applications that do not integrate fuzzy logic to give meaningful results. “Leaf Doctor” is an example of an application that is recommended to employ our proposed method in its future updates. Our proposed method was tested using a software called Fiji image J for colour threshold segmentation. The fuzzy logic inference system was modelled and tested using a software called LabVIEW.