Next Article in Journal
Evaluation of Air Temperature, Photoperiod and Light Intensity Conditions to Produce Cucumber Scions and Rootstocks in a Plant Factory with Artificial Lighting
Next Article in Special Issue
A Sustainable Viticulture Method Adapted to the Cold Climate Zone in China
Previous Article in Journal
In-Vivo In-Vitro Screening of Ocimum basilicum L. Ecotypes with Differential UV-B Radiation Sensitivity
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Artificial Intelligence and Novel Sensing Technologies for Assessing Downy Mildew in Grapevine

by
Inés Hernández
1,2,
Salvador Gutiérrez
3,
Sara Ceballos
1,2,
Rubén Iñíguez
1,2,
Ignacio Barrio
1,2 and
Javier Tardaguila
1,2,*
1
Televitis Research Group, University of La Rioja, 26006 Logroño, Spain
2
Institute of Grapevine and Wine Sciences, Consejo Superior de Investigaciones Científicas, Gobierno de La Rioja, University of La Rioja, 26007 Logroño, Spain
3
Department of Computer Science and Artificial Intelligence, University of Granada, 18071 Granada, Spain
*
Author to whom correspondence should be addressed.
Horticulturae 2021, 7(5), 103; https://doi.org/10.3390/horticulturae7050103
Submission received: 16 April 2021 / Revised: 29 April 2021 / Accepted: 4 May 2021 / Published: 8 May 2021
(This article belongs to the Special Issue Advances in Viticulture Production)

Abstract

:
Plant diseases and pests cause a large loss of world agricultural production. Downy mildew is a major disease in grapevine. Conventional techniques for plant diseases evaluations are time-consuming and require expert personnel. This work investigates novel sensing technologies and artificial intelligence applications for assessing downy mildew in grapevine under laboratory conditions. In our methodology, machine vision is applied to assess downy mildew sporulation, while hyperspectral imaging is used to explore its potential capability towards early detection of this disease. Image analysis applied to RGB leaf disc images is used to estimate downy mildew (Plamopara viticola) severity in grapevine (Vitis vinifera L. cv Tempranillo). A determination coefficient (R2) of 0.76 ** and a root mean square error (RMSE) of 20.53% are observed in the correlation between downy mildew severity by computer vision and expert’s visual assessment. Furthermore, an accuracy of 81% is achieved to detect downy mildew early using hyperspectral images. These results indicate that non-invasive sensing technologies and computer vision can be applied for assessing and quantify sporulation of downy mildew in grapevine leaves. The severity of this key disease is evaluated in grapevine under laboratory conditions. In conclusion, computer vision, hyperspectral imaging and machine learning could be applied for important disease detection in grapevine.

1. Introduction

Plant diseases, pests and weeds cause large losses of food production in agriculture. Traditionally, plant diseases are identified by visual observations by the growers in the field of biological techniques in the laboratory [1]. Nevertheless, these techniques are time-consuming, susceptible to human error and/or require qualified personnel [1,2]. Non-invasive sensing technologies as RGB (Red, Green and Blue) images, thermography, multispectral and hyperspectral imaging have been postulated as potential non-invasive tools for detecting plant diseases in agriculture with several advantages versus conventional methods [1,2,3], even for detecting diseases when the symptoms are not visible [4]. Computer vision, machine learning and artificial intelligence technologies can be applied to identify, classify and quantify crop diseases in data-driven agriculture [5,6,7,8].
In the European project NoPest (Novel Pesticides for a Sustainable Agriculture), non-destructive proximal sensing technologies and artificial intelligence are being developed for assessing fungal diseases in key crops as grapevine and potato. Downy mildew is a key grapevine disease in world viticulture. Nowadays, the evaluation of this disease has been based mostly on visual assessment of leaves in the vineyards or histological analyses at the laboratory [9]. Computer vision and artificial intelligence could be very useful to recognise and quantify some diseases in grapevine [10,11,12].
This work examines non-invasive imaging technologies and artificial intelligence applications for assessing downy mildew in grapevine under laboratory conditions. Computer vision was applied to evaluate downy mildew severity, while hyperspectral imaging was used to detect this disease early in grapevine.

2. Materials and Methods

This approach could be separated into three main sections (Figure 1): (i) RGB and hyperspectral image acquisition under laboratory conditions; (ii) image processing using computer vision techniques and hyperspectral preprocessing to improve image features and assess downy mildew sporulation; and (iii) machine learning modelling, used to detect the disease early using hyperspectral imaging.

2.1. Image Acquisition under Laboratory Conditions

Under laboratory conditions, leaf discs from grapevine (Vitis vinifera L., cv. Tempranillo) plants were placed in Petri dishes with the abaxial side up. Two groups of the study were defined: One group was infected with the downy mildew agent (Plasmopara viticola), and the other group was used as control. Images were taken in the laboratory every day until nine days after inoculation using a digital RGB camera (Canon EOS 5D, Japan). Additionally, a push broom hyperspectral visible (VIS) camera (spectral range from 400 to 1000 nm) was used to detect downy mildew early under lab conditions. For validation, the percentage of leaf area showing downy mildew sporulation in the leaf discs was visually evaluated by a panel of eight experts, providing a disease severity value for each disc. The average evaluation of each disc was used as a reference.

2.2. Processing of RGB and Hyperspectral Images

The RGB images were processed to detect and quantify downy mildew sporulation in grapevine leaves (Figure 2). The colour space was transformed to HLS (Hue, Lightness, Saturation) to bring the perception of the colour reflected in the digital image closer to the human eye by obtaining colour, brightness and saturation values from red, green and blue values of the images. Saturation values seemed to show different values in areas of the leaves with sporulation and the rest of the leaf, unlike brightness and colour values, thus these values were used to locate sporulation in the leaf. Classical preprocessing techniques were applied to enhance image features that help to analyse downy mildew sporulation, such as median filter, used to smooth the image, and Contrast Limited Adaptive Histogram Equalisation (CLAHE) [13], used to improve the contrast. Hough Transform [14] was used to localise the leaf discs in the image, differentiating them from the rest of the image. Each leaf disc image was segmented with the Otsu method [15], separating the pixels representing downy mildew sporulation and the rest of the leaf. The segmentation of leaf disc images was used to estimate downy mildew severity as the percentage of downy mildew sporulation that appears in the grapevine leaf discs (Figure 3).
On the other hand, hyperspectral images were processed. The values (I) were transformed to reflectance (R) to correct the images using a dark current (DC) and a white reference (WR) values using the following equation:
R = I D C W R D C
Savitzky–Golay filter [16] was also applied to smooth the spectra of each image (Figure 4), with a grade 2 polynomial and a size 15 window. Then, a standardisation was applied.
Leaf disc location on hyperspectral images was obtained by applying the watershed segmentation algorithm [17], once the leaf spectra were separated from the background, as indicated in Figure 1. This spectra separation is detailed in the next section.

2.3. Machine Learning Modelling

For hyperspectral images, a two-stage machine learning analysis was designed for (i) the detection of spectra belonging to leaves (segmentation) and (ii) modelling and prediction using the leaf spectra as input. The segmentation of the leaves was carried out by manually selecting spectra belonging to the positive class (leaf spectra) and negative class (spectra from the background elements). From these data, a binary classifier was trained using Support Vector Machines (SVMs) and applied for the automated segmentation of all the images. The modelling for the pathogen detection was done after leaf spectra extraction in the previous step (by binary classifying spectra according to the disc they belong to, between spectra of infected and non-infected discs) and the training using different models trained with Convolutional Neural Networks (CNNs), k-Nearest Neighbour (KNN), MultiLayer Perceptrons (MLPs) and Partial Least Square-Discriminant Analysis (PLS-DA). The CNN architecture was composed of two convolutional blocks, with a one-dimensional convolutional layer and a max-pooling layer on each one, to obtain features from spectra; and three fully connected blocks, to classify the obtained features in the convolutional blocks.

2.4. Implementation

All the experiments were developed with the Python 3.7.4 programming language. The RGB and hyperspectral images were processed using the OpenCV 4.2.0.32 and scikit-learn 0.22.2 libraries on an Intel Core i7 4770 CPU (16 GB RAM). On the other hand, RGB images taken under field conditions were processed with an NVIDIA GeForce RTX 2080 Ti GPU (11 GB memory), optimising the execution of the CNN developed with the Keras 2.3.1 framework and the Tensorflow 2.1.0 backend.

3. Results and Discussion

Results of two approaches used for assessing downy mildew in grapevine are summarised in Table 1. Computer vision was applied to assess downy mildew severity, while hyperspectral imaging was employed for early detection. A strong and significant relationship (determination coefficient of 0.76** and a root mean square error of 20.53%) was observed between downy mildew severity measured using computer vision and visual assessment by the experts in grapevine leaf discs. As can be seen in Figure 2 and Figure 3, this new method provided a sporulation location for each leaf disc, which adds interpretability to the results, giving information on the estimated severity of downy mildew.
Disease severity estimation of downy mildew using computer vision techniques obtained similar results to expert evaluation. This severity estimation could consider the expert subjectivity, achieving a greater relationship between automatic and manual assessment [11]. The possibility to adopt new sensing technologies for detecting grapevine downy mildew and for evaluating disease severity gives new opportunities for disease assessment in other key commercial crops in agriculture.
Regarding early detection of downy mildew using hyperspectral imaging, classification accuracy between control and downy mildew inoculated discs after several days post inoculation (DPI) using different machine learning models is showed in Figure 5 and Table 1. Artificial neural networks differed from the other models, with an accuracy close to 81% and an f1-score close to 0.80 (Table 1). This implies that the spectral features that differentiate the spectra of infected discs from those from uninfected discs were difficult to find, and the convolution blocks do not provide a better performance of the model, since a shallower neural network, such as an MLP, was capable of achieving results similar to a CNN.
Early detection of downy mildew using machine learning techniques achieved high accuracies in hyperspectral images. This method could be combined with that used to estimate the severity of the disease to obtain a localisation and quantification of the sporulation or by applying machine learning for image segmentation [7,18]. These results indicate that computer vision can be applied for assessing and quantify sporulation of downy mildew in grapevine leaves. Moreover, hyperspectral imaging could be applied to detect this disease early in grapevine.
Both approaches used for assessing downy mildew in grapevine were developed in cv. Tempranillo, however, can be applied to other grapevine varieties with similar results [19]. Better results would be achieved by re-training the models with more grapevine varieties, allowing greater generalisation and achieving a model that is better adapted to different grapevine varieties.
This work tested using two techniques, such as computer vision and hyperspectral imaging, to assess downy mildew in grapevine. The method employed for downy mildew severity estimation, using computer vision techniques, required no training and no modelling, so its computational cost was low. The higher spectral dimension of hyperspectral images allows for the application of more powerful machine learning techniques for better classification. The method for classifying hyperspectral images, despite requiring the training of machine learning models, most of them could be executed with a CPU, training each one of them in less than one hour. On the other hand, for the classification of hyperspectral images using a CNN, a GPU was used to train the CNN, which took several days, due to the computational complexity of this training. Still, but once the model is trained, its prediction can be very fast, classifying more than 100 images in less than a minute. Moreover, RGB cameras, in general, are much cheaper, with higher availability and easiness of use than hyperspectral imaging.
Machine learning and computer vision algorithms can be very useful for assessing downy mildew in grapevine. Our results indicate that downy mildew can be evaluated in grapevine using novel technologies and artificial intelligence techniques in data-driven agriculture.

4. Conclusions

The results exposed in this work indicate that downy mildew in grapevine can be automatically evaluated using emerging technologies. New sensing technologies and machine learning have shown promising results for assessing downy mildew in Tempranillo, but it could be applied to other grapevine varieties. Computer vision, hyperspectral imaging and artificial intelligence can be applied for monitoring downy mildew in grapevine. Machine vision was applied to assess downy mildew severity, and hyperspectral imaging was used to detect this disease early.
Following the example of downy mildew in grapevine, it can be suggested that computer vision and hyperspectral imaging could be used for assessing major diseases in other key crops in precision agriculture, with several advantages versus conventional methods. Artificial intelligence can also be applied in plant pathology for early disease detection and quantify the symptoms.

Author Contributions

Conceptualization, J.T.; methodology, J.T., I.H., R.I., S.C., S.G. and I.B.; validation, I.H., R.I., S.C., S.G. and I.B.; investigation, J.T., I.H., R.I., S.C., I.B. and S.G.; data curation, I.H. and S.G.; writing—original draft preparation, J.T., I.H., R.I., S.C., I.B. and S.G. writing—review and editing, I.H., S.G. and J.T.; supervision, J.T.; funding acquisition, J.T. All authors have read and agreed to the published version of the manuscript.

Funding

This work has been developed as part of the project NoPest (Novel Pesticides for a Sustainable Agriculture), which received funding from the European Union Horizon 2020 FET Open program under Grant agreement ID 828940.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data available on request due to privacy restrictions. The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Ali, M.M.; Bachik, N.A.; Muhadi, N.A.; Yusof, T.N.T.; Gomes, C. Non-destructive techniques of detecting plant diseases: A review. Physiol. Mol. Plant Pathol. 2019, 108, 101426. [Google Scholar] [CrossRef]
  2. Sankaran, S.; Mishra, A.; Ehsani, R.; Davis, C. A review of advanced techniques for detecting plant diseases. Comput. Electron. Agric. 2010, 72, 1–13. [Google Scholar] [CrossRef]
  3. Oerke, E.-C.; Herzog, K.; Toepfer, R. Hyperspectral phenotyping of the reaction of grapevine genotypes toPlasmopara viticola. J. Exp. Bot. 2016, 67, 5529–5543. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Gao, Z.; Khot, L.R.; Naidu, R.A.; Zhang, Q. Early detection of grapevine leafroll disease in a red-berried wine grape cultivar using hyperspectral imaging. Comput. Electron. Agric. 2020, 179, 105807. [Google Scholar] [CrossRef]
  5. Barbedo, J.G.A. Plant disease identification from individual lesions and spots using deep learning. Biosyst. Eng. 2019, 180, 96–107. [Google Scholar] [CrossRef]
  6. Mahlein, A.-K.; Kuska, M.T.; Thomas, S.; Wahabzada, M.; Behmann, J.; Rascher, U.; Kersting, K. Quantitative and qualitative phenotyping of disease resistance of crops by hyperspectral sensors: Seamless interlocking of phytopathology, sensors, and machine learning is needed! Curr. Opin. Plant Biol. 2019, 50, 156–162. [Google Scholar] [CrossRef] [PubMed]
  7. Gonçalves, J.D.; Pinto, F.D.D.; de Queiroz, D.M.; Villar, F.M.D.; Barbedo, J.; del Ponte, E.M. Deep Learning Models for Semantic Segmentation and Automatic Estimation of Severity of Foliar Symptoms Caused by Diseases or Pests. 2020. Available online: https://osf.io/wdb79 (accessed on 9 July 2020). [CrossRef]
  8. Bock, C.H.; Barbedo, J.G.A.; Del Ponte, E.M.; Bohnenkamp, D.; Mahlein, A.-K. From visual estimates to fully automated sensor-based measurements of plant disease severity: Status and challenges for improving accuracy. Phytopathol. Res. 2020, 2, 1–30. [Google Scholar] [CrossRef] [Green Version]
  9. Toffolatti, S.L.; Maddalena, G.; Salomoni, D.; Maghradze, D.; Bianco, P.A.; Failla, O. Evidence of resistance to the downy mildew agent Plasmopara viticola in the Georgian Vitis vinifera germplasm. Vitis -J. Grapevine Res. 2016, 55, 121–128. [Google Scholar] [CrossRef]
  10. Peressotti, E.; Duchêne, E.; Merdinoglu, D.; Mestre, P. A semi-automatic non-destructive method to quantify grapevine downy mildew sporulation. J. Microbiol. Methods 2011, 84, 265–271. [Google Scholar] [CrossRef] [PubMed]
  11. Nagi, R.; Tripathy, S.S. Infected Area Segmentation and Severity Estimation of Grapevine Using Fuzzy Logic. In Advances in Intelligent Systems and Computing; Springer: Berlin/Heidelberg, Germany, 2020; Volume 988, pp. 57–67. [Google Scholar]
  12. Bock, C.H.; Parker, P.E.; Cook, A.Z.; Gottwald, T.R. Visual Rating and the Use of Image Analysis for Assessing Different Symptoms of Citrus Canker on Grapefruit Leaves. Plant Dis. 2008, 92, 530–541. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Pizer, S.M.; Amburn, E.P.; Austin, J.D.; Cromartie, R.; Geselowitz, A.; Greer, T.; Romeny, B.T.H.; Zimmerman, J.B.; Zuiderveld, K. Adaptive histogram equalization and its variations. Comput. Vision Graph. Image Process. 1987, 39, 355–368. [Google Scholar] [CrossRef]
  14. Yuen, H.; Princen, J.; Illingworth, J.; Kittler, J. Comparative study of Hough Transform methods for circle finding. Image Vis. Comput. 1990, 8, 71–77. [Google Scholar] [CrossRef] [Green Version]
  15. Liao, P.S.; Chen, T.S.; Chung, P.C. A fast algorithm for multilevel thresholding. J. Inf. Sci. Eng. 2001, 17, 713–727. [Google Scholar] [CrossRef]
  16. Savitzky, A.; Golay, M.J.E. Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Anal. Chem. 1964, 36, 1627–1639. [Google Scholar] [CrossRef]
  17. Vincent, L.; Soille, P. Watersheds in digital spaces: An efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 1991, 13, 583–598. [Google Scholar] [CrossRef] [Green Version]
  18. Wang, G.; Sun, Y.; Wang, J. Automatic Image-Based Plant Disease Severity Estimation Using Deep Learning. Comput. Intell. Neurosci. 2017, 2017, 2917536. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Hernandez, I.; Gutierrez, S.; Barrio, I.; Toffolatti, S.; Taradaguila, J. Computer Vision and Fuzzy Logic for Evaluating Downy Mildew Severity in Grapevine. 2021; Submitted. [Google Scholar]
Figure 1. Flowchart of the use of novel sensing technologies and artificial intelligence for assessing downy mildew in grapevine leaves: RGB images taken under laboratory conditions (red boxes) were used for downy mildew severity estimation, and hyperspectral images (blue boxes) were used to detect this disease early.
Figure 1. Flowchart of the use of novel sensing technologies and artificial intelligence for assessing downy mildew in grapevine leaves: RGB images taken under laboratory conditions (red boxes) were used for downy mildew severity estimation, and hyperspectral images (blue boxes) were used to detect this disease early.
Horticulturae 07 00103 g001
Figure 2. Flowchart of the use of computer vision for assessing downy mildew severity in grapevine leaves under laboratory conditions.
Figure 2. Flowchart of the use of computer vision for assessing downy mildew severity in grapevine leaves under laboratory conditions.
Horticulturae 07 00103 g002
Figure 3. Assessment of downy mildew severity in grapevine leaves by computer vision. (A) original RGB image, (B) processed image with quantified downy mildew sporulation in leaf discs.
Figure 3. Assessment of downy mildew severity in grapevine leaves by computer vision. (A) original RGB image, (B) processed image with quantified downy mildew sporulation in leaf discs.
Horticulturae 07 00103 g003
Figure 4. Raw spectra of non-infected and infected grapevine leaf discs from 9 DPI (days post inoculation) with downy mildew. The spectra shown summarise all the spectra of each group by their mean (line) and standard deviation (smoothed area).
Figure 4. Raw spectra of non-infected and infected grapevine leaf discs from 9 DPI (days post inoculation) with downy mildew. The spectra shown summarise all the spectra of each group by their mean (line) and standard deviation (smoothed area).
Horticulturae 07 00103 g004
Figure 5. Classification accuracy between control and downy mildew inoculated discs in grapevine after different DPI (days post inoculation) using different machine learning models: Convolutional Neural Network (CNN), k-Nearest Neighbour (KNN), MultiLayer Perceptron (MLP) and Partial Least Square-Discriminant Analysis (PLS-DA). For each DPI, each of the models was trained, using for training the disc spectra preprocessed with reflectance, Savitzky-Golay, a disc-level standardisation and discarding extreme wavelengths. The times of inoculation, washing and appearance of the first sporulation are indicated by vertical lines, separating the different DPIs.
Figure 5. Classification accuracy between control and downy mildew inoculated discs in grapevine after different DPI (days post inoculation) using different machine learning models: Convolutional Neural Network (CNN), k-Nearest Neighbour (KNN), MultiLayer Perceptron (MLP) and Partial Least Square-Discriminant Analysis (PLS-DA). For each DPI, each of the models was trained, using for training the disc spectra preprocessed with reflectance, Savitzky-Golay, a disc-level standardisation and discarding extreme wavelengths. The times of inoculation, washing and appearance of the first sporulation are indicated by vertical lines, separating the different DPIs.
Horticulturae 07 00103 g005
Table 1. Methods and results of severity estimation and early detection of downy mildew in grapevine. Results of the early detection method were selected from the 9th day post inoculation (DPI).
Table 1. Methods and results of severity estimation and early detection of downy mildew in grapevine. Results of the early detection method were selected from the 9th day post inoculation (DPI).
AimImage TypeTechniquesMethod Information
Training TimeTesting TimeResultsDetection Model
Severity estimationRGBComputer visionNoneMiddle-LowR2RMSE-
0.76 **20.53%-
Early detectionHyperspectralHyperspectral preprocessing, computer vision and machine learningMiddleLowAccuracy (%)F1-scoreModel
HighLow820.81CNN
LowLow660.62KNN
MiddleLow810.80MLP
LowLow530.35PLS-DA
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Hernández, I.; Gutiérrez, S.; Ceballos, S.; Iñíguez, R.; Barrio, I.; Tardaguila, J. Artificial Intelligence and Novel Sensing Technologies for Assessing Downy Mildew in Grapevine. Horticulturae 2021, 7, 103. https://doi.org/10.3390/horticulturae7050103

AMA Style

Hernández I, Gutiérrez S, Ceballos S, Iñíguez R, Barrio I, Tardaguila J. Artificial Intelligence and Novel Sensing Technologies for Assessing Downy Mildew in Grapevine. Horticulturae. 2021; 7(5):103. https://doi.org/10.3390/horticulturae7050103

Chicago/Turabian Style

Hernández, Inés, Salvador Gutiérrez, Sara Ceballos, Rubén Iñíguez, Ignacio Barrio, and Javier Tardaguila. 2021. "Artificial Intelligence and Novel Sensing Technologies for Assessing Downy Mildew in Grapevine" Horticulturae 7, no. 5: 103. https://doi.org/10.3390/horticulturae7050103

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop