Weed Classification for Site-Specific Weed Management Using an Automated Stereo Computer-Vision Machine-Learning System in Rice Fields
Abstract
:1. Introduction
2. Materials and Methods
2.1. Plant Material
2.2. Video Data Acquisition
2.3. Pre-Processing and Segmentation
2.4. Feature Extraction
2.5. Effective Feature Selection
2.6. Classification
- High computation speed;
- Ability to efficiently handle noisy inputs;
- Data-driven nature, thanks to learning from the training data.
2.7. Proposed System for the Classification of Rice and Weed Plants Inside Rice Fields
2.8. Arithmetic and Geometric Means
3. Results and Discussion
3.1. Effective Feature Extraction with ANN-PSO
3.2. Classification Using Hybrid Metaheuristic Algorithms
3.2.1. Classification Using Hybrid ANN-BA
3.2.2. Classification Using KNN Classifier
3.2.3. Classification Performance Evaluation by receiver operating characteristic (ROC) Curves
4. Conclusions
Supplementary Materials
Author Contributions
Funding
Conflicts of Interest
References
- Surendrababu, V.; Sumathi, C.; Umapathy, E. Detection of rice leaf diseases using chaos and fractal dimension in image processing. Int. J. Comput. Sci. Eng. 2014, 6, 69. [Google Scholar]
- Asif, M.; Iqbal, M.; Randhawa, H.; Spaner, D. Managing and Breeding Wheat for Organic Systems: Enhancing Competitiveness Against Weeds; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
- Monaco, T.; Weller, C.; Ashton, F. Weed Science Principles and Practices; Jhon Wiley & Sons Inc.: Hoboken, NJ, USA, 2002. [Google Scholar]
- Shinde, A.K.; Shukla, M.Y. Crop detection by machine vision for weed management. Int. J. Adv. Eng. Technol. 2014, 7, 818. [Google Scholar]
- Zheng, Y.; Zhu, Q.; Huang, M.; Guo, Y.; Qin, J. Maize and weed classification using color indices with support vector data description in outdoor fields. Comput. Electron. Agric. 2017, 141, 215–222. [Google Scholar] [CrossRef]
- Oerke, E.-C. Crop losses to pests. J. Agric. Sci. 2006, 144, 31–43. [Google Scholar] [CrossRef]
- Zwerger, P.; Malkomes, H.; Nordmeyer, H.; Söchting, H.; Verschwele, A. Unkrautbekämpfung: Gegenwart und Zukunft–aus deutscher Sicht. Z. Für Pflanzenkrankh. Und PflanzenschutzSonderh. 2004, 19, 27–38. [Google Scholar]
- Choi, K.H.; Han, S.K.; Park, K.-H.; Kim, K.-S.; Kim, S. Vision based guidance line extraction for autonomous weed control robot in paddy field. In Proceedings of the 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China, 6–9 December 2015; pp. 831–836. [Google Scholar]
- Nakai, S.; Yamada, Y. Development of a weed suppression robot for rice cultivation: Weed suppression and posture control. Int. J. Electr. Comput. Electron. Commun. Eng. 2014, 8, 1736–1740. [Google Scholar]
- Cordill, C.; Grift, T.E. Design and testing of an intra-row mechanical weeding machine for corn. Biosyst. Eng. 2011, 110, 247–252. [Google Scholar] [CrossRef]
- Tillett, N.; Hague, T.; Grundy, A.; Dedousis, A. Mechanical within-row weed control for transplanted crops using computer vision. Biosyst. Eng. 2008, 99, 171–178. [Google Scholar] [CrossRef]
- Pallutt, B.; Moll, E. Long-term effects of reduced herbicide doses on weed infestation and grain yield of winter cereals in a 12-year long-term trial. J. Plant Dis. Prot. 2008, 21, 501–508. [Google Scholar]
- Tang, J.-L.; Chen, X.-Q.; Miao, R.-H.; Wang, D. Weed detection using image processing under different illumination for site-specific areas spraying. Comput. Electron. Agric. 2016, 122, 103–111. [Google Scholar] [CrossRef]
- Gianessi, L.P. The increasing importance of herbicides in worldwide crop production. Pest Manag. Sci. 2013, 69, 1099–1105. [Google Scholar] [CrossRef] [PubMed]
- Keller, M.; Böhringer, N.; Möhring, J.; Rueda-Ayala, V.; Gutjahr, C.; Gerhards, R. Changes in weed communities, herbicides, yield levels and effect of weeds on yield in winter cereals based on three decades of field experiments in South-Western Germany. Gesunde Pflanz. 2015, 67, 11–20. [Google Scholar] [CrossRef]
- Keller, M.; Böhringer, N.; Möhring, J.; Rueda-Ayala, V.; Gutjahr, C.; Gerhards, R. Long-term changes in weed occurrence, yield and use of herbicides in maize in south-western G ermany, with implications for the determination of economic thresholds. Weed Res. 2014, 54, 457–466. [Google Scholar] [CrossRef]
- Mayerová, M.; Madaras, M.; Soukup, J. Effect of chemical weed control on crop yields in different crop rotations in a long-term field trial. Crop Prot. 2018, 114, 215–222. [Google Scholar] [CrossRef]
- Cho, S.; Lee, D.; Jeong, J. AE—automation and emerging technologies: Weed–plant discrimination by machine vision and artificial neural network. Biosyst. Eng. 2002, 83, 275–280. [Google Scholar] [CrossRef]
- Berge, T.; Goldberg, S.; Kaspersen, K.; Netland, J. Towards machine vision based site-specific weed management in cereals. Comput. Electron. Agric. 2012, 81, 79–86. [Google Scholar] [CrossRef]
- Dos Santos Ferreira, A.; Freitas, D.M.; da Silva, G.G.; Pistori, H.; Folhes, M.T. Weed detection in soybean crops using ConvNets. Comput. Electron. Agric. 2017, 143, 314–324. [Google Scholar] [CrossRef]
- Pantazi, X.-E.; Moshou, D.; Bravo, C. Active learning system for weed species recognition based on hyperspectral sensing. Biosyst. Eng. 2016, 146, 193–202. [Google Scholar] [CrossRef]
- Yu, J.; Sharpe, S.M.; Schumann, A.W.; Boyd, N.S. Deep learning for image-based weed detection in turfgrass. Eur. J. Agron. 2019, 104, 78–84. [Google Scholar] [CrossRef]
- Gerhards, R.; Oebel, H. Practical experiences with a system for site-specific weed control in arable crops using real-time image analysis and GPS-controlled patch spraying. Weed Res. 2006, 46, 185–193. [Google Scholar] [CrossRef]
- Gibson, K.D.; Dirks, R.; Medlin, C.R.; Johnston, L. Detection of weed species in soybean using multispectral digital images. Weed Technol. 2004, 18, 742–749. [Google Scholar] [CrossRef]
- Onyango, C.M.; Marchant, J. Segmentation of row crop plants from weeds using colour and morphology. Comput. Electron. Agric. 2003, 39, 141–155. [Google Scholar] [CrossRef]
- Sabzi, S.; Abbaspour-Gilandeh, Y. Using video processing to classify potato plant and three types of weed using hybrid of artificial neural network and partincle swarm algorithm. Measurement 2018, 126, 22–36. [Google Scholar] [CrossRef]
- Sabzi, S.; Abbaspour-Gilandeh, Y.; García-Mateos, G. A fast and accurate expert system for weed identification in potato crops using metaheuristic algorithms. Comput. Ind. 2018, 98, 80–89. [Google Scholar] [CrossRef]
- Søgaard, H.T.; Olsen, H.J. Determination of crop rows by image analysis without segmentation. Comput. Electron. Agric. 2003, 38, 141–158. [Google Scholar] [CrossRef]
- Sujaritha, M.; Annadurai, S.; Satheeshkumar, J.; Sharan, S.K.; Mahesh, L. Weed detecting robot in sugarcane fields using fuzzy real time classifier. Comput. Electron. Agric. 2017, 134, 160–171. [Google Scholar] [CrossRef]
- Yang, J.; Zhu, L. Color image segmentation method based on RGB color space. Comput. Mod. 2010, 8, 147–149. [Google Scholar]
- Nguyen, M.L.; Ciesielski, V.; Song, A. Rice leaf detection with genetic programming. In Proceedings of the 2013 IEEE Congress on Evolutionary Computation, Cancun, Mexico, 20–23 June 2013; pp. 1146–1153. [Google Scholar]
- Partel, V.; Kakarla, S.C.; Ampatzidis, Y. Development and evaluation of a low-cost and smart technology for precision weed management utilizing artificial intelligence. Comput. Electron. Agric. 2019, 157, 339–350. [Google Scholar] [CrossRef]
- Bakhshipour, A.; Jafari, A. Evaluation of support vector machine and artificial neural networks in weed detection using shape features. Comput. Electron. Agric. 2018, 145, 153–160. [Google Scholar] [CrossRef]
- Hamuda, E.; Mc Ginley, B.; Glavin, M.; Jones, E. Automatic crop detection under field conditions using the HSV colour space and morphological operations. Comput. Electron. Agric. 2017, 133, 97–107. [Google Scholar] [CrossRef]
- Chen, C.; Zheng, Y.F. Passive and active stereo vision for smooth surface detection of deformed plates. IEEE Trans. Ind. Electron. 1995, 42, 300–306. [Google Scholar] [CrossRef]
- Jin, J.; Tang, L. Corn plant sensing using real-time stereo vision. J. Field Robot. 2009, 26, 591–608. [Google Scholar] [CrossRef]
- Trucco, E.; Verri, A. Introductory Techniques for 3-D Computer Vision; Prentice Hall: Englewood Cliffs, NJ, USA, 1998; Volume 201. [Google Scholar]
- Jeon, H.Y.; Tian, L.F.; Zhu, H. Robust crop and weed segmentation under uncontrolled outdoor illumination. Sensors 2011, 11, 6270–6283. [Google Scholar] [CrossRef] [PubMed]
- Tilneac, M.; Dolga, V.; Grigorescu, S.; Bitea, M. 3D stereo vision measurements for weed-crop discrimination. Elektron. Ir Elektrotechnika 2012, 123, 9–12. [Google Scholar] [CrossRef] [Green Version]
- Meier, U. Growth Stages of Mono-and Dicotyledonous Plants; Blackwell Wissenschafts: Berlin, Germany, 1997. [Google Scholar]
- Muangkasem, A.; Thainimit, S.; Keinprasit, R.; Isshiki, T.; Tangwongkit, R. Weed detection over between-row of sugarcane fields using machine vision with shadow robustness technique for variable rate herbicide applicator. Energy Res. J. 2010, 1, 141–145. [Google Scholar] [CrossRef] [Green Version]
- Hernández-Hernández, J.; García-Mateos, G.; González-Esquiva, J.; Escarabajal-Henarejos, D.; Ruiz-Canales, A.; Molina-Martínez, J.M. Optimal color space selection method for plant/soil segmentation in agriculture. Comput. Electron. Agric. 2016, 122, 124–132. [Google Scholar] [CrossRef]
- Gonzalez, R.C.; Eddins, S.L.; Woods, R.E. Digital Image Publishing Using MATLAB; Prentice Hall: Upper Saddle River, NJ, USA, 2004. [Google Scholar]
- Bakhshipour, A.; Jafari, A.; Nassiri, S.M.; Zare, D. Weed segmentation using texture features extracted from wavelet sub-images. Biosyst. Eng. 2017, 157, 1–12. [Google Scholar] [CrossRef]
- Malemath, V.; Hugar, S. A new approach for weed detection in agriculture using image processing techniques. Int. J. Adv. Sci. Tech. Res. 2016, 3, 356–359. [Google Scholar]
- Kazmi, W.; Garcia-Ruiz, F.J.; Nielsen, J.; Rasmussen, J.; Andersen, H.J. Detecting creeping thistle in sugar beet fields using vegetation indices. Comput. Electron. Agric. 2015, 112, 10–19. [Google Scholar] [CrossRef] [Green Version]
- Kazmi, W.; Garcia-Ruiz, F.; Nielsen, J.; Rasmussen, J.; Andersen, H.J. Exploiting affine invariant regions and leaf edge shapes for weed detection. Comput. Electron. Agric. 2015, 118, 290–299. [Google Scholar] [CrossRef]
- Swain, K.C.; Nørremark, M.; Jørgensen, R.N.; Midtiby, H.S.; Green, O. Weed identification using an automated active shape matching (AASM) technique. Biosyst. Eng. 2011, 110, 450–457. [Google Scholar] [CrossRef]
- Bonev, B.; Escolano, F.; Cazorla, M.A. A novel information theory method for filter feature selection. In Proceedings of the Mexican International Conference on Artificial Intelligence, Aguascalientes, Mexico, 4–10 November 2007; pp. 431–440. [Google Scholar]
- Kavdır, İ. Discrimination of sunflower, weed and soil by artificial neural networks. Comput. Electron. Agric. 2004, 44, 153–160. [Google Scholar] [CrossRef]
- Moshou, D.; Vrindts, E.; De Ketelaere, B.; De Baerdemaeker, J.; Ramon, H. A neural network based plant classifier. Comput. Electron. Agric. 2001, 31, 5–16. [Google Scholar] [CrossRef]
- Singh, A.K.; Rubiya, A.; Raja, B. Classification of rice disease using digital image processing and svm classifier. Int. J. Electr. Electron. Eng. 2015, 7, 294–299. [Google Scholar]
- Eberhart, R.C.; Shi, Y.; Kennedy, J. Swarm Intelligence; Elsevier: Amsterdam, The Netherlands, 2001. [Google Scholar]
- Seeley, T.D. The Wisdom of the Hive: The Social Physiology of Honey bee Colonies; Harvard University Press: Cambridge, MA, USA, 2009. [Google Scholar]
- Camazine, S.; Deneubourg, J.-L.; Franks, N.R.; Sneyd, J.; Bonabeau, E.; Theraula, G. Self-Organization in Biological Systems; Princeton University Press: Princeton, NJ, USA, 2003. [Google Scholar]
- Zhang, S.; Li, X.; Zong, M.; Zhu, X.; Wang, R. Efficient knn classification with different numbers of nearest neighbors. IEEE Trans. Neural Netw. Learn. Syst. 2017, 29, 1774–1785. [Google Scholar] [CrossRef] [PubMed]
- Schmitz, D.; Loss, A.; Oliveira, B.S.; Lovato, P.E.; Veiga, M.d.; Brunetto, G.; Curmi, P.; Comin, J.J. Physical attributes of Cambisol in apple (Malus domestica Borkh. cv. Gala) orchard under different weed management systems in Urubici-Santa Catarina, Brazil. Acta Agronómica 2017, 66, 414–421. [Google Scholar] [CrossRef]
- Sabzi, S.; Gilandeh, Y.A.; Javadikia, H. Developing a Machine Vision System to Detect Weeds from Potato Plant. J. Agric. Sci. 2018, 24, 105–118. [Google Scholar] [CrossRef] [Green Version]
MLP Parameters | PSO Parameters |
---|---|
One input layer | Swarm size: 30 |
One hidden layer with 10 neurons | Maximum iteration: 20 |
One output layer with 3 outputs. | Inertia weight damping ratio: 1 |
Classic Levenberg–Marquardt training function | Maximum variation size: 1 |
Minimum variation size: 0 | |
Inertia rate: 1 | |
Velocity Maximum value: | |
0.1×(VarMax-VarMin) | |
Velocity minimum value: -VelMax |
Category | Selected Effective Features | |||||
---|---|---|---|---|---|---|
Left channel | EXY-YIQ | Elongation Feature | Cluster Prominence-45 | Rn | Inverse Difference-45 | Entropy-45 |
Right channel | Convexity | ExG-RGB | CIVE-HSV | Cluster shade-90 | CIVE-RGB | Difference entropy-0 |
Arithmetic mean | Sum entropy-0 | Information measure of correlation-0 | CIVE-RGB | Autocollelation-90 | Coefficient of variation--90 | WL |
Geometric mean | Inverse difference normalized-135 | WL | CMP | Std-Cb | Entropy | ExM-CMYYY |
Description | Selected Feature Name |
---|---|
Excess yellow from YIQ color space | EXY-YIQ |
Elongation feature = (L − W)/(L + W) L = length and W = width | Elongation feature |
Clumster prominence = ΣiΣj(i + j − μi − μj)4Ng(i,j) Ng = (The normalized co-occurrence matrix) | Cluster prominence |
Rn = R/(R + G + B), (The normalized first component of RGB) | Rn |
Inverse Difference = | Inverse Difference |
Entropy = −ΣΣNg(i,j)log2 Ng(i,j) | Entropy |
A measure of the curvature | Convexity |
ExG-RGB = 2 × Gn − Rn − Bn, (Excess green) | ExG-RGB |
Color index for extracted vegetation cover in HSV color space | CIVE-HSV |
cluster Shade = ΣΣ(i + j − μi − μj)3Ng(i,j) | cluster Shade |
CIVE-RGB = 0.441 × Rn − 0.811 × Gn + 0.385 × Bn + 18.78 (Color index for extracted vegetation cover) | CIVE-RGB |
Difference entropy = −Σpx−y(i) ln [px−y(i)], px-y(k) = | Difference entropy |
Sum Entropy = −Σpx+y(i)log(px;+y(i)) px+y(k) = | Sum Entropy |
IMC = HXY1 = Nx(i) = , Ny(i) = , HX: Entropy of Nx and Hy: Entropy of Ny | Information measure of correlation |
Autocorrelation = ΣΣ(ij)Ng(i,j) | Autocorrelation |
Standard deviation to mean of co-occurrence matrix | Coefficient of variation |
WL = Width/Length | WL |
IDN = | Inverse difference normalized |
CMP = (Compression) A:area, p:perimeter | CMP |
Standard deviation of Cb from YCbCr color space | Std-Cb |
Excess magenta From CMY color space | ExM-CMYYY |
Number of Hidden Layers | Number of Neurons | Transfer Function | Back Propagation Network Training Function | Back Propagation Weight/Bias Learning Function |
---|---|---|---|---|
2 | First layer: 20 Second layer: 12 | First layer: tansig Second layer: satlins | trainrp | learngd |
Left channel | Rice | Narrow-leaf weeds | Wide-leaf weeds |
---|---|---|---|
Rice | 89 | 6 | 2 |
Narrow-leaf weeds | 12 | 67 | 6 |
Wide-leaf weeds | 2 | 1 | 56 |
Accuracy = 87.96% | |||
Right channel | Rice | Narrow-leaf weeds | Wide-leaf weeds |
Rice | 86 | 3 | 1 |
Narrow-leaf weeds | 6 | 73 | 10 |
Wide-leaf weeds | 2 | 4 | 46 |
Accuracy = 88.74% | |||
Arithmetic mean | Rice | Narrow-leaf weeds | Wide-leaf weeds |
Rice | 91 | 5 | 1 |
Narrow-leaf weeds | 6 | 69 | 2 |
Wide-leaf weeds | 1 | 3 | 48 |
Accuracy = 92.02% | |||
Geometric mean | Rice | Narrow-leaf weeds | Wide-leaf weeds |
Rice | 91 | 6 | 0 |
Narrow-leaf weeds | 7 | 67 | 3 |
Wide-leaf weeds | 3 | 2 | 47 |
Accuracy = 90.70% |
Left channel | Rice | Narrow-leaf weeds | Wide-leaf weeds |
---|---|---|---|
Rice | 83 | 8 | 6 |
Narrow-leaf weeds | 10 | 65 | 10 |
Wide-leaf weeds | 0 | 0 | 59 |
Accuracy = 85.89% | |||
Right channel | Rice | Narrow-leaf weeds | Wide-leaf weeds |
Rice | 65 | 19 | 6 |
Narrow-leaf weeds | 17 | 60 | 12 |
Wide-leaf weeds | 0 | 0 | 52 |
Accuracy = 76.62% | |||
Arithmetic mean | Rice | Narrow-leaf weeds | Wide-leaf weeds |
Rice | 83 | 8 | 6 |
Narrow-leaf weeds | 11 | 62 | 4 |
Wide-leaf weeds | 3 | 0 | 49 |
Accuracy = 85.84% | |||
Geometric mean | Rice | Narrow-leaf weeds | Wide-leaf weeds |
Rice | 78 | 15 | 4 |
Narrow-leaf weeds | 12 | 60 | 5 |
Wide-leaf weeds | 0 | 0 | 52 |
Accuracy = 84.07% |
Right Channel | |||||
---|---|---|---|---|---|
Hybrid ANN-BA | Mean | STD | KNN | Mean | STD |
Rice | 0.9446 | 0.0212 | Rice | 0.7224 | 0.0265 |
Narrow-leaf weeds | 0.8596 | 0.0314 | Narrow-leaf weeds | 0.6942 | 0.0272 |
Wide-leaf weeds | 0.9323 | 0.0289 | Wide-leaf weeds | 0.9004 | 0.0315 |
Left Channel | |||||
Hybrid ANN-BA | Mean | STD | KNN | Mean | STD |
Rice | 0.9100 | 0.0275 | Rice | 0.8256 | 0.0238 |
Narrow-leaf weeds | 0.8625 | 0.0275 | Narrow-leaf weeds | 0.7948 | 0.0273 |
Wide-leaf weeds | 0.9132 | 0.0376 | Wide-leaf weeds | 0.8961 | 0.0305 |
Arithmetic mean | |||||
Hybrid ANN-BA | Mean | STD | KNN | Mean | STD |
Rice | 0.9563 | 0.0165 | Rice | 0.8091 | 0.0240 |
Narrow-leaf weeds | 0.9330 | 0.0179 | Narrow-leaf weeds | 0.7993 | 0.0254 |
Wide-leaf weeds | 0.9653 | 0.0211 | Wide-leaf weeds | 0.9214 | 0.0272 |
Geometric mean | |||||
Hybrid ANN-BA | Mean | STD | KNN | Mean | STD |
Rice | 0.9414 | 0.0141 | Rice | 0.7745 | 0.0254 |
Narrow-leaf weeds | 0.9387 | 0.0169 | Narrow-leaf weeds | 0.7625 | 0.0258 |
Wide-leaf weeds | 0.9478 | 0.0200 | Wide-leaf weeds | 0.9493 | 0.0234 |
Hybrid ANN-BA | Rice Class | Narrow-Leaf Weeds Class | Wide-Leaf Weeds Class |
---|---|---|---|
Right Channel | 0.9886 | 0.9376 | 0.9561 |
Left Channel | 0.9462 | 0.9106 | 0.9483 |
Arithmetic mean | 0.9731 | 0.9635 | 0.9765 |
Geometric mean | 0.9668 | 0.9638 | 0.9747 |
KNN | Rice Class | Narrow-Leaf Weeds Class | Wide-Leaf Weeds Class |
---|---|---|---|
Right Channel | 0.8008 | 0.7702 | 0.9497 |
Left Cannel | 0.8931 | 0.8567 | 0.9560 |
Arithmetic mean | 0.8793 | 0.8758 | 0.9424 |
Geometric mean | 0.8556 | 0.8393 | 0.9742 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Dadashzadeh, M.; Abbaspour-Gilandeh, Y.; Mesri-Gundoshmian, T.; Sabzi, S.; Hernández-Hernández, J.L.; Hernández-Hernández, M.; Arribas, J.I. Weed Classification for Site-Specific Weed Management Using an Automated Stereo Computer-Vision Machine-Learning System in Rice Fields. Plants 2020, 9, 559. https://doi.org/10.3390/plants9050559
Dadashzadeh M, Abbaspour-Gilandeh Y, Mesri-Gundoshmian T, Sabzi S, Hernández-Hernández JL, Hernández-Hernández M, Arribas JI. Weed Classification for Site-Specific Weed Management Using an Automated Stereo Computer-Vision Machine-Learning System in Rice Fields. Plants. 2020; 9(5):559. https://doi.org/10.3390/plants9050559
Chicago/Turabian StyleDadashzadeh, Mojtaba, Yousef Abbaspour-Gilandeh, Tarahom Mesri-Gundoshmian, Sajad Sabzi, José Luis Hernández-Hernández, Mario Hernández-Hernández, and Juan Ignacio Arribas. 2020. "Weed Classification for Site-Specific Weed Management Using an Automated Stereo Computer-Vision Machine-Learning System in Rice Fields" Plants 9, no. 5: 559. https://doi.org/10.3390/plants9050559