Development of a Recognition System for Spraying Areas from Unmanned Aerial Vehicles Using a Machine Learning Approach
Abstract
:1. Introduction
2. Materials and Methods
2.1. Mutual Subspace Method (MSM)
2.2. Research Design for Classifiers and MSM
2.3. Field Experiment for Training and Testing with Datasets
2.4. Offline Recognition System
2.5. Online Recognition System
3. Results
3.1. Offline Recognition Performance
3.2. Online Recognition Performance
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Zhang, Y.; Chen, D.; Wang, S.; Tian, L. A promising trend for field information collection: An air-ground multi-sensor monitoring system. Inf. Process. Agric. 2018, 5, 224–233. [Google Scholar] [CrossRef]
- Peña Barragán, J.M.; Kelly, M.; Castro, A.I.D.; López Granados, F. Object-based approach for crop row characterization in UAV images for site-specific weed management. In Proceedings of the 4th Geographic Object-Based Image Analysis (GEOBIA), Rio de Janeiro, Brazil, 7–12 May 2012; p. 426. [Google Scholar]
- Salvatore, M.; Matthew, F.M.; Pauline, E.M.; Richard, L.; Victor, P.M.; Giorgos, M.; Eyal, B.D.; David, H.; Lyndon, E.; Giuseppe, C.; et al. On the use of unmanned aerial systems for environmental monitoring. Remote Sens. 2018, 10, 641. [Google Scholar]
- Bandini, F.; Jakobsen, J.; Olesen, D.; Reyna-Gutierrez, J.A.; Bauer-Gottwein, P. Measuring water level in rivers and lakes from lightweight unmanned aerial vehicles. J. Hydrol. 2017, 548, 237–250. [Google Scholar] [CrossRef]
- Zarco-Tejada, P.J.; González-Dugo, V.; Berni, J.A. Fluorescence, temperature and narrow-band indices acquired from a UAV platform for water stress detection using a micro-hyperspectral imager and a thermal camera. Remote Sens. Environ. 2012, 117, 322–337. [Google Scholar] [CrossRef] [Green Version]
- Hung, C.; Xu, Z.; Sukkarieh, S. Feature learning based approach for weed classification using high resolution aerial images from a digital camera mounted on a UAV. Remote Sens. 2014, 6, 12037–12054. [Google Scholar] [CrossRef]
- Rebetez, J.; Satizábal, H.F.; Mota, M.; Noll, D.; Büchi, L.; Wendling, M.; Cannelle, B.; Pérez-Uribe, A.; Burgos, S. Augmenting a convolutional neural network with local histograms—A case study in crop classification from high-resolution UAV imagery. In Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium, 27–29 April 2016; pp. 27–29. [Google Scholar]
- Peteinatos, G.; Weis, M.; Andújar, D.; Rueda Ayala, V.; Gerhards, R. Potential use of ground-based sensor technologies for weed detection. Pest Manag. Sci. 2014, 70, 190–199. [Google Scholar] [CrossRef] [PubMed]
- Lee, D.H.; Lee, K.S.; Cho, Y.J.; Lee, J.Y.; Chung, S.-O. Development of autonomous sprayer considering tracking performance on geometrical complexity of ground in greenhouse. J. Biosyst. Eng. 2012, 37, 287–295. [Google Scholar] [CrossRef]
- Pimentel, D.; Burgess, M. Small amounts of pesticides reaching target insects. Environ. Dev. Sustain. 2011, 14, 1–2. [Google Scholar] [CrossRef] [Green Version]
- Berni, J.; Zarco-Tejada, P.; Suárez, L.; González-Dugo, V.; Fereres, E. Remote sensing of vegetation from UAV platforms using lightweight multispectral and thermal imaging sensors. Remote Sens. 2009, 38, 722–738. [Google Scholar] [CrossRef]
- Tauro, F.; Selker, J.; van de Giesen, N.; Abrate, T.; Uijlenhoet, R.; Porfiri, M.; Manfreda, S.; Caylor, K.; Moramarco, T.; Benveniste, J. Measurements and observations in the XXI century (MOXXI): Innovation and multi-disciplinarity to sense the hydrological cycle. Hydrol. Sci. J. 2018, 63, 169–196. [Google Scholar] [CrossRef]
- Tauro, F.; Porfiri, M.; Grimaldi, S. Surface flow measurements from drones. J. Hydrol. 2016, 540, 240–245. [Google Scholar] [CrossRef] [Green Version]
- Berni, J.A.; Zarco-Tejada, P.J.; Suárez Barranco, M.D.; Fereres Castiel, E. Thermal and narrow-band multispectral remote sensing for vegetation monitoring from an unmanned aerial vehicle. Inst. Electr. Electron. Eng. 2009, 47, 722–738. [Google Scholar]
- Ludovisi, R.; Tauro, F.; Salvati, R.; Khoury, S.; Mugnozza Scarascia, G.; Harfouche, A. UAV-based thermal imaging for high-throughput field phenotyping of black poplar response to drought. Front. Plant Sci. 2017, 8, 1681. [Google Scholar] [CrossRef]
- Carrio, A.; Sampedro, C.; Rodriguez-Ramos, A.; Campoy, P. A review of deep learning methods and applications for unmanned aerial vehicles. J. Sens. 2017. [Google Scholar] [CrossRef]
- Majidi, B.; Bab-Hadiashar, A. Real time aerial natural image interpretation for autonomous ranger drone navigation. In Proceedings of the IEEE Digital Image Computing: Techniques and Applications (DICTA’05), Queensland, Australia, 6–8 December 2005. [Google Scholar]
- Tellaeche, A.; Burgos-Artizzu, X.P.; Pajares, G.; Ribeiro, A. A vision-based method for weeds identification through the Bayesian decision theory. Pattern Recognit. 2008, 41, 521–530. [Google Scholar] [CrossRef]
- Yan, Z.; Pengbo, G.; Tofael, A. Development of a rescue system for agricultural machinery operators using machine vision. Biosyst. Eng. 2018, 169, 149–164. [Google Scholar]
- Sakano, H.; Mukawa, N.; Nakamura, T. Kernel mutual subspace method and its application for object recognition. Electron. Commun. Jpn. (Part II Electron.) 2005, 88, 45–53. [Google Scholar] [CrossRef]
- Fukui, K.; Yamaguchi, O. Face recognition using multi-viewpoint patterns for robot vision. In Proceedings of the Eleventh International Symposium on Robotics Research, ISRR, Siena, Italy, 19–22 October 2003; Springer: Berlin/Heidelberg, Germany, 2005; pp. 192–201. [Google Scholar]
- Fukui, K.; Yamaguchi, O. The kernel orthogonal mutual subspace method and its application to 3D object recognition. In Proceedings of the Asian Conference on Computer Vision, Tokyo, Japan, 18–22 November 2007; Springer: Berlin/Heidelberg, Germany, 2007; pp. 467–476. [Google Scholar]
- Maeda, K.; Watanabe, S. Pattern matching method with local structure. Trans. IEICE 1985, 68, 345–352. [Google Scholar]
- Watanabe, S.; Pakvasa, N. Subspace method of pattern recognition. In Proceedings of the 1st IJCPR, Washington, DC, USA, Novermber 1973; pp. 25–32. [Google Scholar]
- Iijima, T.; Genchi, H.; Mori, K.-I. A theory of character recognition by pattern matching method. In Learning Systems and Intelligent Robots; Springer: Boston, MA, USA, 1974; pp. 437–450. [Google Scholar]
- Yamaguchi, O.; Fukui, K.; Maeda, K.-I. Face recognition using temporal image sequence. In Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 14–16 April 1998; p. 318. [Google Scholar]
- Fukui, K.; Yamaguchi, O. Subspace methods. In Computer Vision; Springer: Berlin/Heidelberg, Germany, 2014; pp. 777–781. [Google Scholar]
- Scholkopf, B.; Smola, A.; Uuller, K.-R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 1998, 10, 1299–1319. [Google Scholar] [CrossRef]
Targets | Data Sets | Training Image Numbers | Testing Image Numbers | |||
---|---|---|---|---|---|---|
Spray | Nonspray | Offline (Spray + Nonspray) | Online (Spray + Nonspray) | Offline (Spray + Nonspray) | Online | |
Carrot | 120 | 120 | First half (60 + 60) | All (120 + 120) | Last half (60 + 60) | New video (89) |
Cabbage | 198 | 198 | First half (99 + 99) | All (198 + 198) | Last half (99 + 99) | New video (298) |
Onion | 107 | 107 | First half (53 + 53) | All (107 + 107) | Last half (54 + 54) | New video (204) |
Chestnut | 97 | 97 | First half (48 + 48) | All (97 + 97) | Last half (49 + 49) | New video (180) |
Persimmon | 94 | 94 | First half (47 + 47) | All (94 + 94) | Last half (47 + 47) | New video (210) |
Trees and Structures | 118 | 118 | First half (59 + 59) | All (118 + 118) | Last half (59 + 59) | New video (141) |
True condition (offline recognition) | ||||
Spray | Nonspray | ∑Total | ||
Predicted condition (tested by recognition phase) | Spray | True Positive | False Positive | Total Positive |
Nonspray | False Negative | True Negative | Total Negative | |
Accuracy |
True condition (offline recognition) | ||||||
Location (Croplands, Orchards) | Work patterns | Cropland | Orchard | |||
Classifiers | Spray | Nonspray | Spray | Nonspray | ||
Predicted condition (Tested by the recognition phase) | L1 | Spray | 74 | 21 | 35 | 9 |
Nonspray | 16 | 79 | 13 | 31 | ||
Accuracy | 80.5% | 75% | ||||
L2 | Spray | 38 | 11 | 41 | 2 | |
Nonspray | 18 | 31 | 10 | 33 | ||
Accuracy | 70.4% | 86.1% | ||||
L3 | Spray | 56 | 0 | 37 | 18 | |
Nonspray | 31 | 25 | 15 | 40 | ||
Accuracy | 72.3% | 70% |
Croplands and Orchards | Data Sets | Training Image Numbers | Testing Image Numbers | Accuracy | |
---|---|---|---|---|---|
Spray | Nonspray | Offline | Offline | ||
Carrot | 256 | 256 | First half (128 + 128) | Last half (128 + 128) | 73.79% |
Cabbage | 440 | 440 | First half (220 + 220) | Last half (220 + 220) | 81.25% |
Onion | 210 | 210 | First half (105 + 105) | Last half (105 + 105) | 66.32% |
Chestnut | 224 | 224 | First half (112 + 112) | Last half (112 + 112) | 77.31% |
Persimmon | 248 | 248 | First half (124 + 124) | Last half (124 + 124) | 70.94% |
Trees and Structures | 216 | 216 | First half (108 + 108) | Last half (108 + 108) | 64.58% |
Croplands and Orchards | Flying Height (m) | Accuracy (%) | Recognition Time of Classifier (s) |
---|---|---|---|
Carrot | 5 | 65.51 | 0.0031 |
Cabbage | 5 | 60.88 | 0.0048 |
Onion | 5 | 69.00 | 0.0031 |
Chestnut | 15 | 69.10 | 0.0031 |
Persimmon | 15 | 82.21 | 0.0031 |
Trees and Structures | 15 | 74.10 | 0.0031 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gao, P.; Zhang, Y.; Zhang, L.; Noguchi, R.; Ahamed, T. Development of a Recognition System for Spraying Areas from Unmanned Aerial Vehicles Using a Machine Learning Approach. Sensors 2019, 19, 313. https://doi.org/10.3390/s19020313
Gao P, Zhang Y, Zhang L, Noguchi R, Ahamed T. Development of a Recognition System for Spraying Areas from Unmanned Aerial Vehicles Using a Machine Learning Approach. Sensors. 2019; 19(2):313. https://doi.org/10.3390/s19020313
Chicago/Turabian StyleGao, Pengbo, Yan Zhang, Linhuan Zhang, Ryozo Noguchi, and Tofael Ahamed. 2019. "Development of a Recognition System for Spraying Areas from Unmanned Aerial Vehicles Using a Machine Learning Approach" Sensors 19, no. 2: 313. https://doi.org/10.3390/s19020313