Integration of Unmanned Aerial Vehicle Imagery and Machine Learning Technology to Map the Distribution of Conifer and Broadleaf Canopy Cover in Uneven-Aged Mixed Forests
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Site
2.2. Acquisition of UAV Imagery
2.3. Data Analysis
2.3.1. Processing of UAV Imagery
2.3.2. Labelling of Images
2.3.3. U-Net Algorithm
2.3.4. Random Forest Classification Algorithm
2.3.5. Training of the U-Net and RF Models
2.3.6. Evaluation Metrics for the Models
- TP (True Positives): Tree crowns are correctly predicted as positive.
- FP (False Positives): Tree crowns are falsely predicted as positive.
- FN (False Negatives): Tree crowns are falsely predicted as negative when, they are positive.
2.3.7. Algorithm
3. Results
3.1. Discrimination between Coniferous and Broadleaf Canopy Cover Using an Imbalanced UAV Dataset with Machine Learning (U-Net and RF) Models
3.2. Discrimination between Coniferous and Broadleaf Canopy Cover Using a Balanced UAV Dataset with Machine Learning (U-Net and RF) Models
3.3. Mapping the Distribution of Coniferous and Broadleaf Canopy Cover in Uneven-Aged Forests through Integration of UAV Imagery with Machine Learning Technology
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Fraser, B.T.; Congalton, R.G. A Comparison of Methods for Determining Forest Composition from High-Spatial-Resolution Remotely Sensed Imagery. Forests 2021, 12, 1290. [Google Scholar] [CrossRef]
- Korets, M.A.; Ryzhkova, V.A.; Danilova, I.V.; Prokushkin, A.S. Vegetation Cover Mapping Based on Remote Sensing and Digital Elevation Model Data. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.-ISPRS Arch. 2016, 41, 699–704. [Google Scholar] [CrossRef]
- Xie, Y.; Sha, Z.; Yu, M. Remote Sensing Imagery in Vegetation Mapping: A Review. J. Plant Ecol. 2008, 1, 9–23. [Google Scholar] [CrossRef]
- Kaplan, G. Broad-Leaved and Coniferous Forest Classification in Google Earth Engine Using Sentinel Imagery. Environ. Sci. Proc. 2021, 3, 64. [Google Scholar] [CrossRef]
- Onishi, M.; Ise, T. Explainable Identification and Mapping of Trees Using UAV RGB Image and Deep Learning. Sci. Rep. 2021, 11, 903. [Google Scholar] [CrossRef] [PubMed]
- Alonso, L.; Picos, J.; Armesto, J. Forest Cover Mapping and Pinus Species Classification Using Very High-Resolution Satellite Images and Random Forest. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2021, 5, 203–210. [Google Scholar] [CrossRef]
- Kalbi, S.; Fallah, A.; Shataee, S. Forest Stand Types Classification Using Tree-Based Algorithms and SPOT-HRG Data. Int. J. Environ. Resour. Res. 2014, 2, 263–278. [Google Scholar]
- Maturbongs, B.Y.L.; Wing, M.G.; Strîmbu, B.M.; Burnett, J. Forest Inventory Sensivity to UAS-Based Image Processing Algorithms. Ann. For. Res. 2019, 62, 87–108. [Google Scholar] [CrossRef]
- Wessel, M.; Brandmeier, M.; Tiede, D. Evaluation of Different Machine Learning Algorithms for Scalable Classification of Tree Types and Tree Species Based on Sentinel-2 Data. Remote Sens. 2018, 10, 1419. [Google Scholar] [CrossRef]
- Goodbody, T.R.H.; Coops, N.C.; Marshall, P.L.; Tompalski, P.; Crawford, P. Unmanned Aerial Systems for Precision Forest Inventory Purposes: A Review and Case Study. For. Chron. 2017, 93, 71–81. [Google Scholar] [CrossRef]
- Iglhaut, J.; Cabo, C.; Puliti, S.; Piermattei, L.; Connor, J.O.; Rosette, J.; Connor, J.O. Structure from Motion Photogrammetry in Forestry: A Review. Remote Sens. 2019, 5, 155–168. [Google Scholar] [CrossRef]
- Kotivuori, E.; Kukkonen, M.; Mehtätalo, L.; Maltamo, M.; Korhonen, L. Forest Inventories for Small Areas Using Drone Imagery without In-Situ Field Measurements. Remote Sens. Environ. 2020, 237, 111404. [Google Scholar] [CrossRef]
- Puliti, S.; Dash, J.P.; Watt, M.S.; Breidenbach, J.; Pearse, G.D. A Comparison of UAV Laser Scanning, Photogrammetry and Airborne Laser Scanning for Precision Inventory of Small-Forest Properties. Forestry 2020, 93, 150–162. [Google Scholar] [CrossRef]
- Jayathunga, S.; Owari, T.; Tsuyuki, S. The Use of Fixed—Wing UAV Photogrammetry with LiDAR DTM to Estimate Merchantable Volume and Carbon Stock in Living Biomass over a Mixed Conifer—Broadleaf Forest. Int. J. Appl. Earth Obs. Geoinf. 2018, 73, 767–777. [Google Scholar] [CrossRef]
- Velasquez, L.F.; Lleida, U.D.; Mohan, M.M. Remotely Sensed Tree Characterization in Urban Areas: A Review Remotely Sensed Tree Characterization in Urban Areas. Remote Sens. 2021, 13, 4889. [Google Scholar] [CrossRef]
- Zhang, Y.; Yang, W.; Sun, Y.; Chang, C.; Yu, J.; Zhang, W. Fusion of Multispectral Aerial Imagery and Vegetation Indices for Machine Learning-Based Ground Classification. Remote Sens. 2021, 13, 1411. [Google Scholar] [CrossRef]
- Tran, D.Q.; Park, M.; Jung, D.; Park, S. Damage-Map Estimation Using Uav Images and Deep Learning Algorithms for Disaster Management System. Remote Sens. 2020, 12, 4169. [Google Scholar] [CrossRef]
- Schiefer, F.; Kattenborn, T.; Frick, A.; Frey, J.; Schall, P.; Koch, B.; Schmidtlein, S. Mapping Forest Tree Species in High Resolution UAV-Based RGB-Imagery by Means of Convolutional Neural Networks. ISPRS J. Photogramm. Remote Sens. 2020, 170, 205–215. [Google Scholar] [CrossRef]
- Ahmed, I.; Ahmad, M.; Jeon, G. A Real-Time Efficient Object Segmentation System Based on U-Net Using Aerial Drone Images. J. Real Time Image Process. 2021, 18, 1745–1758. [Google Scholar] [CrossRef]
- Jayathunga, S.; Owari, T.; Tsuyuki, S. Evaluating the Performance of Photogrammetric Products Using Fixed-Wing UAV Imagery over a Mixed Conifer-Broadleaf Forest: Comparison with Airborne Laser Scanning. Remote Sens. 2018, 10, 187. [Google Scholar] [CrossRef]
- Moe, K.T.; Owari, T. Sustainability of High-Value Timber Species in Mixed Conifer—Broadleaf Forest Managed under Selection System in Northern Japan. Forests 2020, 11, 484. [Google Scholar] [CrossRef]
- Sivanandam, P.; Turner, D.; Lucieer, A. Drone Data Collection Protocol Using DJI Matrice 300 RTK: Imagery and Lidar. Univ. Tasman. Aust. Terr. Ecosyst. Res. Netw. (TERN) 2022, 1–58. Available online: https://www.tern.org.au/wp-content/uploads/20221103_M300_data_collection.pdf (accessed on 31 October 2023).
- Arzt, M.; Deschamps, J.; Schmied, C.; Pietzsch, T.; Schmidt, D.; Tomancak, P.; Haase, R.; Jug, F. LABKIT: Labeling and Segmentation Toolkit for Big Image Data. Front. Comput. Sci. 2022, 4, 10. [Google Scholar] [CrossRef]
- Pyo, J.C.; Han, K.J.; Cho, Y.; Kim, D.; Jin, D. Generalization of U-Net Semantic Segmentation for Forest Change Detection in South Korea Using Airborne Imagery. Forests 2022, 13, 2170. [Google Scholar] [CrossRef]
- Ronneberge, O.; Fischer, P.; Brox, T. INet: Convolutional Networks for Biomedical Image Segmentation. IEEE Access 2015, 9, 16591–16603. [Google Scholar] [CrossRef]
- Wagner, F.H.; Sanchez, A.; Tarabalka, Y.; Lotte, R.G.; Ferreira, M.P.; Aidar, M.P.M.; Gloor, E.; Phillips, O.L.; Aragão, L.E.O.C. Using the U-Net Convolutional Network to Map Forest Types and Disturbance in the Atlantic Rainforest with Very High Resolution Images. Remote Sens. Ecol. Conserv. 2019, 5, 360–375. [Google Scholar] [CrossRef]
- Cheng, K.; Wang, J. Forest Type Classification Based on Integrated Spectral-Spatial-Temporal Features and Random Forest Algorithm-A Case Study in the Qinling Mountains. Forests 2019, 10, 559. [Google Scholar] [CrossRef]
- Bjerreskov, K.S.; Nord-Larsen, T.; Fensholt, R. Classification of Nemoral Forests with Fusion of Multi-Temporal Sentinel-1 and 2 Data. Remote Sens. 2021, 13, 950. [Google Scholar] [CrossRef]
- Ye, N.; Morgenroth, J.; Xu, C.; Chen, N. Indigenous Forest Classification in New Zealand—A Comparison of Classifiers and Sensors. Int. J. Appl. Earth Obs. Geoinf. 2021, 102, 102395. [Google Scholar] [CrossRef]
- Tamiminia, H.; Salehi, B.; Mahdianpari, M.; Beier, C.M.; Klimkowski, D.J.; Volk, T.A. Random Forest Outperformed Convolutional Neural Networks for Shrub Willow Above Ground Biomass Estimation Using Multi-Spectral Uas Imagery. In Proceedings of the International Geoscience and Remote Sensing Symposium (IGARSS), Brussels, Belgium, 11–16 July 2021; pp. 8269–8272. [Google Scholar] [CrossRef]
- Kaler, P. Study of Grayscale Image in Image Processing. Ijritcc 2016, 4, 309–311. [Google Scholar]
- Datta, L. A Survey on Activation Functions and Their Relation with Xavier and He Normal Initialization. arXiv 2020, arXiv:2004.06632. [Google Scholar]
- Zhang, B.; Zhao, L.; Zhang, X. Three-Dimensional Convolutional Neural Network Model for Tree Species Classification Using Airborne Hyperspectral Images. Remote Sens. Environ. 2020, 247, 111938. [Google Scholar] [CrossRef]
- Chen, C.; Jing, L.; Li, H.; Tang, Y. A New Individual Tree Species Classification Method Based on the Resu-Net Model. Forests 2021, 12, 1202. [Google Scholar] [CrossRef]
- Yu, K.; Hao, Z.; Post, C.J.; Mikhailova, E.A.; Lin, L.; Zhao, G.; Tian, S.; Liu, J. Comparison of Classical Methods and Mask R-CNN for Automatic Tree Detection and Mapping Using UAV Imagery. Remote Sens. 2022, 14, 295. [Google Scholar] [CrossRef]
- Solórzano, J.V.; Mas, J.F.; Gao, Y.; Gallardo-Cruz, J.A. Land Use Land Cover Classification with U-Net: Advantages of Combining Sentinel-1 and Sentinel-2 Imagery. Remote Sens. 2021, 13, 3600. [Google Scholar] [CrossRef]
- Deng, Z.; Sun, H.; Zhou, S.; Zhao, J.; Lei, L.; Zou, H. Multi-Scale Object Detection in Remote Sensing Imagery with Convolutional Neural Networks. ISPRS J. Photogramm. Remote Sens. 2018, 145, 3–22. [Google Scholar] [CrossRef]
- Sothe, C.; La Rosa, L.E.C.; De Almeida, C.M.; Gonsamo, A.; Schimalski, M.B.; Castro, J.D.B.; Feitosa, R.Q.; Dalponte, M.; Lima, C.L.; Liesenberg, V.; et al. Evaluating a Convolutional Neural Network for Feature Extraction and Tree Species Classification Using Uav-Hyperspectral Images. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2020, 5, 193–199. [Google Scholar] [CrossRef]
- Abdollahnejad, A.; Panagiotidis, D. Tree Species Classification and Health Status Assessment for a Mixed Broadleaf-Conifer Forest with Uas Multispectral Imaging. Remote Sens. 2020, 12, 3722. [Google Scholar] [CrossRef]
- Yang, R.; Wang, L.; Tian, Q.; Xu, N.; Yang, Y. Estimation of the Conifer-Broadleaf Ratio in Mixed Forests Based on Time-Series Data. Remote Sens. 2021, 13, 4426. [Google Scholar] [CrossRef]
- Kellenberger, B.; Marcos, D.; Tuia, D. Detecting Mammals in UAV Images: Best Practices to Address a Substantially Imbalanced Dataset with Deep Learning. Remote Sens. Environ. 2018, 216, 139–153. [Google Scholar] [CrossRef]
- Diez, Y.; Kentsch, S.; Fukuda, M.; Caceres, M.L.L.; Moritake, K.; Cabezas, M. Deep Learning in Forestry Using Uav-Acquired Rgb Data: A Practical Review. Remote Sens. 2021, 13, 2837. [Google Scholar] [CrossRef]
- Karatas, G.; Demir, O.; Sahingoz, O.K. Increasing the Performance of Machine Learning-Based IDSs on an Imbalanced and Up-to-Date Dataset. IEEE Access 2020, 8, 32150–32162. [Google Scholar] [CrossRef]
- Pei, H.; Owari, T.; Tsuyuki, S.; Zhong, Y. Application of a Novel Multiscale Global Graph Convolutional Neural Network to Improve the Accuracy of Forest Type Classification Using Aerial Photographs. Remote Sens. 2023, 15, 1001. [Google Scholar] [CrossRef]
- Safonova, A.; Tabik, S.; Alcaraz-Segura, D.; Rubtsov, A.; Maglinets, Y.; Herrera, F. Detection of Fir Trees (Abies sibirica) Damaged by the Bark Beetle in Unmanned Aerial Vehicle Images with Deep Learning. Remote Sens. 2019, 11, 643. [Google Scholar] [CrossRef]
- Marinelli, D.; Paris, C.; Bruzzone, L. An Approach Based on Deep Learning for Tree Species Classification in LiDAR Data Acquired in Mixed Forest. IEEE Geosci. Remote Sens. Lett. 2022, 19, 1–5. [Google Scholar] [CrossRef]
- Shia, W.C.; Chen, D.R. Classification of Malignant Tumors in Breast Ultrasound Using a Pretrained Deep Residual Network Model and Support Vector Machine. Comput. Med. Imaging Graph. 2021, 87, 101829. [Google Scholar] [CrossRef]
- Onishi, M.; Watanabe, S.; Nakashima, T.; Ise, T. Practicality and Robustness of Tree Species Identification Using UAV RGB Image and Deep Learning in Temperate Forest in Japan. Remote Sens. 2022, 14, 1243. [Google Scholar] [CrossRef]
- Zhong, H.; Lin, W.; Liu, H.; Ma, N.; Liu, K.; Cao, R.; Wang, T.; Ren, Z. Identification of Tree Species Based on the Fusion of UAV Hyperspectral Image and LiDAR Data in a Coniferous and Broad-Leaved Mixed Forest in Northeast China. Front. Plant Sci. 2022, 13, 964769. [Google Scholar] [CrossRef]
Machine Learning Models | Sub-Compartments | Class | Pixels |
---|---|---|---|
U-Net | 42B | Conifer | 11,601,650 |
Broadleaf | 54,573,657 | ||
16AB | Conifer | 33,209,707 | |
Broadleaf | 33,204,081 | ||
RF | 42B | Conifer | 2,941,142 |
Broadleaf | 8,665,505 | ||
16AB | Conifer | 5,155,835 | |
Broadleaf | 5,103,208 |
Machine Learning Models | Validation Accuracy | IoU (Mean) | IoU (Conifer) | IoU (Broadleaf) | |
---|---|---|---|---|---|
U-Net | 50 epochs | 0.88 | 0.57 | 0.00 | 0.78 |
700 epochs | 0.93 | 0.80 | 0.60 | 0.86 | |
1000 epochs | 0.93 | 0.79 | 0.54 | 0.86 | |
RF | 0.69 | 0.33 | 0.04 | 0.68 |
Machine Learning Models | Validation Accuracy | IoU (Mean) | IoU (Conifer) | IoU (Broadleaf) | |
---|---|---|---|---|---|
U-Net | 50 epochs | 0.75 | 0.61 | 0.62 | 0.55 |
300 epochs | 0.83 | 0.70 | 0.70 | 0.69 | |
500 epochs | 0.79 | 0.66 | 0.67 | 0.61 | |
RF | 0.59 | 0.43 | 0.40 | 0.42 |
Machine Learning Models | Sub-Compartments 42B | Sub-Compartments 16AB | ||
---|---|---|---|---|
Precision | Recall | Precision | Recall | |
U-Net | 0.92 | 0.85 | 0.83 | 0.83 |
RF | 0.60 | 0.44 | 0.61 | 0.60 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Htun, N.M.; Owari, T.; Tsuyuki, S.; Hiroshima, T. Integration of Unmanned Aerial Vehicle Imagery and Machine Learning Technology to Map the Distribution of Conifer and Broadleaf Canopy Cover in Uneven-Aged Mixed Forests. Drones 2023, 7, 705. https://doi.org/10.3390/drones7120705
Htun NM, Owari T, Tsuyuki S, Hiroshima T. Integration of Unmanned Aerial Vehicle Imagery and Machine Learning Technology to Map the Distribution of Conifer and Broadleaf Canopy Cover in Uneven-Aged Mixed Forests. Drones. 2023; 7(12):705. https://doi.org/10.3390/drones7120705
Chicago/Turabian StyleHtun, Nyo Me, Toshiaki Owari, Satoshi Tsuyuki, and Takuya Hiroshima. 2023. "Integration of Unmanned Aerial Vehicle Imagery and Machine Learning Technology to Map the Distribution of Conifer and Broadleaf Canopy Cover in Uneven-Aged Mixed Forests" Drones 7, no. 12: 705. https://doi.org/10.3390/drones7120705