Deep Learning Model Transfer in Forest Mapping Using Multi-Source Satellite SAR and Optical Images
Abstract
:1. Introduction
2. Materials and Methods
2.1. UNet Model and Its Derivative/Improved Models
2.2. Logic of Transfer Learning
2.3. Our Approach
2.4. Study Sites, Satellite SAR and Optical Data
2.4.1. Satellite Image Data
- Satellite optical data were represented by an ESA Sentinel-2 image acquired on 12 July 2018 over the Salla site and 11 August 2018 over the Kouvola site. The Level 2A surface reflectance product systematically generated by ESA and distributed in tiles of 110 × 110 km2 was used. The multi-spectral instrument on board Sentinel-2 satellites has 13 spectral bands with 10 m (four bands), 20 m (six bands) and 60 m (three bands) spatial resolutions. We used bands 2, 3, 4, 5, 8, 11, 12 in our analysis, which were found most useful for monitoring boreal forest in prior studies [38].
- SAR data are represented by an annual composite of 39 Sentinel-1 images acquired during 2018 in the same geometry. The original dual-polarization Sentinel-1A images available as GRD (ground range detected) products were radiometrically terrain-flattened and orthorectified with VTT in-house software using local digital elevation model available from National Land Survey of Finland [39]. Final preprocessed images were in gamma-naught format [40].
- L-band SAR imagery was represented by JAXA mosaic produced from dual-pol ALOS-2 PALSAR-2 images acquired during 2018.
- Interferometric SAR layers were represented by TanDEM-X images collected during summer 2018. TanDEM-X canopy height model is calculated via subtracting of TanDEM-X phase and topographic phase (calculated from local topographic map) in slant range followed by phase-to-height conversion and geocoding obtained height product. It is later called interferometric canopy height model (ICHM) in the paper. Additionally, TanDEM-X coherence magnitude was used as an image feature layer. ESA SNAP software was used for calculating TanDEM-X image layers.
2.4.2. Reference Data
2.5. Implementation Details
2.6. Accuracy Metrics
3. Results
3.1. Prediction Performance over Pretraining (Salla) Site
3.2. Prediction Performance over Target (Kouvola) Site
3.3. Model Stability with Scarce or Missing Reference Data in the Kouvola Site
- Only 35 plots (5% of original training sample) are used in model training (model transfer).
- Small-biomass plots with forest heights less than 10 m are completely removed from the training dataset.
- Tall forest plots with heights exceeding 25 m are completely removed from the training dataset.
4. Discussion
4.1. Overall Discussion on Performance across Various Models and EO Datasets
4.2. Comparison with Prior Studies
4.3. Outlook
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
ALS | Airborne Laser Scanning |
ALOS | Advanced Land-Observing Satellite |
CNN | Convolutional Neural Network |
CPR | Cross Pseudo Regression |
DL | Deep Learning |
EO | Earth Observation |
ESA | European Space Agency |
GRD | Ground Range Detected |
ML | Machine Learning |
MLR | Multiple Linear Regression |
PALSAR | Phased-Array L-band Synthetic-Aperture Radar |
PCA | Principal Component Analysis |
RF | Random Forest |
RMSE | Root Mean Squared Error |
RNN | Recurrent Neural Network |
SAR | Synthetic Aperture Radar |
TanDEM-X | TerraSAR-X add-on for Digital Elevation Measurement |
References
- Herold, M.; Carter, S.; Avitabile, V.; Espejo, A.B.; Jonckheere, I.; Lucas, R.; McRoberts, R.E.; Næsset, E.; Nightingale, J.; Petersen, R.; et al. The role and need for space-based forest biomass-related measurements in environmental management and policy. Surv. Geophys. 2019, 40, 757–778. [Google Scholar] [CrossRef]
- McRoberts, R.E.; Tomppo, E.O. Remote sensing support for national forest inventories. Remote Sens. Environ. 2007, 110, 412–419. [Google Scholar] [CrossRef]
- GFOI. Integrating Remote-Sensing and Ground-Based Observations for Estimation of Emissions and Removals of Greenhouse Gases in Forests: Methods and Guidance from the Global Forest Observations Initiative; Group on Earth Observations: Geneva, Switzerland, 2014. [Google Scholar]
- Rodríguez-Veiga, P.; Quegan, S.; Carreiras, J.; Persson, H.; Fransson, J.; Hoscilo, A.; Ziółkowski, D.; Stereńczak, K.; Lohberger, S.; Stängel, M.; et al. Forest biomass retrieval approaches from Earth observation in different biomes. Int. J. Appl. Earth Obs. Geoinf. 2019, 77, 53–68. [Google Scholar] [CrossRef]
- Astola, H.; Häme, T.; Sirro, L.; Molinier, M.; Kilpi, J. Comparison of Sentinel-2 and Landsat 8 imagery for forest variable prediction in boreal region. Remote Sens. Environ. 2019, 223, 257–273. [Google Scholar] [CrossRef]
- Schmullius, C.; Thiel, C.; Pathe, C.; Santoro, M. Radar time series for land cover and forest mapping. In Remote Sensing Time Series: Revealing Land Surface Dynamics; Kuenzer, C., Dech, S., Wagner, W., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 323–356. [Google Scholar]
- Antropov, O.; Rauste, Y.; Häme, T.; Praks, J. Polarimetric ALOS PALSAR time series in mapping biomass of boreal forests. Remote Sens. 2017, 9, 999. [Google Scholar] [CrossRef]
- Olesk, A.; Praks, J.; Antropov, O.; Zalite, K.; Arumäe, T.; Voormansik, K. Interferometric SAR coherence models for characterization of hemiboreal forests using TanDEM-X data. Remote Sens. 2016, 8, 700. [Google Scholar] [CrossRef]
- Kugler, F.; Lee, S.K.; Hajnsek, I.; Papathanassiou, K.P. Forest height estimation by means of Pol-InSAR data inversion: The role of the vertical wavenumber. IEEE Trans. Geosci. Remote Sens. 2015, 53, 5294–5311. [Google Scholar] [CrossRef]
- Persello, C.; Wegner, J.D.; Hänsch, R.; Tuia, D.; Ghamisi, P.; Koeva, M.; Camps-Valls, G. Deep learning and Earth Observation to support the sustainable development goals: Current approaches, open challenges, and future opportunities. IEEE Geosci. Remote Sens. Mag. 2022, 10, 172–200. [Google Scholar] [CrossRef]
- Zhu, X.X.; Montazeri, S.; Ali, M.; Hua, Y.; Wang, Y.; Mou, L.; Shi, Y.; Xu, F.; Bamler, R. Deep learning meets SAR: Concepts, models, pitfalls, and perspectives. IEEE Geosci. Remote Sens. Mag. 2021, 9, 143–172. [Google Scholar] [CrossRef]
- Zhu, X.X.; Tuia, D.; Mou, L.; Xia, G.S.; Zhang, L.; Xu, F.; Fraundorfer, F. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geosci. Remote Sens. Mag. 2017, 5, 8–36. [Google Scholar] [CrossRef]
- Astola, H.; Seitsonen, L.; Halme, E.; Molinier, M.; Lönnqvist, A. Deep neural networks with transfer learning for forest variable estimation using Sentinel-2 imagery in boreal forest. Remote Sens. 2021, 13, 2392. [Google Scholar] [CrossRef]
- Illarionova, S.; Trekin, A.; Ignatiev, V.; Oseledets, I. Tree species mapping on Sentinel-2 satellite imagery with weakly supervised classification and object-wise sampling. Forests 2021, 12, 1413. [Google Scholar] [CrossRef]
- Ge, S.; Gu, H.; Su, W.; Praks, J.; Antropov, O. Improved semisupervised UNet deep learning model for forest height mapping with satellite SAR and optical Data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2022, 15, 5776–5787. [Google Scholar] [CrossRef]
- Ge, S.; Su, W.; Gu, H.; Rauste, Y.; Praks, J.; Antropov, O. Improved LSTM model for boreal forest height mapping using Sentinel-1 time series. Remote Sens. 2022, 14, 5560. [Google Scholar] [CrossRef]
- Bolyn, C.; Lejeune, P.; Michez, A.; Latte, N. Mapping tree species proportions from satellite imagery using spectral–spatial deep learning. Remote Sens. Environ. 2022, 280, 113205. [Google Scholar] [CrossRef]
- Wang, S.; Chen, W.; Xie, S.M.; Azzari, G.; Lobell, D.B. Weakly supervised deep learning for segmentation of remote sensing imagery. Remote Sens. 2020, 12, 207. [Google Scholar] [CrossRef]
- Illarionova, S.; Shadrin, D.; Ignatiev, V.; Shayakhmetov, S.; Trekin, A.; Oseledets, I. Estimation of the canopy height model from multispectral satellite imagery with convolutional neural networks. IEEE Access 2022, 10, 34116–34132. [Google Scholar] [CrossRef]
- Lang, N.; Schindler, K.; Wegner, J.D. Country-wide high-resolution vegetation height mapping with Sentinel-2. Remote Sens. Environ. 2019, 233, 111347. [Google Scholar] [CrossRef]
- Bueso-Bello, J.-L.; Carcereri, D.; Martone, M.; González, C.; Posovszky, P.; Rizzoli, P. Deep learning for mapping tropical forests with TanDEM-X bistatic InSAR data. Remote Sens. 2022, 14, 3981. [Google Scholar] [CrossRef]
- Bjork, S.; Anfinsen, S.N.; Næsset, E.; Gobakken, T.; Zahabu, E. On the potential of sequential and nonsequential regression models for Sentinel-1-based biomass prediction in Tanzanian Miombo forests. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2022, 15, 4612–4639. [Google Scholar] [CrossRef]
- Zhao, X.; Hu, J.; Mou, L.; Xiong, Z.; Zhu, X.X. Cross-city Landuse classification of remote sensing images via deep transfer learning. Int. J. Appl. Earth Obs. Geoinf. 2023, 122, 103358. [Google Scholar] [CrossRef]
- Zhang, Y.; Guo, X.; Leung, H.; Li, L. Cross-task and cross-domain SAR target recognition: A meta-transfer learning approach. Pattern Recognit. 2023, 138, 109402. [Google Scholar] [CrossRef]
- Javed, A.; Kim, T.; Lee, C.; Oh, J.; Han, Y. Deep learning-based detection of urban forest cover change along with overall urban changes using very-high-resolution satellite images. Remote Sens. 2023, 15, 4285. [Google Scholar] [CrossRef]
- Reis, H.C.; Turk, V. Detection of forest fire using deep convolutional neural networks with transfer learning approach. Appl. Soft Comput. 2023, 143, 110362. [Google Scholar] [CrossRef]
- Zhuang, F.; Qi, Z.; Duan, K.; Xi, D.; Zhu, Y.; Zhu, H.; Xiong, H.; He, Q. A comprehensive survey on transfer learning. Proc. IEEE 2020, 109, 43–76. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, 5–9 October 2015; Proceedings, Part III 18. Springer: Berlin, Germany, 2015; pp. 234–241. [Google Scholar] [CrossRef]
- Šćepanović, S.; Antropov, O.; Laurila, P.; Rauste, Y.; Ignatenko, V.; Praks, J. Wide-area land cover mapping with Sentinel-1 imagery using deep learning semantic segmentation models. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 10357–10374. [Google Scholar] [CrossRef]
- Gazzea, M.; Solheim, A.; Arghandeh, R. High-resolution mapping of forest structure from integrated SAR and optical images using an enhanced U-net method. Sci. Remote Sens. 2023, 8, 100093. [Google Scholar] [CrossRef]
- Huang, Z.; Pan, Z.; Lei, B. Transfer learning with deep convolutional neural network for SAR target classification with limited labeled data. Remote Sens. 2017, 9, 907. [Google Scholar] [CrossRef]
- Wurm, M.; Stark, T.; Zhu, X.X.; Weigand, M.; Taubenböck, H. Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks. ISPRS J. Photogramm. Remote Sens. 2019, 150, 59–69. [Google Scholar] [CrossRef]
- Englhart, S.; Keuck, V.; Siegert, F. Modeling aboveground biomass in tropical forests using multi-frequency SAR data—A comparison of methods. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2012, 5, 298–306. [Google Scholar] [CrossRef]
- Ge, S.; Tomppo, E.; Rauste, Y.; McRoberts, R.E.; Praks, J.; Gu, H.; Su, W.; Antropov, O. Sentinel-1 time series for predicting growing stock volume of boreal forest: Multitemporal analysis and feature selection. Remote Sens. 2023, 15, 3489. [Google Scholar] [CrossRef]
- Antropov, O.; Miettinen, J.; Häme, T.; Yrjö, R.; Seitsonen, L.; McRoberts, R.E.; Santoro, M.; Cartus, O.; Duran, N.M.; Herold, M.; et al. Intercomparison of Earth Observation data and methods for forest mapping in the context of forest carbon monitoring. In Proceedings of the IGARSS 2022—2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 17–22 July 2022; pp. 5777–5780. [Google Scholar] [CrossRef]
- Berninger, A.; Lohberger, S.; Stängel, M.; Siegert, F. SAR-based estimation of above-ground biomass and its changes in tropical forests of Kalimantan using L- and C-band. Remote Sens. 2018, 10, 831. [Google Scholar] [CrossRef]
- Tomppo, E.; Olsson, H.; Ståhl, G.; Nilsson, M.; Hagner, O.; Katila, M. Combining national forest inventory field plots and remote sensing data for forest databases. Remote Sens. Environ. 2008, 112, 1982–1999. [Google Scholar] [CrossRef]
- Miettinen, J.; Carlier, S.; Häme, L.; Mäkelä, A.; Minunno, F.; Penttilä, J.; Pisl, J.; Rasinmäki, J.; Rauste, Y.; Seitsonen, L.; et al. Demonstration of large area forest volume and primary production estimation approach based on Sentinel-2 imagery and process based ecosystem modelling. Int. J. Remote Sens. 2021, 42, 9467–9489. [Google Scholar] [CrossRef]
- Rauste, Y.; Lonnqvist, A.; Molinier, M.; Henry, J.B.; Hame, T. Ortho-rectification and terrain correction of polarimetric SAR data applied in the ALOS/Palsar context. In Proceedings of the 2007 IEEE International Geoscience and Remote Sensing Symposium, Barcelona, Spain, 23–28 July 2007; pp. 1618–1621. [Google Scholar] [CrossRef]
- Small, D. Flattening gamma: Radiometric terrain correction for SAR imagery. IEEE Trans. Geosci. Remote Sens. 2011, 49, 3081–3093. [Google Scholar] [CrossRef]
- Huang, W.; Min, W.; Ding, J.; Liu, Y.; Hu, Y.; Ni, W.; Shen, H. Forest height mapping using inventory and multi-source satellite data over Hunan Province in southern China. For. Ecosyst. 2022, 9, 100006. [Google Scholar] [CrossRef]
- Luo, Y.; Qi, S.; Liao, K.; Zhang, S.; Hu, B.; Tian, Y. Mapping the forest height by fusion of ICESat-2 and multi-source remote sensing imagery and topographic information: A case study in Jiangxi province, China. Forests 2023, 14, 454. [Google Scholar] [CrossRef]
- Zhang, N.; Chen, M.; Yang, F.; Yang, C.; Yang, P.; Gao, Y.; Shang, Y.; Peng, D. Forest height mapping using feature selection and machine learning by integrating multi-source satellite data in Baoding city, North China. Remote Sens. 2022, 14, 4434. [Google Scholar] [CrossRef]
- Praks, J.; Hallikainen, M.; Antropov, O.; Molina, D. Boreal forest tree height estimation from interferometric TanDEM-X images. In Proceedings of the 2012 IEEE International Geoscience and Remote Sensing Symposium, Munich, Germany, 22–27 July 2012; pp. 1262–1265. [Google Scholar] [CrossRef]
- Li, W.; Niu, Z.; Shang, R.; Qin, Y.; Wang, L.; Chen, H. High-resolution mapping of forest canopy height using machine learning by coupling ICESat-2 LiDAR with Sentinel-1, Sentinel-2 and Landsat-8 data. Int. J. Appl. Earth Obs. Geoinf. 2020, 92, 102163. [Google Scholar] [CrossRef]
- Becker, A.; Russo, S.; Puliti, S.; Lang, N.; Schindler, K.; Wegner, J.D. Country-wide retrieval of forest structure from optical and SAR satellite imagery with deep ensembles. ISPRS J. Photogramm. Remote Sens. 2023, 195, 269–286. [Google Scholar] [CrossRef]
- Chen, H.; Cloude, S.R.; Goodenough, D.G. Forest canopy height estimation using Tandem-X coherence data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 3177–3188. [Google Scholar] [CrossRef]
- Olesk, A.; Voormansik, K.; Vain, A.; Noorma, M.; Praks, J. Seasonal differences in forest height estimation from interferometric TanDEM-X coherence data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 5565–5572. [Google Scholar] [CrossRef]
- Praks, J.; Antropov, O.; Olesk, A.; Voormansik, K. Forest height estimation from TanDEM-X images with semi-empirical coherence models. In Proceedings of the IGARSS 2018—2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 8805–8808. [Google Scholar] [CrossRef]
- Chen, H.; Cloude, S.R.; Goodenough, D.G.; Hill, D.A.; Nesdoly, A. Radar forest height estimation in mountainous terrain using Tandem-X coherence data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 3443–3452. [Google Scholar] [CrossRef]
- Schlund, M.; Magdon, P.; Eaton, B.; Aumann, C.; Erasmi, S. Canopy height estimation with TanDEM-X in temperate and boreal forests. Int. J. Appl. Earth Obs. Geoinf. 2019, 82, 101904. [Google Scholar] [CrossRef]
RMSE, m | RMSE% | Bias | R2 | |
---|---|---|---|---|
Sentinel-2 data | ||||
MLR | 4.64 | 30.6 | 0.30 | 0.645 |
RF | 4.57 | 30.3 | 0.14 | 0.655 |
kNN | 4.58 | 30.3 | 0.28 | 0.650 |
SeUNet, non-fine-tuned | 6.84 | 45.8 | −2.75 | 0.245 |
SeUNet, fine-tuned | 4.55 | 30.3 | −0.77 | 0.667 |
Sentinel-2 and Sentinel-1 data | ||||
MLR | 4.60 | 30.4 | 0.30 | 0.650 |
RF | 4.37 | 28.9 | 0.15 | 0.685 |
kNN | 4.46 | 29.5 | 0.26 | 0.670 |
SeUNet, non-fine-tuned | 7.23 | 48.4 | −2.48 | 0.157 |
SeUNet, fine-tuned | 4.27 | 28.6 | 0.53 | 0.71 |
Multi-source EO data | ||||
MLR | 3.16 | 20.9 | 0.23 | 0.835 |
RF | 2.91 | 19.3 | 0.10 | 0.860 |
kNN | 2.98 | 19.7 | −0.40 | 0.853 |
SeUNet, non-fine-tuned | 5.67 | 38.0 | −2.32 | 0.481 |
SeUNet, fine-tuned | 2.70 | 18.1 | −0.25 | 0.882 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ge, S.; Antropov, O.; Häme, T.; McRoberts, R.E.; Miettinen, J. Deep Learning Model Transfer in Forest Mapping Using Multi-Source Satellite SAR and Optical Images. Remote Sens. 2023, 15, 5152. https://doi.org/10.3390/rs15215152
Ge S, Antropov O, Häme T, McRoberts RE, Miettinen J. Deep Learning Model Transfer in Forest Mapping Using Multi-Source Satellite SAR and Optical Images. Remote Sensing. 2023; 15(21):5152. https://doi.org/10.3390/rs15215152
Chicago/Turabian StyleGe, Shaojia, Oleg Antropov, Tuomas Häme, Ronald E. McRoberts, and Jukka Miettinen. 2023. "Deep Learning Model Transfer in Forest Mapping Using Multi-Source Satellite SAR and Optical Images" Remote Sensing 15, no. 21: 5152. https://doi.org/10.3390/rs15215152
APA StyleGe, S., Antropov, O., Häme, T., McRoberts, R. E., & Miettinen, J. (2023). Deep Learning Model Transfer in Forest Mapping Using Multi-Source Satellite SAR and Optical Images. Remote Sensing, 15(21), 5152. https://doi.org/10.3390/rs15215152