Mapping Crop Rotation by Using Deeply Synergistic Optical and SAR Time Series
Abstract
:1. Introduction
- (1)
- A new hybrid deep learning network, proposed to capture both crop types and seasonal changes for crop rotation mapping;
- (2)
- Test results validating the advantage of a synergy SAR-optical time series and incorporating the use of the attention mechanism in LSTM over traditional and ablation models.
2. Study Area and Materials
2.1. Study Area
2.2. Satellite Data and Pre-Processing
2.3. Ground-Truth Data
3. Crop Rotation Mapping (CRM)
3.1. Robust Temporal Features Extraction with Conv1D
3.2. Crop Rotation Patterns Depiction with LSTM
3.3. Crop Rotation Mapping with a Multi-Temporal Attention Mechanism
4. Experiments
4.1. Competing Models and VIs
- RF(S2): A random forest (RF) classifier with Sentinel-2 time series inputs was used to derive the baseline performance of classical non-deep learning architecture. RF has been considered as the most common and effective land cover mapping approach [44] because of its high performance in the classification task, combined with optical time-series data in recent mapping works.
- LSTM(S2): A deep learning classifier that used an LSTM unit with the attention mechanism with Sentinel-2 multi-temporal inputs, following the lead of popular frameworks [45,46]. To adopt such a model, we employed 14 Sentinel-2 images to the LSTM layer to capture dynamic growing information, plus an attention layer to allocate independent weight to each timestamp. Here, LSTM(S2) represented the baseline of time series processing with deep-learning architecture.
- CRM(S2): An ablation model that only considered the optical source, because optimal temporal profiles could better express crop rotation cycles and record crop growth compared to unexplainable SAR profiles. Such a structure was introduced not only to disentangle the benefits of convolutional operation compared to LSTM(S2) but also to demonstrate the necessity of synergizing multiple satellite sources compared to the original CRM model.
- CRM(NoAtt): An ablation model excluding the use of the attention mechanism. Here, CRM(NoAtt) was conducted to prove the importance of assigning weights according to different temporal characteristics.
4.2. Model Training
4.3. Model Evaluation
5. Results
5.1. Sensitivity Analysis of VIs as Model Input
5.2. Ablation Analysis of Multi-Source Data and Attention Mechanism
5.3. Crop Rotation Mapping Results
6. Discussion
6.1. Evaluation of End-To-End Crop Rotation Mapping
6.2. Necessity of Establishing the Dual-Stream Model for Multi-Source Data
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Panigrahy, S.; Sharma, S.A. Mapping of Crop Rotation Using Multidate Indian Remote Sensing Satellite Digital Data. ISPRS J. Photogramm. Remote Sens. 1997, 52, 85–91. [Google Scholar] [CrossRef]
- Zhao, J.; Yang, Y.; Zhang, K.; Jeong, J.; Zeng, Z.; Zang, H. Does Crop Rotation Yield More in China? A Meta-Analysis. Field Crop. Res. 2020, 245, 107659. [Google Scholar] [CrossRef]
- Wenbin, W.; Yu, Q.; You, L.; Chen, K.; Tang, H.; Liu, J. Global Cropping Intensity Gaps: Increasing Food Production without Cropland Expansion. Land Use Policy 2018, 76, 515–525. [Google Scholar] [CrossRef]
- Barbieri, P.; Pellerin, S.; Seufert, V.; Nesme, T. Changes in Crop Rotations Would Impact Food Production in an Organically Farmed World. Nat. Sustain. 2019, 2, 378–385. [Google Scholar] [CrossRef]
- Kuenzer, C.; Knauer, K. Remote Sensing of Rice Crop Areas. Int. J. Remote Sens. 2013, 34, 2101–2139. [Google Scholar] [CrossRef]
- Merlos, F.A.; Hijmans, R.J. The Scale Dependency of Spatial Crop Species Diversity and Its Relation to Temporal Diversity. Proc. Natl. Acad. Sci. USA 2020, 117, 26176–26182. [Google Scholar] [CrossRef] [PubMed]
- Konduri, V.S.; Kumar, J.; Hargrove, W.W.; Hoffman, F.M.; Ganguly, A.R. Mapping Crops within the Growing Season across the United States. Remote Sens. Environ. 2020, 251, 112048. [Google Scholar] [CrossRef]
- Multi-Data Approach for remote sensing-based regional crop rotation mapping: A case study for the Rur catchment, Germany. Int. J. Appl. Earth Obs. Geoinf. 2017, 61, 55–69. [CrossRef]
- Han, J.; Zhang, Z.; Luo, Y.; Cao, J.; Zhang, L.; Zhang, J.; Li, Z. The RapeseedMap10 Database: Annual Maps of Rapeseed at a Spatial Resolution of 10 m Based on Multi-Source Data. Earth Syst. Sci. Data 2021, 13, 2857–2874. [Google Scholar] [CrossRef]
- Chen, Y.; Lu, D.; Moran, E.; Batistella, M.; Dutra, L.V.; Sanches, I.D.; da Silva, R.F.B.; Huang, J.; Luiz, A.J.B.; de Oliveira, M.A.F. Mapping Croplands, Cropping Patterns, and Crop Types Using MODIS Time-Series Data. Int. J. Appl. Earth Obs. Geoinf. 2018, 69, 133–147. [Google Scholar] [CrossRef]
- Li, R.; Xu, M.; Chen, Z.; Gao, B.; Cai, J.; Shen, F.; He, X.; Zhuang, Y.; Chen, D. Phenology-Based Classification of Crop Species and Rotation Types Using Fused MODIS and Landsat Data: The Comparison of a Random-Forest-Based Model and a Decision-Rule-Based Model. Soil Tillage Res. 2021, 206, 104838. [Google Scholar] [CrossRef]
- Wang, S.; Azzari, G.; Lobell, D.B. Crop Type Mapping without Field-Level Labels: Random Forest Transfer and Unsupervised Clustering Techniques. Remote Sens. Environ. 2019, 222, 303–317. [Google Scholar] [CrossRef]
- Zhong, L.; Hawkins, T.; Biging, G.; Gong, P. A Phenology-Based Approach to Map Crop Types in the San Joaquin Valley, California. Int. J. Remote Sens. 2011, 32, 7777–7804. [Google Scholar] [CrossRef]
- Liu, J.; Li, L.; Huang, X.; Liu, Y.; Li, T. Mapping Paddy Rice in Jiangsu Province, China, Based on Phenological Parameters and a Decision Tree Model. Front. Earth Sci. 2019, 13, 111–123. [Google Scholar] [CrossRef]
- Deschamps, B.; McNairn, H.; Shang, J.; Jiao, X. Towards Operational Radar-Only Crop Type Classification: Comparison of a Traditional Decision Tree with a Random Forest Classifier. Can. J. Remote Sens. 2012, 38, 60–68. [Google Scholar] [CrossRef]
- Yang, N.; Liu, D.; Feng, Q.; Xiong, Q.; Zhang, L.; Ren, T.; Zhao, Y.; Zhu, D.; Huang, J. Large-Scale Crop Mapping Based on Machine Learning and Parallel Computation with Grids. Remote Sens. 2019, 11, 1500. [Google Scholar] [CrossRef] [Green Version]
- Hao, P.; Zhan, Y.; Wang, L.; Niu, Z.; Shakir, M. Feature Selection of Time Series MODIS Data for Early Crop Classification Using Random Forest: A Case Study in Kansas, USA. Remote Sens. 2015, 7, 5347–5369. [Google Scholar] [CrossRef] [Green Version]
- Zheng, B.; Myint, S.W.; Thenkabail, P.S.; Aggarwal, R.M. A Support Vector Machine to Identify Irrigated Crop Types Using Time-Series Landsat NDVI Data. Int. J. Appl. Earth Obs. Geoinf. 2015, 34, 103–112. [Google Scholar] [CrossRef]
- You, N.; Dong, J.; Huang, J.; Du, G.; Zhang, G.; He, Y.; Yang, T.; Di, Y.; Xiao, X. The 10-m Crop Type Maps in Northeast China during 2017–2019. Sci. Data 2021, 8, 41. [Google Scholar] [CrossRef]
- Peña, J.M.; Gutiérrez, P.A.; Hervás-Martínez, C.; Six, J.; Plant, R.E.; López-Granados, F. Object-Based Image Classification of Summer Crops with Machine Learning Methods. Remote Sens. 2014, 6, 5019–5041. [Google Scholar] [CrossRef] [Green Version]
- Pan, Y.; Li, L.; Zhang, J.; Liang, S.; Zhu, X.; Sulla-Menashe, D. Winter Wheat Area Estimation from MODIS-EVI Time Series Data Using the Crop Proportion Phenology Index. Remote Sens. Environ. 2012, 119, 232–242. [Google Scholar] [CrossRef]
- Guan, X.; Huang, C.; Liu, G.; Meng, X.; Liu, Q. Mapping Rice Cropping Systems in Vietnam Using an NDVI-Based Time-Series Similarity Measurement Based on DTW Distance. Remote Sens. 2016, 8, 19. [Google Scholar] [CrossRef] [Green Version]
- Csillik, O.; Belgiu, M.; Asner, G.P.; Kelly, M. Object-Based Time-Constrained Dynamic Time Warping Classification of Crops Using Sentinel-2. Remote Sens. 2019, 11, 1257. [Google Scholar] [CrossRef] [Green Version]
- Maus, V.; Câmara, G.; Cartaxo, R.; Sanchez, A.; Ramos, F.M.; de Queiroz, G.R. A Time-Weighted Dynamic Time Warping Method for Land-Use and Land-Cover Mapping. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2016, 9, 3729–3739. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep Learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef]
- Sun, Z.; Di, L.; Fang, H.; Burgess, A. Deep Learning Classification for Crop Types in North Dakota. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 2200–2213. [Google Scholar] [CrossRef]
- Yang, S.; Gu, L.; Li, X.; Jiang, T.; Ren, R. Crop Classification Method Based on Optimal Feature Selection and Hybrid CNN-RF Networks for Multi-Temporal Remote Sensing Imagery. Remote Sens. 2020, 12, 3119. [Google Scholar] [CrossRef]
- Zhong, L.; Hu, L.; Zhou, H. Deep Learning Based Multi-Temporal Crop Classification. Remote Sens. Environ. 2019, 221, 430–443. [Google Scholar] [CrossRef]
- Crisóstomo de Castro Filho, H.; Abílio de Carvalho Júnior, O.; Ferreira de Carvalho, O.L.; Pozzobon de Bem, P.; dos Santos de Moura, R.; Olino de Albuquerque, A.; Rosa Silva, C.; Guimarães Ferreira, P.H.; Fontes Guimarães, R.; Trancoso Gomes, R.A. Rice Crop Detection Using LSTM, Bi-LSTM, and Machine Learning Models from Sentinel-1 Time Series. Remote Sens. 2020, 12, 2655. [Google Scholar] [CrossRef]
- Sun, Z.; Di, L.; Fang, H. Using Long Short-Term Memory Recurrent Neural Network in Land Cover Classification on Landsat and Cropland Data Layer Time Series. Int. J. Remote Sens. 2019, 40, 593–614. [Google Scholar] [CrossRef]
- Rußwurm, M.; Körner, M. Temporal Vegetation Modelling Using Long Short-Term Memory Networks for Crop Identification from Medium-Resolution Multi-Spectral Satellite Images. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, Hawaii, 21–26 July 2017; pp. 1496–1504. [Google Scholar]
- Cué La Rosa, L.E.; Queiroz Feitosa, R.; Nigri Happ, P.; Del’Arco Sanches, I.; Ostwald Pedro da Costa, G.A. Combining Deep Learning and Prior Knowledge for Crop Mapping in Tropical Regions from Multitemporal SAR Image Sequences. Remote Sens. 2019, 11, 2029. [Google Scholar] [CrossRef] [Green Version]
- Wei, S.; Zhang, H.; Wang, C.; Wang, Y.; Xu, L. Multi-Temporal SAR Data Large-Scale Crop Mapping Based on U-Net Model. Remote Sens. 2019, 11, 68. [Google Scholar] [CrossRef] [Green Version]
- Adrian, J.; Sagan, V.; Maimaitijiang, M. Sentinel SAR-Optical Fusion for Crop Type Mapping Using Deep Learning and Google Earth Engine. ISPRS J. Photogramm. Remote Sens. 2021, 175, 215–235. [Google Scholar] [CrossRef]
- Liao, C.; Wang, J.; Xie, Q.; Baz, A.A.; Huang, X.; Shang, J.; He, Y. Synergistic Use of Multi-Temporal RADARSAT-2 and VENµS Data for Crop Classification Based on 1D Convolutional Neural Network. Remote Sens. 2020, 12, 832. [Google Scholar] [CrossRef] [Green Version]
- Ienco, D.; Interdonato, R.; Gaetano, R.; Ho Tong Minh, D. Combining Sentinel-1 and Sentinel-2 Satellite Image Time Series for Land Cover Mapping via a Multi-Source Deep Learning Architecture. ISPRS J. Photogramm. Remote Sens. 2019, 158, 11–22. [Google Scholar] [CrossRef]
- Mou, L.; Bruzzone, L.; Zhu, X.X. Learning Spectral-Spatial-Temporal Features via a Recurrent Convolutional Neural Network for Change Detection in Multispectral Imagery. IEEE Trans. Geosci. Remote Sens. 2019, 57, 924–935. [Google Scholar] [CrossRef] [Green Version]
- Rawat, A.; Kumar, A.; Upadhyay, P.; Kumar, S. Deep Learning-Based Models for Temporal Satellite Data Processing: Classification of Paddy Transplanted Fields. Ecol. Inform. 2021, 61, 101214. [Google Scholar] [CrossRef]
- Li, C.; Li, H.; Li, J.; Lei, Y.; Li, C.; Manevski, K.; Shen, Y. Using NDVI Percentiles to Monitor Real-Time Crop Growth. Comput. Electron. Agric. 2019, 162, 357–363. [Google Scholar] [CrossRef]
- Camps-Valls, G.; Campos-Taberner, M.; Moreno-Martínez, Á.; Walther, S.; Duveiller, G.; Cescatti, A.; Mahecha, M.D.; Muñoz-Marí, J.; García-Haro, F.J.; Guanter, L.; et al. A Unified Vegetation Index for Quantifying the Terrestrial Biosphere. Sci. Adv. 2021, 7, eabc7447. [Google Scholar] [CrossRef] [PubMed]
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
- Zhou, Y.; Luo, J.; Feng, L.; Yang, Y.; Chen, Y.; Wu, W. Long-Short-Term-Memory-Based Crop Classification Using High-Resolution Optical Images and Multi-Temporal SAR Data. GIScience Remote Sens. 2019, 56, 1170–1191. [Google Scholar] [CrossRef]
- Britz, D.; Guan, M.Y.; Luong, M.-T. Efficient Attention Using a Fixed-Size Memory Representation. arXiv 2017, arXiv:1707.00110. [Google Scholar]
- Tatsumi, K.; Yamashiki, Y.; Canales Torres, M.A.; Taipe, C.L.R. Crop Classification of Upland Fields Using Random Forest of Time-Series Landsat 7 ETM+ Data. Comput. Electron. Agric. 2015, 115, 171–179. [Google Scholar] [CrossRef]
- He, T.; Xie, C.; Liu, Q.; Guan, S.; Liu, G. Evaluation and Comparison of Random Forest and A-LSTM Networks for Large-Scale Winter Wheat Identification. Remote Sens. 2019, 11, 1665. [Google Scholar] [CrossRef] [Green Version]
- Qiu, C.; Mou, L.; Schmitt, M.; Zhu, X.X. Local Climate Zone-Based Urban Land Cover Classification from Multi-Seasonal Sentinel-2 Images with a Recurrent Residual Network. ISPRS J. Photogramm. Remote Sens. 2019, 154, 151–162. [Google Scholar] [CrossRef]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2017, arXiv:1412.6980. [Google Scholar]
- Zhang, M.; Lin, H.; Wang, G.; Sun, H.; Fu, J. Mapping Paddy Rice Using a Convolutional Neural Network (CNN) with Landsat 8 Datasets in the Dongting Lake Area, China. Remote Sens. 2018, 10, 1840. [Google Scholar] [CrossRef] [Green Version]
- Zhang, X.; Wu, B.; Ponce-Campos, G.E.; Zhang, M.; Chang, S.; Tian, F. Mapping Up-to-Date Paddy Rice Extent at 10 M Resolution in China through the Integration of Optical and Synthetic Aperture Radar Images. Remote Sens. 2018, 10, 1200. [Google Scholar] [CrossRef] [Green Version]
- Cai, Y.; Lin, H.; Zhang, M. Mapping Paddy Rice by the Object-Based Random Forest Method Using Time Series Sentinel-1/Sentinel-2 Data. Adv. Space Res. 2019, 64, 2233–2244. [Google Scholar] [CrossRef]
- Xiao, X.; Boles, S.; Liu, J.; Zhuang, D.; Frolking, S.; Li, C.; Salas, W.; Moore, B. Mapping Paddy Rice Agriculture in Southern China Using Multi-Temporal MODIS Images. Remote Sens. Environ. 2005, 95, 480–492. [Google Scholar] [CrossRef]
- Dong, J.; Xiao, X.; Menarguez, M.A.; Zhang, G.; Qin, Y.; Thau, D.; Biradar, C.; Moore, B. Mapping Paddy Rice Planting Area in Northeastern Asia with Landsat 8 Images, Phenology-Based Algorithm and Google Earth Engine. Remote Sens. Environ. 2016, 185, 142–154. [Google Scholar] [CrossRef] [Green Version]
- Ye, Q.; Yang, X.; Dai, S.; Chen, G.; Li, Y.; Zhang, C. Effects of Climate Change on Suitable Rice Cropping Areas, Cropping Systems and Crop Water Requirements in Southern China. Agric. Water Manag 2015, 159, 35–44. [Google Scholar] [CrossRef]
- Joshi, N.; Baumann, M.; Ehammer, A.; Fensholt, R.; Grogan, K.; Hostert, P.; Jepsen, M.R.; Kuemmerle, T.; Meyfroidt, P.; Mitchard, E.T.A.; et al. A Review of the Application of Optical and Radar Remote Sensing Data Fusion to Land Use Mapping and Monitoring. Remote Sens. 2016, 8, 70. [Google Scholar] [CrossRef] [Green Version]
- Zhao, W.; Qu, Y.; Chen, J.; Yuan, Z. Deeply Synergistic Optical and SAR Time Series for Crop Dynamic Monitoring. Remote Sens. Environ. 2020, 247, 111952. [Google Scholar] [CrossRef]
F1 Score | Accuracy | Kappa Coefficient | |||||||
---|---|---|---|---|---|---|---|---|---|
Model | NDVI | kNDVI | EVI | NDVI | kNDVI | EVI | NDVI | kNDVI | EVI |
RF(S2) | 0.727 | 0.726 | 0.637 | 0.781 | 0.776 | 0.714 | 0.704 | 0.699 | 0.607 |
LSTM(S2) | 0.786 | 0.762 | 0.671 | 0.813 | 0.805 | 0.725 | 0.752 | 0.742 | 0.636 |
CRM(S2) | 0.802 | 0.775 | 0.805 | 0.830 | 0.807 | 0.836 | 0.774 | 0.744 | 0.784 |
CRM(NoAtt) | 0.796 | 0.810 | 0.820 | 0.832 | 0.840 | 0.857 | 0.776 | 0.789 | 0.811 |
CRM | 0.841 | 0.850 | 0.843 | 0.865 | 0.876 | 0.871 | 0.821 | 0.836 | 0.828 |
CF-SR | F-SR | F-DR | RS-SR | RS-DC | Others | |
---|---|---|---|---|---|---|
RF(S2) | 0.86 | 0.59 | 0.93 | 0.75 | 0.51 | 0.72 |
LSTM(S2) | 0.88 | 0.71 | 0.95 | 0.73 | 0.74 | 0.80 |
CRM(S2) | 0.86 | 0.69 | 0.94 | 0.73 | 0.71 | 0.79 |
CRM(NoAtt) | 0.90 | 0.71 | 0.92 | 0.71 | 0.83 | 0.81 |
CRM | 0.92 | 0.81 | 0.95 | 0.74 | 0.84 | 0.84 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, Y.; Zhao, W.; Chen, S.; Ye, T. Mapping Crop Rotation by Using Deeply Synergistic Optical and SAR Time Series. Remote Sens. 2021, 13, 4160. https://doi.org/10.3390/rs13204160
Liu Y, Zhao W, Chen S, Ye T. Mapping Crop Rotation by Using Deeply Synergistic Optical and SAR Time Series. Remote Sensing. 2021; 13(20):4160. https://doi.org/10.3390/rs13204160
Chicago/Turabian StyleLiu, Yiqing, Wenzhi Zhao, Shuo Chen, and Tao Ye. 2021. "Mapping Crop Rotation by Using Deeply Synergistic Optical and SAR Time Series" Remote Sensing 13, no. 20: 4160. https://doi.org/10.3390/rs13204160
APA StyleLiu, Y., Zhao, W., Chen, S., & Ye, T. (2021). Mapping Crop Rotation by Using Deeply Synergistic Optical and SAR Time Series. Remote Sensing, 13(20), 4160. https://doi.org/10.3390/rs13204160