Domain-Guided Machine Learning for Remotely Sensed In-Season Crop Growth Estimation
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Area and Data
2.2. Data Standardization
2.3. Neural Network Design
2.4. NN Loss, Validation and Evaluation
2.5. Visualization of NN Operation
3. Results
3.1. Model Validation
3.2. Model Evaluation
3.3. UMAP Visualization of Layer Activations
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Ray, D.K.; Gerber, J.S.; MacDonald, G.K.; West, P.C. Climate variation explains a third of global crop yield variability. Nat. Commun. 2015, 6, 5989. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Iizumi, T.; Ramankutty, N. Changes in yield variability of major crops for 1981–2010 explained by climate change. Environ. Res. Lett. 2016, 11, 034003. [Google Scholar] [CrossRef]
- Mehrabi, Z.; Ramankutty, N. Synchronized failure of global crop production. Nat. Ecol. Evol. 2019, 3, 780–786. [Google Scholar] [CrossRef] [Green Version]
- Shen, Y.; Wu, L.; Di, L.; Yu, G.; Tang, H.; Yu, G.; Shao, Y. Hidden Markov Models for Real-Time Estimation of Corn Progress Stages Using MODIS and Meteorological Data. Remote Sens. 2013, 5, 1734–1753. [Google Scholar] [CrossRef] [Green Version]
- Zeng, L.; Wardlow, B.D.; Wang, R.; Shan, J.; Tadesse, T.; Hayes, M.J.; Li, D. A hybrid approach for detecting corn and soybean phenology with time-series MODIS data. Remote Sens. Environ. 2016, 181, 237–250. [Google Scholar] [CrossRef]
- Seo, B.; Lee, J.; Lee, K.D.; Hong, S.; Kang, S. Improving remotely-sensed crop monitoring by NDVI-based crop phenology estimators for corn and soybeans in Iowa and Illinois, USA. Field Crops Res. 2019, 238, 113–128. [Google Scholar] [CrossRef]
- Diao, C. Remote sensing phenological monitoring framework to characterize corn and soybean physiological growing stages. Remote Sens. Environ. 2020, 248, 111960. [Google Scholar] [CrossRef]
- Ghamghami, M.; Ghahreman, N.; Irannejad, P.; Pezeshk, H. A parametric empirical Bayes (PEB) approach for estimating maize progress percentage at field scale. Agric. For. Meteorol. 2020, 281, 107829. [Google Scholar] [CrossRef]
- Orynbaikyzy, A.; Gessner, U.; Conrad, C. Crop type classification using a combination of optical and radar remote sensing data: A review. Int. J. Remote Sens. 2019, 40, 6553–6595. [Google Scholar] [CrossRef]
- Jia, X.; Khandelwal, A.; Mulla, D.J.; Pardey, P.G.; Kumar, V. Bringing automated, remote-sensed, machine learning methods to monitoring crop landscapes at scale. Agric. Econ. 2019, 50, 41–50. [Google Scholar] [CrossRef]
- Kerner, H.; Sahajpal, R.; Skakun, S.; Becker-Reshef, I.; Barker, B.; Hosseini, M.; Puricelli, E.; Gray, P. Resilient In-Season Crop Type Classification in Multispectral Satellite Observations using Growth Stage Normalization. arXiv 2020, arXiv:2009.10189. [Google Scholar]
- Teimouri, N.; Dyrmann, M.; Jørgensen, R.N. A Novel Spatio-Temporal FCN-LSTM Network for Recognizing Various Crop Types Using Multi-Temporal Radar Images. Remote Sens. 2019, 11, 990. [Google Scholar] [CrossRef] [Green Version]
- Weiss, M.; Jacob, F.; Duveiller, G. Remote sensing for agricultural applications: A meta-review. Remote Sens. Environ. 2020, 236, 111402. [Google Scholar] [CrossRef]
- USDA National Agricultural Statistics Service Cropland Data Layer. 2008–2019. Type: Dataset. Available online: http://nassgeodata.gmu.edu/CropScape/ (accessed on 5 November 2020).
- Karpatne, A.; Atluri, G.; Faghmous, J.H.; Steinbach, M.; Banerjee, A.; Ganguly, A.; Shekhar, S.; Samatova, N.; Kumar, V. Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data. IEEE Trans. Knowl. Data Eng. 2017, 29, 2318–2331. [Google Scholar] [CrossRef]
- Khandelwal, A.; Xu, S.; Li, X.; Jia, X.; Stienbach, M.; Duffy, C.; Nieber, J.; Kumar, V. Physics Guided Machine Learning Methods for Hydrology. arXiv 2020, arXiv:2012.02854. [Google Scholar]
- Willard, J.; Jia, X.; Xu, S.; Steinbach, M.; Kumar, V. Integrating Physics-Based Modeling with Machine Learning: A Survey. arXiv 2020, arXiv:2003.04919. [Google Scholar]
- Karpatne, A.; Watkins, W.; Read, J.; Kumar, V. Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling. arXiv 2018, arXiv:1710.11431. [Google Scholar]
- Hu, X.; Hu, H.; Verma, S.; Zhang, Z.L. Physics-Guided Deep Neural Networks for Power Flow Analysis. IEEE Trans. Power Syst. 2020, 36, 2082–2092. [Google Scholar] [CrossRef]
- Shahhosseini, M.; Hu, G.; Huber, I.; Archontoulis, S.V. Coupling machine learning and crop modeling improves crop yield prediction in the US Corn Belt. Sci. Rep. 2021, 11, 1606. [Google Scholar] [CrossRef] [PubMed]
- Chen, T.S.; Aoike, T.; Yamasaki, M.; Kajiya-Kanegae, H.; Iwata, H. Predicting rice heading date using an integrated approach combining a machine learning method and a crop growth model. Front. Genet. 2020, 11, 1643. [Google Scholar] [CrossRef] [PubMed]
- Rong, M.; Zhang, D.; Wang, N. A Lagrangian Dual-based Theory-guided Deep Neural Network. arXiv 2020, arXiv:2008.10159. [Google Scholar]
- Wang, N.; Zhang, D.; Chang, H.; Li, H. Deep learning of subsurface flow via theory-guided neural network. J. Hydrol. 2020, 584, 124700. [Google Scholar] [CrossRef] [Green Version]
- USDA National Agricultural Statistics Service. World Agricultural Production; Technical Report WAP 4-21; USDA Foreign Agricultural Service: Washington, DC, USA, 2021. [Google Scholar]
- USDA National Agricultural Statistics Service. NASS—Quick Stats. 2017: Corn—Acres Planted. Type: Dataset. Available online: https://data.nal.usda.gov/dataset/nass-quick-stats (accessed on 15 March 2021).
- Wang, S.; Di Tommaso, S.; Deines, J.M.; Lobell, D.B. Mapping twenty years of corn and soybean across the US Midwest using the Landsat archive. Sci. Data 2020, 7, 307. [Google Scholar] [CrossRef]
- Lizaso, J.I.; Boote, K.J.; Jones, J.W.; Porter, C.H.; Echarte, L.; Westgate, M.E.; Sonohat, G. CSM-IXIM: A New Maize Simulation Model for DSSAT Version 4.5. Agron. J. 2011, 103, 766–779. [Google Scholar] [CrossRef]
- Kiniry, J.R. Maize Phasic Development. In Agronomy Monographs; Hanks, J., Ritchie, J.T., Eds.; American Society of Agronomy, Crop Science Society of America, Soil Science Society of America: Madison, WI, USA, 2015; pp. 55–70. [Google Scholar] [CrossRef]
- Thornton, M.; Shrestha, R.; Wei, Y.; Thornton, P.; Kao, S.; Wilson, B. Daymet: Daily Surface Weather Data on a 1-km Grid for North America, Version 4; ORNL Distributed Active Archive Center: Oak Ridge, TN, USA, 2020. [Google Scholar] [CrossRef]
- Myneni, Y.R.; Knyazikhin, T.P. MCD15A3H MODIS/Terra+Aqua Leaf Area Index/FPAR 4-day L4 Global 500m SIN Grid V006. NASA EOSDIS Land Processes DAAC. 2015. Available online: https://lpdaac.usgs.gov/products/mcd15a3hv006/ (accessed on 5 November 2020).
- Staff, S.S. Gridded Soil Survey Geographic (gSSURGO) Database for the United States of America and the Territories, Commonwealths, and Island Nations served by the USDA-NRCS; Technical Report; USDA National Resource Conversvation Service: Washington, DC, USA, 2020. [Google Scholar]
- Chen, J.; Jönsson, P.; Tamura, M.; Gu, Z.; Matsushita, B.; Eklundh, L. A simple method for reconstructing a high-quality NDVI time-series data set based on the Savitzky–Golay filter. Remote Sens. Environ. 2004, 91, 332–344. [Google Scholar] [CrossRef]
- Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef] [PubMed]
- Yu, Y.; Si, X.; Hu, C.; Zhang, J. A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures. Neural Comput. 2019, 31, 1235–1270. [Google Scholar] [CrossRef]
- Khaki, S.; Wang, L.; Archontoulis, S.V. A CNN-RNN Framework for Crop Yield Prediction. Front. Plant Sci. 2020, 10, 1750. [Google Scholar] [CrossRef] [PubMed]
- Graves, A. Long short-term memory. In Supervised Sequence Labelling with Recurrent Neural Networks; Springer: Berlin/Heidelberg, Germany, 2012; pp. 37–45. [Google Scholar]
- Warrington, I.J.; Kanemasu, E.T. Corn Growth Response to Temperature and Photoperiod I. Seedling Emergence, Tassel Initiation, and Anthesis. Agron. J. 1983, 75, 749–754. [Google Scholar] [CrossRef]
- Warrington, I.J.; Kanemasu, E.T. Corn Growth Response to Temperature and Photoperiod II. Leaf-Initiation and Leaf-Appearance Rates. Agron. J. 1983, 75, 755–761. [Google Scholar] [CrossRef]
- NeSmith, D.; Ritchie, J. Short- and Long-Term Responses of Corn to a Pre-Anthesis Soil Water Deficit. Agron. J. 1992, 84, 107–113. [Google Scholar] [CrossRef]
- Çakir, R. Effect of water stress at different development stages on vegetative and reproductive growth of corn. Field Crops Res. 2004, 89, 1–16. [Google Scholar] [CrossRef]
- Jones, C.; Ritchie, J.; Kiniry, J.; Godwin, D.; Otter, S. The CERES wheat and maize models. In Proceedings of the International Symposium on Minimum Data Sets for Agrotechnology Transfer, ICRISAT Center, Patancheru, India, 21–26 March 1983; pp. 95–100. [Google Scholar]
- Holzworth, D.P.; Huth, N.I.; deVoil, P.G.; Zurcher, E.J.; Herrmann, N.I.; McLean, G.; Chenu, K.; van Oosterom, E.J.; Snow, V.; Murphy, C.; et al. APSIM–Evolution towards a new generation of agricultural systems simulation. Environ. Model. Softw. 2014, 62, 327–350. [Google Scholar] [CrossRef]
- Young, T.; Hazarika, D.; Poria, S.; Cambria, E. Recent Trends in Deep Learning Based Natural Language Processing. IEEE Comput. Intell. Mag. 2018, 13, 55–75. [Google Scholar] [CrossRef]
- Liu, Y.; Sun, C.; Lin, L.; Wang, X. Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention. arXiv 2016, arXiv:1605.09090. [Google Scholar]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention Is All You Need. arXiv 2017, arXiv:1706.03762. [Google Scholar]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
- McInnes, L.; Healy, J.; Melville, J. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv 2020, arXiv:1802.03426. [Google Scholar]
- Becht, E.; McInnes, L.; Healy, J.; Dutertre, C.A.; Kwok, I.W.H.; Ng, L.G.; Ginhoux, F.; Newell, E.W. Dimensionality reduction for visualizing single-cell data using UMAP. Nat. Biotechnol. 2019, 37, 38–44. [Google Scholar] [CrossRef] [PubMed]
- USDA-National Agricultural Statistics Service Upper Midwest Region, Iowa Field Office. 2010 Iowa Agricultural Statistics; Technical Report; USDA: Washington, DC, USA, 2010. [Google Scholar]
- USDA-National Agricultural Statistics Service Upper Midwest Region, Iowa Field Office. 2013 Iowa Agricultural Statistics; Technical Report; USDA: Washington, DC, USA, 2013. [Google Scholar]
- USDA-National Agricultural Statistics Service Upper Midwest Region, Iowa Field Office. 2015 Iowa Agricultural Statistics; Technical Report; USDA: Washington, DC, USA, 2015. [Google Scholar]
- USDA-National Agricultural Statistics Service Upper Midwest Region, Iowa Field Office. 2020 Iowa Agricultural Statistics; Technical Report; USDA: Washington, DC, USA, 2020. [Google Scholar]
- USDA National Agricultural Statistics Service. NASS—Quick Stats. 2017: Corn—Crop Condition. Type: Dataset. Available online: https://data.nal.usda.gov/dataset/nass-quick-stats (accessed on 15 March 2021).
- Meyes, R.; Lu, M.; de Puiseau, C.W.; Meisen, T. Ablation Studies in Artificial Neural Networks. arXiv 2019, arXiv:1901.08644. [Google Scholar]
- Peerlinck, A.; Sheppard, J.; Senecal, J. AdaBoost with Neural Networks for Yield and Protein Prediction in Precision Agriculture. In Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 14–19 July 2019; IEEE: Budapest, Hungary, 2019; pp. 1–8. [Google Scholar] [CrossRef]
- Kussul, N.; Lavreniuk, M.; Shelestov, A.; Yailymov, B. Along the season crop classification in Ukraine based on time series of optical and SAR images using ensemble of neural network classifiers. In Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 10–15 July 2016; IEEE: New York, NY, USA, 2016; pp. 7145–7148. [Google Scholar]
Parameter | Source | Wavelength | Spat. Res. | Temp. Res. | Orig. Units | Input Units | Reference |
---|---|---|---|---|---|---|---|
Solar Radiation | ORNL DayMet | - | 1 km | Daily | W/m2 | MJ/m2/week | [29] |
Temperature | ORNL DayMet | - | 1 km | Daily | degrees C | AGDD | [29] |
Rainfall | ORNL DayMet | - | 1 km | Daily | mm/day | mm/week | [29] |
FPAR | NASA MODIS Aqua/Terra | VIS/NIR | 250 m+ | 4-day | - | - | [30] |
Soil Hydrologic Conductivity | USDA-NRSC gSSURGO | - | 30 m | Constant | μm/s | μm/s | [31] |
Soil Bulk Density | USDA-NRSC gSSURGO | - | 30 m | Constant | g/cm3 | g/cm3 | [31] |
Year | AGDD (Full Season) (°C) | Precipitation (April–June) (mm) | Precipitation (Full Season) (mm) | Solar Radiation (April–June) (W/m2/day) | Solar Radiation (Full Season) (W/m2/day) |
---|---|---|---|---|---|
2009 | −277.1 (−2.285) | −52.7 (−0.825) | +43.7 (+0.388) | +19.81 (+0.476) | +5.94 (+0.115) |
2012 | +135.4 (+1.116) | −65.8 (−1.029) | −193.4 (−1.718) | +24.00 (+0.576) | +84.54 (+1.635) |
2014 | −127.0 (−1.047) | +101.2 (+1.584) | +104.5 (+0.928) | −20.81 (−0.500) | −11.94 (−0.231) |
2019 | −80.6 (−0.664) | +15.9 (+0.249) | +99.9 (+0.887) | −23.50 (−0.565) | −87.77 (−1.697) |
Year | Emerged (days) | Silking (days) | Grainfill (days) | Mature (days) | Harvested (days) |
---|---|---|---|---|---|
2009 | −2 (−0.397) | +4 (+0.774) | +11 (+1.58) | +9 (+1.00) | +20 (+1.98) |
2012 | −6 (−1.19) | −10 (−1.93) | −10 (−1.42) | −16 (−2.01) | −25 (−2.48) |
2014 | +3 (+0.60) | −1 (−0.19) | −4 (−0.57) | +6 (+0.75) | +6 (+0.60) |
2019 | +11 (+2.18) | +2 (+0.39) | +2 (+0.29) | +16 (+2.01) | +13 (+1.29) |
Model | NS-Emerged | NS-Silking | NS-Grainfill | NS-Mature | NS-Harvested |
---|---|---|---|---|---|
HMM | 0.874 (0.041) | 0.624 (0.166) | 0.785 (0.060) | 0.688 (0.160) | 0.931 (0.062) |
Dense NN | 0.949 (0.041) | 0.863 (0.123) | 0.890 (0.062) | 0.869 (0.087) | 0.966 (0.034) |
Sequential NN | 0.979 (0.024) | 0.890 (0.108) | 0.884 (0.132) | 0.906 (0.075) | 0.982 (0.028) |
DgNN | 0.984 (0.013) | 0.914 (0.077) | 0.923 (0.063) | 0.935 (0.043) | 0.988 (0.014) |
Model | NS-Emerged | NS-Silking | NS-Grainfill | NS-Mature | NS-Harvested |
---|---|---|---|---|---|
HMM | 0.891 (0.049) | 0.475 (0.247) | 0.732 (0.106) | 0.245 (0.468) | 0.786 (0.117) |
Dense NN | 0.932 (0.016) | 0.761 (0.094) | 0.745 (0.185) | 0.635 (0.375) | 0.917 (0.062) |
Sequential NN | 0.947 (0.048) | 0.823 (0.048) | 0.841 (0.097) | 0.758 (0.186) | 0.936 (0.031) |
DgNN | 0.964 (0.015) | 0.790 (0.127) | 0.877 (0.026) | 0.870 (0.126) | 0.976 (0.021) |
Test Year | 2009 | 2012 | 2014 | 2019 | Total |
---|---|---|---|---|---|
Dense NN | 14 | 9 | 14 | 15 | 44 |
Sequential NN | 18 | 19 | 18 | 15 | 70 |
DgNN | 19 | 32 | 24 | 22 | 97 |
Model | NS-Emerged | NS-Silking | NS-Grainfill | NS-Mature | NS-Harvested |
---|---|---|---|---|---|
2009 | |||||
HMM | 0.855 | 0.767 | 0.865 | 0.454 | 0.609 |
Dense NN | 0.915 | 0.610 | 0.781 | 0.904 | 0.933 |
Sequential NN | 0.993 | 0.896 | 0.880 | 0.856 | 0.898 |
DgNN | 0.987 | 0.917 | 0.907 | 0.952 | 0.991 |
2012 | |||||
HMM | 0.868 | 0.671 | 0.571 | -0.537 | 0.763 |
Dense NN | 0.930 | 0.802 | 0.436 | -0.010 | 0.814 |
Sequential NN | 0.956 | 0.836 | 0.677 | 0.448 | 0.914 |
DgNN | 0.954 | 0.908 | 0.837 | 0.659 | 0.942 |
2014 | |||||
HMM | 0.975 | 0.258 | 0.721 | 0.703 | 0.921 |
Dense NN | 0.958 | 0.767 | 0.848 | 0.869 | 0.975 |
Sequential NN | 0.867 | 0.789 | 0.933 | 0.933 | 0.978 |
DgNN | 0.967 | 0.706 | 0.891 | 0.980 | 0.994 |
2019 | |||||
HMM | 0.864 | 0.203 | 0.770 | 0.359 | 0.851 |
Dense NN | 0.924 | 0.865 | 0.915 | 0.778 | 0.947 |
Sequential NN | 0.973 | 0.771 | 0.873 | 0.797 | 0.954 |
DgNN | 0.948 | 0.627 | 0.875 | 0.891 | 0.978 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Worrall, G.; Rangarajan, A.; Judge, J. Domain-Guided Machine Learning for Remotely Sensed In-Season Crop Growth Estimation. Remote Sens. 2021, 13, 4605. https://doi.org/10.3390/rs13224605
Worrall G, Rangarajan A, Judge J. Domain-Guided Machine Learning for Remotely Sensed In-Season Crop Growth Estimation. Remote Sensing. 2021; 13(22):4605. https://doi.org/10.3390/rs13224605
Chicago/Turabian StyleWorrall, George, Anand Rangarajan, and Jasmeet Judge. 2021. "Domain-Guided Machine Learning for Remotely Sensed In-Season Crop Growth Estimation" Remote Sensing 13, no. 22: 4605. https://doi.org/10.3390/rs13224605