Overcoming the Challenges of Thermal Infrared Orthomosaics Using a Swath-Based Approach to Correct for Dynamic Temperature and Wind Effects
Abstract
:1. Introduction
2. Materials and Methods
2.1. Description of Study Area
2.2. Thermal Data Acquisition
2.3. Field Data Collection
2.4. Thermal Data Pre-Processing
2.5. Orthomosaic Comparison and Development of the Novel Swath-Based Method
- Mosaic: orthophotos are decomposed into high- and low-frequency components. A weighted average is calculated separately for low-frequency and high-frequency components (with different weights), which are subsequently combined into the final orthomosaic, where pixels closer to nadir have higher importance.
- Average: the weighted average pixel value from all available overlapping orthophotos is assigned to the corresponding pixel in the final orthomosaic.
- Disable: each pixel value in the resulting orthomosaic is selected from a single orthophoto among all overlapping orthophotos based on the photo having the view closest to nadir. While the pixel value is not modified and each pixel represents the initially observed temperature at nadir, neighboring pixels may come from different photos.
3. Results
3.1. Calibration of Orthophotos to Remove Inconsistencies
3.2. Flight Direction Analysis
3.3. Novel Swath-Based Correction for Orthomosaicking
3.4. Spatial Intercomparison of Orthomosaics Produced with Different Methods
4. Discussion
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Basso, B.; Antle, J. Digital agriculture to design sustainable agricultural systems. Nature 2020, 3, 254–256. [Google Scholar] [CrossRef]
- Gebbers, R.; Adamchuk, V.I. Precision Agriculture and Food Security. Science 2010, 327, 828–831. [Google Scholar] [CrossRef]
- Whitcraft, A.K.; Becker-Reshef, I.; Justice, C.O.; Gifford, L.; Kavvada, A.; Jarvis, I. No Pixel Left behind: Toward Integrating Earth Observations for Agriculture into the United Nations Sustainable Development Goals Framework. Remote Sens. Environ. 2019, 235, 111470. [Google Scholar] [CrossRef]
- McCabe, M.F.; Rodell, M.; Alsdorf, D.E.; Miralles, D.G.; Uijlenhoet, R.; Wagner, W.; Lucieer, A.; Houborg, R.; Verhoest, N.E.C.; Franz, T.E.; et al. The future of Earth observation in hydrology. Hydrol. Earth Syst. Sci. 2017, 21, 3879–3914. [Google Scholar] [CrossRef] [Green Version]
- Tmušić, G.; Manfreda, S.; Aasen, H.; James, M.R.; Gonçalves, G.; Ben-Dor, E.; Brook, A.; Polinova, M.; Arranz, J.J.; Mészáros, J.; et al. Current Practices in UAS-based Environmental Monitoring. Remote Sens. 2020, 12, 1001. [Google Scholar] [CrossRef] [Green Version]
- Colomina, I.; Molina, P. Unmanned Aerial Systems for Photogrammetry and Remote Sensing: A Review. ISPRS J. Photogramm. Remote Sens. 2014, 92, 79–97. [Google Scholar] [CrossRef] [Green Version]
- Anderson, K.; Gaston, K.J. Lightweight Unmanned Aerial Vehicles will Revolutionize Spatial Ecology. Front. Ecol. Environ. 2013, 11, 138–146. [Google Scholar] [CrossRef] [Green Version]
- Vasterling, M.; Meyer, U. Challenges and Opportunities for UAV-Borne Thermal Imaging. In Thermal Infrared Remote Sensing: Sensors, Methods, Applications; Springer: Dordrecht, The Netherlands, 2013. [Google Scholar]
- Aasen, H.; Honkavaara, E.; Lucieer, A.; Zarco-Tejada, P.J. Quantitative Remote Sensing at Ultra-High Resolution with UAV Spectroscopy: A Review of Sensor Technology, Measurement Procedures, and Data Correctionworkflows. Remote Sens. 2018, 10, 1091. [Google Scholar] [CrossRef] [Green Version]
- Adão, T.; Hruška, J.; Pádua, L.; Bessa, J.; Peres, E.; Morais, R.; Sousa, J. Hyperspectral Imaging: A Review on UAV-Based Sensors, Data Processing and Applications for Agriculture and Forestry. Remote Sens. 2017, 9, 1110. [Google Scholar] [CrossRef] [Green Version]
- Zarco-Tejada, P.J.; Berni, J.A.J.; Suárez, L.; Sepulcre-Cantó, G.; Morales, F.; Miller, J.R. Imaging Chlorophyll Fluorescence with an Airborne Narrow-Band Multispectral Camera for Vegetation Stress Detection. Remote Sens. Environ. 2009, 113, 1262–1275. [Google Scholar] [CrossRef]
- Lin, Y.; Hyyppä, J.; Jaakkola, A. Mini-UAV-Borne LIDAR for Fine-Scale Mapping. IEEE Geosci. Remote Sens. Lett. 2011, 8, 426–430. [Google Scholar] [CrossRef]
- Malbéteau, Y.; Parkes, S.; Aragon, B.; Rosas, J.; McCabe, M.F. Capturing the diurnal cycle of land surface temperature using an unmanned aerial vehicle. Remote Sens. 2018, 10, 1407. [Google Scholar] [CrossRef] [Green Version]
- Ziliani, M.G.; Parkes, S.D.; Hoteit, I.; McCabe, M.F. Intra-Season Crop Height Variability at Commercial Farm Scales Using a Fixed-Wing UAV. Remote Sens. 2018, 10, 2007. [Google Scholar] [CrossRef] [Green Version]
- Berni, J.A.J.; Zarco-Tejada, P.J.; Suárez, L.; Fereres, E.; Suarez, L.; Fereres, E. Thermal and Narrowband Multispectral Remote Sensing for Vegetation Monitoring from an Unmanned Aerial Vehicle. IEEE Trans. Geosci. Remote Sens. 2009, 47, 722–738. [Google Scholar] [CrossRef] [Green Version]
- Baluja, J.; Diago, M.P.; Balda, P.; Zorer, R.; Meggio, F.; Morales, F.; Tardaguila, J. Assessment of Vineyard Water Status Variability by Thermal and Multispectral Imagery Using an Unmanned Aerial Vehicle (UAV). Irrig. Sci. 2012, 30, 511–522. [Google Scholar] [CrossRef]
- Manfreda, S.; McCabe, M.F.; Miller, P.E.; Lucas, R.; Madrigal, V.P.; Mallinis, G.; Dor, E.B.; Helman, D.; Estes, L.; Ciraolo, G.; et al. On the Use of Unmanned Aerial Systems for Environmental Monitoring. Remote Sens. 2018, 10, 641. [Google Scholar] [CrossRef] [Green Version]
- Gago, J.; Douthe, C.; Coopman, R.E.; Gallego, P.P.; Ribas-Carbo, M.; Flexas, J.; Escalona, J.; Medrano, H. UAVs Challenge to Assess Water Stress for Sustainable Agriculture. Agric. Water Manag. 2015, 153, 9–19. [Google Scholar] [CrossRef]
- Bellvert, J.; Zarco-Tejada, J.P.J.; Marsal, J.; Girona, J.; González-Dugo, V.; Fereres, E. Vineyard Irrigation Scheduling Based on Airborne Thermal Imagery and Water Potential Thresholds. Aust. J. Grape Wine Res. 2016, 22, 307–315. [Google Scholar] [CrossRef] [Green Version]
- Santesteban, L.G.; Gennaro, S.F.D.; Herrero-Langreo, A.; Miranda, C.; Royo, J.B.; Matese, A. High-Resolution UAV-Based Thermal Imaging to Estimate the Instantaneous and Seasonal Variability of Plant Water Status within a Vineyard. Agric. Water Manag. 2017, 183, 49–59. [Google Scholar] [CrossRef]
- Tattaris, M.; Reynolds, M.P.; Chapman, S.C. A Direct Comparison of Remote Sensing Approaches for High-Throughput Phenotyping in Plant Breeding. Front. Plant Sci. 2016, 7, 1131. [Google Scholar] [CrossRef]
- Yang, G.; Liu, J.; Zhao, C.; Li, Z.; Huang, Y.; Yu, H.; Xu, B.; Yang, X.; Zhu, D.; Zhang, X.; et al. Unmanned Aerial Vehicle Remote Sensing for Field-Based Crop Phenotyping: Current Status and Perspectives. Front. Plant Sci. 2017, 8, 1111. [Google Scholar] [CrossRef]
- Ludovisi, R.; Tauro, F.; Salvati, R.; Khoury, S.; Mugnozza, G.S.; Harfouche, A. UAV-Based Thermal Imaging for High-Throughput Field Phenotyping of Black Poplar Response to Drought. Front. Plant Sci. 2017, 8, 1681. [Google Scholar] [CrossRef]
- Gómez-Candón, D.; Virlet, N.; Labbé, S.; Jolivot, A.; Regnard, J.L. Field Phenotyping of Water Stress at Tree Scale by UAV-Sensed Imagery: New Insights for Thermal Acquisition and Calibration. Precis. Agric. 2016, 17, 786–800. [Google Scholar] [CrossRef]
- Maimaitijiang, M.; Sagan, V.; Sidike, P.; Hartling, S.; Esposito, F.; Fritschi, F.B. Soybean Yield Prediction from UAV Using Multimodal Data Fusion and Deep Learning. Remote Sens. Environ. 2020, 237, 111599. [Google Scholar] [CrossRef]
- Zarco-Tejada, P.J.; Camino, C.; Beck, P.S.A.; Calderon, R.; Hornero, A.; Hernández-Clemente, R.; Kattenborn, T.; Montes-Borrego, M.; Susca, L.; Morelli, M.; et al. Previsual symptoms of Xylella fastidiosa infection revealed in spectral plant-trait alterations. Nat. Plants 2018, 4, 432–439. [Google Scholar] [CrossRef]
- Calderón, R.; Navas-Cortés, J.A.; Lucena, C.; Zarco-Tejada, P.J. High-Resolution Airborne Hyperspectral and Thermal Imagery for Early Detection of Verticillium Wilt of Olive Using Fluorescence, Temperature and Narrow-Band Spectral Indices. Remote Sens. Environ. 2013, 139, 231–245. [Google Scholar] [CrossRef]
- Sagan, V.; Maimaitijiang, M.; Sidike, P.; Eblimit, K.; Peterson, K.T.; Hartling, S.; Esposito, F.; Khanal, K.; Newcomb, M.; Pauli, D.; et al. UAV-Based High Resolution Thermal Imaging for Vegetation Monitoring, and Plant Phenotyping Using ICI 8640 P, FLIR Vue Pro R 640, and Thermomap Cameras. Remote Sens. 2019, 11, 330. [Google Scholar] [CrossRef] [Green Version]
- Aragon, B.; Johansen, K.; Parkes, S.; Malbeteau, Y.; Al-Mashharawi, S.; Al-Amoudi, T.; Andrade, C.F.; Turner, D.; Lucieer, A.; McCabe, M.F. A Calibration Procedure for Field and UAV-Based Uncooled Thermal Infrared Instruments. Sensors 2020, 20, 3316. [Google Scholar] [CrossRef]
- Kelly, J.; Kljun, N.; Olsson, P.O.; Mihai, L.; Liljeblad, B.; Weslien, P.; Klemedtsson, L.; Eklundh, L. Challenges and Best Practices for Deriving Temperature Data from an Uncalibrated UAV Thermal Infrared Camera. Remote Sens. 2019, 11, 567. [Google Scholar] [CrossRef] [Green Version]
- Perich, G.; Hund, A.; Anderegg, J.; Roth, L.; Boer, M.P.; Walter, A.; Liebisch, F.; Aasen, H. Assessment of Multi-Image Unmanned Aerial Vehicle Based High-Throughput Field Phenotyping of Canopy Temperature. Front. Plant Sci. 2020, 11, 150. [Google Scholar] [CrossRef]
- Zhang, L.; Yaxiao, N.; Zhang, H.; Han, W.; Li, G.; Tang, J.; Peng, X. Maize canopy temperature extracted from UAV thermal and RGB imagery and its application in water stress monitoring. Front. Plant Sci. 2019, 10, 1270. [Google Scholar] [CrossRef]
- Lin, D.; Maas, H.-G.; Westfeld, P.; Budzier, H.; Gerlach, G. An advanced radiometric calibration approach for uncooled thermal cameras. Photogramm. Rec. 2017, 33, 30–48. [Google Scholar] [CrossRef]
- Budzier, H.; Gerlach, G. Calibration of Uncooled Thermal Infrared Cameras. J. Sens. Sens. Syst. 2015, 4, 187–197. [Google Scholar] [CrossRef] [Green Version]
- Ribeiro-Gomes, K.; Hernández-López, D.; Ortega, J.F.; Ballesteros, R.; Poblete, T.; Moreno, M.A. Uncooled Thermal Camera Calibration and Optimization of the Photogrammetry Process for UAV Applications in Agriculture. Sensors 2017, 17, 2173. [Google Scholar] [CrossRef]
- Nugent, P.W.; Shaw, J.A.; Pust, N.J. Correcting for Focal-Plane-Array Temperature Dependence in Microbolometer Infrared Cameras Lacking Thermal Stabilization. Opt. Eng. 2013, 52, 061304. [Google Scholar] [CrossRef] [Green Version]
- Turner, D.; Lucieer, A.; Malenovský, Z.; King, D.; Robinson, S. Spatial Co-Registration of Ultra-High Resolution Visible, Multispectral and Thermal Images Acquired with a Micro-UAV over Antarctic Moss Beds. Remote Sens. 2014, 6, 4003–4024. [Google Scholar] [CrossRef] [Green Version]
- Acorsi, M.G.; Gimenez, L.M.; Martello, M. Assessing the performance of a low-cost thermal camera in proximal and aerial conditions. Remote Sens. 2020, 12, 3591. [Google Scholar] [CrossRef]
- Turner, D.; Lucieer, A.; Wallace, L. Direct Georeferencing of Ultrahigh-Resolution UAV Imagery. IEEE Trans. Geosci. Remote Sens. 2014, 52, 2738–2745. [Google Scholar] [CrossRef]
- Snavely, N.; Seitz, S.M.; Szeliski, R. Modeling the World from Internet Photo Collections. Int. J. Comput. Vis. 2008, 80, 189–210. [Google Scholar] [CrossRef] [Green Version]
- Turner, D.; Lucieer, A.; Watson, C. An Automated Technique for Generating Georectified Mosaics from Ultra-High Resolution Unmanned Aerial Vehicle (UAV) Imagery, Based on Structure from Motion (SFM) Point Clouds. Remote Sens. 2012, 4, 1392–1410. [Google Scholar] [CrossRef] [Green Version]
- Kustas, W.P.; Norman, J.M. Use of Remote Sensing for Evapotranspiration Monitoring over Land Surfaces. Hydrol. Sci. J. 1996, 41, 495–516. [Google Scholar] [CrossRef]
- Wan, Z.; Dozier, J. A Generalized Split-Window Algorithm for Retrieving Land-Surface Temperature from Space. IEEE Trans. Geosci. Remote Sens. 1996, 34, 892–905. [Google Scholar] [CrossRef] [Green Version]
- Mesas-Carrascosa, F.-J.; Pérez-Porras, F.; Larriva, J.E.M.D.; Frau, C.M.; Agüera-Vega, F.; Carvajal-Ramírez, F.; Martínez-Carricondo, P.; García-Ferrer, A. Drift Correction of Lightweight Microbolometer Thermal Sensors On-Board Unmanned Aerial Vehicles. Remote Sens. 2018, 10, 615. [Google Scholar] [CrossRef] [Green Version]
- Johansen, K.; Morton, M.J.L.; Malbeteau, Y.M.; Aragon, B.; Al-Mashharawi, S.K.; Ziliani, M.G.; Angel, Y.; Fiene, G.M.; Negrão, S.S.C.; Mousa, M.A.A.; et al. Unmanned Aerial Vehicle-Based Phenotyping Using Morphometric and Spectral Analysis Can Quantify Responses of Wild Tomato Plants to Salinity Stress. Front. Plant Sci. 2019, 30, 370. [Google Scholar] [CrossRef]
- Ihuoma, S.O.; Madramootoo, C.A. Recent advances in crop water stress detection. Comput. Electron. Agric. 2017, 141, 267–275. [Google Scholar] [CrossRef]
- De Oliveira, A.F.; Dettori, F.R.I.; Azzena, M.; Nieddu, G. UV Light Acclimation Capacity of Leaf Photosynthetic and Photochemical Behaviour in Near-isohydric and Anisohydric Grapevines in Hot and Dry Environments. S. Afr. J. Enol. Vitic. 2019, 40, 188–204. [Google Scholar] [CrossRef] [Green Version]
- Guo, Q.; Zhu, Y.; Tang, Y.; Hou, C.; He, Y.; Zhuang, J.; Zheng, Y.; Luo, S. CFD simulation and experimental verification of the spatial and temporal distributions of the downwash airflow of a quad-rotor agricultural UAV in hover. Comput. Electron. Agric. 2020, 172, 105343. [Google Scholar] [CrossRef]
- Johansen, K.; Morton, M.J.L.; Malbeteau, Y.M.; Aragon, B.; Al-Mashharawi, S.K.; Ziliani, M.G.; Angel, Y.; Fiene, G.M.; Negrão, S.S.C.; Mousa, M.A.A.; et al. Predicting Biomass and Yield in a Tomato Phenotyping Experiment Using UAV Imagery and Random Forest. Front. Artif. Intell. 2020, 3, 28. [Google Scholar] [CrossRef]
- McCabe, M.F.; Balick, L.K.; Theiler, J.; Gillespie, A.R.; Mushkin, A. Linear Mixing in Thermal Infrared Temperature Retrieval. Int. J. Remote Sens. 2008, 29, 5047–5061. [Google Scholar] [CrossRef]
- Niu, H.; Hollenbeck, D.; Zhao, T.; Wang, D.; Chen, Y. Evapotranspiration estimation with small UAVs in precision agriculture. Sensors 2020, 20, 6427. [Google Scholar] [CrossRef]
- Awais, M.; Li, W.; Cheema, M.J.M.; Hussain, S.; Algarni, T.S.; Liu, C.; Ali, A. Remotely sensed identification of canopy characteristics using UAV-based imagery under unstable environmental conditions. Environ. Technol. Innov. 2021, 22, 101465. [Google Scholar] [CrossRef]
- Ezenne, G.I.; Jupp, L.; Mentel, S.K.; Tanner, J.L. Current and potential capabilities of UAS for crop water productivity in precision agriculture. Agric. Water Manag. 2019, 218, 158–164. [Google Scholar] [CrossRef]
- Han, X.; Thomasson, J.A.; Swaminathan, V.; Wang, T.; Siegfried, J.; Raman, R.; Rajan, N.; Neely, H. Field-based calibration of unmanned aerial vehicle thermal infrared imagery with temperature-controlled references. Sensors 2020, 20, 7098. [Google Scholar] [CrossRef] [PubMed]
Survey Date | Starting Time | Flight Duration (min) | Flight Direction (°) | Air Temperature (°C) | Mean Wind Speed (m/s) | Max Wind Speed (m/s) | Min Wind Speed (m/s) |
---|---|---|---|---|---|---|---|
9 November 2017 | 08:27 | 16 | 66/246 | 28.3 | 0.8 | 2.2 | 0.1 |
08:49 | 15 | 156/336 | 28.9 | 0.9 | 2.3 | 0.0 | |
11:06 | 16 | 66/246 | 32.4 | 2.0 | 4.1 | 0.2 | |
12:58 | 16 | 66/246 | 34.3 | 2.0 | 4.7 | 0.1 | |
20 December 2017 | 08:01 | 16 | 66/246 | 21.9 | 2.2 | 4.6 | 0.6 |
08:24 | 15 | 156/336 | 22.9 | 2.0 | 4.5 | 0.5 | |
09:52 | 14 | 66/246 | 27.1 | 1.8 | 3.7 | 0.3 | |
10:19 | 15 | 156/336 | 28.0 | 1.1 | 2.6 | 0.1 | |
11:56 | 15 | 66/246 | 30.6 | 1.2 | 4.2 | 0.1 | |
7 January 2018 | 09:16 | 15 | 66/246 | 23.1 | 3.6 | 8.3 | 0.3 |
09:40 | 15 | 156/336 | 23.8 | 3.0 | 6.8 | 0.4 | |
12:43 | 15 | 66/246 | 28.9 | 2.8 | 5.5 | 0.7 |
Survey Date | Starting Time | Flight Direction (°) | Mean Wind Speed (m/s) | MAD Before Flight Direction Normalization | MAD After Flight Direction Normalization | MAD Difference |
---|---|---|---|---|---|---|
9 November 2017 | 08:27 | 66/246 | 0.8 | 0.80 | 0.62 | 0.18 |
08:49 | 156/336 | 0.9 | 0.95 | 0.67 | 0.28 | |
11:06 | 66/246 | 2.0 | 1.79 | 1.19 | 0.6 | |
12:58 | 66/246 | 2.0 | 1.36 | 1.01 | 0.35 | |
20 December 2017 | 08:01 | 66/246 | 2.2 | 1.86 | 0.90 | 0.96 |
08:24 | 156/336 | 2.0 | 0.74 | 0.57 | 0.17 | |
09:52 | 66/246 | 1.8 | 1.96 | 1.22 | 0.74 | |
10:19 | 156/336 | 1.1 | 1.04 | 0.79 | 0.25 | |
11:56 | 66/246 | 1.2 | 1.35 | 1.15 | 0.2 | |
7 January 2018 | 09:16 | 66/246 | 3.6 | 1.79 | 1.35 | 0.44 |
09:40 | 156/336 | 3.0 | 0.66 | 0.62 | 0.04 | |
12:43 | 66/246 | 2.8 | 1.53 | 1.19 | 0.34 |
Mosaicking Method | R2 | MAE (°C) | MD (°C) | RMSE (°C) |
---|---|---|---|---|
Average | 0.99 | 1.39 | 1.07 | 1.66 |
Disable | 0.99 | 1.34 | 0.69 | 1.63 |
Mosaic | 0.99 | 1.38 | 0.96 | 1.63 |
Swath | 0.99 | 1.07 | 0.47 | 1.23 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Malbéteau, Y.; Johansen, K.; Aragon, B.; Al-Mashhawari, S.K.; McCabe, M.F. Overcoming the Challenges of Thermal Infrared Orthomosaics Using a Swath-Based Approach to Correct for Dynamic Temperature and Wind Effects. Remote Sens. 2021, 13, 3255. https://doi.org/10.3390/rs13163255
Malbéteau Y, Johansen K, Aragon B, Al-Mashhawari SK, McCabe MF. Overcoming the Challenges of Thermal Infrared Orthomosaics Using a Swath-Based Approach to Correct for Dynamic Temperature and Wind Effects. Remote Sensing. 2021; 13(16):3255. https://doi.org/10.3390/rs13163255
Chicago/Turabian StyleMalbéteau, Yoann, Kasper Johansen, Bruno Aragon, Samir K. Al-Mashhawari, and Matthew F. McCabe. 2021. "Overcoming the Challenges of Thermal Infrared Orthomosaics Using a Swath-Based Approach to Correct for Dynamic Temperature and Wind Effects" Remote Sensing 13, no. 16: 3255. https://doi.org/10.3390/rs13163255