Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method
Abstract
:1. Introduction
2. Methodology
2.1. Data Overview
2.2. Acceleration Techniques
2.2.1. Summary of the Cluster Low-Streams Regression (CLSR) Method
2.2.2. Double Cluster Low-Streams Regression Method
2.3. CLSR Method: Improvement to Aerosol Scheme
3. Results and Discussion
3.1. Single CLSR vs. Double CLSR: Accuracy Results
- More than 70% and 60% of the residuals are below 0.01% for the single and double CLSR methods, respectively, for all bands, with the exception of the water vapour band.
- The residuals of the water vapour band present a wider distribution in comparison with the other spectral bands.
- The probability densities are almost indistinguishable for both acceleration methods, demonstrating that both techniques provide accurate results among the different spectral bands.
3.2. Single CLSR vs. Double CLSR: Computational Performance
3.3. Computational Performance: State-of-the-Art Acceleration Techniques
- For simulations in the Hartley–Huggins band, PCA techniques, linear embedding methods (LEM) and double CLSR have been applied. The double CLSR does not further improve the performance, since the computational burden is due to the MS RTM computations (see Section 3.2). The highest acceleration factor is provided by the method described in [12], in which PCA is applied to both optical parameters and spectral radiances. The performance enhancement in this case is up to 18 times.
- There are several studies in which fast RTMs for the O A- and CO bands (either weak or strong) have been designed. In general, all considered techniques provide acceleration factors of about 2–3 orders of magnitude, including those based on artificial neural networks (NN) [37].
- The water vapour band represents a challenge for acceleration techniques due to its complicated spectral structure. Therefore, the accuracy of the acceleration techniques is lower than for the O A-band. For this band, the double CLSR method provides an acceleration factor of about 3 orders of magnitude, while the k-distribution [32] and PCA-based RTMs [19] achieve lower acceleration factors, of one order of magnitude.
3.4. Further Improvements to Aerosol Schemes
3.5. Combined Application of the Single CLSR vs. Double CLSR Method for Aerosol Scenarios
- The residual distributions of the single CLSR method are narrower than those of the double CLSR method, meaning that the single CLSR method is more accurate. However, in general, the residuals are below 0.01% for both methods and all spectral bands, except for the water vapour band, where the residual distributions are slightly wider and still below 0.05%. The distributions are not biased.
- For the Hartley–Huggins, O A- and CO bands, the residuals are below 0.05% for both single and double CLSR methods. Regarding the water vapour band, the residuals are below 0.05% and 0.1% for the single and double CLSR method, respectively. Similar accuracies were achieved in Kopparla et al. [19] for the water vapour band using the PCA-based RTM.
- In the case of the low aerosol load, the probability density functions are similar for all -configurations. However, as the aerosol load increases, the residual distributions for the configuration provided by the single and double CLSR methods sometimes become biased for the water vapour band.
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
AOD | Aerosol Optical Depth |
CLSR | Cluster Low-Streams Interpolation |
DOME | Discrete Ordinates with Matrix Exponential |
HITRAN | High-Resolution Transmission Molecular Absorption Database |
LBL | Line-By-Line |
LEM | Linear Embedding Methods |
LSI | Low-Streams Interpolation |
LUT | LookUp Table |
MS | Multi-Stream |
NN | Neural Network |
OPAC | Optical Properties of Aerosols and Clouds |
PCA | Principal Component Analysis |
Py4CAts | Python for Computational Atmospheric Spectroscopy |
RTM | Radiative Transfer Model |
SDCOMP | Spectral Data Compression |
SS | Single-Scattering |
SSA | Single Scattering Albedo |
TOA | Top-of-the-Atmosphere |
TS | Two-Stream |
UV | Ultraviolet |
References
- Goody, R.; West, R.; Chen, L.; Crisp, D. The correlated-k method for radiation calculations in nonhomogeneous atmospheres. J. Quant. Spectrosc. Radiat. Transf. 1989, 42, 539–550. [Google Scholar] [CrossRef]
- Fu, Q.; Liou, K. On the correlated k-distribution method for radiative transfer in nonhomogeneous atmospheres. J. Atmos. Sci. 1992, 49, 2139–2156. [Google Scholar] [CrossRef] [Green Version]
- Kato, S.; Ackerman, T.P.; Mather, J.H.; Clothiaux, E.E. The k-distribution method and correlated-k approximation for a shortwave radiative transfer model. J. Quant. Spectrosc. Radiat. Transf. 1999, 62, 109–121. [Google Scholar] [CrossRef]
- Zhang, F.; Zhu, M.; Li, J.; Li, W.; Di, D.; Shi, Y.N.; Wu, K. Alternate Mapping Correlated k-Distribution Method for Infrared Radiative Transfer Forward Simulation. Remote Sens. 2019, 11, 994. [Google Scholar] [CrossRef] [Green Version]
- O’Dell, C.W. Acceleration of multiple-scattering, hyperspectral radiative transfer calculations via low-streams interpolation. J. Geophys. Res. 2010, 115. [Google Scholar] [CrossRef]
- Natraj, V.; Jiang, X.; Shia, R.; Huang, X.; Margolis, J.; Yung, Y. Application of the principal component analysis to high spectral resolution radiative transfer: A case study of the O2 A-band. J. Quant. Spectrosc. Radiat. Transf. 2005, 95, 539–556. [Google Scholar] [CrossRef]
- Kopparla, P.; Natraj, V.; Spurr, R.; Shia, R.L.; Crisp, D.; Yung, Y.L. A fast and accurate PCA based radiative transfer model: Extension to the broadband shortwave region. J. Quant. Spectrosc. Radiat. Transf. 2016, 173, 65–71. [Google Scholar] [CrossRef]
- Efremenko, D.; Doicu, A.; Loyola, D.; Trautmann, T. Acceleration techniques for the discrete ordinate method. J. Quant. Spectrosc. Radiat. Transf. 2013, 114, 73–81. [Google Scholar] [CrossRef]
- Efremenko, D.; Doicu, A.; Loyola, D.; Trautmann, T. Optical property dimensionality reduction techniques for accelerated radiative transfer performance: Application to remote sensing total ozone retrievals. J. Quant. Spectrosc. Radiat. Transf. 2014, 133, 128–135. [Google Scholar] [CrossRef]
- Liu, X.; Smith, W.L.; Zhou, D.K.; Larar, A. Principal component-based radiative transfer model for hyperspectral sensors: Theoretical concept. Appl. Opt. 2006, 45, 201–208. [Google Scholar] [CrossRef]
- Molina García, V.; Sasi, S.; Efremenko, D.; Doicu, A.; Loyola, D. Radiative transfer models for retrieval of cloud parameters from EPIC/DSCOVR measurements. J. Quant. Spectrosc. Radiat. Transf. 2018, 213, 228–240. [Google Scholar] [CrossRef] [Green Version]
- del Águila, A.; Efremenko, D.S.; Molina García, V.; Xu, J. Analysis of two dimensionality reduction techniques for fast simulation of the spectral radiances in the Hartley-Huggins band. Atmosphere 2019, 10, 142. [Google Scholar] [CrossRef] [Green Version]
- del Águila, A.; Efremenko, D.S.; Trautmann, T. A review of dimensionality reduction techniques for processing hyper-spectral optical signal. Light Eng. 2019, 27, 85–98. [Google Scholar] [CrossRef] [Green Version]
- del Águila, A.; Efremenko, D.S.; Molina García, V.; Kataev, M.Y. Cluster Low-Streams Regression Method for Hyperspectral Radiative Transfer Computations: Cases of O2 A- and CO2 Bands. Remote Sens. 2020, 12, 1250. [Google Scholar] [CrossRef] [Green Version]
- Vincent, R.A.; Dudhia, A. Fast radiative transfer using monochromatic look-up tables. J. Quant. Spectrosc. Radiat. Transf. 2017, 186, 254–264. [Google Scholar] [CrossRef] [Green Version]
- Duan, M.; Min, Q.; Li, J. A fast radiative transfer model for simulating high-resolution absorption bands. J. Geophys. Res. Atmos. 2005, 110. [Google Scholar] [CrossRef] [Green Version]
- Liu, C.; Yao, B.; Natraj, V.; Kopparla, P.; Weng, F.; Le, T.; Shia, R.L.; Yung, Y.L. A Spectral Data Compression (SDCOMP) Radiative Transfer Model for High-Spectral-Resolution Radiation Simulations. J. Atmos. Sci. 2020, 77, 2055–2066. [Google Scholar] [CrossRef] [Green Version]
- Doicu, A.; Efremenko, D.; Trautmann, T. A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method. Remote Sens. 2020, 12, 3703. [Google Scholar] [CrossRef]
- Kopparla, P.; Natraj, V.; Limpasuvan, D.; Spurr, R.; Crisp, D.; Shia, R.L.; Somkuti, P.; Yung, Y.L. PCA-based radiative transfer: Improvements to aerosol scheme, vertical layering and spectral binning. J. Quant. Spectrosc. Radiat. Transf. 2017, 198, 104–111. [Google Scholar] [CrossRef]
- Schreier, F.; Gimeno García, S.; Hochstaffl, P.; Städt, S. Py4CAtS—PYthon for Computational Atmospheric Spectroscopy. Atmosphere 2019, 10, 262. [Google Scholar] [CrossRef] [Green Version]
- Gordon, I.; Rothman, L.; Hill, C.; Kochanov, R.; Tan, Y.; Bernath, P.; Birk, M.; Boudon, V.; Campargue, A.; Chance, K.; et al. The HITRAN2016 molecular spectroscopic database. J. Quant. Spectrosc. Radiat. Transf. 2017, 203, 3–69. [Google Scholar] [CrossRef]
- Bodhaine, B.; Wood, N.; Dutton, E.; Slusser, J. On Rayleigh optical depth calculations. J. Atmos. Ocean. Technol. 1999, 16, 1854–1861. [Google Scholar] [CrossRef]
- Anderson, G.; Clough, S.; Kneizys, F.; Chetwynd, J.; Shettle, E. AFGL Atmospheric Constituent Profiles (0.120 km); Air ForceGeophysics Lab.: Hanscom AFB, MA, USA, 1986; p. 46. [Google Scholar]
- Doicu, A.; Trautmann, T. Discrete-ordinate method with matrix exponential for a pseudo-spherical atmosphere: Scalar case. J. Quant. Spectrosc. Radiat. Transf. 2009, 110, 146–158. [Google Scholar] [CrossRef]
- Efremenko, D.S.; Molina García, V.; Gimeno García, S.; Doicu, A. A review of the matrix-exponential formalism in radiative transfer. J. Quant. Spectrosc. Radiat. Transf. 2017, 196, 17–45. [Google Scholar] [CrossRef] [Green Version]
- Wiscombe, W.J. The Delta-M Method: Rapid Yet Accurate Radiative Flux Calculations for Strongly Asymmetric Phase Functions. J. Atmos. Sci. 1977, 34, 1408–1422. [Google Scholar] [CrossRef] [Green Version]
- Nakajima, T.; Tanaka, M. Algorithms for radiative intensity calculations in moderately thick atmospheres using a truncation approximation. J. Quant. Spectrosc. Radiat. Transf. 1988, 40, 51–69. [Google Scholar] [CrossRef]
- Fischer, J.; Grassl, H. Radiative transfer in an atmosphere-ocean system: An azimuthally dependent matrix-operator approach. Appl. Opt. 1984, 23, 1032. [Google Scholar] [CrossRef]
- Chalhoub, E.; Garcia, R. The equivalence between two techniques of angular interpolation for the discrete-ordinates method. J. Quant. Spectrosc. Radiat. Transf. 2000, 64, 517–535. [Google Scholar] [CrossRef]
- Hess, M.; Koepke, P.; Schult, I. Optical properties of aerosols and clouds: The software package OPAC. Bull. Am. Meteorol. Soc. 1998, 79, 831–844. [Google Scholar] [CrossRef]
- Huang, Y.; Natraj, V.; Zeng, Z.C.; Kopparla, P.; Yung, Y.L. Quantifying the impact of aerosol scattering on the retrieval of methane from airborne remote sensing measurements. Atmos. Meas. Tech. 2020, 13, 6755–6769. [Google Scholar] [CrossRef]
- Fomin, B. A k-distribution technique for radiative transfer simulation in inhomogeneous atmosphere: 2. FKDM, fast k-distribution model for the shortwave. J. Geophys. Res. 2005, 110. [Google Scholar] [CrossRef] [Green Version]
- Natraj, V.; Shia, R.L.; Yung, Y.L. On the use of principal component analysis to speed up radiative transfer calculations. J. Quant. Spectrosc. Radiat. Transf. 2010, 111, 810–816. [Google Scholar] [CrossRef]
- Spurr, R.; Natraj, V.; Lerot, C.; Roozendael, M.V.; Loyola, D. Linearization of the Principal Component Analysis method for radiative transfer acceleration: Application to retrieval algorithms and sensitivity studies. J. Quant. Spectrosc. Radiat. Transf. 2013, 125, 1–17. [Google Scholar] [CrossRef]
- Efremenko, D.S.; Loyola, D.G.; Spurr, R.J.; Doicu, A. Acceleration of radiative transfer model calculations for the retrieval of trace gases under cloudy conditions. J. Quant. Spectrosc. Radiat. Transf. 2014, 135, 58–65. [Google Scholar] [CrossRef]
- Somkuti, P.; Boesch, H.; Natraj, V.; Kopparla, P. Application of a PCA-Based Fast Radiative Transfer Model to XCO2 Retrievals in the Shortwave Infrared. J. Geophys. Res. Atmos. 2017, 122, 10,477–10,496. [Google Scholar] [CrossRef]
- Le, T.; Liu, C.; Yao, B.; Natraj, V.; Yung, Y.L. Application of machine learning to hyperspectral radiative transfer simulations. J. Quant. Spectrosc. Radiat. Transf. 2020, 246, 106928. [Google Scholar] [CrossRef]
- Sasi, S.; Natraj, V.; Molina García, V.; Efremenko, D.; Loyola, D.; Doicu, A. Model Selection in Atmospheric Remote Sensing with an Application to Aerosol Retrieval from DSCOVR/EPIC, Part 1: Theory. Remote Sens. 2020, 12, 3724. [Google Scholar] [CrossRef]
- Sasi, S.; Natraj, V.; Molina García, V.; Efremenko, D.; Loyola, D.; Doicu, A. Model Selection in Atmospheric Remote Sensing with Application to Aerosol Retrieval from DSCOVR/EPIC. Part 2: Numerical Analysis. Remote Sens. 2020, 12, 3656. [Google Scholar] [CrossRef]
- Vicent, J.; Verrelst, J.; Rivera-Caicedo, J.P.; Sabater, N.; Muñoz-Marí, J.; Camps-Valls, G.; Moreno, J. Emulation as an Accurate Alternative to Interpolation in Sampling Radiative Transfer Codes. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 4918–4931. [Google Scholar] [CrossRef]
- Vicent Servera, J.; Alonso, L.; Martino, L.; Sabater, N.; Verrelst, J.; Camps-Valls, G.; Moreno, J. Gradient-Based Automatic Lookup Table Generator for Radiative Transfer Models. IEEE Trans. Geosci. Remote Sens. 2019, 57, 1040–1048. [Google Scholar] [CrossRef]
Band | Spectral Range (nm) | Spectral Resolution (nm) | Number of Spectral Points |
---|---|---|---|
Hartley–Huggins | 280–335 | 0.18 | 300 |
O A | 755–775 | 0.0010 | 20,000 |
Water vapour | 770–1000 | 0.0058 | 40,000 |
CO | 1590–1620 | 0.0015 | 20,000 |
RTM | LBL | Single CLSR | Double CLSR | |
---|---|---|---|---|
Number of calls | MS | 300 | 20 | 20 |
TS | 0 | 300 | 32 | |
SS | 0 | 0 | 300 | |
Computation time (s) | MS | 35 | 2.32 | 2.32 |
TS | 0 | 0.048 | 0.005 | |
SS | 0 | 0 | 0.006 | |
Total computational time (s) | 35 | 2.37 | 2.33 | |
Acceleration factor | – | 14.8 | 15.0 |
RTM | LBL | Single CLSR | Double CLSR | |
---|---|---|---|---|
Number of calls | MS | 20,000 | 20 | 20 |
TS | 0 | 20,000 | 32 | |
SS | 0 | 0 | 20,000 | |
Computation time (s) | MS | 2320 | 2.32 | 2.32 |
TS | 0 | 3.2 | 0.005 | |
SS | 0 | 0 | 0.4 | |
Total computational time (s) | 2320 | 5.52 | 2.725 | |
Acceleration factor | – | 420 | 850 |
RTM | LBL | Single CLSR | Double CLSR | |
---|---|---|---|---|
Number of calls | MS | 40,000 | 20 | 20 |
TS | 0 | 40,000 | 32 | |
SS | 0 | 0 | 40,000 | |
Computation time (s) | MS | 4640 | 2.32 | 2.32 |
TS | 0 | 6.4 | 0.005 | |
SS | 0 | 0 | 0.8 | |
Total computational time (s) | 4640 | 8.72 | 3.13 | |
Acceleration factor | – | 532 | 1482 |
Acceleration Technique | Band/Spectral Region | Acceleration Factor | Reference |
---|---|---|---|
k-distribution | HO, CO, O, and O | 10× * | Fomin [32] |
double-k approach | O A | 1000 | Duan [16] |
LSI | O A, CO weak, CO strong | 45 , 210 | O’Dell [5] |
PCA | O A, CO weak, CO strong | 50 | Natraj et al. [33] |
PCA | 290–340 nm | 10 | Spurr et al. [34] |
LEM | 325–335 nm | 10 | Efremenko et al. [9] |
PCA | 325–335 nm | 2 | Efremenko et al. [35] |
PCA | 300–3000 nm | 10× | Kopparla et al. [19] |
PCA | O A, CO weak, CO strong | 100× | Somkuti et al. [36] |
k-distribution + PCA | O A | 342 | Molina García et al. [11] |
PCA | Hartley-Huggins | 18 | del Águila et al. [12] |
NN | O A, CO weak, CO strong | 250 | Le et al. [37] |
LEM | NO (425–450 nm) | 12 e | Doicu et al. [18] |
CLSR | O A, CO weak | 505 | del Águila et al. [14] |
SDCOMP | 750–920 nm | 1000 | Liu et al. [17] |
double CLSR | Hartley-Huggins | 15 | This study |
O A, CO weak | 850 | ||
Water vapour | 1500 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
del Águila, A.; Efremenko, D.S. Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method. Remote Sens. 2021, 13, 434. https://doi.org/10.3390/rs13030434
del Águila A, Efremenko DS. Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method. Remote Sensing. 2021; 13(3):434. https://doi.org/10.3390/rs13030434
Chicago/Turabian Styledel Águila, Ana, and Dmitry S. Efremenko. 2021. "Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method" Remote Sensing 13, no. 3: 434. https://doi.org/10.3390/rs13030434