# Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Data Overview

#### 2.2. Acceleration Techniques

#### 2.2.1. Summary of the Cluster Low-Streams Regression (CLSR) Method

#### 2.2.2. Double Cluster Low-Streams Regression Method

**Step 1**: We compute the LBL spectra ${\left\{{I}_{\mathrm{SS}}\left({\lambda}_{i}\right)\right\}}_{i=1}^{N}$ by using the SS RTM and apply sorting and clustering to the space of SS radiances. Assuming a regression model between SS and TS radiances within each cluster z, we obtain

**Step 2**: We apply the CLSR method as described in the previous section (Section 2.2.1) using the TS spectra computed in Step 1.

#### 2.3. CLSR Method: Improvement to Aerosol Scheme

## 3. Results and Discussion

#### 3.1. Single CLSR vs. Double CLSR: Accuracy Results

- More than 70% and 60% of the residuals are below 0.01% for the single and double CLSR methods, respectively, for all bands, with the exception of the water vapour band.
- The residuals of the water vapour band present a wider distribution in comparison with the other spectral bands.
- The probability densities are almost indistinguishable for both acceleration methods, demonstrating that both techniques provide accurate results among the different spectral bands.

#### 3.2. Single CLSR vs. Double CLSR: Computational Performance

#### 3.3. Computational Performance: State-of-the-Art Acceleration Techniques

- For simulations in the Hartley–Huggins band, PCA techniques, linear embedding methods (LEM) and double CLSR have been applied. The double CLSR does not further improve the performance, since the computational burden is due to the MS RTM computations (see Section 3.2). The highest acceleration factor is provided by the method described in [12], in which PCA is applied to both optical parameters and spectral radiances. The performance enhancement in this case is up to 18 times.
- There are several studies in which fast RTMs for the O${}_{2}$ A- and CO${}_{2}$ bands (either weak or strong) have been designed. In general, all considered techniques provide acceleration factors of about 2–3 orders of magnitude, including those based on artificial neural networks (NN) [37].
- The water vapour band represents a challenge for acceleration techniques due to its complicated spectral structure. Therefore, the accuracy of the acceleration techniques is lower than for the O${}_{2}$ A-band. For this band, the double CLSR method provides an acceleration factor of about 3 orders of magnitude, while the k-distribution [32] and PCA-based RTMs [19] achieve lower acceleration factors, of one order of magnitude.

#### 3.4. Further Improvements to Aerosol Schemes

#### 3.5. Combined Application of the Single CLSR vs. Double CLSR Method for Aerosol Scenarios

- The residual distributions of the single CLSR method are narrower than those of the double CLSR method, meaning that the single CLSR method is more accurate. However, in general, the residuals are below 0.01% for both methods and all spectral bands, except for the water vapour band, where the residual distributions are slightly wider and still below 0.05%. The distributions are not biased.
- For the Hartley–Huggins, O${}_{2}$ A- and CO${}_{2}$ bands, the residuals are below 0.05% for both single and double CLSR methods. Regarding the water vapour band, the residuals are below 0.05% and 0.1% for the single and double CLSR method, respectively. Similar accuracies were achieved in Kopparla et al. [19] for the water vapour band using the PCA-based RTM.
- In the case of the low aerosol load, the probability density functions are similar for all ${\mathbf{X}}_{i}$-configurations. However, as the aerosol load increases, the residual distributions for the ${\mathbf{X}}_{1}$ configuration provided by the single and double CLSR methods sometimes become biased for the water vapour band.

## 4. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

AOD | Aerosol Optical Depth |

CLSR | Cluster Low-Streams Interpolation |

DOME | Discrete Ordinates with Matrix Exponential |

HITRAN | High-Resolution Transmission Molecular Absorption Database |

LBL | Line-By-Line |

LEM | Linear Embedding Methods |

LSI | Low-Streams Interpolation |

LUT | LookUp Table |

MS | Multi-Stream |

NN | Neural Network |

OPAC | Optical Properties of Aerosols and Clouds |

PCA | Principal Component Analysis |

Py4CAts | Python for Computational Atmospheric Spectroscopy |

RTM | Radiative Transfer Model |

SDCOMP | Spectral Data Compression |

SS | Single-Scattering |

SSA | Single Scattering Albedo |

TOA | Top-of-the-Atmosphere |

TS | Two-Stream |

UV | Ultraviolet |

## References

- Goody, R.; West, R.; Chen, L.; Crisp, D. The correlated-k method for radiation calculations in nonhomogeneous atmospheres. J. Quant. Spectrosc. Radiat. Transf.
**1989**, 42, 539–550. [Google Scholar] [CrossRef] - Fu, Q.; Liou, K. On the correlated k-distribution method for radiative transfer in nonhomogeneous atmospheres. J. Atmos. Sci.
**1992**, 49, 2139–2156. [Google Scholar] [CrossRef] [Green Version] - Kato, S.; Ackerman, T.P.; Mather, J.H.; Clothiaux, E.E. The k-distribution method and correlated-k approximation for a shortwave radiative transfer model. J. Quant. Spectrosc. Radiat. Transf.
**1999**, 62, 109–121. [Google Scholar] [CrossRef] - Zhang, F.; Zhu, M.; Li, J.; Li, W.; Di, D.; Shi, Y.N.; Wu, K. Alternate Mapping Correlated k-Distribution Method for Infrared Radiative Transfer Forward Simulation. Remote Sens.
**2019**, 11, 994. [Google Scholar] [CrossRef] [Green Version] - O’Dell, C.W. Acceleration of multiple-scattering, hyperspectral radiative transfer calculations via low-streams interpolation. J. Geophys. Res.
**2010**, 115. [Google Scholar] [CrossRef] - Natraj, V.; Jiang, X.; Shia, R.; Huang, X.; Margolis, J.; Yung, Y. Application of the principal component analysis to high spectral resolution radiative transfer: A case study of the O
_{2}A-band. J. Quant. Spectrosc. Radiat. Transf.**2005**, 95, 539–556. [Google Scholar] [CrossRef] - Kopparla, P.; Natraj, V.; Spurr, R.; Shia, R.L.; Crisp, D.; Yung, Y.L. A fast and accurate PCA based radiative transfer model: Extension to the broadband shortwave region. J. Quant. Spectrosc. Radiat. Transf.
**2016**, 173, 65–71. [Google Scholar] [CrossRef] - Efremenko, D.; Doicu, A.; Loyola, D.; Trautmann, T. Acceleration techniques for the discrete ordinate method. J. Quant. Spectrosc. Radiat. Transf.
**2013**, 114, 73–81. [Google Scholar] [CrossRef] - Efremenko, D.; Doicu, A.; Loyola, D.; Trautmann, T. Optical property dimensionality reduction techniques for accelerated radiative transfer performance: Application to remote sensing total ozone retrievals. J. Quant. Spectrosc. Radiat. Transf.
**2014**, 133, 128–135. [Google Scholar] [CrossRef] - Liu, X.; Smith, W.L.; Zhou, D.K.; Larar, A. Principal component-based radiative transfer model for hyperspectral sensors: Theoretical concept. Appl. Opt.
**2006**, 45, 201–208. [Google Scholar] [CrossRef] - Molina García, V.; Sasi, S.; Efremenko, D.; Doicu, A.; Loyola, D. Radiative transfer models for retrieval of cloud parameters from EPIC/DSCOVR measurements. J. Quant. Spectrosc. Radiat. Transf.
**2018**, 213, 228–240. [Google Scholar] [CrossRef] [Green Version] - del Águila, A.; Efremenko, D.S.; Molina García, V.; Xu, J. Analysis of two dimensionality reduction techniques for fast simulation of the spectral radiances in the Hartley-Huggins band. Atmosphere
**2019**, 10, 142. [Google Scholar] [CrossRef] [Green Version] - del Águila, A.; Efremenko, D.S.; Trautmann, T. A review of dimensionality reduction techniques for processing hyper-spectral optical signal. Light Eng.
**2019**, 27, 85–98. [Google Scholar] [CrossRef] [Green Version] - del Águila, A.; Efremenko, D.S.; Molina García, V.; Kataev, M.Y. Cluster Low-Streams Regression Method for Hyperspectral Radiative Transfer Computations: Cases of O2 A- and CO2 Bands. Remote Sens.
**2020**, 12, 1250. [Google Scholar] [CrossRef] [Green Version] - Vincent, R.A.; Dudhia, A. Fast radiative transfer using monochromatic look-up tables. J. Quant. Spectrosc. Radiat. Transf.
**2017**, 186, 254–264. [Google Scholar] [CrossRef] [Green Version] - Duan, M.; Min, Q.; Li, J. A fast radiative transfer model for simulating high-resolution absorption bands. J. Geophys. Res. Atmos.
**2005**, 110. [Google Scholar] [CrossRef] [Green Version] - Liu, C.; Yao, B.; Natraj, V.; Kopparla, P.; Weng, F.; Le, T.; Shia, R.L.; Yung, Y.L. A Spectral Data Compression (SDCOMP) Radiative Transfer Model for High-Spectral-Resolution Radiation Simulations. J. Atmos. Sci.
**2020**, 77, 2055–2066. [Google Scholar] [CrossRef] [Green Version] - Doicu, A.; Efremenko, D.; Trautmann, T. A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method. Remote Sens.
**2020**, 12, 3703. [Google Scholar] [CrossRef] - Kopparla, P.; Natraj, V.; Limpasuvan, D.; Spurr, R.; Crisp, D.; Shia, R.L.; Somkuti, P.; Yung, Y.L. PCA-based radiative transfer: Improvements to aerosol scheme, vertical layering and spectral binning. J. Quant. Spectrosc. Radiat. Transf.
**2017**, 198, 104–111. [Google Scholar] [CrossRef] - Schreier, F.; Gimeno García, S.; Hochstaffl, P.; Städt, S. Py4CAtS—PYthon for Computational Atmospheric Spectroscopy. Atmosphere
**2019**, 10, 262. [Google Scholar] [CrossRef] [Green Version] - Gordon, I.; Rothman, L.; Hill, C.; Kochanov, R.; Tan, Y.; Bernath, P.; Birk, M.; Boudon, V.; Campargue, A.; Chance, K.; et al. The HITRAN2016 molecular spectroscopic database. J. Quant. Spectrosc. Radiat. Transf.
**2017**, 203, 3–69. [Google Scholar] [CrossRef] - Bodhaine, B.; Wood, N.; Dutton, E.; Slusser, J. On Rayleigh optical depth calculations. J. Atmos. Ocean. Technol.
**1999**, 16, 1854–1861. [Google Scholar] [CrossRef] - Anderson, G.; Clough, S.; Kneizys, F.; Chetwynd, J.; Shettle, E. AFGL Atmospheric Constituent Profiles (0.120 km); Air ForceGeophysics Lab.: Hanscom AFB, MA, USA, 1986; p. 46. [Google Scholar]
- Doicu, A.; Trautmann, T. Discrete-ordinate method with matrix exponential for a pseudo-spherical atmosphere: Scalar case. J. Quant. Spectrosc. Radiat. Transf.
**2009**, 110, 146–158. [Google Scholar] [CrossRef] - Efremenko, D.S.; Molina García, V.; Gimeno García, S.; Doicu, A. A review of the matrix-exponential formalism in radiative transfer. J. Quant. Spectrosc. Radiat. Transf.
**2017**, 196, 17–45. [Google Scholar] [CrossRef] [Green Version] - Wiscombe, W.J. The Delta-M Method: Rapid Yet Accurate Radiative Flux Calculations for Strongly Asymmetric Phase Functions. J. Atmos. Sci.
**1977**, 34, 1408–1422. [Google Scholar] [CrossRef] [Green Version] - Nakajima, T.; Tanaka, M. Algorithms for radiative intensity calculations in moderately thick atmospheres using a truncation approximation. J. Quant. Spectrosc. Radiat. Transf.
**1988**, 40, 51–69. [Google Scholar] [CrossRef] - Fischer, J.; Grassl, H. Radiative transfer in an atmosphere-ocean system: An azimuthally dependent matrix-operator approach. Appl. Opt.
**1984**, 23, 1032. [Google Scholar] [CrossRef] - Chalhoub, E.; Garcia, R. The equivalence between two techniques of angular interpolation for the discrete-ordinates method. J. Quant. Spectrosc. Radiat. Transf.
**2000**, 64, 517–535. [Google Scholar] [CrossRef] - Hess, M.; Koepke, P.; Schult, I. Optical properties of aerosols and clouds: The software package OPAC. Bull. Am. Meteorol. Soc.
**1998**, 79, 831–844. [Google Scholar] [CrossRef] - Huang, Y.; Natraj, V.; Zeng, Z.C.; Kopparla, P.; Yung, Y.L. Quantifying the impact of aerosol scattering on the retrieval of methane from airborne remote sensing measurements. Atmos. Meas. Tech.
**2020**, 13, 6755–6769. [Google Scholar] [CrossRef] - Fomin, B. A k-distribution technique for radiative transfer simulation in inhomogeneous atmosphere: 2. FKDM, fast k-distribution model for the shortwave. J. Geophys. Res.
**2005**, 110. [Google Scholar] [CrossRef] [Green Version] - Natraj, V.; Shia, R.L.; Yung, Y.L. On the use of principal component analysis to speed up radiative transfer calculations. J. Quant. Spectrosc. Radiat. Transf.
**2010**, 111, 810–816. [Google Scholar] [CrossRef] - Spurr, R.; Natraj, V.; Lerot, C.; Roozendael, M.V.; Loyola, D. Linearization of the Principal Component Analysis method for radiative transfer acceleration: Application to retrieval algorithms and sensitivity studies. J. Quant. Spectrosc. Radiat. Transf.
**2013**, 125, 1–17. [Google Scholar] [CrossRef] - Efremenko, D.S.; Loyola, D.G.; Spurr, R.J.; Doicu, A. Acceleration of radiative transfer model calculations for the retrieval of trace gases under cloudy conditions. J. Quant. Spectrosc. Radiat. Transf.
**2014**, 135, 58–65. [Google Scholar] [CrossRef] - Somkuti, P.; Boesch, H.; Natraj, V.; Kopparla, P. Application of a PCA-Based Fast Radiative Transfer Model to XCO2 Retrievals in the Shortwave Infrared. J. Geophys. Res. Atmos.
**2017**, 122, 10,477–10,496. [Google Scholar] [CrossRef] - Le, T.; Liu, C.; Yao, B.; Natraj, V.; Yung, Y.L. Application of machine learning to hyperspectral radiative transfer simulations. J. Quant. Spectrosc. Radiat. Transf.
**2020**, 246, 106928. [Google Scholar] [CrossRef] - Sasi, S.; Natraj, V.; Molina García, V.; Efremenko, D.; Loyola, D.; Doicu, A. Model Selection in Atmospheric Remote Sensing with an Application to Aerosol Retrieval from DSCOVR/EPIC, Part 1: Theory. Remote Sens.
**2020**, 12, 3724. [Google Scholar] [CrossRef] - Sasi, S.; Natraj, V.; Molina García, V.; Efremenko, D.; Loyola, D.; Doicu, A. Model Selection in Atmospheric Remote Sensing with Application to Aerosol Retrieval from DSCOVR/EPIC. Part 2: Numerical Analysis. Remote Sens.
**2020**, 12, 3656. [Google Scholar] [CrossRef] - Vicent, J.; Verrelst, J.; Rivera-Caicedo, J.P.; Sabater, N.; Muñoz-Marí, J.; Camps-Valls, G.; Moreno, J. Emulation as an Accurate Alternative to Interpolation in Sampling Radiative Transfer Codes. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2018**, 11, 4918–4931. [Google Scholar] [CrossRef] - Vicent Servera, J.; Alonso, L.; Martino, L.; Sabater, N.; Verrelst, J.; Camps-Valls, G.; Moreno, J. Gradient-Based Automatic Lookup Table Generator for Radiative Transfer Models. IEEE Trans. Geosci. Remote Sens.
**2019**, 57, 1040–1048. [Google Scholar] [CrossRef]

**Figure 1.**Overview of the optical properties (AOD, SSA and g) obtained from the OPAC database for the aerosol types: tropospheric, clean continental, urban, desert and polluted. Each vertical line corresponds with the middle wavelength of the spectral bands in order: Hartley–Huggins, O${}_{2}$ A, water vapour and CO${}_{2}$ bands.

**Figure 3.**TOA radiances computed by using the MS RTM for the absorption bands: Hartley–Huggins, O${}_{2}$ A-, water vapour and CO${}_{2}$ bands. Blue lines correspond to the case without aerosol, while the black lines correspond to the specific case of desert aerosol.

**Figure 4.**Probability density function of the residuals for the single CLSR (grey) and the double CLSR (blue) methods for the Hartley–Huggins, O${}_{2}$ A-, water vapour and CO${}_{2}$ bands and for the tropospheric aerosol case.

**Figure 5.**Same as Figure 4, but for the cumulated probability of the residuals.

**Figure 6.**Mean absolute relative error in % for the four spectral bands, as well as for all aerosol types. Each bar colour corresponds to an aerosol type, while each group of bars corresponds to one of the ${\mathbf{X}}_{i}$-matrix ($i=0,1,2$) (Equations (8)–(10)) used in the CLSR method. Note that the scale of the water vapour band is different from the rest of the spectral bands.

**Figure 7.**Probability density of the residuals for the single CLSR method and for several matrix configurations: ${\mathbf{X}}_{0}$, ${\mathbf{X}}_{1}$ and ${\mathbf{X}}_{2}$. Each plot represents the absorption bands: Hartley–Huggins, O${}_{2}$ A-, water vapour and CO${}_{2}$ bands. The case presented corresponds to the tropospheric aerosol.

**Figure 8.**Same as Figure 7, but for the double CLSR method.

**Table 1.**Spectral ranges, resolutions and number of spectral points for the absorption bands used in this study.

Band | Spectral Range (nm) | Spectral Resolution (nm) | Number of Spectral Points |
---|---|---|---|

Hartley–Huggins | 280–335 | 0.18 | 300 |

O${}_{2}$ A | 755–775 | 0.0010 | 20,000 |

Water vapour | 770–1000 | 0.0058 | 40,000 |

CO${}_{2}$ | 1590–1620 | 0.0015 | 20,000 |

**Table 2.**Summary of number of calls, computational time and acceleration factors for the Hartley–Huggins band. The computational times marked in red indicate the computational burden.

RTM | LBL | Single CLSR | Double CLSR | |
---|---|---|---|---|

Number of calls | MS | 300 | 20 | 20 |

TS | 0 | 300 | 32 | |

SS | 0 | 0 | 300 | |

Computation time (s) | MS | 35 | 2.32 | 2.32 |

TS | 0 | 0.048 | 0.005 | |

SS | 0 | 0 | 0.006 | |

Total computational time (s) | 35 | 2.37 | 2.33 | |

Acceleration factor | – | 14.8 | 15.0 |

RTM | LBL | Single CLSR | Double CLSR | |
---|---|---|---|---|

Number of calls | MS | 20,000 | 20 | 20 |

TS | 0 | 20,000 | 32 | |

SS | 0 | 0 | 20,000 | |

Computation time (s) | MS | 2320 | 2.32 | 2.32 |

TS | 0 | 3.2 | 0.005 | |

SS | 0 | 0 | 0.4 | |

Total computational time (s) | 2320 | 5.52 | 2.725 | |

Acceleration factor | – | 420 | 850 |

**Table 4.**Same as for Table 2 but for the water vapour band.

RTM | LBL | Single CLSR | Double CLSR | |
---|---|---|---|---|

Number of calls | MS | 40,000 | 20 | 20 |

TS | 0 | 40,000 | 32 | |

SS | 0 | 0 | 40,000 | |

Computation time (s) | MS | 4640 | 2.32 | 2.32 |

TS | 0 | 6.4 | 0.005 | |

SS | 0 | 0 | 0.8 | |

Total computational time (s) | 4640 | 8.72 | 3.13 | |

Acceleration factor | – | 532 | 1482 |

**Table 5.**Selected acceleration techniques with the corresponding spectral region or band, acceleration factor (computed with respect to the LBL model) and their reference. Note that the acceleration factors are sometimes given in orders of magnitude compared to the LBL approach. One or two order of magnitude are indicated as 10× and 100×, respectively. The references are ordered chronologically.

Acceleration Technique | Band/Spectral Region | Acceleration Factor | Reference |
---|---|---|---|

k-distribution | H${}_{2}$O, CO${}_{2}$, O${}_{3}$, and O${}_{2}$ | 10× * | Fomin [32] |

double-k approach | O${}_{2}$ A | 1000 | Duan [16] |

LSI | O${}_{2}$ A, CO${}_{2}$ weak, CO${}_{2}$ strong | 45 ${}^{\mathrm{b}}$, 210 ${}^{\mathrm{c}}$ | O’Dell [5] |

PCA | O${}_{2}$ A, CO${}_{2}$ weak, CO${}_{2}$ strong | 50 | Natraj et al. [33] |

PCA | 290–340 nm | 10 | Spurr et al. [34] |

LEM | 325–335 nm | 10 | Efremenko et al. [9] |

PCA | 325–335 nm | 2 | Efremenko et al. [35] |

PCA | 300–3000 nm | 10× | Kopparla et al. [19] |

PCA | O${}_{2}$ A, CO${}_{2}$ weak, CO${}_{2}$ strong | 100× | Somkuti et al. [36] |

k-distribution + PCA | O${}_{2}$ A | 342 | Molina García et al. [11] |

PCA ${}^{\mathrm{a}}$ | Hartley-Huggins | 18 | del Águila et al. [12] |

NN | O${}_{2}$ A, CO${}_{2}$ weak, CO${}_{2}$ strong | 250 ${}^{\mathrm{d}}$ | Le et al. [37] |

LEM | NO${}_{2}$ (425–450 nm) | 12 ^{e} | Doicu et al. [18] |

CLSR | O${}_{2}$ A, CO${}_{2}$ weak | 505 | del Águila et al. [14] |

SDCOMP ${}^{\mathrm{a}}$ | 750–920 nm | 1000 ${}^{\mathrm{d}}$ | Liu et al. [17] |

double CLSR | Hartley-Huggins | 15 | This study |

O${}_{2}$ A, CO${}_{2}$ weak | 850 | ||

Water vapour | 1500 |

^{e}Relative to the k-distribution method. * It is estimated from the information found in the reference, but the value does not appear explicitly.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

del Águila, A.; Efremenko, D.S.
Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method. *Remote Sens.* **2021**, *13*, 434.
https://doi.org/10.3390/rs13030434

**AMA Style**

del Águila A, Efremenko DS.
Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method. *Remote Sensing*. 2021; 13(3):434.
https://doi.org/10.3390/rs13030434

**Chicago/Turabian Style**

del Águila, Ana, and Dmitry S. Efremenko.
2021. "Fast Hyper-Spectral Radiative Transfer Model Based on the Double Cluster Low-Streams Regression Method" *Remote Sensing* 13, no. 3: 434.
https://doi.org/10.3390/rs13030434