Next Article in Journal
Self-Organized Model Fitting Method for Railway Structures Monitoring Using LiDAR Point Cloud
Next Article in Special Issue
A Proof-of-Concept Algorithm for the Retrieval of Total Column Amount of Trace Gases in a Multi-Dimensional Atmosphere
Previous Article in Journal
Identification before-after Forest Fire and Prediction of Mangrove Forest Based on Markov-Cellular Automata in Part of Sembilang National Park, Banyuasin, South Sumatra, Indonesia
Previous Article in Special Issue
Cluster Low-Streams Regression Method for Hyperspectral Radiative Transfer Computations: Cases of O2 A- and CO2 Bands
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method

German Aerospace Center (DLR), Remote Sensing Technology Institute, 82234 Oberpfaffenhofen, Germany
*
Author to whom correspondence should be addressed.
Remote Sens. 2020, 12(22), 3703; https://doi.org/10.3390/rs12223703
Submission received: 12 October 2020 / Revised: 9 November 2020 / Accepted: 10 November 2020 / Published: 11 November 2020

Abstract

:
A spectral acceleration approach for the spherical harmonics discrete ordinate method (SHDOM) is designed. This approach combines the correlated k-distribution method and some dimensionality reduction techniques applied on the optical parameters of an atmospheric system. The dimensionality reduction techniques used in this study are the linear embedding methods: principal component analysis, locality pursuit embedding, locality preserving projection, and locally embedded analysis. Through a numerical analysis, it is shown that relative to the correlated k-distribution method, PCA in conjunction with a second-order of scattering approximation yields an acceleration factor of 12. This implies that SHDOM equipped with this acceleration approach is efficient enough to perform spectral integration of radiance fields in inhomogeneous multi-dimensional media.

Graphical Abstract

1. Introduction

The retrieval of trace gas products from UV/VIS spectrometers is strongly affected by the presence of clouds. For the forward-model simulation of satellite measurements from instruments with high spatial resolution, it is important to account for the sub-pixel cloud inhomogeneity, or at the least, to assess this effect on the radiances at the top of the atmosphere, and so, on the retrieval results. For the new generation of atmospheric composition UV–VIS-NIR sensors, such as Sentinel-5P, and Sentinel-4 and −5, fast and accurate models accounting for the cloud inhomogeneities are crucial.
The spherical harmonics discrete ordinate method (SHDOM) developed by Evans [1,2] is one of the most efficient and widely used multi-dimensional deterministic (explicit) methods in the atmospheric sciences. SHDOM adopts (i) both spherical harmonics and discrete ordinates representation of the radiance field during different parts of the solution algorithm, and (ii) a sort of successive order of scattering solution method (Picard iteration [3]). The spherical harmonics are employed for computing the source function including the scattering integral, while the discrete ordinates are used to integrate the radiative transfer equation spatially. Specifically, the radiative transfer equation is solved iteratively by (i) transforming the source function from spherical harmonics to discrete ordinates, (ii) integrating the product of the source function and transmission along discrete ordinates, (iii) transforming the discrete ordinate radiances to spherical harmonics, and (iv) calculating the source function from the radiance in the spherical harmonics space. To achieve higher accuracy with a limited amount of memory, an adaptive grid is implemented. By this technique, regions where the source function is changing more rapidly have a higher density of grid points.
The computation of the radiative transfer through an atmosphere consisting of gas molecules is a demanding task because it requires an accurate modeling of the spectral absorption and scattering signatures of the gases plus additional particulate material under consideration. SHDOM uses a correlated k-distribution approach [4] for the integration over a spectral band. Note that in the version of March 1996, the parameterization of Fu and Liou [5] is the basis of the broadband integration of the gaseous line absorption, while longwave and shortwave Rapid Radiative Transfer Model (RRTM)-based k-distribution programs were added in 2003.
In addition to the k-distribution method, dimensionality reduction techniques applied on the optical properties of an atmospheric system become a standard broadband acceleration tool for one-dimensional radiative transfer [6,7,8]. Natraj et al. [9,10] designed a radiative transfer model based on the principal component analysis, while Efremenko et al. [11] generalized this model to include local linear embedding methods, as for example, locality pursuit embedding, locality preserving projection, and locally embedded analysis.
The aim of this paper was two-fold. First, to extend the applicability range of dimensionality reduction techniques to multi-dimensional radiative transfer, and second, to design a spectral acceleration approach for SHDOM that combines the correlated k-distribution method with some dimensionality reduction techniques. Although in the literature these methods are used separately, we apply them together. In this way, we expect to increase the computation speed of the k-distribution-based SHDOM.

2. The Spectral Acceleration Approach

Let us consider the solar radiative transfer in a rectangular prism of lengths L x , L y and L z as shown in Figure 1. The top and bottom faces of the prism are denoted by S t and S b , respectively, while S 1 x ( x = 0 ), S 2 x ( x = L x ), S 1 y ( y = 0 ), and S 2 y ( y = L y ) correspond to the lateral faces. The boundary-value problem for the total radiance at point r in direction Ω consists of the inhomogeneous differential equation
d I d s ( λ , r , Ω ) = σ ext ( λ , r ) I ( λ , r , Ω ) + σ sct ( λ , r ) 4 π Ω P ( λ , r , Ω , Ω ) I ( λ , r , Ω ) d Ω ,
the top-of-atmosphere boundary condition
I ( λ , r t , Ω ) = F 0 ( λ ) μ 0 δ ( Ω Ω 0 ) , r t S t ,
and the Lambertian surface boundary condition
I ( λ , r b , Ω + ) = A ( λ ) π Ω I ( λ , r b , Ω ) | μ | d Ω , r b S b .
At the horizontal boundaries, periodic boundary conditions are assumed, i.e.,
I ( λ , r 1 x , Ω ) = I ( λ , r 2 x , Ω ) , I ( λ , r 1 y , Ω ) = I ( λ , r 2 y , Ω ) ,
for r i x S i x and r i y S i y with i = 1 , 2 . In Equations (1)–(4), λ is the wavelength, σ ext ( λ , r ) and σ sct ( λ , r ) the extinction and scattering coefficients, respectively, P ( λ , · ) the phase function, A ( λ ) the surface albedo, Ω 0 = ( μ 0 , φ 0 ) with μ 0 > 0 , the solar direction, F 0 ( λ ) the solar flux, r 2 x = r 1 x + L x i and r 2 y = r 1 y + L y j with ( i , j , k ) being the Cartesian unit vectors. Moreover, Ω + and Ω stand for an upward and a downward direction, respectively, Ω is the unit sphere, while Ω + and Ω denote the upper and lower unit hemispheres, respectively,
An instrument placed at point r m measures the radiance at the top of the atmosphere in direction Ω m 0 = ( μ m 0 , φ m 0 ) , μ m 0 > 0 . Denoting by g ( λ ) the slit function of the instrument, s the slit width, and  S tm the footprint of the instrument on the top face S t , the spectral signal measured by the instrument at wavelength λ m in the spectral interval [ λ min , λ max ] is given by
I ( λ m ) = λ min s / 2 λ max + s / 2 g ( λ m λ ) I FOV ( λ ) d λ ,
where
I FOV ( λ ) = S t h ( r t ) I ( λ , r t , Ω m ( r t ) ) d S t ,
is the signal integrated over the field of view of the instrument at wavelength λ , Ω m ( r t ) the unit vector along the line connecting the points r t S tm and r m , i.e., Ω m ( r t ) = ( r m r t ) / | r m r t | (see Figure 1), and
h ( r t ) = 1 A tm , r t S tm 0 , otherwise ,
and A tm the characteristic function and the area of the instrument footprint, respectively. Assuming that the distance from the top of the atmosphere to the instrument is large, we approximate I ( λ , r , Ω m ( r ) ) I ( λ , r , Ω m 0 ) in Equation (6).
For an atmosphere consisting of gas molecules and a cloud, we suppose that
  • the optical coefficients of the gas molecules depend on the altitude level and the wavelength, and
  • the optical coefficients of the cloud depend on the spatial coordinates but not on the wavelength.
The extinction coefficient is then computed as
σ ext ( λ , r ) = σ ext cloud ( r ) + σ sct mol ( λ , z ) + σ abs gas ( λ , z ) ,
where σ ext cloud ( r ) is the extinction coefficient in the cloud, σ sct mol ( λ , z ) the molecular scattering coefficient due to Rayleigh scattering, σ abs gas ( λ , z ) the gas absorption coefficient, and ( x , y , z ) the Cartesian coordinates of point r . For the scattering coefficient, we have a similar representation, namely,
σ sct ( λ , r ) = ω cloud σ ext cloud ( r ) + σ sct mol ( λ , z ) ,
where ω cloud is the single-scattering albedo of the cloud.
The goal of our analysis is to design an approach for accelerating the computation of the spectral signal I ( λ m ) at a set of W m spectral points { λ m w } w = 1 W m characterizing the instrument. For this purpose, we combine the correlated k-distribution method with some dimensionality reduction techniques.

2.1. Correlated k-Distribution Method

The correlated k-distribution method [4,12] is based on grouping spectral intervals according to absorption coefficient strength. In this section we provide a description of the correlated k-distribution method, which will enable us to connect this method with dimensionality reduction techniques.
Let { λ ¯ k } k = 1 W ¯ be a discrete set of W ¯ equally spaced wavelengths in the interval [ λ min s / 2 , λ max + s / 2 ] with
λ ¯ 1 Δ λ 2 = λ min s 2 and λ W ¯ + Δ λ 2 = λ max + s 2 ,
where Δ λ the discretization step. The forward-model spectral set { λ ¯ k } k = 1 W ¯ is finer than the measurement spectral set { λ m w } w = 1 W m ; in practice, W ¯ is chosen as a multiple of W m , i.e., W ¯ = c W m with say, c being an integer greater than 20 . The convolution integral giving the expression of the spectral signal measured by the instrument can be approximated by
I ( λ m ) = λ min s / 2 λ max + s / 2 g ( λ m λ ) I FOV ( λ ) d λ k = 1 W ¯ g ( λ m λ ¯ k ) λ ¯ k Δ λ / 2 λ ¯ k + Δ λ / 2 I FOV ( λ ) d λ .
As gas absorption has stronger spectral variation than molecular and particulate scattering, we write formally (at a given altitude level z) I FOV ( λ ) = I FOV ( σ abs gas ( λ ) ) . The accurate technique for computing the integral in Equation (10) is a line-by-line (LBL) calculation of the gas absorption coefficient versus wavelength. Contrarily, the correlated k-distribution method is based on the fact that for a homogeneous atmosphere, the transmission within a spectral interval is independent of the LBL variation of σ abs gas with respect to the wavelength, but rather depends only on the distribution of σ abs gas within the spectral interval [13]. In this regard, let F = F ( σ abs , k gas ) be the cumulative density function of σ abs gas ( λ ) in the spectral interval [ λ ¯ k Δ λ / 2 , λ ¯ k + Δ λ / 2 ] , and σ abs , k gas ( F ) the quantile function (i.e., the inverse function of the cumulative distribution function). The spectral signal measured by the instrument can then be computed as
I ( λ m ) = Δ λ k = 1 W ¯ g ( λ m λ ¯ k ) 0 1 I FOV ( σ abs , k gas ( F ) ) d F = Δ λ k = 1 W ¯ g ( λ m λ ¯ k ) l = 1 N q ϖ l I FOV ( σ abs , k gas ( F l ) ) ,
where { F l , ϖ l } l = 1 N q is a set of N q quadrature points and weights in the interval [ 0 , 1 ] . The σ abs , k gas ( F l ) can be computed by inverting the cumulative density functions of the LBL gas absorption coefficients, or in the case of the “exponential sum fitting of transmittance” method [14], by solving a nonlinear least squares problem.
In the final step, we define the sets of wavelengths and weights { λ w } w = 1 W and { ω w } w = 1 W with W = W ¯ N q , respectively, through the relations λ w = λ ¯ k and ω w = Δ λ ϖ l , where w = l + ( k 1 ) N q for k = 1 , , W ¯ , l = 1 , , N q . Accordingly, we set σ abs gas ( λ w ) = σ abs , k gas ( F l ) . Note that { λ w } w = 1 W contains W ¯ groups of N q identical wavelengths. By this construction, Equation (11) becomes
I ( λ m ) = w = 1 W ω w g ( λ m λ w ) I FOV ( σ abs gas ( λ w ) ) .
Note that Equation (12) is a quadrature rule for the convolution integral (10) in the case of the correlated k-distribution method.
The computation of the spectral signal by means of the correlated k-distribution method requires the computation of the absorption coefficients σ abs gas ( λ w , z ) at the set of wavelengths { λ w } w = 1 W , and so, W monochromatic radiative transfer calculations. For applications with c 20 and N q 4 , W = W ¯ N q = c W m N q can be in the order of hundreds, and a further acceleration is needed. This can be achieved by applying dimensionality reduction techniques on the optical parameters.

2.2. Dimensionality Reduction of Atmospheric Optical Parameters

Following Ref. [11], the dimensionality reduction problem of the optical parameters can be formulated as follows. For each wavelength λ w , we define an N-dimensional vector x w by
x w T = ln σ abs , 1 gas ( λ w ) , , ln σ abs , L gas ( λ w ) , ln σ sct , 1 mol ( λ w ) , , ln σ sct , L mol ( λ w ) , ln F 0 ( λ w ) ,
where σ abs , k gas and σ sct , k mol are the optical coefficients in the kth level, N = 2 N z + 1 , and N z is the number of altitude levels. For simplicity, the surface albedo A was not included in x w (we assumed that the variations of A are negligible over the width of a molecular absorption band). Thus, the wavelength variability of the optical parameters is encapsulated in the vector x w R N , and there is a one-to-one correspondence between the discrete set of wavelengths { λ w } w = 1 W , and the discrete set of optical parameters { x w } w = 1 W . For the N-dimensional data set { x w } w = 1 W , x ¯ = ( 1 / W ) w = 1 W x w is the sample mean of the data. The goal of a dimensionality reduction technique is to find an M-dimensional subspace ( M < N ) spanned by a set of linear independent vectors { a k } k = 1 M , such that the centered data x w x ¯ belong to this subspace; that is,
x w x ¯ + k = 1 M y w k a k = x ¯ + A y w , w = 1 , , W .
In Equation (14), the matrix A = [ a k ] k = 1 M R N × M , encapsulating the column vectors a k , represents the inverse transform from the low-dimensional space to the high-dimensional space, and y w k is the kth component of y w R M . The vector y w is given by the forward transform from the high-dimensional space to the low-dimensional space,
y w = A ( x w x ¯ ) ,
where A = ( A T A ) 1 A T R M × N is the pseudoinverse of A .
An approximate model for estimating the integrated signal of the instrument at wavelength λ is of the form
ln I FOV ( λ ) I FOV a ( λ ) = f ( λ ) ,
where I FOV a ( λ ) is the integrated signal computed by an approximate radiative transfer model, and  f ( λ ) a correction factor. The correction factor f ( λ ) can be calculated efficiently and accurately by means of dimensionality reduction techniques applied on the optical parameters. Let us assume that the scalar function f ( x w ) given by Equation (16) is not too nonlinear in x w . The computational process is organized as follows.
  • For (cf. Equation (14))
    Δ x w = x w x ¯ = k = 1 M y w k a k = A y w ,
    f ( x w ) is approximated by a second-order Taylor expansion around x ¯ ; that is,
    f ( x w ) f ( x ¯ + Δ x w ) f ( x ¯ ) + Δ x w T f ( x ¯ ) + 1 2 Δ x w T 2 f ( x ¯ ) Δ x w ,
    where f and 2 f are the gradient and the Hessian of f, respectively.
  • For a k = | | a k | | a ^ k and in view of Equation (17), the second and third terms in Equation (18) are written as
    Δ x w T f ( x ¯ ) = k = 1 M y w k | | a k | | f a ^ k ( x ¯ )
    and
    Δ x w T 2 f ( x ¯ ) Δ x w = k = 1 M l = 1 M y w k y w l | | a k | | | | a l | | 2 f a ^ k a ^ l ( x ¯ ) ,
    respectively.
  • The mixed directional derivatives in Equation (20) are neglected, while the first- and second-order directional derivatives are approximated by means of central differences; that is,
    f a ^ k ( x ¯ ) f ( x ¯ + h a ^ k ) f ( x ¯ h a ^ k ) 2 h ,
    and
    2 f a ^ k 2 ( x ¯ ) f ( x ¯ + h a ^ k ) 2 f ( x ¯ ) + f ( x ¯ h a ^ k ) h 2 ,
    respectively.
  • For h = | | a k | | , Equations (18)–(22) yield the computational formula:
    f ( x w ) f ( x ¯ ) + 1 2 k = 1 M [ f ( x ¯ + a k ) f ( x ¯ a k ) ] y w k + 1 2 k = 1 M [ f ( x ¯ + a k ) 2 f ( x ¯ ) + f ( x ¯ a k ) ] y w k 2 .
From Equation (23), it is apparent that the computation of f ( x w ) requires 2 M + 1 calls of the exact and approximate models, while the computation of I FOV ( λ w ) requires, in addition, W calls of the approximate model. In this regard and taking into account that M W , we are led to a substantial reduction of the computational time.
The most widely used dimensionality reduction techniques are the linear embedding methods which are summarized below.
  • The principal component analysis (PCA) [15] performs a dimensionality reduction by projecting the original N-dimensional data on the M-dimensional subspace spanned by the dominant singular vectors of the data covariance matrix.
  • The locality pursuit embedding (LPE) [16] performs a principal component analysis on local nearest neighbor patches to reveal the tangent space structure on the M-dimensional subspace.
  • The locality preserving projection (LPP) [17] solves a variational problem that optimally preserves the neighborhood structure of the data set.
  • The locally embedded analysis (LEA) [18] uses an embedding strategy based on a linear eigenspace analysis to minimize the local reconstruction error.
It should be pointed out that PCA preserves only the global structure of the data, and may fail to preserve the local structure if the data lie on a nonlinear manifold. In contrast, LPE, LPP, and LEA are local linear approaches, which optimally preserve local neighborhood information (the local structure of the data) in a certain sense.
The PCA, LPE, LPP, and LEA methods are illustrated in Algorithms A1–A4 of Appendix A. The outputs of these algorithms are the linear mapping A and the vectors of parameters y w , w = 1 , , W . Note that in the PCA language, the column vectors a k , k = 1 , , M of A are the so-called empirical orthogonal functions, while the vectors of parameters y w , w = 1 , , W are the principal components.
A last problem which has to be solved is the choice of an approximate model. Two special features of SHDOM facilitate this choice, namely, the use (i) of spherical harmonics and discrete ordinates to represent the radiance field, and (ii) of a sort of successive order of scattering solution method (Picard iteration). In a first approximate model, we choose 2 zenith discrete angles per hemisphere to represent the radiance field; analogous to the one-dimensional radiative transfer, the resulting method is called the four-stream approximation. In a second approximate model, we consider only 2 steps of the method of Picard iteration; the resulting method is called the second-order of scattering approximation.
In summary, the spectral acceleration approach includes two steps. The first step is a pre-processing step, consisting in the computation of the sets of wavelengths { λ w } w = 1 W , the absorption coefficients σ abs gas ( λ w , z ) , and the weighting factors { ω w } w = 1 W in the framework of the correlated k-distribution method. The second step is a computational step, consisting of the calculation of
  • the data vectors x w , w = 1 , , W , the empirical orthogonal functions a k , k = 1 , , M , and the principal components y w , w = 1 , , W in the framework of dimensionality reduction techniques,
  • the correction factor f ( x w ) , w = 1 , , W , and the integrated signal I FOV ( λ w ) , w = 1 , , W according to Equations (23) and (16), respectively, and finally,
  • the spectral signal measured by the instrument I ( λ mw ) , w = 1 , , W m by means of Equation (10).

3. Numerical Analysis

Before presenting some numerical results, we describe an implementation of SHDOM that is devoted to the retrieval of atmospheric trace gas concentrations from space-borne spectral measurements of radiation reflected through the Earth’s atmosphere.

3.1. SHDOM Implementation

For an atmosphere consisting of gas molecules, we have to consider a domain of analysis with a large vertical extent, especially when dealing with trace retrievals. The reason is that gas molecules have a vertical concentration profile up to several tens of kilometers. Unfortunately, for large problems, SHDOM is limited by the amount of memory available for calculation, in particular on smaller computers. To handle this problem, in our implementation of SHDOM, we divide the domain of analysis into a number of subdomains along the vertical direction, and for each subdomain, we store the radiance and source function in separate files. The optical properties and the direct solar beam are calculated at each point on the base grid of the entire domain of analysis, while (i) the radiative transfer equation integration along each discrete ordinate during the solution iterations and (ii) the integration along the viewing directions for the intensity output are done successively moving from one subdomain to the neighboring one. Each subdomain has its own grid obtained by the cell splitting (adaptive grid) procedure. At the boundaries of a subdomain, the radiances are calculated not only at the grid points of the subdomain itself, but also at the grid points corresponding to the boundaries of the neighboring subdomains (Figure 2). This is done by integrating the radiative transfer equation and by applying the standard interpolation approach (the entering and exiting values of the extinction and extinction-source function product are computed using bilinear interpolation of the four gridpoint values of the faces pierced by a ray). To ensure the continuity condition for the radiance field, the boundary values of the radiance are passed between the neighboring subdomains. The data are transferred to and from files.
This implementation of SHDOM does not give exactly the same results as the standard implementation because of (i) the splitting decision for the cells situated at the boundaries of a subdomain, and (ii) the differences between the rays along which the radiative transfer equation is integrated (a ray is traced backward until the transmission falls below some minimum specified value only inside a subdomain). However, as compared to the overall accuracy of the algorithm, these errors are small. On the other hand, this implementation increases the computation time by the times which are needed (i) to write and read data to and from files, and (ii) to compute the radiances at the additional grid points on the boundaries of each subdomain. It should be pointed out that the number of subdomains is set by the user and depends on the problem to be solved.
The code can be reformulated as a parallel code by using the Message Passing Interface (MPI). In this context, it should be pointed out that a parallel implementation of SHDOM, but in which the domain of analysis is divided along the horizontal directions, has been described in Ref. [19]. However, in the case of horizontal splitting, the transfer of data between different neighboring subdomains is more challenging for the implementation.

3.2. Numerical Results

The computational domain is a rectangular prism of lengths L x = L y = 15 km and L z = 40  km. The discretization steps along the horizontal directions are Δ x = Δ y = 0.5 km; thus, the corresponding numbers of base grid points are N x = N y = 30 . Along the vertical direction, the domain is discretized into 8 subdomains; the corresponding altitude levels and discretization steps are listed in Table 1. The cloud is homogeneous in the vertical direction and is placed between 3 and 4 km (second subdomain). The cloud extinction field is modeled as
σ ext cloud ( x , y ) = σ 0 f ( x , y ) ,
where σ 0 = 6 km 1 and f ( x , y ) is the indicator function taking the values f ( x , y ) = 1 inside the cloud and f ( x , y ) = 0 inside the clear sky region. The indicator function f ( x , y ) is generated by a two-dimensional broken cloud model [20] with a cloud fraction of about 0.4 . In order to avoid abrupt changes of the extinction field in the horizontal plane, σ ext cloud ( x , y ) is smoothed at the boundary of a cloudy region (by linear interpolation along two discretization steps in each direction). To simplify the analysis, a Henyey–Greenstein phase function with an asymmetry parameter g = 0.8 is considered, while the single-scattering albedo is chosen as ω cloud = 0.99 . The number of discrete zenith and azimuth angles are N μ = 16 and N φ = 2 N μ , respectively, the solar and instrument zenith angles are θ 0 = 60 and θ m 0 = 30 , respectively, the relative azimuth angle is Δ φ = 0 , and a Lambertian reflecting surface with the surface albedo A = 0.2 is assumed. The footprint of the detector is considered to be a square of length 2 a = 10 Δ x centered at x 0 = y 0 = L x / 2 , and z 0 = L z . The calculations are performed for a mid-latitude summer atmosphere [21] and a wavelength-dependent slit function corresponding to the TROPOspheric Monitoring Instrument (TROPOMI).
SHDOM is run by using an adaptive grid with a splitting accuracy of 10 4 . The solution accuracy is 10 4 , and the simulations are performed for periodic boundary conditions by using the delta-M approximation along with the untruncated phase function single-scattering solution (TMS correction of Nakajima and Tanaka [22]). The dimensionality reduction parameter M, which determines the approximation accuracy of the linear embedding methods, is chosen as M = 4 . The simulations were performed on a computer Intel Core i5-3340M CPU 2.70GHz with 8 GB RAM.

3.2.1. Oxygen A-Band Test Problem

In our first example, we consider, in addition to the scattering and absorption by a cloud, molecular Rayleigh scattering, and the absorption in the Oxygen A-band. The measurement spectral grid contains 107 spectral points between 758 nm and 771 nm, while the number of spectral points for the correlated k-distribution method is W = 936 . For this test, a two-dimensional slice at y = 7 km is selected from a three-dimensional broken-cloud field (Figure 3). Figure 4 illustrates the spectral signal computed by using SHDOM with the correlated k-distribution method. Using these results as a reference, we show in Figure 5 the relative differences in the spectral signal corresponding to the linear embedding methods, and in Table 2, the RMS of the relative difference and the computational time in hours, minutes, and seconds. The computation time of SHDOM with the correlated k-distribution method was 36 minutes and 28 s. Several conclusions can be formulated:
  • For the dimensionality reduction parameter M = 4 , the relative differences are smaller than 1.25 × 10 3 over the entire spectral domain, while the RMS values are smaller than 6.4 × 10 4 .
  • The computation time corresponding to the second-order of scattering approximation is smaller than that corresponding to the four-stream approximation.
  • For the second-order of scattering approximation, the fastest linear embedding method is PCA and the most accurate is LPP.
  • Relative to the correlated k-distribution method, the acceleration factor is of about 7–12.

3.2.2. NO 2 -Test Problem

In the second example, the absorption of NO 2 , ozone ( O 3 ), oxygen dimer ( O 4 ), and water vapor is considered. The measurement spectral grid contains 119 spectral points between 425 nm and 450 nm, while the number of spectral points for the correlated k-distribution method is W = 268 . Note that in the correlated k-distribution method, the auxiliary gases are treated as weak absorbers meaning that for these gases, the average values of the LBL cross sections are used in the calculation. The test problem is three-dimensional, and the indicator function of the broken cloud is illustrated in Figure 6. In Table 3, we indicate the amount of memory required for storing the radiance and source function for each subdomain. As a result that the domain of analysis is not excessively large, the total amount of disk memory is 3.168 GB. As a result, we infer that SHDOM can be also run without splitting the domain of analysis. Taking these results as a reference, we found that the relative errors due to the domain splitting procedure are smaller than 10 3 . The spectral signal computed by using SHDOM with the correlated k-distribution method is shown in Figure 7, the relative differences in the spectral signal corresponding to the linear embedding methods are plotted in Figure 8, and finally, the RMS of the relative difference and the computational time in hours, minutes, and seconds are given in Table 4. For this test example, the computation time of SHDOM with the correlated k-distribution method was 12 h, 18 min, and 32 s. As in the first test example, we see that
  • for the dimensionality reduction parameter M = 4 , the relative differences are smaller than 5 × 10 4 over the entire spectral domain, while the RMS values are smaller than 2.2 × 10 4 ;
  • the second-order of scattering approximation is faster than the four-stream approximation;
  • for the second-order of scattering approximation, the fastest linear embedding method is PCA and the most accurate is LPE;
  • the acceleration factor is about 9–12.

4. Conclusions

A spectral acceleration approach for SHDOM has been designed. This approach combines the correlated k-distribution method and the linear embedding methods PCA, LPE, LPP, and LEA.
We have performed a numerical analysis of these techniques for a two-dimensional atmosphere in the Oxygen A-band, and a three-dimensional atmosphere consisting of nitrogen, ozone, oxygen dimer, and water vapor in a spectral interval ranging from 425 nm and 450 nm. The main conclusions of our analysis are as follows.
  • SHDOM with the correlated k-distribution and linear embedding methods has a sufficiently high accuracy. The relative differences in the spectral signal are smaller than 1.25 × 10 3 (over the entire spectral domain) in the case of a two-dimensional atmosphere, and 5.0 × 10 4 in the case of a three-dimensional atmosphere. In three of the four cases, the most accurate linear embedding method is LPE.
  • The linear embedding methods based on the second-order of scattering approximation are faster than those based on the four-stream approximation. Specifically, in the case of a two-dimensional atmosphere, the computation time of the linear embedding methods is on average about 4 min and 30 s for the four-stream approximation, and 3 min and 20 s for the second-order of scattering approximation, while in the case of a three-dimensional atmospheres, the corresponding times are 1 h and 21 min for the four-stream approximation, and 1 h and 2 min for the second-order of scattering approximation. The fastest linear embedding methods is PCA followed by LPE. For the test examples considered in our numerical analysis, PCA in conjunction with a second-order of scattering approximation yields an acceleration factor of 12 relative to the correlated k-distribution method.
In conclusion, from the point of view of accuracy and efficiency, it appears that LPE based on the second-order of scattering approximation is the most suitable acceleration method. Note that the accuracy and efficiency of the linear embedding methods are determined by the dimensionality reduction parameter M (as the accuracy and the computation time increase with M, an optimal value for M should be a compromise between accuracy and efficiency).
SHDOM equipped with the spectral acceleration approach can be used to analyze the impact of cloud inhomogeneities on trace gas retrievals [23]. In particular, for a specific cloudy scene, a spectral signal can be simulated by SHDOM and then included in a one-dimensional retrieval algorithm to derive for example, the total column amount of a trace gas. By comparing the retrieved total column with the true value, a bias due to cloud effects can be deduced [24,25], and possible correction strategies can be explored. Going one step further, a multi-dimensional trace gas retrieval algorithm can be designed. In this case, the spectral acceleration approach should be adapted to a linearized version of SHDOM (based on a forward or a forward-adjoint approach as described in Ref. [26]). This is the topic of a companion paper.

Author Contributions

Conceptualization, A.D.; software, A.D. and D.S.E.; formal analysis, D.S.E. and T.T.; writing—original draft preparation, A.D.; writing—review and editing, D.S.E. and T.T. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Appendix A

The PCA, LPE, LPP, and LEA methods are described in Ref. [11]. For convenience of the reader, the related algorithms are summarized below.
Algorithm A1: PCA.
Step 1.
Stack all centered data x w x w x ¯ , w = 1 , , W , into the columns of a matrix X R N × W , i.e., X = [ x w ] w = 1 W .
Step 2.
Calculate the covariance matrix C x = ( 1 / W ) X X T R N × N .
Step 3.
Calculate a singular value decomposition of the covariance matrix C x = U Σ U T , where  Σ = diag [ σ k ] k = 1 N R N × N is the diagonal matrix of the singular values σ 1 > σ 2 > > σ N > 0 , and U = [ u k ] k = 1 N R N × N is the orthogonal matrix of the singular vectors u k .
Step 4.
Take A = U M = [ u k ] k = 1 M R N × M yielding A = U M T R M × N , or alternatively, consider the scaled orthogonal vectors u ¯ k = σ k u k , and take
A = U ¯ M = U M Σ M 1 / 2 R N × M yielding A = U ¯ M = Σ M 1 / 2 U M T R M × N , with  Σ M = diag [ σ k ] k = 1 M .
Step 5.
Compute the vectors of parameters y w = A x w for w = 1 , , W .
Algorithm A2: LPE.
Step 1.
Stack all centered data x w x w x ¯ , w = 1 , , W , into the columns of a matrix
X R N × W , i.e., X = [ x w ] w = 1 W .
Step 2.
For each w = 1 , W , consider the sets X ( w ) = { x N ( j ) ( w ) } j = 1 k containing the k nearest neighbors to x w with respect to the Euclidean distance.
Step 3.
Set L = 0 R W × W .
Step 4.
For w = 1 , W , compute the diagonal matrix D w = diag [ I v X ( w ) ] v = 1 W R W × W , where the indicator function I v X ( w ) is defined by I v X ( w ) = 1 if x v X ( w ) , and  I v X ( w ) = 0  otherwise.
Step 5.
For all w = 1 , W , update the matrix L recursively as L L + D w ( 1 / k ) D w 1 1 T D w T , where 1 R W is a column vector taking 1 at each entry.
Step 6.
Calculate the symmetric and positive semi-definite matrix A = X L X T R N × N , compute a singular value decomposition of A , A = U Σ U T , and as in Algorithm A1, take A = U M Σ M 1 / 2 R N × M and A = Σ M 1 / 2 U M R M × N .
Step 7.
Compute the vectors of parameters y w = A x w for w = 1 , , W .
Algorithm A3: LPP.
Step 2.
Stack all centered data x w x w x ¯ , w = 1 , , W , into the columns of a matrix
X R N × W , i.e., X = [ x w ] w = 1 W .
Step 2.
For w = 1 , W , consider the sets X ( w ) = { x N ( j ) ( w ) } j = 1 k containing the k nearest neighbors to x w with respect to the Euclidean distance.
Step 3.
For w , v = 1 , W , compute the elements W w v of the sparse matrix W R W × W as W w v = exp ( | | x w x v | | 2 / t ) if x v X ( w ) , and W w v = 0 otherwise. Alternatively, chose  W w v = 1 if x v X ( w ) , and W w v = 0 otherwise.
Step 4.
Calculate the elements D v of the diagonal matrix D = diag [ D v ] v = 1 W R W × W as
D v = w W w v , and the Laplacian matrix L = D W R W × W .
Step 5.
For A = X L X T R N × N and B = X D X T R N × N , solve the generalized eigenvalue problem A z k = λ k B z k as follows: (i) compute a singular value decomposition of the matrix B = U Σ U T , (ii) calculate the matrix D = Σ 1 / 2 U T R N × N and its inverse D 1 = U Σ 1 / 2 R N × N , and finally, (iii) compute the matrix C = D A D T R N × N and its singular value decomposition C = V Λ V T . The eigenvalues λ k are the singular values of C , while the eigenvectors z k are given by z k = D T v k , where v k are the singular vectors of C .
Step 6.
Construct the matrix V M = [ v k + N M ] k = 1 M R N × M , where v k is the kth column vector of V R N × N , and take A = D 1 V M R N × M and A = V M T D R M × N .
Step 7.
Compute the vectors of parameters y w = A x w for w = 1 , , W .
Algorithm A4: LEA.
Step 1.
Stack all centered data x w x w x ¯ , w = 1 , , W , into the columns of a matrix X R N × W , i.e., X = [ x w ] w = 1 W .
Step 2.
For each w = 1 , W , consider the sets X ( w ) = { x N ( j ) ( w ) } j = 1 k containing the k nearest neighbors to x w with respect to the Euclidean distance.
Step 3.
Calculate the elements of the sparse matrix W R W × W as follows: (i) for w = 1 , W , calculate the elements G i j of the matrix G R k × k as G i j = ( x w x N ( i ) ( w ) ) T ( x w x N ( j ) ( w ) ) , and (ii) set W w N ( j ) = w j with w j being the elements of the column vector w = G 1 1 / ( 1 T G 1 1 ) R k .
Step 4.
Calculate the elements D v of the diagonal matrix D = diag [ D v ] v = 1 W R W × W as D v = w W w v .
Step 5.
As in Algorithm A3, solve the generalized eigenvalue problem A z k = λ k B z k , for  A = X ( D W ) T ( D W ) X T R N × N and B = X ( D T D + W T W ) X T R N × N , and let  C = V Λ V T .
Step 6.
Construct the matrix V M = [ v k + N M ] k = 1 M R N × M , where v k is the kth column vector of V R N × N , and take A = D 1 V M R N × M and A = V M T D R M × N .
Step 7.
Compute the vectors of parameters y w = A x w for w = 1 , , W .

References

  1. Evans, K. The spherical harmonic discrete ordinate method for three-dimensional atmospheric radiative transfer. J. Atmos. Sci. 1998, 55, 429–446. [Google Scholar] [CrossRef]
  2. Spherical Harmonic Discrete Ordinate Method (SHDOM) for Atmospheric Radiative Transfer. Available online: http://coloradolinux.com/shdom/ (accessed on 11 May 2020).
  3. Kuo, K.S.; Weger, R.; Welch, R.; Cox, S. The Picard iterative approximation to the solution of the integral equation of radiative transfer—Part II. Three-dimensional geometry. J. Quant. Spectrosc. Radiat. Transf. 1996, 55, 195–213. [Google Scholar] [CrossRef]
  4. Goody, R.; Yung, Y. Atmospheric Radiation: Theoretical Basis; Oxford University Press: Oxford, UK, 1996. [Google Scholar]
  5. Fu, Q.; Liou, K.N. On the Correlated k-Distribution Method for Radiative Transfer in Nonhomogeneous Atmospheres. J. Atmos. Sci. 1992, 49, 2139–2156. [Google Scholar] [CrossRef] [Green Version]
  6. Natraj, V. A review of fast radiative transfer techniques. In Light Scattering Reviews 8; Springer: Berlin/Heidelberg, Germany, 2013; pp. 475–504. [Google Scholar] [CrossRef]
  7. Del Águila, A.; Efremenko, D.S.; Trautmann, T. A Review of Dimensionality Reduction Techniques for Processing Hyper-Spectral Optical Signal. Light Eng. 2019, 85–98. [Google Scholar] [CrossRef] [Green Version]
  8. Del Águila, A.; Efremenko, D.; Molina García, V.; Xu, J. Analysis of Two Dimensionality Reduction Techniques for Fast Simulation of the Spectral Radiances in the Hartley-Huggins Band. Atmosphere 2019, 10, 142. [Google Scholar] [CrossRef] [Green Version]
  9. Natraj, V.; Jiang, X.; Shia, R.; Huang, X.; Margolis, J.; Yung, Y. Application of the principal component analysis to high spectral resolution radiative transfer: A case studyof the O2 A-band. J. Quant. Spectrosc. Radiat. Transf. 2005, 95, 539–556. [Google Scholar] [CrossRef]
  10. Natraj, V.; Shia, R.; Yung, Y. On the use of principal component analysis to speed up radiative transfer calculations. J. Quant. Spectrosc. Radiat. Transf. 2010, 111, 810–816. [Google Scholar] [CrossRef]
  11. Efremenko, D.; Doicu, A.; Loyola, D.; Trautmann, T. Optical property dimensionality reduction techniques for accelerated radiative transfer performance: Application to remote sensing total ozone retrievals. J. Quant. Spectrosc. Radiat. Transf. 2014, 133, 128–135. [Google Scholar] [CrossRef]
  12. Goody, R.; West, R.; Chen, L.; Crisp, D. The correlated k-method for radiation calculations in nonhomogeneous atmosphere. J. Quant. Spectrosc. Radiat. Transf. 1989, 42, 539–550. [Google Scholar] [CrossRef]
  13. Ambartzumyan, V. The effect of the absorption lines on the radiative equilibrium of the outer layers of the stars. Publ. Obs. Astron. Univ. Leningr. 1936, 6, 7–18. [Google Scholar]
  14. Wiscombe, W.; Evans, J. Exponential-sum fitting of radiative transmission functions. J. Comput. Phys. 1997, 24, 416–444. [Google Scholar] [CrossRef]
  15. Jolliffe, I. Principal Component Analysis; Springer: Berlin/Heidelberg, Germany, 2002. [Google Scholar] [CrossRef]
  16. Min, W.; Lu, K.; He, X. Locality pursuit embedding. Pattern Recognit. 2004, 37, 781–788. [Google Scholar] [CrossRef]
  17. He, X.; Niyogi, P. Locality Preserving Projections. In NIPS’03: Proceedings of the 16th International Conference on Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2003; pp. 153–160. Available online: https://papers.nips.cc/paper/2359-locality-preserving-projections.pdf (accessed on 1 June 2020).
  18. Fu, Y.; Huang, T. Locally Linear Embedded Eigenspace Analysis. Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign. 2005. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.93.8003&rep=rep1&type=pdf (accessed on 29 May 2020).
  19. Pincus, R.; Evans, K.F. Computational Cost and Accuracy in Calculating Three-Dimensional Radiative Transfer: Results for New Implementations of Monte Carlo and SHDOM. J. Atmos. Sci. 2009, 66, 3131–3146. [Google Scholar] [CrossRef] [Green Version]
  20. Alexandrov, M.; Marshak, A.; Ackerman, A. Cellular Statistical Models of Broken Cloud Fields. Part I: Theory. J. Atmos. Sci. 2010, 67, 2125–2151. [Google Scholar] [CrossRef]
  21. Anderson, G.; Clough, S.; Kneizys, F.; Chetwynd, J.; Shettle, E. AFGL Atmospheric Constituent Profiles (0–120 km); AFGL-TR-86-0110; Air Force Geophysics Laboratory: Hanscom Air Force Base, MA, USA, 1986. [Google Scholar]
  22. Nakajima, T.; Tanaka, M. Algorithms for radiative intensity calculations in moderately thick atmos using a truncation approximation. J. Quant. Spectrosc. Radiat. Transf. 1988, 40, 51–69. [Google Scholar] [CrossRef]
  23. Efremenko, D.S.; Doicu, A.; Loyola, D.; Trautmann, T. Fast Stochastic Radiative Transfer Models for Trace Gas and Cloud Property Retrievals Under Cloudy Conditions. In Springer Series in Light Scattering; Kokhanovsky, A., Ed.; Springer: Berlin/Heidelberg, Germany, 2018; pp. 231–277. [Google Scholar] [CrossRef]
  24. Tarasenkov, M.V.; Kirnos, I.V.; Belov, V.V. Observation of the Earth’s surface from the space through a gap in a cloud field. Atmos. Ocean. Opt. 2017, 30, 39–43. [Google Scholar] [CrossRef]
  25. Zhuravleva, T.B.; Nasrtdinov, I.M.; Russkova, T.V. Influence of 3D cloud effects on spatial-angular characteristics of the reflected solar radiation field. Atmos. Ocean. Opt. 2017, 30, 103–110. [Google Scholar] [CrossRef]
  26. Doicu, A.; Efremenko, D. Linearizations of the Spherical Harmonic Discrete Ordinate Method (SHDOM). Atmosphere 2019, 10, 292. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Scheme of the radiative transfer problem.
Figure 1. Scheme of the radiative transfer problem.
Remotesensing 12 03703 g001
Figure 2. At the lower boundary of the subdomain k + 1 , the downward radiances are also computed at the points A, B, C, and D, which are grid points on the upper boundary of the subdomain k. During the downward iteration step, these boundary values are passed to the subdomain k.
Figure 2. At the lower boundary of the subdomain k + 1 , the downward radiances are also computed at the points A, B, C, and D, which are grid points on the upper boundary of the subdomain k. During the downward iteration step, these boundary values are passed to the subdomain k.
Remotesensing 12 03703 g002
Figure 3. Upper panel: indicator function f ( x i , y j ) with x i = i Δ x and y j = j Δ y for the O 2 A-band test problem. Lower panel: a slice of the indicator function f ( x ) at y = 7 km.
Figure 3. Upper panel: indicator function f ( x i , y j ) with x i = i Δ x and y j = j Δ y for the O 2 A-band test problem. Lower panel: a slice of the indicator function f ( x ) at y = 7 km.
Remotesensing 12 03703 g003
Figure 4. Spectral signal for the O 2 A-band test problem. The differences between SHDOM with the correlated k-distribution method with and without linear embedding methods are not visible in this plot.
Figure 4. Spectral signal for the O 2 A-band test problem. The differences between SHDOM with the correlated k-distribution method with and without linear embedding methods are not visible in this plot.
Remotesensing 12 03703 g004
Figure 5. Relative differences in the spectral signal for the linear embedding methods and the O 2 A-band test problem. The plots in the left panel correspond to the four-stream approximation, while the plots in the right panel correspond to the second-order of scattering approximation.
Figure 5. Relative differences in the spectral signal for the linear embedding methods and the O 2 A-band test problem. The plots in the left panel correspond to the four-stream approximation, while the plots in the right panel correspond to the second-order of scattering approximation.
Remotesensing 12 03703 g005
Figure 6. Indicator function f ( x i , y j ) with x i = i Δ x and y j = j Δ y for the NO 2 -test problem.
Figure 6. Indicator function f ( x i , y j ) with x i = i Δ x and y j = j Δ y for the NO 2 -test problem.
Remotesensing 12 03703 g006
Figure 7. Spectral signal for the NO 2 -test problem.
Figure 7. Spectral signal for the NO 2 -test problem.
Remotesensing 12 03703 g007
Figure 8. Relative differences in the spectral for the linear embedding methods and the NO 2 -test problem. The plots in the left panel correspond to the four-stream approximation, while the plots in the right panel correspond to the second-order of scattering approximation.
Figure 8. Relative differences in the spectral for the linear embedding methods and the NO 2 -test problem. The plots in the left panel correspond to the four-stream approximation, while the plots in the right panel correspond to the second-order of scattering approximation.
Remotesensing 12 03703 g008
Table 1. Discretization of the domain of analysis along the vertical direction.
Table 1. Discretization of the domain of analysis along the vertical direction.
Subdomain12345678
Altitude levels [km]0–33–44–77–1010–1414–2222–3030–40
Discretization step [km]0.50.10.50.51.02.02.05
Table 2. Root mean square (RMS) of relative difference and the computational time (CPU) in hours:minutes:seconds corresponding to the linear embedding methods for the O 2 A-band test problem.
Table 2. Root mean square (RMS) of relative difference and the computational time (CPU) in hours:minutes:seconds corresponding to the linear embedding methods for the O 2 A-band test problem.
Method Four-StreamSecond-Order Scattering
PCARMS 6.37 × 10 4 5.19 × 10 5
CPU0:04:210:03:01
LEARMS 5.69 × 10 4 4.00 × 10 5
CPU0:04:450:03:17
LPERMS 5.37 × 10 4 3.29 × 10 5
CPU0:04:350:03:05
LPPRMS 6.00 × 10 4 3.14 × 10 5
CPU0:05:010:03:53
Table 3. Amount of memory required for storing the radiance and source function for each subdomain.
Table 3. Amount of memory required for storing the radiance and source function for each subdomain.
Subdomain12345678
Memory [MB]54992454948622822822852
Table 4. RMS of relative difference and the computational time (CPU) in hours:minutes:seconds corresponding to the linear embedding methods for the NO 2 -test problem.
Table 4. RMS of relative difference and the computational time (CPU) in hours:minutes:seconds corresponding to the linear embedding methods for the NO 2 -test problem.
MethodFour-StreamSecond-Order Scattering
PCARMS 3.84 × 10 5 1.25 × 10 4
CPU1:21:241:02:56
LEARMS 7.41 × 10 5 1.34 × 10 4
CPU1:22:181:03:48
LPERMS 3.73 × 10 5 1.12 × 10 4
CPU1:21:531:03:21
LPPRMS 7.22 × 10 5 1.26 × 10 4
CPU1:22:431:04:09
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Doicu, A.; Efremenko, D.S.; Trautmann, T. A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method. Remote Sens. 2020, 12, 3703. https://doi.org/10.3390/rs12223703

AMA Style

Doicu A, Efremenko DS, Trautmann T. A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method. Remote Sensing. 2020; 12(22):3703. https://doi.org/10.3390/rs12223703

Chicago/Turabian Style

Doicu, Adrian, Dmitry S. Efremenko, and Thomas Trautmann. 2020. "A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method" Remote Sensing 12, no. 22: 3703. https://doi.org/10.3390/rs12223703

APA Style

Doicu, A., Efremenko, D. S., & Trautmann, T. (2020). A Spectral Acceleration Approach for the Spherical Harmonics Discrete Ordinate Method. Remote Sensing, 12(22), 3703. https://doi.org/10.3390/rs12223703

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop