Linearizations of the Spherical Harmonic Discrete Ordinate Method (SHDOM)

Abstract

Linearizations of the spherical harmonic discrete ordinate method (SHDOM) by means of a forward and a forward-adjoint approach are presented. Essentially, SHDOM is specialized for derivative calculations and radiative transfer problems involving the delta-M approximation, the TMS correction, and the adaptive grid splitting, while practical formulas for computing the derivatives in the spherical harmonics space are derived. The accuracies and efficiencies of the proposed methods are analyzed for several test problems.

1. Introduction

Accurate and efficient modeling of radiative transfer is important for the retrieval of earth atmospheric constituent information by means of remote sensing. Real clouds are an inhomogeneous three-dimensional scattering medium. For the new generation of satellite spectrometers with a relative high spatial resolution (such as Sentinel 5 Precursor, Sentinel 4 and Sentinel 5 [1]), it is important to account for the sub-pixel cloud inhomogeneities, or at least, to assess their effect on the radiances at the top of the atmosphere, and in particular, on the trace gas and cloud retrieval results. So far, the majority of operational retrieval algorithms are based on the one-dimensional (1D) radiative transfer model, which neglects all horizontal variability of the atmosphere. However, 1D approximation can lead to the significant errors in the calculated radiances and retrieved parameters [2,3,4,5,6]. In this regard, three-dimensional (3D) radiative trasnfer models are required.

The SHDOM method developed by Evans is a widely used deterministic method to solve the 3D radiative transfer equation [7]. The radiative transfer equation is solved iteratively by using the spherical harmonic and the discrete ordinate representations of the radiance field. Each iteration consists of four steps: (1) the transformation of the source function from spherical harmonics to discrete ordinates; (2) the integration of the source function along discrete ordinate directions to compute the radiance field; (3) the transformation of the discrete ordinate radiances to spherical harmonics; and (4) the calculation of the source function from the radiance in spherical harmonics space. An important feature of SHDOM is the adaptive grid technique. This technique improves the solution accuracy by increasing the spatial resolution in regions where the source function is changing more rapidly. The difference scheme employed in SHDOM is a characteristic scheme. In the so-called long characteristic method, the integration is performed along the whole characteristic of the differential operator, while in the short characteristic method, the integration is performed along a segment of the characteristic located inside a cell. SHDOM has been modfied for use on multiple processors [8], and to include polarization [9,10,11].

In atmospheric remote sensing and data assimilation, it is necessary to compute the partial derivatives of the outgoing radiances with respect to some atmospheric parameters of interest. For this purpose, a linearized forward approach, relying on an analytical computation of the derivatives, or a linearized forward-adjoint approach, relying on the application of the adjoint radiative transfer theory, can be used. For a plane-parallel geometry, scalar and vector linearized forward approaches were described in Refs. [12,13], while linearized forward-adjoint approaches were proposed in [14,15,16,17,18,19,20,21]. The forward-adjoint method is extremely efficient because only two radiative transfer calculations are required for derivatives calculations.

A theoretical framework for three-dimensional remote sensing of the atmosphere with adjoint methods was described by Martin et al. [22]. In a subsequent paper, Martin and Hasekamp [23] used the linearized-forward approach to retrieve cloud extinction fields from multi-angle measurements. Although, the analysis was restricted to two-dimensional problems (a two-dimensional domain and a set of directions confined to the unit circle and characterized by one angular variable), the efficiency of adjoint methods for multi-dimensional remote sensing of the atmosphere and surface was demonstrated.

In this paper, linearizations of SHDOM by using a forward and a forward-adjoint approach are presented. After formulating the problem in Section 2, we discuss in Section 3 the fundamentals of the linearized SHDOM with a forward-adjoint approach. The accuracy and efficiency of the linearized models is analyzed in Section 4, while the final section of our paper contains a few concluding remarks.

2. Problem Formulation

We consider the solar radiative transfer in a rectangular cuboid of lengths $L_{\mathrm{x}}$, $L_{\mathrm{y}}$ and $L_{\mathrm{z}}$ as shown in Figure 1. The top and the bottom faces of the prism are denoted by $S_{\mathrm{t}}$ and $S_{\mathrm{b}}$, respectively, while the lateral faces are denoted by $S_{1\mathrm{x}}$ ($x=0$), $S_{2\mathrm{x}}$ ($x=L_{\mathrm{x}}$), $S_{1\mathrm{y}}$ ($y=0$), and $S_{2\mathrm{y}}$ ($y=L_{\mathrm{y}}$). The boundary-value problem for the total radiance at point $\mathbf{r}$ in direction $\mathsf{\Omega }$ consists of the inhomogeneous differential equation (with the thermal emission term neglected)

$$\begin{array}{cc}\hfill \frac{\mathrm{d}I}{\mathrm{d}s}(\mathbf{r},\mathsf{\Omega })& =-{\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)I(\mathbf{r},\mathsf{\Omega })+\frac{{\sigma }_{\mathrm{sct}}\left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime },\hfill \end{array}$$

(1)

the top-of-atmosphere boundary condition

$$I({\mathbf{r}}_{\mathrm{t}},{\mathsf{\Omega }}^{-})=\frac{F_0}{|{\mu }_0|}\delta ({\mathsf{\Omega }}^{-}-{\mathsf{\Omega }}_0),{\mathbf{r}}_{\mathrm{t}}\in S_{\mathrm{t}},$$

(2)

and the surface boundary condition

$$\begin{array}{cc}\hfill I({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+})& =\frac{1}{\pi }{\int }_{{\mathsf{\Omega }}^{-}}\rho ({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+},{\mathsf{\Omega }}^{-})\left|{\mu }^{-}\right|I({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{-})\mathrm{d}{\mathsf{\Omega }}^{-},{\mathbf{r}}_{\mathrm{b}}\in S_{\mathrm{b}}.\hfill \end{array}$$

(3)

At the horizontal boundaries we assume periodic boundary conditions, i.e.,

$$\begin{gathered} \begin{array}{cc}\hfill I(x=0,y,z,\mathsf{\Omega })& =I(x=L_{\mathrm{x}},y,z,\mathsf{\Omega }),\hfill \\ \\ \hfill I(x,y=0,z,\mathsf{\Omega })& =I(x,y=L_{\mathrm{y}},z,\mathsf{\Omega }).\hfill \end{array} \end{gathered}$$

(4)

Here, ${\sigma }_{\mathrm{ext}}$ and ${\sigma }_{\mathrm{sct}}=\omega {\sigma }_{\mathrm{ext}}$ are the extinction and scattering coefficients, respectively, $\omega$ is the single-scattering albedo, P is the phase function, $\rho$ is the bidirectional surface reflection function, ${\mathsf{\Omega }}_0=({\mu }_0,{\phi }_0)$ with ${\mu }_0<0$, is the solar direction, $F_0$ is the solar flux, ${\mathsf{\Omega }}^{+}$ and ${\mathsf{\Omega }}^{-}$ denote an upward and a downward direction, respectively, $\mathsf{\Omega }$ is the unit sphere, and ${\mathsf{\Omega }}^{+}$ and ${\mathsf{\Omega }}^{-}$ stand for the upper and lower unit hemispheres, respectively.

Let us define the forward transport operator $\mathcal{L}$ by

$$\begin{array}{cc}\hfill \left(\mathcal{L}I\right)(\mathbf{r},\mathsf{\Omega })& =\frac{\mathrm{d}I}{\mathrm{d}s}(\mathbf{r},\mathsf{\Omega })+{\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)I(\mathbf{r},\mathsf{\Omega })\hfill \\ & -\frac{{\sigma }_{\mathrm{sct}}\left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }\hfill \\ & -\frac{1}{\pi }\delta \left(z\right)H\left(\mu \right)\mu {\int }_{\mathsf{\Omega }}\rho (\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })H(-{\mu }^{\prime })\left|{\mu }^{\prime }\right|I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime },\hfill \end{array}$$

(5)

and the forward source function $Q(\mathbf{r},\mathsf{\Omega })$ by

$$Q(\mathbf{r},\mathsf{\Omega })=F_0\delta (z-L_{\mathrm{z}})\delta (\mathsf{\Omega }-{\mathsf{\Omega }}_0),$$

(6)

where $\delta$ is the Dirac delta function and H is the Heaviside step function. It can be shown that if the radiation field $I(\mathbf{r},\mathsf{\Omega })$ solves the inhomogeneous operator equation

$$\left(\mathcal{L}I\right)(\mathbf{r},\mathsf{\Omega })=Q(\mathbf{r},\mathsf{\Omega }),$$

(7)

with the boundary conditions (4),

$$I({\mathbf{r}}_{\mathrm{t}},{\mathsf{\Omega }}^{-})=0,{\mathbf{r}}_{\mathrm{t}}\in S_{\mathrm{t}},\mathrm{and}I({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+})=0,{\mathbf{r}}_{\mathrm{b}}\in S_{\mathrm{b}},$$

(8)

then $I(\mathbf{r},\mathsf{\Omega })$ solves the radiative transfer Equation (1) with the boundary conditions (2), (3), and (4). The converse result is also true.

The adjoint transport operator ${\mathcal{L}}^{†}$ is defined through the relation

$$〈\mathcal{L}I,I^{†}〉=〈I,{\mathcal{L}}^{†}{I}^{†}〉,$$

(9)

where the scalar product of the fields $I_1$ and $I_2$ is given by

$$〈I_1,I_2〉={\int }_D{\int }_{\mathsf{\Omega }}{I}_1(\mathbf{r},\mathsf{\Omega })I_2(\mathbf{r},\mathsf{\Omega })\mathrm{d}\mathsf{\Omega }\mathrm{d}V$$

(10)

and D is the domain of the rectangular cuboid. The expression of the adjoint operator, under the assumptions that (i) I satisfies the boundary conditions (4) and (8), and (ii) $I^{†}$ satisfies the boundary conditions (4),

$$I^{†}({\mathbf{r}}_{\mathrm{t}},{\mathsf{\Omega }}^{+})=0,{\mathbf{r}}_{\mathrm{t}}\in S_{\mathrm{t}},\mathrm{and}{I}^{†}({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{-})=0,{\mathbf{r}}_{\mathrm{b}}\in S_{\mathrm{b}},$$

(11)

is given by

$$\begin{array}{cc}\hfill \left({\mathcal{L}}^{†}{I}^{†}\right)(\mathbf{r},\mathsf{\Omega })& =-\frac{\mathrm{d}{I}^{†}}{\mathrm{d}s}(\mathbf{r},\mathsf{\Omega })+{\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)I^{†}(\mathbf{r},\mathsf{\Omega })\hfill \\ & -\frac{{\sigma }_{\mathrm{sct}}\left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},{\mathsf{\Omega }}^{\prime },\mathsf{\Omega })I^{†}(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }\hfill \\ & -\frac{1}{\pi }\delta \left(z\right)H(-\mu )\left|\mu \right|{\int }_{\mathsf{\Omega }}\rho (\mathbf{r},{\mathsf{\Omega }}^{\prime },\mathsf{\Omega })H\left({\mu }^{\prime }\right){\mu }^{\prime }{I}^{†}(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }.\hfill \end{array}$$

(12)

Note that the requirement of homogeneous and periodic boundary conditions for both I and $I^{†}$ is essential when computing the adjoint of the differential operator $\mathrm{d}/\mathrm{d}s$ by using the Gauss theorem over the domain D.

In our analysis we consider a situation which is typical for atmospheric remote sensing. Atmospheric radiation can be measured e.g., by a spectrometer, which in the following is called detector. The detector, which measures the radiance at the top of the atmosphere in the direction ${\mathsf{\Omega }}_{\mathrm{m}}^0=({\mu }_{\mathrm{m}}^0,{\phi }_{\mathrm{m}}^0)$, ${\mu }_{\mathrm{m}}^0>0$, is represented by a point with position vector ${\mathbf{r}}_{\mathrm{D}}=(x_{\mathrm{D}},y_{\mathrm{D}},z_{\mathrm{D}})$ and is characterized by a characteristic function of its field of view ${\chi }_{\mathrm{FOV}}={\chi }_{\mathrm{FOV}}({\mathsf{\Omega }}_{\mathrm{m}}·{\mathsf{\Omega }}_{\mathrm{m}}^0)$. The signal measured by the detector, can be written as

$$\mathcal{I}={\int }_{\mathsf{\Omega }}{\chi }_{\mathrm{FOV}}({\mathsf{\Omega }}_{\mathrm{m}}·{\mathsf{\Omega }}_{\mathrm{m}}^0)I({\mathbf{r}}_{\mathrm{t}}\left({\mathsf{\Omega }}_{\mathrm{m}}\right),{\mathsf{\Omega }}_{\mathrm{m}})\mathrm{d}{\mathsf{\Omega }}_{\mathrm{m}},$$

(13)

where $I({\mathbf{r}}_{\mathrm{t}}\left({\mathsf{\Omega }}_{\mathrm{m}}\right),{\mathsf{\Omega }}_{\mathrm{m}})$ is the radiance at the top of the atmosphere in direction ${\mathsf{\Omega }}_{\mathrm{m}}$. For the characteristic function ${\chi }_{\mathrm{FOV}}$, we consider the representation

$${\chi }_{\mathrm{FOV}}\left(\mathsf{\Theta }\right)=\left\{\begin{array}{c}{\displaystyle \frac{1}{{\mathsf{\Omega }}_{\mathrm{FOV}}}},0\le \mathsf{\Theta }\le {\mathsf{\Theta }}_{\mathrm{FOV}}\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right.,$$

(14)

where ${\mathsf{\Theta }}_{\mathrm{FOV}}$ and ${\mathsf{\Omega }}_{\mathrm{FOV}}\approx \pi {\mathsf{\Theta }}_{\mathrm{FOV}}^2$ are the collecting polar and solid angles of the detector, respectively, and $cos\mathsf{\Theta }={\mathsf{\Omega }}_{\mathrm{m}}·{\mathsf{\Omega }}_{\mathrm{m}}^0$. Let $S_{\mathrm{tm}}=S_{\mathrm{tm}}({\mathbf{r}}_{\mathrm{D}},{\mathsf{\Omega }}_{\mathrm{m}}^0)$ be the footprint of the detector on the top face $S_{\mathrm{t}}$, i.e., an ellipse centered at ${\mathbf{r}}_{\mathrm{t}}^0=(x_{\mathrm{t}}^0,y_{\mathrm{t}}^0,L_{\mathrm{z}})$ with the semi-major and semi-minor axes $b=R_{\mathrm{D}}{\mathsf{\Theta }}_{\mathrm{FOV}}/{\mu }_{\mathrm{m}}^0$ and $a=R_{\mathrm{D}}{\mathsf{\Theta }}_{\mathrm{FOV}}$, respectively, where $R_{\mathrm{D}}=|{\mathbf{r}}_{\mathrm{D}}-{\mathbf{r}}_{\mathrm{t}}^0|$. Using the relation $\mathrm{d}{\mathsf{\Omega }}_{\mathrm{m}}=({\mu }_{\mathrm{m}}^0/R_{\mathrm{D}}^2)\mathrm{d}S$, where $\mathrm{d}S$ is the surface element on the top face $S_{\mathrm{t}}$, we express the measured signal as

$$\mathcal{I}={\int }_{S_{\mathrm{t}}}h\left({\mathbf{r}}_{\mathrm{t}}\right)I({\mathbf{r}}_{\mathrm{t}},{\mathsf{\Omega }}_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right))\mathrm{d}{S}_{\mathrm{t}},$$

(15)

where

$${\mathsf{\Omega }}_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)=\frac{{\mathbf{r}}_{\mathrm{D}}-{\mathbf{r}}_{\mathrm{t}}}{|{\mathbf{r}}_{\mathrm{D}}-{\mathbf{r}}_{\mathrm{t}}|}=({\mu }_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right),{\phi }_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)),$$

(16)

is the unit vector along the line connecting the points ${\mathbf{r}}_{\mathrm{t}}\in S_{\mathrm{tm}}$ and ${\mathbf{r}}_{\mathrm{D}}$,

$$h\left({\mathbf{r}}_{\mathrm{t}}\right)=\left\{\begin{array}{c}{\displaystyle \frac{1}{A_{\mathrm{tm}}}},{\mathbf{r}}_{\mathrm{t}}\in S_{\mathrm{tm}}\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right.$$

(17)

is the characteristic function associated to the detector footprint, and $A_{\mathrm{tm}}=\pi ab=R_{\mathrm{D}}^2{\mathsf{\Omega }}_{\mathrm{FOV}}/{\mu }_{\mathrm{m}}^0$ is the area of the ellipse.

The radiative transfer in an inhomogeneous three-dimensional medium is usually solved by a numerical method. Considering a spatial discretization of the domain of analysis, the extinction coefficient ${\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)$, the single-scattering albedo $\omega (\mathbf{r}$), the Legendre expansion coefficients of the phase function ${\chi }_n\left(\mathbf{r}\right)$ (Appendix A), are specified at every grid point of the so-called base grid and trilinearly interpolated within each cell. Furthermore, the bidirectional surface reflection function $\rho ({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+},{\mathsf{\Omega }}^{-})$ is specified at every grid point on the bottom surface and in all upward and downward discrete ordinate directions.

3. Linearized SHDOM

In atmospheric remote sensing we are interested in the retrieval of say, $N_{\mathrm{p}}$ atmospheric parameters ${\xi }_p$, $p=1,\dots ,N_{\mathrm{p}}$, which determine the fields ${\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)$, $\omega \left(\mathbf{r}\right)$, ${\chi }_n\left(\mathbf{r}\right)$, and $\rho ({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+},{\mathsf{\Omega }}^{-})$. The retrieval requires the knowledge of the weighting functions, i.e., the partial derivatives of the measured radiance with respect to the atmospheric parameters being retrieved, i.e., $\partial \mathcal{I}/\partial {\xi }_p$.

3.1. Linearized Forward Approach

The linearized forward approach relies on the analytical differentiation technique and is based on the fact that the radiance can be regarded as the composition of differentiable functions. By applying the chain rule repeatedly to these functions, derivatives of arbitrary order can be computed analytically. After choosing the independent variables with respect to which the derivatives of the radiance are required, the derivatives of each function are computed recursively, i.e., by repeatedly substituting the derivatives of the inner function in the chain rule.

The pertinent equations which are differentiated are given in Appendix A. Essentially, the analytical differentiation technique is applied to all steps of the algorithm including the transformation of the source function from spherical harmonics to discrete ordinates, the integration of the source function along discrete ordinate directions, the transformation of the discrete ordinate radiances to spherical harmonics, the calculation of the source function from the radiance in spherical harmonics space, the adaptive grid procedure, and the acceleration step. The benefit of this method is that no assumptions rather than those of the forward model have to be made. The disadvantage is that not only the radiance and the source function have to be stored at all grid points, but also their derivatives with respect to the atmospheric parameters of interest. As a result, the amount of required memory may exceed the size of RAM (several Gbytes).

3.2. Linearized Forward-Adjoint Approach

In view of Equation (15), the measured signal can be written as

$$\mathcal{I}=〈Q^{†},I〉,$$

(18)

where

$$\begin{array}{cc}\hfill Q^{†}(\mathbf{r},\mathsf{\Omega })& =F_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)\delta (z-L_{\mathrm{z}})\delta (\mathsf{\Omega }-{\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right)),\hfill \end{array}$$

(19)

with

$$F_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)=h\left({\mathbf{r}}_{\mathrm{t}}\right),$$

(20)

is the adjoint source function. Here,

$${\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right)=\frac{{\mathbf{r}}_{\mathrm{D}}-\mathbf{r}}{|{\mathbf{r}}_{\mathrm{D}}-\mathbf{r}|}=({\mu }_{\mathrm{m}}\left(\mathbf{r}\right),{\phi }_{\mathrm{m}}\left(\mathbf{r}\right))$$

(21)

is the unit vector along the line joining the points $\mathbf{r}$ and ${\mathbf{r}}_{\mathrm{D}}$, and ${\mathbf{r}}_{\mathrm{t}}=\mathbf{r}+s{\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right)\in S_{\mathrm{tm}}$ with $s=(L_{\mathrm{z}}-z)/{\mu }_{\mathrm{m}}\left(\mathbf{r}\right)$ and $\mathbf{r}=(x,y,z)$, is the intersection point of this line with the top face $S_{\mathrm{t}}$ (Figure 1). Obviously, because the points $\mathbf{r}$, ${\mathbf{r}}_{\mathrm{t}}$, and ${\mathbf{r}}_{\mathrm{D}}$ are on the same line, we have ${\mathsf{\Omega }}_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)={\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right)$, and so, ${\mu }_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)={\mu }_{\mathrm{m}}\left(\mathbf{r}\right)$.

The main result of the adjoint radiative transfer theory states that if (i) I solves the forward problem consisting in the operator equation $\mathcal{L}I=Q$ and the boundary conditions (4) and (8), and (ii) $I^{†}$ solves the adjoint problem consisting in the operator equation ${\mathcal{L}}^{†}{I}^{†}=Q^{†}$ and the boundary conditions (4) and (11), then the measured signal $\mathcal{I}$ can be estimated either by the scalar product of the radiance field I (as determined by Q) and the adjoint source function $Q^{†}$, or by the scalar product of the adjoint radiance field $I^{†}$ (as determined by $Q^{†}$) and the forward source function Q, i.e.,

$$\mathcal{I}=〈Q^{†},I〉=〈{\mathcal{L}}^{†}{I}^{†},I〉=〈I^{†},\mathcal{L}I〉=〈I^{†},Q〉.$$

(22)

The partial derivative of the measured signal with respect to some atmospheric parameter ${\xi }_p$, can be computed as

$$\frac{\partial \mathcal{I}}{\partial {\xi }_p}=〈Q^{†},\frac{\partial I}{\partial {\xi }_p}〉=〈{\mathcal{L}}^{†}{I}^{†},\frac{\partial I}{\partial {\xi }_p}〉=〈I^{†},\mathcal{L}\frac{\partial I}{\partial {\xi }_p}〉,$$

(23)

where in the first step we assumed that $\partial Q^{†}/\partial {\xi }_p=0$, while in the last step we used the fact that $\partial I/\partial {\xi }_p$ satisfies the boundary conditions (4) and (8). Taking the partial derivative of the operator equation $\mathcal{L}I=Q$ with respect to ${\xi }_p$ and assuming $\partial Q/\partial {\xi }_p=0$ gives

$$\frac{\partial \mathcal{I}}{\partial {\xi }_p}=-〈I^{†},\frac{\partial \mathcal{L}}{\partial {\xi }_p}I〉.$$

(24)

It should be pointed out that in general, the linearized forward-adjoint approach can be used to compute the weighted integral of the partial derivative of the radiance field $\partial I/\partial {\xi }_p$ defined by

$$D_p=〈Q^{†},\frac{\partial I}{\partial {\xi }_p}〉.$$

(25)

In this case, employing the same arguments as in the derivation of Equations (23) and (24), but without assuming that $\partial Q^{†}/\partial {\xi }_p=0$, we get

$$D_p=-〈I^{†},\frac{\partial \mathcal{L}}{\partial {\xi }_p}I〉.$$

(26)

Thus, identifying the adjoint source function from Equation (25) and solving the corresponding adjoint radiative transfer equation ${\mathcal{L}}^{†}{I}^{†}=Q^{†}$, we can compute the weighted integral $D_p$ by means of Equation (26).

The forward and adjoint radiative transfer equations $\mathcal{L}I=Q$ and ${\mathcal{L}}^{†}{I}^{†}=Q^{†}$, respectively, are related to each other. Replacing $\mathsf{\Omega }$ by $-\mathsf{\Omega }$ in the expression of the adjoint operator ${\mathcal{L}}^{†}$, gives ${\mathcal{L}}^{†}(-\mathsf{\Omega })I^{†}(\mathbf{r},-\mathsf{\Omega })=Q^{†}(\mathbf{r},-\mathsf{\Omega })$. Defining the pseudo-forward field ${\widehat{I}}^{†}$ by the relation ${\widehat{I}}^{†}(\mathbf{r},\mathsf{\Omega })=I^{†}(\mathbf{r},-\mathsf{\Omega })$ and using the symmetry properties of the phase function $P(\mathbf{r},-\mathsf{\Omega },-{\mathsf{\Omega }}^{\prime })=P(\mathbf{r},{\mathsf{\Omega }}^{\prime },\mathsf{\Omega })$ and of the normalized reflection function $\rho ({\mathbf{r}}_{\mathrm{b}},-\mathsf{\Omega },-{\mathsf{\Omega }}^{\prime })=\rho ({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{\prime },\mathsf{\Omega })$, we find the identity ${\mathcal{L}}^{†}(-\mathsf{\Omega })I^{†}(\mathbf{r},-\mathsf{\Omega })=\mathcal{L}\left(\mathsf{\Omega }\right){\widehat{I}}^{†}(\mathbf{r},\mathsf{\Omega })$. Thus, the pseudo-forward field ${\widehat{I}}^{†}$ solves the same type of radiative transfer equation as the radiance field I, i.e., $\mathcal{L}{\widehat{I}}^{†}={\widehat{Q}}^{†}$, where the pseudo-forward source function, defined by ${\widehat{Q}}^{†}(\mathbf{r},\mathsf{\Omega })=Q^{†}(\mathbf{r},-\mathsf{\Omega })$, is

$$\begin{array}{cc}\hfill {\widehat{Q}}^{†}(\mathbf{r},\mathsf{\Omega })& ={\widehat{F}}_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)\delta (z-L_{\mathrm{z}})\delta (\mathsf{\Omega }-{\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right)),\hfill \end{array}$$

(27)

with ${\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right)=-{\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right)$, ${\widehat{\mu }}_{\mathrm{m}}\left(\mathbf{r}\right)=-{\mu }_{\mathrm{m}}\left(\mathbf{r}\right)$, and

$${\widehat{F}}_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)=F_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)=h\left({\mathbf{r}}_{\mathrm{t}}\right).$$

(28)

In other words, ${\widehat{I}}^{†}(\mathbf{r},\mathsf{\Omega })$ solves the inhomogeneous differential equation

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{\widehat{I}}^{†}}{\mathrm{d}s}(\mathbf{r},\mathsf{\Omega })& =-{\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right){\widehat{I}}^{†}(\mathbf{r},\mathsf{\Omega })+\frac{{\sigma }_{\mathrm{sct}}\left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime }){\widehat{I}}^{†}(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }\hfill \end{array}$$

(29)

with the top-of-atmosphere boundary condition

$${\widehat{I}}^{†}({\mathbf{r}}_{\mathrm{t}},{\mathsf{\Omega }}^{-})=\frac{{\widehat{F}}_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)}{|{\widehat{\mu }}_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)|}\delta ({\mathsf{\Omega }}^{-}-{\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)),{\mathbf{r}}_{\mathrm{t}}\in S_{\mathrm{t}},$$

(30)

and the surface boundary condition

$$\begin{array}{cc}\hfill {\widehat{I}}^{†}({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+})& =\frac{1}{\pi }{\int }_{{\mathsf{\Omega }}^{-}}\rho ({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{+},{\mathsf{\Omega }}^{-})\left|{\mu }^{-}\right|{\widehat{I}}^{†}({\mathbf{r}}_{\mathrm{b}},{\mathsf{\Omega }}^{-})\mathrm{d}{\mathsf{\Omega }}^{-},{\mathbf{r}}_{\mathrm{b}}\in S_{\mathrm{b}}.\hfill \end{array}$$

(31)

We are now concern with the computation of the partial derivative $\partial \mathcal{I}/\partial {\xi }_p$ by means of Equation (24). Taking the variation of the forward transport operator $\mathcal{L}$ under the assumption that the phase function does not depend on the atmospheric parameter ${\xi }_p$, yields (cf. Equation (5))

$$\begin{array}{cc}\hfill \left(\delta \mathcal{L}I\right)(\mathbf{r},\mathsf{\Omega })& =\delta {\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)I(\mathbf{r},\mathsf{\Omega })\hfill \\ & -\delta {\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)\frac{\omega \left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }\hfill \\ & -{\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)\frac{\delta \omega \left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }\hfill \\ & -\frac{1}{\pi }\delta \left(z\right)H\left(\mu \right)\mu {\int }_{\mathsf{\Omega }}\delta \rho (\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })H(-{\mu }^{\prime })\left|{\mu }^{\prime }\right|I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }.\hfill \end{array}$$

(32)

In this section we describe the computation of the partial derivative of the measured signal with respect to a parameter that determine the extinction field; the partial derivatives with respect to parameters that specify the single-scattering albedo and the bidirectional reflection function are given in Appendix B.

To compute the partial derivatives of the forward operator we need to find appropriate interpolation schemes for the extinction coefficient, source function, and radiance field within a grid cell. A major problem arises from the fact that SHDOM uses the integral form of the radiative transfer equation for radiance calculation, and in particular, the assumption that the extinction/source function product varies linearly along the characteristic. Unfortunately, when dealing with the differential form of the radiative transfer equation, this assumption can not be met; instead, we may assume that the extinction/source function product varies linearly within a cell. Thus, different interpolation schemes are used for radiance and derivative calculations.

In the following, we assume that the partial derivatives $\partial {\sigma }_{\mathrm{ext}}\left({\mathbf{r}}_i\right)/\partial {\xi }_p$ at all grid points ${\mathbf{r}}_i$ are known. Let c be the global index of a cell containing a grid point with global index j, and let $\overline{j}$ be the local index of this point within cell c (Figure 2); we define the map g as $(\overline{j},c)\stackrel{g}{⟶}j$, or equivalently, $j=g(\overline{j},c)$. For $\mathbf{r}\in D_c$, where $D_c$ is the domain of cell c, we assume that the extinction and extinction/source function product vary linearly within the cell, i.e.

$${\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)=\sum _{\overline{i}}{L}_{\overline{i}}\left(\mathbf{R}\right){\sigma }_{\mathrm{ext}}\left({\mathbf{r}}_{g(\overline{i},c)}\right)$$

(33)

and

$${\sigma }_{\mathrm{ext}}\left(\mathbf{r}\right)J(\mathbf{r},\mathsf{\Omega })=\sum _{\overline{i}}{L}_{\overline{i}}\left(\mathbf{R}\right){\sigma }_{\mathrm{ext}}\left({\mathbf{r}}_{g(\overline{i},c)}\right)J({\mathbf{r}}_{g(\overline{i},c)},\mathsf{\Omega }),$$

(34)

respectively. Here,

$$J(\mathbf{r},\mathsf{\Omega })=\frac{\omega \left(\mathbf{r}\right)}{4\pi }{\int }_{\mathsf{\Omega }}P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })I(\mathbf{r},{\mathsf{\Omega }}^{\prime })\mathrm{d}{\mathsf{\Omega }}^{\prime }$$

(35)

is the source function, $L_{\overline{i}}$ are the first-order interpolation basis functions for a rectangular cuboid element (Appendix C), $\mathbf{R}=\mathbf{r}-{\rho }_c$ is the position vector of a point in a local coordinate system attached to cell c, ${\rho }_c=(1/8){\sum }_{\overline{i}}{\mathbf{r}}_{g(\overline{i},c)}$ is the position vector of the center point of cell c, ${\mathbf{r}}_{g(\overline{i},c)}={\mathbf{r}}_i$ is the position vector of the grid point with global index $i=g(\overline{i},c)$, and the sum ${\sum }_{\overline{i}}$ is taken over all grid points of a cell. Setting ${\sigma }_{\mathrm{ext}g(\overline{i},c)}={\sigma }_{\mathrm{ext}}\left({\mathbf{r}}_{g(\overline{i},c)}\right)$, we obtain

$$\left(\frac{\partial \mathcal{L}}{\partial {\xi }_p}I\right)(\mathbf{r},\mathsf{\Omega })=\sum _{\overline{i}}{L}_{\overline{i}}\left(\mathbf{R}\right)[I(\mathbf{r},\mathsf{\Omega })-J({\mathbf{r}}_{g(\overline{i},c)},\mathsf{\Omega })]\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p},$$

(36)

implying

$$\frac{\partial \mathcal{I}}{\partial {\xi }_p}=-\sum _c\sum _{\overline{i}}{\int }_{\mathsf{\Omega }}{\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right){\widehat{I}}^{†}(\mathbf{r},-\mathsf{\Omega })[I(\mathbf{r},\mathsf{\Omega })-J({\mathbf{r}}_{g(\overline{i},c)},\mathsf{\Omega })]\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}\mathrm{d}V\mathrm{d}\mathsf{\Omega }.$$

(37)

In the next step, we split the total radiance $I(\mathbf{r},\mathsf{\Omega })$ into the diffuse and direct radiances, i.e.,

$$\begin{array}{cc}\hfill I(\mathbf{r},\mathsf{\Omega })& =I_{\mathrm{d}}(\mathbf{r},\mathsf{\Omega })+\delta (\mathsf{\Omega }-{\mathsf{\Omega }}_0)T_0\left(\mathbf{r}\right),\hfill \end{array}$$

(38)

where

$$T_0\left(\mathbf{r}\right)=\frac{F_0}{|{\mu }_0|}{\mathrm{e}}^{-{\tau }_{\mathrm{ext}}(\mathbf{r},{\mathbf{r}}_0|{\mathsf{\Omega }}_0)}$$

(39)

is the transmission along the solar direction ${\mathsf{\Omega }}_0$ from the starting point ${\mathbf{r}}_0=\mathbf{r}-s{\mathsf{\Omega }}_0\in S_{\mathrm{t}}$ to the end point $\mathbf{r}$ (the forward direct beam), and

$${\tau }_{\mathrm{ext}}(\mathbf{r},{\mathbf{r}}_0|{\mathsf{\Omega }}_0)={\int }_0^s{\sigma }_{\mathrm{ext}}({\mathbf{r}}_0+s^{\prime }{\mathsf{\Omega }}_0)\mathrm{d}{s}^{\prime }$$

(40)

is the optical depth. Similarly, we split the pseudo-forward radiance ${\widehat{I}}^{†}(\mathbf{r},\mathsf{\Omega })$ into its diffuse and direct components, i.e.,

$$\begin{array}{cc}\hfill {\widehat{I}}^{†}(\mathbf{r},\mathsf{\Omega })& ={\widehat{I}}_{\mathrm{d}}^{†}(\mathbf{r},\mathsf{\Omega })+\delta (\mathsf{\Omega }-{\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right)){\widehat{T}}_{\mathrm{m}}^{†}\left(\mathbf{r}\right),\hfill \end{array}$$

(41)

where

$${\widehat{T}}_{\mathrm{m}}^{†}\left(\mathbf{r}\right)=\frac{{\widehat{F}}_0^{†}\left({\mathbf{r}}_{\mathrm{t}}\right)}{|{\widehat{\mu }}_{\mathrm{m}}\left({\mathbf{r}}_{\mathrm{t}}\right)|}{\mathrm{e}}^{-{\tau }_{\mathrm{ext}}(\mathbf{r},{\mathbf{r}}_{\mathrm{t}}|{\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right))},$$

(42)

is the transmission along the detector direction ${\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right)$ from the starting point ${\mathbf{r}}_{\mathrm{t}}=\mathbf{r}-s{\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right)\in S_{\mathrm{tm}}$ to the end point $\mathbf{r}$ (the pseudo-forward direct beam). Approximating

$${\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right)=-{\widehat{\mathsf{\Omega }}}_{\mathrm{m}}\left(\mathbf{r}\right)\approx {\mathsf{\Omega }}_{\mathrm{m}}\left({\rho }_{\mathrm{c}}\right):={\mathsf{\Omega }}_{\mathrm{m}c},\mathbf{r}\in D_c,$$

(43)

we are led to the following representation for $\partial \mathcal{I}/\partial {\xi }_p$ in terms of diffuse radiances:

$$\frac{\partial \mathcal{I}}{\partial {\xi }_p}=T_A+T_B+T_C+T_D+T_E,$$

(44)

where

$$\begin{array}{cc}\hfill T_A& =-\sum _c\sum _{\overline{i}}{\int }_{\mathsf{\Omega }}{\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right){\widehat{I}}_{\mathrm{d}}^{†}(\mathbf{r},-\mathsf{\Omega })I_{\mathrm{d}}(\mathbf{r},\mathsf{\Omega })\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}\mathrm{d}V\mathrm{d}\mathsf{\Omega },\hfill \end{array}$$

(45)

$$\begin{array}{cc}\hfill T_B& =\sum _c\sum _{\overline{i}}{\int }_{\mathsf{\Omega }}{\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right){\widehat{I}}_{\mathrm{d}}^{†}(\mathbf{r},-\mathsf{\Omega })J({\mathbf{r}}_{g(\overline{i},c)},\mathsf{\Omega })\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}\mathrm{d}V\mathrm{d}\mathsf{\Omega },\hfill \end{array}$$

(46)

and

$$\begin{array}{cc}\hfill T_C& =-\sum _c\sum _{\overline{i}}{\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right){\widehat{I}}_{\mathrm{d}}^{†}(\mathbf{r},-{\mathsf{\Omega }}_0)T_0\left(\mathbf{r}\right)\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}\mathrm{d}V,\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill T_D& =-\sum _c\sum _{\overline{i}}{\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right){\widehat{T}}_{\mathrm{m}}^{†}\left(\mathbf{r}\right)I_{\mathrm{d}}(\mathbf{r},{\mathsf{\Omega }}_{\mathtt{m}c})\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}\mathrm{d}V,\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill T_E& =\sum _c\sum _{\overline{i}}{\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right){\widehat{T}}_{\mathrm{m}}^{†}\left(\mathbf{r}\right)J({\mathbf{r}}_{g(\overline{i},c)},{\mathsf{\Omega }}_{\mathtt{m}c})\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}\mathrm{d}V.\hfill \end{array}$$

(49)

The above integrals are computed in the spherical harmonics space by assuming the expansions

$$\begin{array}{cc}\hfill I_{\mathrm{d}}(\mathbf{r},\mathsf{\Omega })& =\sum _{mn}{I}_{mn}\left(\mathbf{r}\right)Y_{mn}\left(\mathsf{\Omega }\right),\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill {\widehat{I}}_{\mathrm{d}}^{†}(\mathbf{r},\mathsf{\Omega })& =\sum _{mn}{\widehat{I}}_{mn}^{†}\left(\mathbf{r}\right)Y_{mn}\left(\mathsf{\Omega }\right),\hfill \end{array}$$

(51)

$$\begin{array}{cc}\hfill J(\mathbf{r},\mathsf{\Omega })& =\sum _{mn}{J}_{mn}\left(\mathbf{r}\right)Y_{mn}\left(\mathsf{\Omega }\right),\hfill \end{array}$$

(52)

where $Y_{mn}\left(\mathsf{\Omega }\right)$ are the orthonormal real-valued spherical harmonic functions (Appendix A), and the sum ${\sum }_{mn}$ should be understood as

$$\sum _{mn}=\sum _{m=-M}^M\sum _{n=\left|m\right|}^N,$$

with N and M being the maximum expansion order and number of azimuthal modes, respectively. The linear approximation

$$f\left(\mathbf{r}\right)=\sum _{\overline{j}}{L}_{\overline{j}}\left(\mathbf{R}\right)f\left({\mathbf{r}}_{g(\overline{j},c)}\right)=\sum _{\overline{j}}{L}_{\overline{j}}\left(\mathbf{R}\right)f_{g(\overline{j},c)},$$

(53)

where $f\left(\mathbf{r}\right)$ stands for $I_{mn}\left(\mathbf{r}\right)$, ${\widehat{I}}_{mn}^{†}\left(\mathbf{r}\right)$, $T_0\left(\mathbf{r}\right)$, and ${\widehat{T}}_{\mathrm{m}}^{†}\left(\mathbf{r}\right)$, together with the identity $Y_{mn}(-\mathsf{\Omega })={(-1)}^n{Y}_{mn}\left(\mathsf{\Omega }\right)$, then yield

$$\begin{array}{cc}\hfill T_A& =-\sum _c\sum _{mn}{(-1)}^n\sum _{\overline{i},\overline{j},\overline{k}}{\widehat{I}}_{mn,g(\overline{j},c)}^{†}{I}_{mn,g(\overline{k},c)}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j},\overline{k}},\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill T_B& =\sum _c\sum _{mn}{(-1)}^n\sum _{\overline{i},\overline{j}}{\widehat{I}}_{mn,g(\overline{j},c)}^{†}{J}_{mn,g(\overline{i},c)}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j}},\hfill \end{array}$$

(55)

and

$$\begin{array}{cc}\hfill T_C& =-\sum _c\sum _{\overline{i},\overline{j},\overline{k}}{\widehat{I}}_{\mathrm{d}}^{†}({\mathbf{r}}_{g(\overline{j},c)},-{\mathsf{\Omega }}_0)T_{0g(\overline{k},c)}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j},\overline{k}},\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill T_D& =-\sum _c\sum _{\overline{i},\overline{j},\overline{k}}{I}_{\mathrm{d}}({\mathbf{r}}_{g(\overline{j},c)},{\mathsf{\Omega }}_{\mathrm{m}c}){\widehat{T}}_{\mathrm{m}g(\overline{k},c)}^{†}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j},\overline{k}},\hfill \end{array}$$

(57)

$$\begin{array}{cc}\hfill T_E& =\sum _c\sum _{mn}{Y}_{mn}\left({\mathsf{\Omega }}_{\mathrm{m}c}\right)\sum _{\overline{i},\overline{j}}{\widehat{T}}_{\mathrm{m}g(\overline{j},c)}^{†}{J}_{mn,g(\overline{i},c)}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j}}.\hfill \end{array}$$

(58)

Here, the interpolations coefficients $L_{\overline{i},\overline{j},\overline{k}}$ and $L_{\overline{i},\overline{j}}$ are given, respectively, by

$$\begin{array}{cc}\hfill L_{\overline{i},\overline{j},\overline{k}}& ={\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right)L_{\overline{j}}\left(\mathbf{R}\right)L_{\overline{k}}\left(\mathbf{R}\right)\mathrm{d}V,\hfill \end{array}$$

(59)

$$\begin{array}{cc}\hfill L_{\overline{i},\overline{j}}& ={\int }_{D_c}{L}_{\overline{i}}\left(\mathbf{R}\right)L_{\overline{j}}\left(\mathbf{R}\right)\mathrm{d}V,\hfill \end{array}$$

(60)

while for $i=g(\overline{i},c)$ and ${\omega }_i=\omega \left({\mathbf{r}}_i\right)$, the expansion coefficients of the source function in Equations (55) and (58) are computed as

$$J_{mn,i}={\omega }_i\frac{{\chi }_{n,i}}{2n+1}{I}_{mn,i}+{\omega }_i\frac{{\chi }_{n,i}}{2n+1}{Y}_{mn}\left({\mathsf{\Omega }}_0\right)T_{0i},$$

(61)

where ${\chi }_{n,i}={\chi }_n\left({\mathbf{r}}_i\right)$ are the Legendre expansion coefficients of the phase function (Appendix A), i.e.,

$$P(\mathbf{r},\mathsf{\Omega },{\mathsf{\Omega }}^{\prime })=4\pi \sum _{mn}\frac{{\chi }_n\left(\mathbf{r}\right)}{2n+1}{Y}_{mn}\left(\mathsf{\Omega }\right)Y_{mn}\left({\mathsf{\Omega }}^{\prime }\right).$$

(62)

Further comments and algorithm details are given below:

1.

The input parameters of the numerical algorithm are the partial derivatives $\partial {\sigma }_{\mathrm{ext}}\left({\mathbf{r}}_i\right)/\partial {\xi }_p$ and $\partial \omega \left({\mathbf{r}}_i\right)/\partial {\xi }_p$ at all grid points ${\mathbf{r}}_i$, and eventually, the partial derivatives $\partial \rho ({\mathbf{r}}_{\mathrm{b}i},{\mathsf{\Omega }}_{ij}^{+},{\mathsf{\Omega }}_{pq}^{-})/\partial {\xi }_p$ at all grid points on the bottom surface ${\mathbf{r}}_{\mathrm{b}i}$ and in all upward and downward discrete ordinate directions ${\mathsf{\Omega }}_{ij}^{+}$ and ${\mathsf{\Omega }}_{pq}^{-}$, respectively.

2.

The interpolation coefficients $L_{\overline{i},\overline{j},\overline{k}}$ and $L_{\overline{i},\overline{j}}$ are computed analytically in a local coordinate system attached to the grid cell.

3.

The radiances ${\widehat{I}}_{\mathrm{d}}^{†}({\mathbf{r}}_{g(\overline{j},c)},-{\mathsf{\Omega }}_0)$ and $I_{\mathrm{d}}({\mathbf{r}}_{g(\overline{j},c)},{\mathsf{\Omega }}_{\mathrm{m}c})$ in the expressions of $T_C$ and $T_D$, respectively, are computed by the source integration method.

4.

For solar problems with the delta-M method [24], the radiative transfer equation is expressed in terms of the scaled quantities

$${\overline{\sigma }}_{\mathrm{ext}}=(1-f\omega ){\sigma }_{\mathrm{ext}},\overline{\omega }=\frac{1-f}{1-f\omega }\omega ,$$

(63)

where f is the truncation fraction. At each grid point, we switch from the partial derivatives $\partial {\sigma }_{\mathrm{ext}}/\partial {\xi }_p$ and $\partial \omega /\partial {\xi }_p$ to $\partial {\overline{\sigma }}_{\mathrm{ext}}/\partial {\xi }_p$ and $\partial \overline{\omega }/\partial {\xi }_p$, respectively, whereby the latter two are computed from Equation (63) by using the chain rule.

5.

The partial derivatives of the signal correction in the TMS method of Nakajima and Tanaka [25] are computed analytically (Appendix A).

6.

During the adaptive grid procedure, a base grid cell, also called a “parent cell”, is split into “child cells” to achieve higher spatial resolution. In this regard, the sums in Equations (54)–(58) are taken over all child cells.

7.

According to the main result of the adjoint radiative transfer theory, the measured signal can be computed by using Equation (15) or (22); in the latter case, the computational formula is

$$\begin{array}{cc}\hfill \mathcal{I}& =F_0{\int }_{S_{\mathrm{t}}}{\widehat{I}}_{\mathrm{d}}^{†}(x_{\mathrm{t}},y_{\mathrm{t}},L_{\mathrm{z}},-{\mathsf{\Omega }}_0)\mathrm{d}{S}_{\mathrm{t}}.\hfill \end{array}$$

(64)

8.

The code uses two alternative interpolation schemes for the source function: a linear variation of the extinction/source function product within a cell, i.e., Equation (34), and a linear variation of the source function within a cell, i.e.,

$$\begin{array}{cc}\hfill J(\mathbf{r},\mathsf{\Omega })& =\sum _{\overline{i}}{L}_{\overline{i}}\left(\mathbf{R}\right)J({\mathbf{r}}_{g(\overline{i},c)},\mathsf{\Omega }),\hfill \end{array}$$

(65)

for $\mathbf{r}\in D_c$. In the second case, the terms $T_B$ and $T_E$ are computed, respectively, as

$$\begin{array}{cc}\hfill T_B& =\sum _c\sum _{mn}{(-1)}^n\sum _{\overline{i},\overline{j},\overline{k}}{\widehat{I}}_{mn,g(\overline{j},c)}^{†}{J}_{mn,g(\overline{k},c)}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j},\overline{k}},\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill T_E& =\sum _c\sum _{mn}{Y}_{mn}\left({\mathsf{\Omega }}_{\mathrm{m}c}\right)\sum _{\overline{i},\overline{j},\overline{k}}{\widehat{T}}_{\mathrm{m}g(\overline{j},c)}^{†}{J}_{mn,g(\overline{k},c)}\frac{\partial {\sigma }_{\mathrm{ext}g(\overline{i},c)}}{\partial {\xi }_p}{L}_{\overline{i},\overline{j},\overline{k}}.\hfill \end{array}$$

(67)

9.

In the case of satellite remote sensing we may assume that (i) the distance from the top of the atmosphere to the detector $R_{\mathrm{D}}$ is sufficiently large, so that we can approximate $I(\mathbf{r},{\mathsf{\Omega }}_{\mathrm{m}}\left(\mathbf{r}\right))\approx I(\mathbf{r},{\mathsf{\Omega }}_{\mathrm{m}}^0)$, and (ii) the footprint of the detector is a rectangle of lengths $2a$ and $2b$ centered at ${\mathbf{r}}_{\mathrm{t}}^0$. The second assumption is made because the domain is discretized in rectangular cuboid elements. In this context, the measured signal is computed as

$$\mathcal{I}=\frac{1}{4ab}{\int }_{S_{\mathrm{t}}}\overline{h}(x_{\mathrm{t}},y_{\mathrm{t}})I(x_{\mathrm{t}},y_{\mathrm{t}},L_{\mathrm{z}},{\mathsf{\Omega }}_{\mathrm{m}}^0)\mathrm{d}{S}_{\mathrm{t}},$$

(68)

where the normalized characteristic function $\overline{h}(x_{\mathrm{t}},y_{\mathrm{t}})$ takes the value 1 inside the footprint of the detector and 0 otherwise.

10.

The solution of the adjoint problem is a challenging task due to the spatial discontinuity of the pseudo-forward direct beam. In SHDOM, this type of problem is handled by the adaptive grid procedure. Essentially, the adaptive grid supplies extra spatial resolution along the boundaries of the pseudo-forward direct beam, so that there are not large discontinuities in the grid. In order to reduce the number of adaptive grid cells, the step characteristic function of the detector can be replaced by a smooth function, e.g., trapezoidal, Gaussian, cosine [23]. In the first case, the normalized characteristic function reads as

$$\overline{h}(x_{\mathrm{t}},y_{\mathrm{t}})=\left\{\begin{array}{c}{\overline{h}}_0(x_{\mathrm{t}},\Delta h_{\mathrm{x}}){\overline{h}}_0(y_{\mathrm{t}},\Delta h_{\mathrm{y}}),(x_{\mathrm{t}},y_{\mathrm{t}})\in S_{\mathrm{tm}}\hfill \\ 0,\mathrm{otherwise}\hfill \end{array}\right.,$$

(69)

where, for example, ${\overline{h}}_0(x_{\mathrm{t}},\Delta h_{\mathrm{x}})$ is given by

$${\overline{h}}_0(x_{\mathrm{t}},\Delta h_{\mathrm{x}})=c_{\mathrm{x}}\times \left\{\begin{array}{c}{\displaystyle \frac{x_{\mathrm{t}}-x_{\mathrm{t}min}^0}{\Delta h_{\mathrm{x}}}},x_{\mathrm{t}min}^0\le x_{\mathrm{t}}<x_{\mathrm{t}min}^0+\Delta h_{\mathrm{x}}\hfill \\ 1,x_{\mathrm{t}min}^0+\Delta h_{\mathrm{x}}\le x_{\mathrm{t}}\le x_{\mathrm{t}max}^0-\Delta h_{\mathrm{x}}\hfill \\ {\displaystyle \frac{x_{\mathrm{t}max}^0-x_{\mathrm{t}}}{\Delta h_{\mathrm{x}}}},x_{\mathrm{t}max}^0-\Delta h_{\mathrm{x}}<x_{\mathrm{t}}\le x_{\mathrm{t}max}^0\hfill \end{array}\right.$$

(70)

with $c_{\mathrm{x}}=2a/(2a-\Delta h_{\mathrm{x}})$, $x_{\mathrm{t}min}^0=x_{\mathrm{t}}^0-a$, and $x_{\mathrm{t}max}^0=x_{\mathrm{t}}^0+a$.

11.

In the linearized forward-adjoint approach, the forward and adjoint problems are solved successively on the same grid; in this way, the interpolation between different grids is avoided. Actually, in the first step, the adjoint problem is solved by using the adaptive grid procedure with a prescribed splitting accuracy ${\epsilon }_{\mathrm{split}}$ (Appendix A), and in the second step, the forward problem is solved on the resulting grid without splitting. In the linearized forward approach, the forward problem is solved first by using the adaptive grid procedure, and then, the partial derivatives are computed on the same grid; in this way, a less amount of memory for storing the derivatives of the radiance and the source function with respect to the atmospheric parameters of interest is required.

4. Numerical Simulations

In this section we analyze the accuracy and efficiency of the linearized versions of SHDOM.

4.1. Example 1

We consider a base grid with $N_{\mathrm{x}}=N_{\mathrm{y}}=20$ and $N_{\mathrm{z}}=11$ points, and the discretization steps $\Delta x=\Delta y=\Delta z=0.1$ km. For periodic boundary conditions, when the vertical boundaries in x and y have additional planes of grid points that duplicate the grid points in the planes $x=0$ and $y=0$, respectively, the domain of analysis has the sizes $L_{\mathrm{x}}=N_{\mathrm{x}}\Delta x=2$ km, $L_{\mathrm{y}}=N_{\mathrm{y}}\Delta y=2$ km, and $L_{\mathrm{z}}=(N_{\mathrm{z}}-1)\Delta z=1$ km. The extinction field is Gaussian, i.e.,

$${\sigma }_{\mathrm{ext}}(x,y,z)={\sigma }_{\mathrm{ext}}^{\max }f(x,y,z),$$

(71)

$$f(x,y,z)=f_{\mathrm{G}}(x,x_0,s_{\mathrm{x}})f_{\mathrm{G}}(y,y_0,s_{\mathrm{y}})f_{\mathrm{G}}(z,z_0,s_{\mathrm{z}}),$$

(72)

$$f_{\mathrm{G}}(u,u_0,s_{\mathrm{u}})=exp\left[-\frac{{(u-u_0)}^2}{2s_{\mathrm{u}}^2}\right],u=x,y,z,$$

(73)

with $x_0=L_{\mathrm{x}}/2$, $y_0=L_{\mathrm{y}}/2$, $z_0=L_{\mathrm{z}}/2$, $s_{\mathrm{x}}=L_{\mathrm{x}}/4$, $s_{\mathrm{y}}=L_{\mathrm{y}}/4$, and $s_{\mathrm{z}}=L_{\mathrm{z}}/4$. The parameters ${\sigma }_{\mathrm{ext}}^{\max }$ is 6 ${\mathrm{km}}^{-1}$, in which case, the peak optical depth is $3.6$. The extinction field in the plane $y=L_{\mathrm{y}}/2$ is illustrated in Figure 3. The number of discrete zenith and azimuthal angles are $N_{\mu }=16$ and $N_{\phi }=2N_{\mu }$, respectively, the single-scattering albedo is $\omega =0.9$, and a Henyey–Greenstein phase function [26] with the asymmetry parameter $g=0.8$ is considered. In all our simulations, the solar zenith angle is ${\theta }_0={120}^{\circ }$, and a Lambertian reflecting surface with the surface albedo $A=0.2$ is chosen. The coordinates of the center of the detector footprint are $x_{\mathrm{t}}^0=L_{\mathrm{x}}/2$, $y_{\mathrm{t}}^0=L_{\mathrm{y}}/2$, and $z_{\mathrm{t}}^0=L_{\mathrm{z}}$, the detector azimuthal angle is ${\phi }_{\mathrm{m}}^0=0^{\circ }$, the lengths of the rectangular footprint are $a=5\Delta x$, and $b=[a/(\Delta y{\mu }_{\mathrm{m}}^0)]\Delta y$, where $\left[x\right]$ is the integer part of x, and the lengths specifying the slope of the characteristic function in Equation (70) are $\Delta h_{\mathrm{x}}=0.5\Delta x$ and $\Delta h_{\mathrm{y}}=0.5\Delta y$.

In the following, we compute the partial derivatives of the measured signal with respect to the extinction coefficients ${\sigma }_{\mathrm{ext}p}$ at $N_{\mathrm{p}}=11$ base grid points ($x=x_{\mathrm{t}}^0$, $y=y_{\mathrm{t}}^0$, $z=\overline{p}\Delta z$), $\overline{p}=1,\dots ,N_{\mathrm{p}}$. Obviously, if p is the global index of the grid point with the local index $\overline{p}$ in the derivative list, we have

$$\frac{\partial {\sigma }_{\mathrm{ext}i}}{\partial {\sigma }_{\mathrm{ext}p}}={\delta }_{ip},\frac{\partial {\omega }_i}{\partial {\sigma }_{\mathrm{ext}p}}=0.$$

(74)

First, we compare the accuracies of the measured signal and of its derivatives. The accuracy of SHDOM radiances is determined by the angular and spatial resolution, i.e., the number of discrete ordinates in zenith angle $N_{\mu }$ and, for a specified base grid, the splitting accuracy ${\epsilon }_{\mathrm{split}}$. In Figure 4, we plot the partial derivatives computed with the linearized forward-adjoint approach and the linearized approach for different values of the splitting accuracy ${\epsilon }_{\mathrm{split}}$. Note that the influence of $N_{\mu }$ on the results accuracy is rather low. Taking the values corresponding to ${\epsilon }_{\mathrm{split}}^0=5\times {10}^{-4}$ as a reference, we illustrate in

Table 1. Relative errors ${\epsilon }_{\partial \mathcal{I}}$ and ${\epsilon }_{\mathcal{I}}$ for different values of the splitting accuracy ${\epsilon }_{\mathrm{split}}$. The results correspond to the linearized forward-adjoint approach (FAA) and the linearized forward approach (FA).

${\mathit{\epsilon }}_{\mathit{split}}$	FAA		FA
	${\mathit{\epsilon }}_{\partial \mathcal{I}}$	${\mathit{\epsilon }}_{\mathcal{I}}$	${\mathit{\epsilon }}_{\partial \mathcal{I}}$	${\mathit{\epsilon }}_{\mathcal{I}}$
$8\times {10}^{-4}$	$1.06\times {10}^{-2}$	$1.16\times {10}^{-4}$	$3.00\times {10}^{-2}$	$8.70\times {10}^{-5}$
$1\times {10}^{-3}$	$1.42\times {10}^{-2}$	$1.49\times {10}^{-4}$	$3.83\times {10}^{-2}$	$2.07\times {10}^{-4}$
$3\times {10}^{-3}$	$4.22\times {10}^{-2}$	$8.36\times {10}^{-4}$	$6.22\times {10}^{-2}$	$7.05\times {10}^{-4}$
$5\times {10}^{-3}$	$5.58\times {10}^{-2}$	$1.04\times {10}^{-3}$	$7.75\times {10}^{-2}$	$1.42\times {10}^{-3}$

, the relative error in the partial derivatives, expressed as an rms difference normalized by the mean of the quantity over the $N_{\mathrm{p}}$ comparison points, i.e.,

$${\epsilon }_{\partial \mathcal{I}}={\displaystyle \frac{\sqrt{{\sum }_{\overline{p}}{\left[{\left({\displaystyle \frac{\partial \mathcal{I}}{\partial {\sigma }_{\mathrm{ext}p}}}\right)}_{{\epsilon }_{\mathrm{split}}}-{\left({\displaystyle \frac{\partial \mathcal{I}}{\partial {\sigma }_{\mathrm{ext}p}}}\right)}_{{\epsilon }_{\mathrm{split}}^0}\right]}^2}}{\sqrt{{\sum }_{\overline{p}}{\left({\displaystyle \frac{\partial \mathcal{I}}{\partial {\sigma }_{\mathrm{ext}p}}}\right)}_{{\epsilon }_{\mathrm{split}}^0}^2}}},$$

(75)

and the relative error in the measured signal, i.e.,

$${\epsilon }_{\mathcal{I}}=|{\mathcal{I}}_{{\epsilon }_{\mathrm{split}}}-{\mathcal{I}}_{{\epsilon }_{\mathrm{split}}^0}|/{\mathcal{I}}_{{\epsilon }_{\mathrm{split}}^0}.$$

(76)

The results show that for the same ${\epsilon }_{\mathrm{split}}$,

1.

the accuracy of the measured signal is higher than that of its derivatives, and

2.

the accuracy of the linearized forward-adjoint approach is higher than that of the linearized forward approach.

However, in the next calculations, to avoid a large CPU time and computer memory, we take ${\epsilon }_{\mathrm{split}}={10}^{-3}$. For this splitting accuracy, we show in Figure 5, the adaptive grids in the plane $y=L_{\mathrm{y}}/2$. From these plots we infer that

1.

for the linearized forward-adjoint approach, the number of adaptive grid cells is higher in the domain “seen” by the detector along the direction ${\theta }_{\mathrm{m}}^0$ and in particular, along the boundaries of the pseudo-forward direct beam,

2.

for the linearized forward approach, the number of adaptive grid cells is higher in the entire domain along the solar direction ${\theta }_0={120}^{\circ }$ and when large discontinuities in the source function are present, and

3.

in general, the number of adaptive grid cells for the linearized forward approach is grater than that for the linearized forward-adjoint approach.

In Figure 6 we plot the partial derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ for different values of the detector zenith angle ${\theta }_{\mathrm{m}}^0$. The results are computed with the linearized forward-adjoint approach (FAA), the linearized forward approach (FA), and the finite-difference approach (FDA). In the latter case, a centered finite difference scheme is used for derivative calculations. The partial derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ at $N_{\mathrm{p}}=19$ base grid points ($x=\overline{p}\Delta x$, $y=y_{\mathrm{t}}^0$, $z=L_{\mathrm{z}}/2$), $\overline{p}=1\dots .,N_{\mathrm{p}}$, for different values of the solar azimuthal angle ${\phi }_0$ are plotted in Figure 7. Taking the values computed by the finite-difference approach as a reference, we illustrate in

Table 2. The first part of the table shows the relative errors ${\epsilon }_{\partial \mathcal{I}}$ in the partial derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ at the base grid points ($x=x_{\mathrm{t}}^0$, $y=y_{\mathrm{t}}^0$, $z=\overline{p}\Delta z$), $\overline{p}=1,\dots ,11$, for different values of the detector zenith angle ${\theta }_{\mathrm{m}}^0$, while the second part shows the relative errors ${\epsilon }_{\partial \mathcal{I}}$ in the partial derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ at the base grid points ($x=\overline{p}\Delta x$, $y=y_{\mathrm{t}}^0$, $z=L_{\mathrm{z}}/2$), $\overline{p}=1,\dots ,19$, for different values of the solar azimuthal angle ${\phi }_0$. The results are computed with the linearized forward-adjoint approach (FAA) and the linearized forward approach (FA).

${\mathit{\theta }}_{\mathbf{m}}^0$	${\mathit{\phi }}_0$	${\mathit{\epsilon }}_{\mathit{\partial }\mathcal{I}}$
		$\mathbf{FAA}$	$\mathbf{FA}$
$0^{°}$	$0^{°}$	$2.50\times {10}^{-2}$	$4.02\times {10}^{-3}$
${15}^{°}$	$0^{°}$	$2.23\times {10}^{-2}$	$4.28\times {10}^{-3}$
${30}^{°}$	$0^{°}$	$2.42\times {10}^{-2}$	$3.22\times {10}^{-3}$
${45}^{°}$	$0^{°}$	$2.05\times {10}^{-2}$	$1.29\times {10}^{-3}$
${45}^{°}$	$0^{°}$	$7.35\times {10}^{-3}$	$9.74\times {10}^{-4}$
${45}^{°}$	${30}^{°}$	$1.64\times {10}^{-2}$	$6.52\times {10}^{-4}$
${45}^{°}$	${60}^{°}$	$1.56\times {10}^{-2}$	$3.97\times {10}^{-4}$
${45}^{°}$	${90}^{°}$	$1.21\times {10}^{-2}$	$3.10\times {10}^{-3}$

the corresponding relative errors in the partial derivatives, defined, for example, in the case of the linearized forward-adjoint approach as

$${\epsilon }_{\partial \mathcal{I}}^{\mathrm{FAA}}={\displaystyle \frac{\sqrt{{\sum }_{\overline{p}}{\left[{\left({\displaystyle \frac{\partial \mathcal{I}}{\partial {\sigma }_{\mathrm{ext}p}}}\right)}_{\mathrm{FAA}}-{\left({\displaystyle \frac{\partial \mathcal{I}}{\partial {\sigma }_{\mathrm{ext}p}}}\right)}_{\mathrm{FDA}}\right]}^2}}{\sqrt{{\sum }_{\overline{p}}{\left({\displaystyle \frac{\partial \mathcal{I}}{\partial {\sigma }_{\mathrm{ext}p}}}\right)}_{\mathrm{FDA}}^2}}}.$$

(77)

The relative errors corresponding to the linearized forward-adjoint approach are smaller than $2.5\times {10}^{-2}$, while the relative errors corresponding to the linearized forward approach are smaller than $4.3\times {10}^{-3}$. The relative large errors of the linearized forward-adjoint approach can be explained as follows.

.1

As it can be inferred from Figure 5, the linearized forward-adjoint approach uses a rougher discretization grid (depending on the detector zenith angle ${\theta }_{\mathrm{m}}^0$) as compared to the forward and finite-difference approaches (which use the same grid).

2.

In the linearized forward-adjoint approach, the extinction/source function product is assumed to vary linearly within the cell, while in the forward and finite-difference approaches, the extinction/source function product is assumed to vary linearly along the characteristic.

4.2. Example 2

In the second example we consider in addition to the scattering and absorption by a cloud, molecular Rayleigh scattering, and the absorption by ${\mathrm{NO}}_2$ at the wavelength $\lambda =443$ nm. The difference to the previous geometry is that now we consider a base grid with $N_{\mathrm{z}}=16$ points along the z axis, implying that the height of the domain of analysis is $L_{\mathrm{z}}=(N_{\mathrm{z}}-1)\Delta z=1.5$ km. The cloud extinction field is given by Equations (71)–(73), while the cloud single-scattering albedo and the phase function are computed by Mie theory [27] for a water-cloud model with a Gamma size distribution

$$P\left(a\right)\propto a^{\alpha }exp\left[-\alpha \left(\frac{a}{a_{\mathrm{mod}}}\right)\right]$$

(78)

of parameters $a_{\mathrm{eff}}=10$$\mu \mathrm{m}$, $a_{\mathrm{mod}}=2a_{\mathrm{eff}}/3$, and $\alpha =6$. Here, a is the particle radius, and the droplet size ranges between 0.02 and 50.0 $\mathsf{\mu }$m. The computed value for the cloud single-scattering albedo is ${\omega }_{\mathrm{cld}}=0.865$, the Rayleigh cross-section and depolarization ratios are calculated as in [28], while the pressure and temperature profiles correspond to the US standard model atmosphere [29]. The number density and absorption cross section profiles for ${\mathrm{NO}}_2$, $n\left(z\right)$ and $C_{\mathrm{abs}}\left(z\right)$, respectively, are shown in Figure 8, and the absorption coefficient is calculated as ${\sigma }_{\mathrm{abs}}\left(z\right)=n\left(z\right)C_{\mathrm{abs}}\left(z\right)$. The peak optical depth is $5.4$.

In the following, we compute the partial derivative of the measured signal with respect to maximum value of the extinction field ${\sigma }_{\mathrm{ext}}^{\max }$ in Equation (71); in this case, we approximate $P\approx P_{\mathrm{cld}}$ and use

$$\frac{\partial {\sigma }_{\mathrm{ext}i}}{\partial {\sigma }_{\mathrm{ext}}^{\max }}=f(x_i,y_i,z_i)\mathrm{and}\frac{\partial {\omega }_i}{\partial {\sigma }_{\mathrm{ext}}^{\max }}=\frac{{\omega }_{\mathrm{cld}}-{\omega }_i}{{\sigma }_{\mathrm{ext}i}}\frac{\partial {\sigma }_{\mathrm{ext}i}}{\partial {\sigma }_{\mathrm{ext}}^{\max }}.$$

(79)

We also compute the partial derivative of the measured signal with respect to the total column of ${\mathrm{NO}}_2$, $X=(k_{\mathrm{B}}{T}_0/p_0){\int }_0^{L_{\mathrm{z}}}n\left(z\right)\mathrm{d}z$, where $k_{\mathrm{B}}$ is the Boltzmann constant, and $p_0$ and $T_0$ are the standard pressure and temperature. Assuming that the number density profile for ${\mathrm{NO}}_2$ is the a priori profile $n_{\mathrm{a}}\left(z\right)$ scaled by a constant, i.e., $n\left(z\right)=\alpha n_{\mathrm{a}}\left(z\right)$, where $\alpha$ is the scaling factor, we get

$$\frac{\partial {\sigma }_{\mathrm{ext}i}}{\partial X}=\frac{1}{X_{\mathrm{a}}}{\sigma }_{\mathrm{abs}}\left(z_i\right)\mathrm{and}\frac{\partial {\omega }_i}{\partial X}=-\frac{{\omega }_i}{{\sigma }_{\mathrm{ext}i}}\frac{\partial {\sigma }_{\mathrm{ext}i}}{\partial X},$$

(80)

implying $\partial {\sigma }_{\mathrm{sct}i}/\partial X=0$.

In

Table 3. Partial derivative $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}}^{\max }$, and the relative errors ${\epsilon }_{\mathcal{I}}$ and ${\epsilon }_{\partial \mathcal{I}}$ for different values of the detector zenith angle ${\theta }_{\mathrm{m}}^0$. The results correspond to the solar azimuthal angle ${\phi }_0=0^{\circ }$ and are computed with the linearized forward-adjoint approach (FAA) and the linearized forward approach (FA).

${\mathit{\theta }}_{\mathbf{m}}^0$	$\mathit{\partial }\mathcal{I}/\mathit{\partial }{\mathit{\sigma }}_{\mathbf{ext}}^{\mathbf{max}}$	${\mathit{\epsilon }}_{\mathcal{I}}$	${\mathit{\epsilon }}_{\mathit{\partial }\mathcal{I}}$
			FAA	FA
$0^{\circ }$	$5.776\times {10}^{-4}$	$4.20\times {10}^{-4}$	$3.46\times {10}^{-3}$	$2.03\times {10}^{-3}$
${15}^{\circ }$	$1.439\times {10}^{-3}$	$5.53\times {10}^{-4}$	$4.86\times {10}^{-3}$	$2.76\times {10}^{-4}$
${30}^{\circ }$	$2.930\times {10}^{-3}$	$2.30\times {10}^{-5}$	$3.40\times {10}^{-3}$	$2.53\times {10}^{-4}$
${45}^{\circ }$	$5.499\times {10}^{-3}$	$8.19\times {10}^{-5}$	$1.81\times {10}^{-3}$	$1.52\times {10}^{-4}$

and

Table 4. Partial derivative $\partial \mathcal{I}/\partial X$, and the relative errors ${\epsilon }_{\mathcal{I}}$ and ${\epsilon }_{\partial \mathcal{I}}$ for different values of the detector zenith angle ${\theta }_{\mathrm{m}}^0$. The results correspond to the solar azimuthal angle ${\phi }_0=0^{°}$ and are computed with the linearized forward-adjoint approach (FAA) and the linearized forward approach (FA).

${\mathit{\theta }}_{\mathbf{m}}^0$	$\mathit{\partial }\mathcal{I}/\mathit{\partial }\mathit{X}$	${\mathit{\epsilon }}_{\mathcal{I}}$	${\mathit{\epsilon }}_{\mathit{\partial }\mathcal{I}}$
			FAA	FA
$0^{°}$	$-1.184\times {10}^{-3}$	$4.20\times {10}^{-4}$	$1.65\times {10}^{-2}$	$3.42\times {10}^{-3}$
${15}^{°}$	$-1.284\times {10}^{-3}$	$5.53\times {10}^{-4}$	$2.27\times {10}^{-2}$	$3.70\times {10}^{-3}$
${30}^{°}$	$-1.428\times {10}^{-3}$	$2.30\times {10}^{-5}$	$2.26\times {10}^{-2}$	$2.82\times {10}^{-3}$
${45}^{°}$	$-2.529\times {10}^{-3}$	$8.19\times {10}^{-5}$	$2.20\times {10}^{-2}$	$3.06\times {10}^{-3}$

, we illustrate the partial derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}}^{\max }$ and $\partial \mathcal{I}/\partial X,$ and the corresponding relative errors using the values computed by the finite-difference approach as a reference. Also shown are the relative errors of the measured signal computed by Equation (22) (or, equivalently, by Equation (64)) with respect to the measured signal computed by Equation (15). In these simulations, the number of discrete zenith angles is $N_{\mu }=20$, and the splitting accuracy is ${\epsilon }_{\mathrm{split}}=3\times {10}^{-3}$. The relative errors corresponding to the linearized forward-adjoint approach are now smaller than $2.3\times {10}^{-2}$, while the relative errors corresponding to the linearized forward approach are smaller than $3.8\times {10}^{-3}$. It should be pointed out that the small errors in the measured signal certify that the adjoint problem is correctly solved.

In

Table 5. CPU time in min:sec for the test problems considered in this study and corresponding to the linearized forward-adjoint approach (FAA), the linearized forward approach (FA), and the finite-difference approach (FDA). The detector zenith angle is ${\theta }_{\mathrm{m}}^0={45}^{°}$ and the solar azimuthal angle is ${\phi }_0=0^{°}$.

Partial Derivatives	$\mathbf{FAA}$	$\mathbf{FA}$	$\mathbf{FDA}$
$\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ (Figure 6)	1:16	27:12	138:19
$\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ (Figure 7)	1:47	32:34	153:26
$\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}}^{\max }$	0:32	2:21	2:24
$\partial \mathcal{I}/\partial X$	0:31	2:20	2:20

we compare the computational efficiency of all linearization methods. Note that the code is parallelized with the OpenMP interface. The simulations are performed on a server Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30 GHz with 56 cores, while the CPU times are given for the parallelized implementation. Clearly, the most efficient method is the linearized forward-adjoint approach, while the most time-consuming method is the finite-difference approach.

Finally, in

Table 6. The first part of the table shows the number of adaptive cells $N_{\mathrm{cells}}$ and the final number of grid points $N_{\mathrm{points}}$ for computing the derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}p}$ with the splitting accuracy ${\epsilon }_{\mathrm{split}}={10}^{-3}$, while the second part of the table shows $N_{\mathrm{cells}}$ and $N_{\mathrm{points}}$ for computing the derivatives $\partial \mathcal{I}/\partial {\sigma }_{\mathrm{ext}}^{\max }$ and $\partial \mathcal{I}/\partial X$ with the splitting accuracy ${\epsilon }_{\mathrm{split}}=3\times {10}^{-3}$. The results correspond to the linearized forward-adjoint approach (FAA) and the linearized forward approach (FA).

${\mathit{\epsilon }}_{\mathbf{split}}$	${\mathit{\theta }}_{\mathbf{m}}^0$	FAA		FA
		${\mathit{N}}_{\mathbf{cells}}$	${\mathit{N}}_{\mathbf{points}}$	${\mathit{N}}_{\mathbf{cells}}$	${\mathit{N}}_{\mathbf{points}}$
${10}^{-3}$	$0^{°}$	$\mathrm{66,652}$	$\mathrm{40,727}$	$\mathrm{293,492}$	$\mathrm{162,279}$
${10}^{-3}$	${15}^{°}$	$\mathrm{81,174}$	$\mathrm{50,381}$	$\mathrm{293,492}$	$\mathrm{162,279}$
${10}^{-3}$	${30}^{°}$	$\mathrm{95,152}$	$\mathrm{59,595}$	$\mathrm{293,492}$	$\mathrm{162,279}$
${10}^{-3}$	${45}^{°}$	$\mathrm{115,186}$	$\mathrm{73,332}$	$\mathrm{293,492}$	$\mathrm{162,279}$
$3\times {10}^{-3}$	$0^{°}$	$\mathrm{17,804}$	$\mathrm{13,196}$	$\mathrm{54,164}$	$\mathrm{33,576}$
$3\times {10}^{-3}$	${15}^{°}$	$\mathrm{20,094}$	$\mathrm{14,942}$	$\mathrm{54,164}$	$\mathrm{33,576}$
$3\times {10}^{-3}$	${30}^{°}$	$\mathrm{22,674}$	$\mathrm{16,989}$	$\mathrm{54,164}$	$\mathrm{33,576}$
$3\times {10}^{-3}$	${45}^{°}$	$\mathrm{25,806}$	$\mathrm{19,607}$	$\mathrm{54,164}$	$\mathrm{33,576}$

, we show the numbers of adaptive cells and grid points for the test problems considered in this study. Because in the linearized forward-adjoint approach, the numbers of cells and points are smaller than in the linearized forward approach, we infer that the first method is not only faster but also less memory demanding than the second one.

5. Conclusions

Two linearization methods for SHDOM have been discussed. The first method is an analytical linearization approach, while the second method is based on the adjoint radiative transfer theory. In the latter case, practical formulas for computing the derivatives of the measured signal in the spherical harmonics space have been derived. Essentially, SHDOM has been specialized for derivative calculations and radiative transfer problems involving the delta-M approximation, the TMS correction, and the adaptive grid splitting.

Our numerical analysis leads to the following conclusions.

1.

The linearized SHDOM with analytical derivatives is an accurate approach. However, the method is time-consuming and demands a large computer memory. The reason is that not only the source function has to be stored as a spherical harmonic series at each grid point, but also its derivatives with respect to the atmospheric parameters of interest.

2.

The linearized SHDOM with a forward-adjoint approach requires less storage for derivatives calculation, are much faster, but relatively less accurate. The main reason for this lower accuracy is that different interpolation schemes are used for radiance and derivative calculations.

According to our numerical analysis, the relative errors of the linearized forward-adjoint approach are smaller than 2.5%. In this regard, it should be pointed out that in the framework of the Gauss-Newton method, errors of about 2–3% in the Jacobian do not play a significant role. The effect is somehow similar to the computation of the root of an equation by the secant method instead of the Newton method. However, if the errors in the Jacobian are important for the retrieval, the regularized total least squares, which accounts on the errors in both the Jacobian and the data, can be used.

In our analysis, we considered the case of a fixed detector position ${\mathbf{r}}_{\mathrm{D}}$ and a single-angle measurement ${\mathsf{\Omega }}_{\mathrm{m}}^0$. For multi-angle measurements and eventually, several detector positions, the solution of several adjoint problems (corresponding to each pair $({\mathbf{r}}_{\mathrm{D}},{\mathsf{\Omega }}_{\mathrm{m}}^0)$) can be avoided by employing the approach proposed in Ref. [23]. The main ideas of this approach, which can be applied for multi-dimensional remote sensing of cloud extinction fields from airborne radiometer measurements, are discussed in Appendix D. However, it should be pointed out that the computational cost for the retrieval of 3D extinction fields is extremely high. A reduction of the computation time can be achieved if a two-dimensional geometry (a constant extinction field along the y-axis, and a set of directions on the unit sphere) is considered. In this case, the application of the forward-adjoint approach requires that both the solar and detector azimuthal angles are zero (the solar and detector directions are in the $xz$-plane).

Potential application of the linearized SHDOM include the following.

1.

Retrieval of cloud model parameters. For broken clouds [30], the indicator function $f(x,y,z)$ in Equation (71) takes the values 1 inside the cloud and 0 inside the clear sky region. As in [31], the extinction field can be parametrized in terms of ${\sigma }_{\mathrm{ext}}^{\max }$ (cf. Equation (71)), the cloud top height $h_{\mathrm{t}}$, and the cloud bottom height $h_{\mathrm{b}}$. The cloud optical thickness and the cloud geometrical parameters can be retrieved in the oxygen A-band by using the correlated k-distribution method for the broadband integration of the gaseous line absorption. Note that SHDOM is able to compute monochromatic and broadband radiative transfer (with a k distribution).

2.

Trace gas retrievals under cloudy conditions. The retrieval of total column of trace gases, e.g., ${\mathrm{O}}_3$, ${\mathrm{NO}}_2$, etc., can be performed by means of the differential optical absorption spectroscopy (DOAS) technique [32]. This approach requires the knowledge of the air mass factor (AMF), i.e., the partial derivative of the measured signal with respect to the total column of the trace gas at a specific wavelength. The main goal of the analysis is the computation of the air mass factor when clouds are outside the footprint of the instrument (effect of horizontal cloud edges on AMF calculation).