Multicriteria Accuracy Assessment of Digital Elevation Models (DEMs) Produced by Airborne P-Band Polarimetric SAR Tomography in Tropical Rainforests

Abstract

The penetration capability of P-band radar waves through dense vegetation, along with the ability of tomography to separate the contributions of different layers in a vertical reflectivity profile, make P-band radar tomography a promising tool for digital terrain modeling in forested areas, specifically in dense tropical forests under which terrain topography remains poorly known. This paper aims to assess the overall quality of a digital terrain model (DTM) produced using tomographic processing of airborne P-band SAR imagery acquired during the TropiSAR campaign in French Guiana. Many quality descriptors are used to evaluate the quality of this DTM. Position and slope accuracies are computed based on a lidar DTM considered as the reference, and the impact of several parameters on these accuracies is studied, namely, slope, slope orientation, off-nadir angle and local incidence angle. The realism of the landforms is also studied according to geomorphological criteria. The results of this multicriteria accuracy assessment show the high potential of P-band SAR tomography in depicting the topography under forests, despite the intrinsic limitations related to the slant range geometry: the absolute elevation error is around 2 m; the slope is overestimated with an error of about 16°, mainly due to a processing artifact for which easy and direct solutions exist. Indeed, this error is equal to about 3° in flat artifact-free areas. These errors vary depending on the acquisition parameters and the local topography. The shapes are globally well preserved. These results are also discussed in the frame of the upcoming BIOMASS mission developed by the European Space Agency (ESA) and expected to be launched in 2024.

1. Introduction

Digital elevation models (DEMs) are essential data for most geoscientific studies. They represent the terrain topography and have a large potential in the quantification of morphological and hydrological indices [1]. Topographic mapping in forested areas is a difficult task since the ground is hidden for most mapping techniques, and this is particularly true for tropical dense forests, such as in the Amazon region [2]. Classical techniques such as field surveys are limited by the lack of visibility and reduced coverage. Photogrammetry, which has been operational and widely used by all mapping agencies since the Second World War, is limited in forested areas since it only describes the 3D geometry of the upper area of the canopy. Airborne lidar has become the most successful technique in forested areas [3,4]. Indeed, it provides a dense and accurate point cloud that represents both the canopy and the terrain. Thus, it enables the computation of both DSM (digital surface model, i.e., canopy) and DTM (digital terrain model, i.e., ground). This method has been widely used in a variety of applications such as hydrology and archeology [5]. The performances of airborne lidar and airborne photogrammetry for digital terrain modeling are comparable over bare soil or low vegetation [6], although photogrammetry has the advantage of providing images together with the elevation model.

Synthetic aperture radar (SAR) imagery acquired from airborne or spaceborne sensors can also be processed to derive terrain models. This allows higher coverage capabilities than in the case of lidar-based technique limited by beamwidth and power constraints. Several processing methods have long been used to produce DEMs from SAR imagery, namely, radargrammetry [7,8,9], radarclinometry [10,11] and radar interferometry [12,13], the latter being the most precise. Interferometry has recently seen new advances, such as PolInSAR (based on the use of the signal polarization) [14,15] and multibaseline SAR tomography (TomoSAR) [16,17,18].

The most accurate radar systems available nowadays operate with short wavelengths (i.e., X- and C-bands), which have two severe limitations in forested areas:

(*)

The radar signal is mainly backscattered by the upper layer of the canopy so that it does not reach the ground. This limitation is partially overcome with longer wavelengths.

(**)

The radar signal is decorrelated by the foliage movement and backscattering changes between two acquisitions in repeat pass interferometry [19]. This limitation may be overcome by the implementation of a dedicated dual-antenna single-pass InSAR system (e.g., SRTM [13]) or a tandem configuration (e.g., TanDEM-X [20]), in which the same radar echo is received simultaneously by the two antennas [21]. Another approach is to use longer wavelengths, such as L-band, used in JERS and ALOS satellites [22], and P-band (wavelength of 69 cm), which will be implemented for the first time in space with the BIOMASS satellite [23,24,25,26].

By using long radar wavelengths (mainly L- and P-bands) over forested areas, the echo is produced by multiple interactions of the radar signal with different layers of the forest cover, from the ground to the top of the canopy. The ability of P-band radar waves to reach the ground through the forest canopy makes them suitable for terrain modeling in dense forest areas and has shown great potential for DTM extraction in different studies [15,16,18,27,28,29,30]. The possibility of implementing a P-band imaging radar on a satellite has been studied since the 1990s [31]. Until now, limited to airborne acquisitions due to frequency allocation restrictions for civilian applications, P-band radar imagery is expected to be available from space for the first time with the ESA BIOMASS mission to be launched in 2024 [23,24,32]. Proposed and designed to estimate forest biomass, especially for dense forests (impenetrable at higher frequencies), the secondary objectives of the BIOMASS mission include the production of wide-scale and wall-to-wall DTM [33].

Image processing methods, such as polarimetric SAR interferometry (PolInSAR) and SAR tomography (TomoSAR), enable the separation of backscattered contributions from the ground and the different parts of the canopy [34], and thereby the extraction of the topography under forested areas. Benefiting from the coherent capabilities of radar processing, SAR tomography gave rise to abundant literature in the last decade, mostly based on airborne L- and P-band radar image acquisitions with multiple incidence angles [35] to image the 3D structure of dense and high forests as well as the underlying topography [16,29,30,36,37,38].

In the framework of the preparatory experiments for the BIOMASS mission, preliminary tests were carried out using data of airborne campaigns in tropical forests, i.e., in French Guiana [17] and Gabon [39], in order to improve and validate the algorithms of biomass inversion. Since digital terrain modeling is foreseen as a product of the BIOMASS mission, the data of airborne surveys have also been processed to test the potential of P-band tomography to model the topography, even though these results do not allow straightforward conclusions about the potential of the BIOMASS mission at a very different spatial resolution. Preliminary studies have confirmed the potential of this product, both in terms of positional accuracy [16,18,29] and landform description [33]. This article aims at extending the previous studies by evaluating the product in forested areas considering that the results will refer to the first steps of DTM production, i.e., the P-band radar acquisition and the tomographic processing, excluding any post-processing step such as slant-to-ground-range projection or subsequent resampling. A wide range of quantitative DTM quality criteria, mainly related to position and shape, are used in order to verify the suitability of the DTM for different geoscientific applications.

A DEM needs to have an acceptable quality, along with a suitable resolution, to provide reliable results for downstream applications [40,41]. That is why an increasing number of studies aiming to assess the quality of DEMs have been published during the last two decades [41,42]. In this paper, we consider two types of quality criteria, namely, external (based on the comparison with external reference data) and internal (based on the detection of artifacts or unrealistic landforms) criteria. Based on these criteria, the following questions will be addressed: What are the main insights of P-band SAR tomography in terms of three-dimensional terrain modeling under tropical rainforests? How accurate are the positions and terrain shapes modeled using this technique? How robust and homogeneous are the performances with respect to terrain morphology and acquisition parameters? The study area, the datasets and the quality assessment methods are firstly presented in Section 2; the results are presented and illustrated in Section 3 and discussed in Section 4.

2. Materials and Methods

2.1. Data

2.1.1. Study Area

The test site is located in French Guiana, near the town of Kourou, on the northern margin of the Guiana Shield, as shown in Figure 1. The surveyed site (52°55′55′′W and 5°16′9′′N) has an area of about 8 km² (2 km × 4 km). It is a typical Amazonian landscape with a hilly relief, a very dense hydrographic network and a dense rainforest. The area has practically no anthropic activities except the presence of a few small buildings. Over this site were acquired P-band airborne radar data in the framework of the ESA TropiSAR campaign [17] for the preparation of the BIOMASS space mission.

2.1.2. Lidar DTM

The lidar DTM, used as a reference, is produced using a small-footprint lidar system. The data was collected in 2009 by Altoa company (Cayenne, French Guiana) using a helicopter-mounted RIEGL LMS-280i (wavelength 0.9 μm) with a ground point density of about 0.1 point/m² [43]. The points representing the ground were interpolated into a triangulated irregular network (TIN), and the final DTM was resampled to a 1 m raster grid.

Airborne lidar is widely used for providing 3D surveys of the forest structure and the underlying topography in forested areas [44,45,46]. The altimetric accuracy is generally decimetric and depends on the morphology and the land cover. For example, in [44], the elevation accuracy of this technique ranges from 17 cm (open areas) to 26 cm (in forests). The lower density of ground points in the forest slightly increases the position error and creates shape errors due to the interpolation effect over longer distances (as shown in Figure 2 in the case of the TIN interpolation), which could bias the results of slope accuracy assessment. Furthermore, a considerable number of sinks exist along the watercourses, which can also alter the extracted stream network if not corrected. In spite of these artifacts, the overall high accuracy of the lidar DTM allows us to consider it as a reference to validate the P-band DTM. The radar and lidar campaigns both took place in 2009, so we consider that no change happened in the landscape.

2.1.3. P-Band DTM

The SAR data processed in the frame of this study were measured in 2009 by the SETHI sensor by ONERA (Palaiseau, France) and consist of a set of 6 fully polarimetric images acquired in a repeat-pass multi-baseline InSAR mode at the P-band. The emitted radar signals occupy a spectral domain centered around ${\mathrm{f}}_{\mathrm{c}}=397.5 \mathrm{MHz}$, and the dimensions of the processed 2D spectrum led to spatial resolutions of 1.2 m and 1.5 m in the slant range and azimuth directions, respectively. A subset of the original stack, whose slant range and azimuth coordinates, delineated in Figure 3, coincide with those of the lidar DTM described below, has been selected. This sub-region, located in the near-to-medium range part of the original swath, is observed by the radar under an elevation angle, $\mathsf{\theta }$, ranging from 20 to 50 degrees, and is characterized by significant topographic features, with hills and valleys.

The principle of tomographic imaging, illustrated in Figure 4, is based on the use of coherent acquisitions performed from slightly shifted positions describing an additional aperture in the direction orthogonal to the radar line of sight. Classical 2D SAR imaging is known to be affected by a cylindrical ambiguity and merges responses originating from scatterers located at a similar distance from the radar, whatever their elevation. The coherent combination of several 2D SAR images acquired with spatial diversity allows the synthesis of 3D reflectivity maps using tomographic processing, which further refines original 2D resolution cells into 3D ones, whose extent in elevation, i.e., vertical resolution, depends on the length of the normalized multi-baseline aperture, the range distance and the carrier frequency [47]. Over the sub-region considered in this study, the vertical resolution ranges from approximately 16 m to 26 m, from near to far range. The polarization diversity may be used to further discriminate contributions having different polarimetric responses. In the case of 3D imaging, several studies showed that polarization could be efficiently used to separate the global response into two components, one of them being associated with the underlying ground response [30,48].

The processing chain set up in this study for estimating ground elevation using P-band tomographic SAR data may be described as follows. The first step consists of estimating residual interferometric phase terms, corresponding to equivalent antenna phase center position errors smaller than a signal’s wavelength, from the SAR images themselves using a phase error model, as proposed in [49]. The joint second-order statistics of the phase-corrected SAR images are then estimated using a sample multi-baseline PolInSAR, or simply PolTomoSAR, covariance matrix, given by:

$${\widehat{\mathit{R}}}_{PS}=\frac{1}{L}{\displaystyle \sum }_{l=1}^L{\mathit{k}}_{PS}\left(l\right){\mathit{k}}_{PS}^H\left(l\right)\in S{P}_{ℂ}^{M{N}_p\times M{N}_p}$$

(1)

where $S{P}_{ℂ}^{Q\times Q}$ is the set of complex-valued $\left(Q\times Q\right)$ positive semi-definite matrices, $M$ is the number of images, $N_p$ represents the number of polarimetric channels and $L$ is the number of realizations used to compute the covariance matrix estimate, which coincides with the maximum likelihood estimate in the case of normally distributed SLC echoes. For a given range-azimuth resolution cell, the acquired signals are gathered into a polarimetric and spatial vector, defined as:

$${\mathit{k}}_{PS}=\left[\begin{array}{c}{\mathit{k}}_1\\ ⋮\\ {\mathit{k}}_{N_p}\end{array}\right]\in {ℂ}^{M{N}_p\times 1}, \mathrm{with} {\mathit{k}}_i=\left[\begin{array}{c}{k}_{i_1}\\ ⋮\\ k_{i_M}\end{array}\right]\in {ℂ}^{M\times 1}$$

(2)

with $M=6$ images and $N_p=3$ polarization channels. The $L$ samples used to compute the covariance matrix are provided by a multi-dimensional $\left(N_{filt}\times N_{filt}\right)$ boxcar filter, with $N_{filt}=17$ pixels, empirically found as a good compromise preserving resolution and radiometric estimation accuracy, corresponding to $N_{eq}=200$ equivalent number of looks. The PolTomoSAR are then decomposed using algebraic approaches detailed in [48] and [50], and the component identified as representing the ground response is focused in the vertical direction using a non-parametric spectral analysis estimator [51]. The ground elevation is determined as the location in elevation corresponding to the maximum of the ground focused response. One may note that, in order to actually assess the intrinsic behavior of DTM estimated from SAR tomography at the P-band, no further processing step, such as smoothing filter regularization, is applied to the estimated elevation map. The final DEM was resampled at 1 m in both slant range and azimuth directions. The processing results are illustrated in Figure 5 with two tomographic profiles estimated in the range direction and displayed in absolute elevation coordinates and after compensation of the lidar DTM estimate.

2.2. Methods

2.2.1. DEM Quality Assessment Approaches

To assess the quality of a DEM, there are two major validation methods, called external and internal [52,53]. The external validation is based on the use of external reference data to assess positional and shape accuracies of the modeled terrain. The internal validation does not need any external data and aims to verify whether the modeled terrain is possible or at least realistic by checking if some physical and statistical rules are validated in the DEM [54,55].

In external validation, the accuracy of a DEM is generally assessed by characterizing the statistical distribution of the elevation difference between the DEM and a number of ground control points, using indicators such as mean, standard deviation, Root Mean Square (RMS), etc. [56]. However, these indicators do not guarantee the quality of terrain shapes, which depend on the accuracy of elevation derivatives such as slope, aspect or curvature to characterize landform elements [57]; most geoscientific applications of DEMs are based on these geomorphic parameters. As the shape descriptors are calculated using DEM values, elevation errors propagate through them and influence their accuracy. The influence of random elevation errors is greater on elevation derivatives as the derivation order increases, and an insignificant input error can, in some cases, give large errors in the derivatives [58,59]. Thus, it is possible to obtain high positional accuracy and a poor shape quality, and vice versa [53]. Indeed, the shape descriptor accuracy depends on both the elevation error and the spatial autocorrelation of this error [60]. Therefore, the overall DEM quality depends not only on the positional accuracy but also on shape accuracy [61].

The elevation error is a result of gross, systematic and random errors and is generally considered to be normally distributed [60,62]. To estimate this error, the elevation difference between the DEM and a sufficiently large sample of reference points is measured, and the RMS is calculated [63]. The value obtained characterizes the elevation quality but does not necessarily reveal the shape uncertainty [64]. Then, to assess the elevation error propagation through the shape descriptors, the spatial autocorrelation of the elevation error has to be evaluated [65]. Finally, an elevation error model for the DEM can be built using the elevation error and its spatial autocorrelation. The error propagation in the shape descriptors can then be assessed analytically when possible or numerically by using a Monte-Carlo simulation [58,66,67].

Different factors can affect the quality of a DEM, related to the production method and its production parameters but also to landscape characteristics [68]. For instance, the acquisition parameters and production parameters have a direct influence on the elevation and its derivatives [69,70,71]. Some production techniques produce errors that are more autocorrelated than others, which provide more accurate results for terrain shape modeling [56,66]. The error autocorrelation can be preserved by processing steps that reduce elevation noise, but this generally leads to degradation of the resolution so that the DTM can no longer depict the smallest terrain shapes. On the other hand, the terrain slope and aspect can also have an impact on the produced DEM [68,72].

Some shape descriptors are scale-free such as the closed depression volume and elevation [59,73], but most of them are scale-dependent [74,75], and their values and quality depend on the mesh size [67,76]. Thus, the choice of the mesh size must be defined according to the foreseen application to have the best results in terms of accuracy [77,78].

The external and internal validation methods used in this study are outlined in the following two sections.

2.2.2. External Validation

External validation consists in comparing the DEM with higher quality external data, typically a set of ground control points or a higher accuracy DEM. The error analysis in this study is based on statistical indicators of the difference between the P-band and lidar DTMs. The overall error in a DEM is composed of different kinds of errors. As Equation (1) shows, the elevation modeled in a DEM (${\widehat{z}}_i$) is the sum of the true elevation on the terrain ($z_i$) with a systematic error or a bias (${\mu }_i$) and a random error [56,66,79]; this latter is composed of a spatially correlated term (${{\epsilon }^{\prime }}_i$) and a spatially white noise term (${{\epsilon }^{″}}_i$).

$${\widehat{z}}_i=z_i+{\mu }_i+{{\epsilon }^{\prime }}_i+{{\epsilon }^{″}}_i$$

(3)

In order to evaluate those errors, a list of quality criteria is used. The mean value of the difference between the P-band and lidar DTMs gives the bias between these two DTMs, whereas the standard deviation provides a relevant description of the random error.

$$\mu =\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N({\widehat{z}}_i-z_i)$$

(4)

$$\sigma =\sqrt{\frac{1}{N}{\displaystyle \sum }_{i=1}^N{\left({\widehat{z}}_i-z_i\right)}^2-{\mu }^2}$$

(5)

The most used statistical indicator in DEM validation is the root mean square error (RMSE) of the elevation (Equation (4)), which includes both systematic and random errors, as shown in Equation (5).

$$\mathrm{RMSE}=\sqrt{\frac{1}{N}{\displaystyle \sum }_{i=1}^N{\left({\widehat{z}}_i-z_i\right)}^2}$$

(6)

$$\mathrm{RMSE}=\sqrt{{\mu }^2+{\sigma }^2}$$

(7)

The mean, standard deviation and RMSE are also calculated for the absolute slope to show the impact on the shape modeling, as it is the most used topographic index to quantify the terrain morphology. The slope is calculated using the third-order finite difference weighted by the reciprocal of squared distance [80] for all the DTMs with a 1 m mesh size. Since some parameters are scale-dependent, DEMs must have the same mesh size in order to be compared [74,75].

The autocorrelated part of the random error is evaluated at different lags, assuming stationary random error terms. It is calculated using the following formula.

$$r\left(h\right)=\frac{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_i\times d_{i+h}-\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_i\times \frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_{i+h}}{\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_i{}^2-{\left(\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_i\right)}^2 }\times \sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_{i+h}{}^2-{\left(\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{d}_{i+h}\right)}^2}}$$

(8)

where n is the total number of point pairs, $d_i$ is the difference between the two DEMs at the position i, $d_{i+h}$ is the difference at the position i + h at a distance h (which is the lag) from the position i. In this study, the lag ranges from 1 to 20 pixels.

All the aforementioned indicators are calculated for the overall P-band DTM with regard to the lidar DTM. The influences of the landscape and the acquisition geometry on the quality of the DTM are analyzed, based on the slope, the effects of which have been widely studied for other techniques [68,72], as well as the aspect, the viewing angle and the local incidence angle.

2.2.3. Internal Validation

Internal validation consists of identifying the evidence of errors and artifacts in the DEM itself, revealed by the presence of unlikely or even impossible terrain shapes. This approach is based on the fact that topographic surfaces have universal properties, so that specific rules are fulfilled everywhere, and their non-satisfaction is considered a consequence of DEM errors. This rule-based approach can be applied without the need for external reference data and is based on both strong and weak requirements [41].

A strong requirement can be expressed by some physical rule, and a terrain that does not fulfill it is impossible. A well-known physical law is that water always goes down. Thus, the occurrence of sinks along the watercourses is a strong indicator of the presence of artifacts in the DEM [54]. The percentage of sinks and their mean depth, along with their spatial distribution, can therefore help provide the location and a quantitative indication of DEM errors.

A weak requirement can be expressed by a statistical rule, and a terrain that does not comply with this rule is improbable but not impossible. For example, terrains modeled by hydric erosion, which is the case of most continental surfaces, have a fractal behavior [81]. A consequence is that they respect Horton’s law, according to which the logarithm of the total number of streams of a given Strahler order decreases linearly as a function of the order [82]. A deviation from this law is also considered a consequence of DEM errors.

To apply this validation method, the percentage of sinks and the mean, standard deviation and RMS of their depth are first calculated. Then, the compliance of the rivers extracted from the DEM to Horton’s law is verified by the calculation of the R² coefficient of the cloud point fitting.

3. Results

3.1. External Validation

3.1.1. Elevation and Slope Errors

The comparison of the P-band radar DTM with the reference lidar DTM provides quality estimations in terms of elevation and slope accuracy.

In Figure 6, the mean value of the elevation error is negligible, whereas the standard deviation is less than 2 m, confirming the potential of P-band radar tomography in terms of absolute elevation accuracy. The slope is higher in the P-band DTM than in the lidar DTM of about 7° on average, and the slope error has a standard deviation and RMSE of about 14° and 16°, respectively. As shown in Section 3.2, these high values of slope error are due to artifacts produced by the application of a 17 × 17 boxcar filter during the tomographic covariance matrix estimation step. The use of smoother filters, with a Gaussian impulse response, for instance, would mitigate this effect [83]. A small region free from this artifact has a higher slope accuracy, i.e., an average slope and standard deviation of about 2°.

The autocorrelation (shown in Figure 7) is high over short distances due to the 17 × 17 boxcar estimation filter, with no significant difference between range and azimuth directions. It decreases as the distance increases, which is the usual behavior of DEM error autocorrelation as a consequence of Tobler’s first law of geography [84,85], and it remains higher in the azimuth direction. This suggests that the shapes are noisier in the range direction, as will be confirmed below.

To better understand the source of the elevation and slope errors, these were extracted according to the morphology of the terrain (slope and aspect), radar acquisition parameters (off-nadir angle and local incidence angle) and a combination of these.

Figure 8 shows the effects of terrain slope on the DTM quality in terms of elevation and slope errors, the reference slope being calculated using the lidar DTM. This figure shows that the mean elevation error is negligible whatever the slope and that the standard deviation increases linearly ($R^2>0.9$) with a difference of around 1 m between elevation errors obtained for the smallest and the steepest slope, which confirms the results obtained with other DTM production techniques [72,86,87]. The slope error also increases according to terrain slope to reach a maximum at around 20°, then it decreases. The standard deviation exhibits a slight increase, almost linear, for slopes less than 20°, which is not consistent with other theoretical studies that suggest a decreasing standard deviation of the slope with an increasing slope [67]. However, the influence of the slope has to be interpreted with care since it is likely to be combined with the influence of the radar viewing angle, as suggested by the following results.

The impact of the off-nadir angle is shown in Figure 9. According to this figure, the following can be observed:

-

The mean value of elevation error is negligible, whatever the off-nadir angle. Similarly, the standard deviation is little influenced by this angle and fluctuates around 2 m.

-

The slope error (mean and standard deviation) is an increasing function of the off-nadir angle, and it reaches large values. This relationship is certainly related to the decrease of vertical resolution as the off-nadir angle rises.

Figure 10 shows the impact of the local incidence angle on elevation and slope errors. This figure shows the following:

-

The mean elevation error is negligible for angles ranging between 30° and 45°, whereas the standard deviation decreases to reach a minimum for a 45° angle. As explained in [88,89,90], the double bounce reflection occurring between the ground and tree trunks is a very energetic and directional scattering mechanism, having a narrow scattering diagram centered around 45°. When the incidence angle reaches this range of values, the dominant double-bounce responses, whose phase center is exactly located at the ground level, can easily be perceived in tomograms and localized in elevation.

-

The mean and standard deviation of the slope error are maximum for small local incidence angles (10–20°), which may be explained both by the weaker contribution of double-bounce scattering and by loss of resolution in areas facing the radar, and they decrease as the local incidence angle increases.

Figure 11 describes the influence of the terrain aspect, i.e., the azimuth direction of the maximum slope, on the elevation and slope errors, considering the flight direction (bottom to top). According to this figure:

-

The mean error in elevation is maximum for slopes facing the radar antenna, probably related to a loss of vertical resolution, while it is almost zero for slopes in the other direction. The standard deviation is higher in the range direction than in the azimuth direction.

-

The slope mean error has the opposite behavior, with a maximum in the areas oriented away from the radar. The standard deviation has the same behavior.

To assess the impact of slopes and local incidence angles, the standard deviation of both elevation and slope are plotted against those parameters, and the results are shown in Figure 12 and Figure 13. According to these figures, the following can be observed:

-

The standard deviation of the elevation error is negligible and isotropic in flat areas, and it gets higher for larger slope values, mainly for slopes facing away from the radar, which is due to a lack of ground response, with either single- or double-bounce contributions, while it preserves an almost constant value for slopes facing the radar, as confirmed by the profile shown in Figure 14.

-

For the largest slope values, the standard deviation is high. The largest local incidence angles produce the maximum error in areas facing the radar, characterized by a lower vertical resolution and by double-bounce scattering contributions with low intensity. The smallest angles produce the highest error in areas facing away from the radar. The intermediate angles (35–45°) lead to the best trade-off, as already shown in Figure 10.

-

The standard deviation of the slope error shows a similar tendency as the elevation error with respect to the slope and aspect. For the same reasons, the results are less reliable for steep slopes facing away from the radar.

All of those results suggest the importance of the radar acquisition parameters and their interaction with the terrain morphology as well as the flight direction in the quality of the produced DEM, specifically in forested areas.

3.2. Internal Validation

The first stage of internal validation is a visual control of the data. Figure 15 shows a combined hypsometry-hillshade representation of the P-band DTM, revealing the presence of random errors in the DTM, as well as systematic artifacts in its slope, due to the use of a raw boxcar filter for estimating the TomoSAR covariance matrix. Using a smoother filtering pattern should reduce this undesirable effect.

Figure 16 shows the variation of the number of streams (in logarithmic scale) as a function of the Strahler order for the streams extracted from the P-band DTM. This graph allows us to verify that the DTM respects Horton’s law, according to which the logarithm of the number of streams decreases linearly as a function of the Strahler order. Thus, a deviation from this law may be interpreted as due to the presence of an error and, therefore, the R² can be used as a quality criterion for the DEM [41]. As expected, the extracted streams show a linear decrease even for the smallest Strahler orders, which are generally more likely to be affected by noise, and the R², suggesting a higher quality of DTM shapes.

Sinks are obstacles in a DTM that prevent the extraction of a reliable and continuous stream network. They are a consequence of the presence of elevation errors in a DTM, and most DTMs include some.

Table 1. Percentage and depth of sinks in P-band DTMs.

Percentage of Sinks (%)		2.02%
Depth of sinks	Mean (m)	0.38
	Standard deviation (m)	0.60
	RMS (m)	0.71

shows the percentage and depth of those sinks in the P-band DTM. The percentage of the sinks is small (~2%), and the depth of those sinks is less than the elevation RMSE. This implies that the P-band DTM has a good agreement with the physical rule that the water always flows downward.

The results show that the shapes of the DTM are in agreement with the expected behavior of most continental surfaces.

4. Discussion

The results of the previous section have shown the potential of a DTM obtained by tomographic processing of P-band airborne radar data while revealing and explaining its limitations.

Our conclusions are based on the study of a particular case and can only be generalized with caution. For example, the study area has a moderate relief which does not allow predicting the performance to be expected from similar data on a mountainous landscape. In addition, we considered the case of a homogeneous forest, different from many terrestrial landscapes where the diversity of vegetation cover would have made the analysis more complex.

Our study is limited, in particular, by the small size of the study area, although the number of points is statistically very significant, and by the fact that we use lidar data as a reference, which has its own error. Indeed, the lidar DTM is highly smoothed due to the interpolation of ground points (which are less dense under forests), so that the slopes are underestimated, while the radar DTM has artifacts and noise that overestimate the slopes; the slope errors that we have calculated are therefore partly attributable to the lidar data. This is what makes the internal validation so interesting, as it allows us to detect DTM errors without the need for reference data. By proposing a multicriteria analysis, which combines requirements on the accuracy of heights and slopes and with respect to shapes, we thus obtain very complete results on the quality of a DTM, which is the original part of our contribution.

It should be noted that this study was based on radar data in slant range geometry, which is not the common geometry of classical DEM products. Although it leads to local variations in range resolution, this choice allows a more intrinsic evaluation of the performance of P-band tomography since it excludes the influence of the projection from slant to ground range or any other resampling step. In other words, the evaluation is based on a raw DTM, resulting from the two basic steps of DTM production, namely, the P-band radar data acquisition and the tomographic processing, without any subsequent post-processing.

Despite these limitations, this study is a basis for further work over larger areas and focuses on more diverse configurations, especially in terms of terrain topography, vegetation cover or acquisition parameters. In the case of the BIOMASS mission, those parameters are different and should give different results. Indeed, the radar bandwidth (6 MHz giving a 25 m range resolution) will not give access to the same topographic features, although the computational algorithm or the overall trends related to physical interactions at P-band (central frequency close to 400 MHz) remain very similar. Specific tomographic techniques will probably have to be implemented in order to cope with the low resolution and low dimensionality of the data [91]. In addition, it is also worth reminding that the exploitation of BIOMASS data will enable the combination of ascending and descending passes, which would significantly improve the mitigation of slant range geometrical effects and motivate further studies based on such configurations. The multicriteria analysis proposed in our study will then have to be conducted with a different interpretation.

5. Conclusions

The study presented in this paper provides an overall quality assessment of a digital elevation model produced by airborne P-band polarimetric SAR tomography in tropical rainforests, based on two validation approaches, namely, an external validation based on the comparison with a lidar DTM considered as a reference, and an internal validation based on the assessment of shape realism. The main conclusions of this multicriteria quality assessment are as follows:

• The P-band radar DTM has a negligible bias, and the elevation RMSE is around 2 m.
• The slope is overestimated with an average error of 7° and a standard deviation of 14°, mainly due to a processing artifact for which easy and direct solutions exist (2° and 2°, respectively, in flat artifact-free areas).
• The elevation and slope errors are more important in the range direction than in the azimuth direction, given the intrinsic effects of the radar slant range geometry.
• The difference between the P-band DTM and the lidar DTM is more important on steep slopes (maximum error around 18–20°); on slopes facing the sensor, the P-band is above the lidar (positive difference), while on slopes looking away from the sensor, the P-band is most often below the lidar (negative difference). In areas with small slopes, the difference is negligible (for both the mean and standard deviation). This is due to the double bounce effect, whose amplitude is maximal for an incidence angle lying close to 45° and over flat areas, over which the tree trunks and the ground are orthogonal. As the terrain slope increases, the error increases due to three effects:

  -

  The loss of resolution for slopes facing the radar, which produces an almost constant error;

  -

  A significant reduction of the amplitude of double-bounce scattering;

  -

  A lack of ground response, with either single- or double-bounce contributions, over slopes facing away from the radar.

• The off-nadir angle has little effect on the elevation error, while the mean and standard deviation of the slope error increase as a function of the off-nadir angle.
• The local incidence angle has little effect on the elevation error, while the mean and standard deviation of the slope error decrease as a function of the local incidence angle.
• Internal validation shows that the hydrography is well preserved, with few (2%) and shallow sinks (0.7 m RMS) and good compliance with Horton’s law.

These results are based on the analysis of a particular dataset over a small area with a specific landscape. They will be consolidated by further studies over larger areas with variable morphologies. This multicriteria accuracy assessment shows the importance of using ad hoc quality criteria, as proposed in this study, and the high potential of the P-Band SAR data in depicting the topography under forests in comparison to state-of-the-art lidar reference, although limited by the slant range geometry for which the combination of ascending/descending passes could solve the problem, and this could be very meaningful in spaceborne acquisitions such as in BIOMASS mission.