Shipborne Mobile Photogrammetry for 3D Mapping and Landslide Detection of the Water-Level Fluctuation Zone in the Three Gorges Reservoir Area, China

Abstract

The water-level fluctuation zone (WLFZ) of the Three Gorges Reservoir is a serious landslide-prone area. However, current remote sensing methods for landslide mapping and detection in the WLFZ are insufficient because of difficulties in data acquisition and lack of facade information. We proposed a novel shipborne mobile photogrammetry approach for 3D mapping and landslide detection in the WLFZ for the first time, containing a self-designed shipborne hardware platform and a data acquisition and processing workflow. To evaluate the accuracy and usability of the resultant 3D models in the WLFZ, four bundle block adjustment (BBA) control configurations were developed and adopted. In the four configurations, the raw Global Navigation Satellite System (GNSS) data, the raw GNSS data and fixed camera height, the GCPs extracted from aerial photogrammetric products, and the mobile Light Detection and Ranging (LiDAR) point cloud were used. A comprehensive accuracy assessment of the 3D models was conducted, and the comparative results indicated the BBA with GCPs extracted from the aerial photogrammetric products was the most practical configuration (RMSE 2.00 m in plane, RMSE 0.46 m in height), while the BBA with the mobile LiDAR point cloud as a control provided the highest georeferencing accuracy (RMSE 0.59 m in plane, RMSE 0.40 m in height). Subsequently, the landslide detection ability of the proposed approach was compared with multisource remote sensing images through visual interpretation, which showed that the proposed approach provided the highest landslide detection rate and unique advantages in small landslide detection as well as in steep terrains due to the more detailed features of landslides provided by the shipborne 3D models. The approach is an effective and flexible supplement to traditional remote sensing methods.

1. Introduction

The Three Gorges Dam on the Yangtze River is the largest hydropower project to date in the world. The Three Gorges Reservoir Area (TGRA) subsequently formed by the project covers more than 20 county-level administrative regions with a total surface water area of 1080 km² at a water level of 175 m and a storage capacity of 39.3 billion m³ [1,2]. Due to the local topography, geomorphology, rock mass structure, valley structure, and climate conditions in the area, the TGRA was historically a serious landslide-prone area prior to the project [3,4,5]. The reservoir started impounding water in 2003, and the water level then reached 135 m, 156 m, and 172 m in 2003, 2006, and 2008, respectively. In 2010, the water level achieved its maximum designed value of 175 m for the first time. Since then on, the water-level fluctuates between 145 m and 175 m above sea level in an anti-seasonal manner (maintained at 175 m in winter and 145 m in summer), mainly in response to annual reservoir regulation, thus forming a water-level fluctuation zone (WLFZ) with a length of approximately 660 km and a total area of 349 km² [6].

As a result of the combined effect of the annual cyclic variation in the water level, rainfall, relocation, and other factors, various types of landside geohazards in the WLFZ occur frequently [2,4,6,7,8,9], posing a very large threat to the properties and lives of the local residents and the safety of the Yangtze River channel. Therefore, conducting landslide research specifically in the WLFZ is of paramount importance in terms of disaster prevention and control as well as for the safety of the reservoir.

As with recent advances in remote sensing data acquisition and processing techniques, remote sensing technology has been widely used in landslide mapping and detection [10,11]. Remote sensing data for landslide mapping and detection can be categorized into five classes by spectral range, including:

• Optical images, e.g., aerial photographs [12], very high-resolution satellite images [13,14,15], unmanned aerial vehicle (UAV) images [16,17,18,19,20,21], terrestrial close-range images [22,23], nighttime light data [24], geostationary satellite data [25], small satellite constellation images [26,27]);
• Thermal infrared data [28,29];
• Hyperspectral images [30];
• Light detection and ranging (LiDAR) data, e.g., airborne LiDAR [31,32,33,34,35], terrestrial laser scanning [36,37], boat-based mobile laser scanning [38];
• Synthetic aperture radar (SAR) data, e.g., spaceborne SAR [39,40,41,42], airborne SAR [43], ground-based SAR [44].

These data can be acquired by spaceborne Earth observation satellites, airborne platforms, and ground-based methods. Various types of remote sensing data can achieve multiscale mapping and detection of different forms of landslides from various perspectives. It should be noted that every type of remote sensing data and technology possesses pros and cons, and a sound choice of data and observation methods should thus be made according to the characteristics of the landslides and the scale of the study area along with the purpose of investigation [45,46]. This situation is especially true in the case that most of the current remote sensing observation methods are insufficient for landslide detection in the WLFZ of the TGRA due to the following aspects:

(1) It is very difficult to acquire usable spaceborne and airborne remote sensing data in the WLFZ because a complete mapping of the WLFZ must be carried out at a relatively low water level of 145 m at which the WLFZ is fully exposed between June and September every year. The wet season in the TGRA is from June to September; the weather conditions are often rainy, cloudy, and foggy during this period; and the water level may dynamically change and rise above 145 m due to the need for flood control.

(2) The terrain of certain segments of the WLFZ in the TGRA is very steep and thus cannot be observed vertically from spaceborne and airborne platforms. For example, the slope of the WLFZ in some segments of Wu Gorge is nearly 90 degrees, and cavities are often present along the WLFZ so that such key information for landslide interpretation is missing from aerial or satellite imagery.

(3) Traditional ground-based survey methods (e.g., close-range photogrammetry, ground-based InSAR, and terrestrial laser scanning) are mainly used for mapping and detecting single landslides and are thus unsuitable for landslide mapping and detection at the regional scale. In addition, the mobile laser scanning method can be used for landslide mapping and detection at the regional scale, but it is relatively more expensive.

This situation also exists in other steep-sided areas around the world, such as the southwest alpine and gorge region in China [47] and the widely distributed coastal cliffs [48]. Thus, a new approach is urgently needed for landslide mapping and detection in these areas to supplement the existing remote sensing technology.

It is well known that structure from motion (SfM) photogrammetry originating from computer vision and traditional photogrammetry can produce highly accurate and detailed three-dimensional (3D) landform models using overlapping images acquired from different perspectives with compact cameras and georeferencing information [49,50]; thus, it has been used widely in the geosciences [50,51,52,53]. In SfM photogrammetry, imagery can be acquired from a range of platforms [54], such as ground-based [55], pole [56], kite [57], blimp [58], UAV [59], and airplane [60] platforms. With the rapid development of UAV platforms in recent years [17,61], SfM photogrammetry with UAV imagery has become a feasible technique for landslide detection and monitoring. Specifically, oblique photogrammetry based on UAVs has been used in the TGRA [62,63] and other similar areas with steep terrain. On the other hand, the Three Gorges Reservoir is an important navigation channel of the Yangtze River, and a large number of ships travel through the Three Gorges Reservoir every day, which provides the possibility and convenience to acquire images from a shipborne platform for SfM photogrammetry. The development of the UAV-based SfM photogrammetry and the convenience of using ships in TGRA provide notable hints for the shipborne SfM photogrammetry. However, the ability of shipborne SfM photogrammetry to detect landslide in the WLFZ of the TGRA has not yet been proven.

Therefore, the aim of this paper is to introduce a low-cost shipborne SfM photogrammetry method to geomorphologists, provide a proof-of-concept investigation of the SfM approach to produce accurate landslide detection results in the WLFZ of the TGRA, and demonstrate the effectiveness of the new approach. The original contributions and novelties of this paper are summarized as follows:

(1) We propose a novel and effective shipborne mobile photogrammetry approach that is composed of an in-house self-designed hardware platform as well as a data acquisition and processing workflow for 3D mapping and landslide detection of the WLFZ in the TGRA.

(2) Due to the difficulties in obtaining ground control points (GCPs) in the field and inaccuracies in the raw onboard global navigation satellite system (GNSS) data, we carried out a comprehensive evaluation of the proposed approach in terms of georeferencing accuracy and usability of the 3D models in the WLFZ using four bundle block adjustment (BBA) control configurations in the data processing workflow: the BBA with the raw GNSS data, the BBA with the raw GNSS data and fixed camera height above the water level, the BBA with the GCPs extracted from the historical aerial photogrammetric products, and the BBA with the survey-grade mobile LiDAR point cloud as a control.

(3) The effectiveness of the proposed approach is fully investigated by comparison with high-resolution satellite images, aerial orthophotographs, and oblique aerial photogrammetric 3D models in terms of landslide detection ability through visual interpretation.

This study shows for the first time the use of shipborne SfM photogrammetry for 3D mapping and landslide detection of the WLFZ in the TGRA. The low cost and logistical simplicity of our approach makes it attractive for landslide studies in the reservoir areas.

2. Materials and Methods

2.1. Study Area

The study area is located in the WLFZ of the Changshou-Zigui segment in the TGRA along the Yangtze River and its watershed, as shown in Figure 1, and it includes the regions of Zigui, Badong, Wushan, Fengjie, Yunyang, Wanzhou, Zhongxian, Shizhu, Fengdu, Fuling, and Changshou. A section of the WLFZ in Wushan with a length of approximately 30 km was used as the assessment area for landslide detection ability assessment of the proposed approach.

Since 2010, the water level in the TGRA has fluctuated periodically between 145 m and 175 m above sea level, as demonstrated in Figure 2. Generally, the water level rises from 145 to 175 m from August or September to the beginning of November each year, starts to drop slowly from 175 m starting in December, and reaches the lowest point of 145 m in June of the following year. Therefore, the time for the full exposure of the WLFZ is very short, that is, from the beginning of June to late August or September. Unfortunately, this period is the wet season in the TGRA, and the water level may rise temporarily due to rainfall events. As a result, the time window for acquiring remote sensing images during low water levels is very short.

The TGRA is located in the transition zone between the second and third steps of China’s topography [3]. The landforms of the TGRA can be divided into two parts. The western part features low mountains formed by fold zones with a gentle slope less than 30°. In the eastern part, three portions consisting of limestone formed the narrow Three Gorges. Between the Three Gorges are two portions consisting of clastic rocks forming wide valleys. The mountain height in the gorge areas is approximately 1000–2000 m, and the river width in these areas is only 200–300 m. The bank slope of the gorge areas is mostly between 30 and 50°, but in some parts of the steepest valleys, such as Wu Gorge and Qutang Gorge, the slope is nearly 90°. Therefore, it is difficult to acquire valid information about the WLFZ in these steep valleys from vertical remote sensing images.

2.2. Data Acquisition

The shipborne photogrammetry platform was designed in-house for the purpose of mobile mapping along the river channel. In this section, we first introduce the system composition and function of our self-designed hardware platform, and then the system parameter configuration pertinent to the mapping process is presented and explained in detail in the context of data acquisition.

2.2.1. Shipborne Mobile Photogrammetry System

The design of our proposed shipborne mobile photogrammetry system, which was inspired by the UAV, is mainly composed of the ship platform, digital camera, and control system (as shown in Figure 3).

(1) Ship platform: The ship with the onboard camera and control system was driven by an operator and traveled along the water channel in the TGRA. Since the elevation range of the WLFZ is within 30 m, a lack of overlapping areas and coverage is likely to occur during image data acquisition, especially when the ship experiences large fluctuations. Therefore, the ship must have a high water discharge rate and high stability. The ship used in this study (Figure 4a) was the Geological Disaster Emergency Response and Investigation Ship, which can carry 22 people and is 20 m in length and 4.3 m in breadth, with a draft of 0.8 m (as shown in

Table 1. Main technical parameters of the ship and cameras.

Parameters	Specifications
Ship’s principal dimensions	length 20 m, breadth 4.3 m, draft 0.8 m
Camera model	Canon EOS 5D Mark III
Camera type	36 × 24 mm CMOS
Effective pixels	22.1 megapixels, 5760 × 3840
Focal lengths	24/35 mm
Pixel size	6.41 μm

) and a speed of 35 to 47 km per hour; the vessel was provided by the Chongqing Institute of Geological Environment Monitoring.

(2) Cameras: Two consumer-grade Canon 5D Mark III cameras (Canon Inc., Tokyo, Japan) with focal lengths of 24/35 mm, 22.1 megapixels, and pixel sizes of 6.41 μm were installed on the ship and linked to a shutter cable to achieve automatic exposure, as shown in Figure 4b–d. To increase the overlapping areas and avoid missing images along the travel direction, the two cameras were positioned side by side, and photos were taken at the same exposure time controlled by a shutter cable.

(3) Control system: The control system was composed of a monitoring and control station and an imaging control unit. The monitoring and control station is similar to the ground station for controlling a UAV and is responsible for mission planning, data replay, and remote control of camera exposure as well as monitoring the ship status. The imaging control unit integrates a low-cost GNSS module, inertial sensor, and an imaging trigger device that receives various remote control commands sent from the monitoring and control station through a radio link and then drives the cameras to be automatically exposed according to the predefined exposure parameters.

2.2.2. Shipborne Image Acquisition Parameters

By utilizing the developed shipborne mobile photogrammetric system, we collected 60,532 images along the Yangtze River and its main tributaries in the Changshou-Zigui segment of the TGRA, while the water level was approximately 145 m in 2017 and 2018.

The main technical parameters of the shipborne image acquisition are shown in

Table 2. Main technical parameters of the shipborne image acquisition.

Parameters	Specifications
	Range	Average
Distance between the ship and the bank (m)	30~400	100
Ground sample distance (m)	0.008~0.107	0.027
Image coverage height (m)	31~410	104
Image coverage width (m)	46~616	155
Camera exposure interval (m)	20~100	50
Forward overlap (%)	60~90	70
Ship’s speed (km/h)	25~38	30

. Along the tortuous coastline of the TGRA, the ship traveled in a relatively straight route that can be adjusted in real time according to the actual conditions of the river channel. The camera exposure positions representing the ship positions were projected onto the aerial orthophotographs of the TGRA, as shown in Figure 5, and it can be seen from Figure 5 that the distance between the ship and the reservoir bank was constantly changing. We measured the approximate distance between the ship and the bank at different locations in the ArcGIS software (ESRI Inc., Redlands, CA, USA). The results showed that the closest distance was only approximately 30 m in some tributaries, and the maximum distance was more than 400 m in the wide river channel of the main stream. The constantly changing distance led to a continuous change in shipborne image resolution ranging from 0.008 m to 0.107 m, which can be calculated based on the known focal length and pixel size of the camera. To ensure that the image forward overlap was high enough for image processing, the operator adjusted the camera exposure interval in the control station from time to time according to the distance between the ship and the bank during the image acquisition. The average camera exposure interval was approximately 50 m. The average forward overlap was approximately 70%.

2.3. Data Processing

2.3.1. Data Processing Workflow

The shipborne images were processed with an SfM photogrammetry workflow to produce point clouds and 3D models of the WLFZ. A number of SfM packages exist, ranging from web-based services to open-source and commercial software [51]. In this paper, we adopted a method built upon the framework that combined an effective BBA with the SfM and multi-view stereo (MVS) techniques to reconstruct the 3D models proposed by the authors of [64]. The method consists of three steps (Figure 6): tie point extraction and relative orientation, BBA, and 3D model reconstruction.

In the step of tie point extraction and relative orientation, the classic scale-invariant feature transform (SIFT) algorithm [65] and the epipolar constraint are applied. Then, the position and attitude of all images in the local coordinate system can be obtained, and a scalable model is built.

Subsequently, the BBA is used to determine the accurate camera parameters, including exterior orientation parameters (EOPs) and interior orientation parameters (IOPs), using the tie point observations and other given information (e.g., the GCPs, initial camera positions, and attitudes).

In the step of 3D model reconstruction, the well-known patch-based multi-view stereo (PMVS) algorithm and Poisson surface reconstruction algorithm were adopted [66].

Among the three steps, the BBA is the most important. The main principles of BBA can be described simply as follows [67]:

Suppose P is a 3D ground point, p is an image point corresponding to the ground point P, and S is the perspective center (camera lens center). Based on the fundamental geometric relationship, the 3D ground point P, the perspective center S and the corresponding image point p are on the same line, and the relationship can be described by Equation (1) [67]:

$$\left[\begin{array}{c}x-\Delta x\\ y-\Delta y\\ -f\end{array}\right]=\left[\begin{array}{ccc}{a}_1& b_1& c_1\\ a_2& b_2& c_2\\ a_3& b_3& c_3\end{array}\right]\left[\begin{array}{c}{X-X}_S\\ {Y-Y}_S\\ {Z-Z}_S\end{array}\right]$$

(1)

where x and y are the coordinates of the image point, f is the focal length of the camera, Δx, Δy are the correction terms for image point coordinates, and they can be expressed by IOPs, lens distortion parameters and principle point translation parameters, X, Y, and Z are the object space coordinates of the ground point P, X_S, Y_S and Z_S are the object space coordinates of the perspective center S, and a_i, b_i and c_i (i = 1, 2, 3) are the nine elements of the rotation matrix formed by three rotation angles phi, omega, and kappa.

Typically, the EOPs (X_S, Y_S, Z_S, phi, omega, kappa) and the object space coordinates of the ground point (X, Y, Z) are unknowns in a BBA. If uncalibrated cameras were used, then the IOPs (f, x₀, y₀, k₁, k₂) are also unknowns. The main parameters given are the observations of image point coordinates (x, y). In practice, a BBA is often executed along with the observations that include the tie points, all initial exterior orientation data as well as GCPs, if any [68].

2.3.2. Configurations of the BBA

The accuracy of the BBA can directly affect the accuracy of the reconstructed 3D models, and higher accuracy achieved in the BBA can significantly improve the efficiency of the MVS. The accuracy of the BBA largely depends on the accuracy of the initial exterior orientation data and the distribution of GCP networks. Because we used low-cost positioning devices as well as low-cost cameras in shipborne mobile photogrammetry systems, reliable and well-distributed GCP networks are essential for obtaining highly accurate BBA results. However, the topography of the WLFZ is very complex, which makes measuring GCPs in the field extremely difficult.

To improve the accuracy of the BBA and the final 3D models, we proposed three other BBA strategies in addition to the original BBA with the raw GNSS data. Therefore, we had a total of four BBA configurations that all belonged to a class of integrated sensor orientation (ISO) approaches [68] for shipborne image processing.

The main difference among the four BBA configurations was that different GNSS data and control strategies were applied. In Configuration 1, the raw GNSS data were used, and there were no GCPs. In Configuration 2, the improved GNSS data were used by fixing the height component of the raw GNSS data, and there were also no GCPs. In Configuration 3, the raw GNSS data and GCPs extracted from the historical aerial photogrammetric products were used. In Configuration 4, the raw GNSS data and GCPs extracted from the historical aerial photogrammetric products were used in the first step, and then the survey-grade mobile LiDAR point cloud was further used as a control.

(1) Configuration 1: BBA with the raw GNSS data

In this simplest configuration, the BBA was conducted using the tie points and the raw GNSS data acquired directly by the low-cost standalone navigation unit of the shipborne mobile photogrammetry system, without any GCPs.

(2) Configuration 2: BBA with the raw GNSS data and fixed camera height

This configuration was developed on Configuration 1 by fixing the height component of the raw GNSS data. A low-cost single-frequency GNSS receiver was used in the shipborne mobile photogrammetry system, and the morphology of some segments of the TGRA has steep valleys; therefore, the accuracy of the raw GNSS data was rather low. Figure 7 gives an example of the height component of the raw GNSS data acquired in Wu Gorge on 2 July 2017. The height component ranged from 127 m to 184 m above sea level and showed apparent errors. Because the water level of the TGRA on that day was approximately 147 m and the distance between the cameras and the water surface of the TGRA was approximately 3 m, the height of the cameras should be approximately 150 m. Even if the ship is bumped by small waves, the height of the cameras should vary between 148 m and 152 m, with no possibility of reaching 127 m or 184 m.

Under the condition that the water level of the TGRA at the time of data acquisition and the distance between the cameras and the water surface were known as prior knowledge, the initial camera height component was then fixed as a constant value in the BBA. This BBA configuration can be applied to all shipborne images in the TGRA.

(3) Configuration 3: BBA with the GCPs extracted from the historical aerial photogrammetric products

The Three Gorges Project is an important project in China such that aerial remote sensing surveys are carried out frequently for monitoring geological disasters in the TGRA, accumulating a wealth of aerial remote sensing data products, including high-precision digital elevation models (DEMs), digital orthophoto maps (DOMs), and oblique aerial photographic 3D models. The feature points in these historical aerial photogrammetric products can be used as GCPs in the BBA procedure.

In this paper, the WLFZ with a length of approximately 2 km in the Wuxia segment of the TGRA was selected for the purpose of accuracy assessment. This is because there were oblique aerial photogrammetric 3D models in this area acquired the same year as the shipborne images. The 3D models were produced using the ContextCapture Center software (Bentley Systems Inc., Exton, PA, USA) from the oblique aerial images acquired by a commercial oblique aerial camera on aircraft with a pixel resolution of approximately 0.15 m. Five evenly distributed features in the oblique photogrammetric 3D models that were also observable in the shipborne images were identified as GCPs. The three-dimensional coordinates of the feature points were read in Acute3DViewer software (Bentley Systems Inc., Exton, PA, USA). It should be noted that the accuracies of the five GCPs were not high enough, with planimetric errors and elevation errors of approximately 2.07 m and 0.46 m, respectively.

(4) Configuration 4: BBA with the survey-grade mobile LiDAR point cloud as a control

According to the cloud control photogrammetry theory proposed in [69], we can also use the LiDAR point cloud as the control of the BBA in addition to extracting control points from historical aerial images. The process of the BBA controlled by the LiDAR point cloud is essentially the process of registering the sparse image matching points resulting from the BBA to the LiDAR point cloud. Therefore, many algorithms for realizing the registration of optical images and LiDAR point clouds can be used for BBAs under the control of LiDAR point clouds, such as the methods proposed in references [70,71].

We adopted a LiDAR point cloud-assisted BBA method proposed by Song [72] in this paper. The main steps can be described simply as follows (Figure 8):

• Conduct the BBA with the GCPs extracted from the historical aerial photogrammetric products, which is Configuration 3 as mentioned above, to produce sparse matching points that are close to the LiDAR points.
• Preprocess the LiDAR points, including LiDAR point cloud segmentation and line feature extraction, to produce the LiDAR points with segmentation information and feature lines.
• Use the iterative closest point (ICP) algorithm to realize the registration of sparse matching points with the LiDAR points produced in step 2 to eliminate the systematic bias between these two data sets.
• Conduct BBA with the LiDAR feature points as controls to obtain the optimized exterior orientation parameters and the interior orientation parameters and update the object coordinates of the sparse matching points.
• Check if the accuracy is high enough (i.e., variation of the image orientation error is less than 0.001 pixels) to export the optimized image orientation parameters and sparse matching points. Otherwise, go to step 3.

During the acquisition of the shipborne images, we used a survey-grade mobile laser scanning system (Leica Pegasus: Two, Leica Geosystems AG, Heerbrugg, Switzerland) to synchronously acquire the LiDAR point cloud of the WLFZ in the Wuxia segment (Figure 9). The horizontal and vertical accuracies of the LiDAR point cloud were approximately 0.020 m and 0.015 m, respectively. The point cloud density was approximately 28 pts/m². We conducted Configuration 4 using shipborne Pegasus: Two LiDAR points as controls.

2.4. Accuracy Assessment

As a new technique for 3D mapping and landslide detection of the WLFZ, a comparison of the georeferencing accuracy of the 3D models with conventional methods is a prerequisite before it can be applied confidently. In this paper, in order to assess the georeferencing accuracy of the four shipborne 3D models produced by the four BBA configurations, the shipborne LiDAR point cloud acquired synchronously with the shipborne images, as shown in Figure 9, was used as the reference data for accuracy validation, the Wuxia segment of the WLFZ was selected as the accuracy validation area, and the mean error (MEAN), the root mean square error (RMSE), and the maximum error (MAX) were used as error metrics.

We extracted 10 CPs from the color LiDAR point cloud manually, compared the coordinates of the CPs in the shipborne LiDAR point cloud and the four shipborne 3D models, and then calculated the MEAN, RMSE, and MAX. The main steps are introduced as follows:

(1) Measure the 3D coordinates of the CPs from the shipborne LiDAR point cloud in the CloudCompare opensource software (version 2.11.3, http://www.cloudcompare.org/ (accessed on 1 December 2020)), $\widetilde{{\mathrm{C}}_i}\left({\tilde{X}}_i,{\tilde{Y}}_i,{\tilde{Z}}_i\right)$ (i = 1,2, …, n, and n is the number of CPs).

(2) Measure the 3D coordinates of the CPs from the generated shipborne photogrammetric 3D models in Acute3D Viewer software,${ \mathrm{C}}_i\left(X_i,Y_i,Z_i\right)$.

(3) Calculate the errors by comparing $\widetilde{{\mathrm{C}}_i}\left({\tilde{X}}_i,{\tilde{Y}}_i,{\tilde{Z}}_i\right)$ and ${\mathrm{C}}_i\left(X_i,Y_i,Z_i\right)$:

$${\Delta }_i={\mathrm{C}}_i\left(X_i,Y_i,Z_i\right)-\widetilde{{\mathrm{C}}_i}\left({\tilde{X}}_i,{\tilde{Y}}_i,{\tilde{Z}}_i\right)=\left(\Delta X_i,\Delta Y_i,\Delta Z_i\right),$$

(2)

where

$$\{\begin{array}{c}\Delta X_i=X_i-{\tilde{X}}_i\\ \Delta Y_i=Y_i-{\tilde{Y}}_i\\ \Delta Z_i=Z_i-{\tilde{Z}}_i\end{array},$$

(3)

then the planimetric errors $\Delta X{Y}_i$ can be calculated,

$$\Delta X{Y}_i=\sqrt{\Delta X_i^2+\Delta Y_i^2},$$

(4)

(4) Calculate the statistics of the errors of the CPs, including the MEAN, RMSE, and MAX,

$$MEAN=\left(\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta X_i,\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta Y_i,\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta X{Y}_i,\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta Z_i\right),$$

(5)

$$RMSE=\left(\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta X_i^2},\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta Y_i^2},\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta X{Y}_i^2},\sqrt{\frac{1}{n}{\displaystyle \sum }_{i=1}^n\Delta Z_i^2}\right),$$

(6)

$$MAX=\left(max\left({\displaystyle \sum }_{i=1}^n\left|\Delta X_i\right|\right),max\left({\displaystyle \sum }_{i=1}^n\left|\Delta Y_i\right|\right),max\left({\displaystyle \sum }_{i=1}^n\left|\Delta X{Y}_i\right|\right),max\left({\displaystyle \sum }_{i=1}^n\left|\Delta Z_i\right|\right)\right)$$

(7)

2.5. Landslide Detection Ability Assessment

In this paper, we evaluated the landslide detection ability of shipborne mobile photogrammetric 3D models by comparing them with high-resolution satellite images, aerial orthophotographs, and oblique aerial photogrammetric 3D models through visual interpretation. A section of the WLFZ in Wu Gorge where the coverage of the four kinds of remote sensing data sets overlapped was used as the assessment area with a length of approximately 30 km. Combining field surveys and visual interpretation of aerial photographs, detailed landslide inventory maps of the assessment area, including 26 landslides, were obtained. Then, the central geographical locations of the landslides were projected on different remote sensing data, and whether the landslides could be clearly recognized in the remote sensing data was determined by an experienced geomorphologist using a visual interpretation method. Finally, we counted the total number of landslides recognized by each remote sensing dataset, and then the detection rate of different remote sensing datasets was computed.

The characteristics of the remote sensing data used for landslide detection ability assessment are described as follows (

Table 3. Characteristics of the remote sensing data used for landslide detection ability assessment.

Data Name	Sensor	Acquisition Time	GSD
High-resolution satellite images	GF-2 satellite	2016.8	0.81 m
Aerial orthophotographs	UCXp-WA aerial camera	2017.9	0.50 m
Oblique photogrammetric 3D models	AMC 5100 oblique aerial camera	2017.5	0.15 m
Shipborne photogrammetric 3D models	self-designed shipborne photogrammetry system	2017.7	0.03 m

):

(1) High-resolution satellite images. The GF-2 satellite images acquired in August 2016 were used. The images were processed using PCI GXL software, including the procedures of image orthorectification, image registration, image fusion, etc., to generate a digital orthophoto with a ground sample distance (GSD) of 0.81 m.

(2) Aerial orthophotographs. Aerial photographs were acquired by a UltraCamXp Wide Angle (Vexcel Imaging GmbH, Graz, Austria) large format digital aerial camera. The raw image resolution at the datum plane (mean elevation plane) in the survey area was approximately 0.3 m. The aerial photographs were processed by Inpho photogrammetry software packages (Trimble Inc., Sunnyvale, CA, USA) to generate orthophotographs with a GSD of 0.5 m.

(3) Oblique photogrammetric 3D models. Oblique aerial photographs were acquired using a PC-6 fixed-wing aircraft equipped with an AMC5100 oblique aerial camera in 2017. For flight safety, the aircraft’s flight altitude was relatively high, and the image resolution on the terrain reference plane was approximately 0.15 m. The oblique aerial photographs were processed with ContextCapture Center software to generate 3D models of the survey area.

(4) Shipborne photogrammetric 3D models. The shipborne images were obtained by the shipborne mobile photogrammetry system described in this paper and processed using BBA Configuration 4. The average image resolution was approximately 0.03 m.

3. Results

3.1. 3D Models and Accuracy

The images acquired by the shipborne mobile photogrammetry system in the TGRA were processed with the data processing workflow and the four BBA configurations introduced in this paper. These four configurations could all obtain 3D models of the WLFZ in the TGRA. Figure 10 shows the shipborne photogrammetric 3D model of the left bank of Wushan from the Yangtze River Bridge to Hengshixi, and Figure 11 is the 3D model of the Gongjiafang segment of the WLFZ where a landslide-prone area exists. The 3D model clearly shows the geomorphic characteristics of the WLFZ.

In the accuracy validation area, four 3D models were obtained with the four different BBA configurations. Ten features were identified in the shipborne LiDAR data and used as accuracy CPs. The distribution of the CPs is shown in Figure 11. The error statistics under different data processing configurations are shown in

Table 4. Error statistics under different data processing configurations (Unit: m).

Point ID	Configuration 1				Configuration 2				Configuration 3				Configuration 4
	∆X	∆Y	∆XY	∆Z	∆X	∆Y	∆XY	∆Z	∆X	∆Y	∆XY	∆Z	∆X	∆Y	∆XY	∆Z
CP#1	5.98	−5.83	8.35	31.69	3.22	1.27	3.46	−1.35	−1.41	−1.27	1.90	0.84	−0.20	−0.42	0.46	−0.50
CP#2	5.60	−3.80	6.76	25.53	3.47	1.40	3.74	−1.26	−1.60	−0.78	1.78	0.66	−0.28	−0.08	0.29	−0.34
CP#3	4.97	−2.07	5.38	21.20	3.69	1.21	3.88	−1.70	−1.79	−0.70	1.92	0.07	−0.26	−0.21	0.33	−0.29
CP#4	6.09	−5.57	8.25	24.20	3.65	1.16	3.83	−1.38	−2.08	−0.63	2.17	0.69	−0.47	−0.16	0.50	−0.22
CP#5	4.84	−1.41	5.04	19.14	3.99	1.07	4.13	−1.21	−2.21	−0.46	2.26	0.18	−0.53	−0.28	0.60	0.05
CP#6	7.91	−10.84	13.42	29.47	3.80	0.56	3.84	−1.58	−1.65	−0.94	1.89	0.33	−0.64	−0.14	0.65	−0.72
CP#7	5.50	−3.15	6.34	20.45	4.12	0.60	4.17	−1.41	−1.94	−0.65	2.04	0.27	−0.67	−0.43	0.79	−0.15
CP#8	6.36	−6.99	9.45	24.58	4.02	0.32	4.03	−1.07	−1.69	−0.93	1.93	0.15	−0.59	−0.31	0.67	−0.19
CP#9	4.84	−2.10	5.27	19.76	4.12	0.40	4.14	−1.26	−1.75	−0.76	1.91	0.31	−0.39	−0.50	0.64	−0.14
CP#10	5.43	−4.17	6.85	23.81	4.16	0.30	4.17	−1.19	−1.86	−0.99	2.11	0.37	−0.39	−0.64	0.75	−0.70
MEAN	5.75	−4.59	7.51	23.98	3.82	0.83	3.94	−1.34	−1.80	−0.81	1.99	0.39	−0.44	−0.32	0.57	−0.32
RMSE	5.82	5.33	7.89	24.30	3.84	0.93	3.95	1.35	1.81	0.84	2.00	0.46	0.47	0.36	0.59	0.40
MAX	7.91	−10.84	13.42	31.69	4.16	1.40	4.17	−1.70	−2.21	−1.27	2.26	0.84	−0.67	−0.64	0.79	−0.72

, which shows that the accuracy of Configuration 1 is the lowest, and the accuracies of Configuration 2, Configuration 3, and Configuration 4 increase in ascending order.

The absolute georeferencing accuracies (indicated by the RMSE) of the 3D models generated under different processing configurations varied greatly, but the relative accuracies were still very high. For the length between the two endpoints shown in Figure 12, the measurement results in the 3D models generated under the four configurations were 199.96 m, 199.78 m, 199.68 m, and 199.45 m, respectively, which indicated small differences.

3.2. Landslide Detection Results

The total number of recognized landslides and the detection rates are illustrated in

Table 5. Landslide detection results of different remote sensing data.

Data Name	Number of Landslides Recognized	Detection Rate
High-resolution satellite images	12	46.15%
Aerial orthophotographs	19	73.08%
Oblique photogrammetric 3D models	25	96.42%
Shipborne photogrammetric 3D models	26	100%

. In the shipborne photogrammetric 3D models, all 26 landslides in the assessment area were recognized. The oblique aerial photogrammetric 3D models also achieved high detection results with 25 recognized landslides (i.e., a detection rate of 96.42%). In the high-resolution satellite images and aerial orthophotographs, only 12 and 19 landslides were recognized, respectively.

The number of landslides recognized by different remote sensing data mainly depends on the size and topographic characteristics of the landslides. We therefore divided the landslides in the assessment area into three categories, that is, large landslides, small landslides, and landslides in steep terrain areas, to further demonstrate the landslide detection abilities of different remote sensing data.

Large landslides (e.g., areas larger than 1000 m²), as displayed in Figure 13, are visually recognizable in all remote sensing data sets and can be identified directly as landslides without ground validation. However, it is relatively easier to identify landslides from 3D models generated by oblique aerial photographs and shipborne images because the 3D visualization effect of 3D models allows us to more easily detect topographic features distinctive of landslides [45].

For small landslides (e.g., areas less than 500 m²), as displayed in Figure 14, although some tone and texture features could be found from satellite images and aerial orthophotographs, it was still difficult to directly identify landslides based on these features because the pixels corresponding to the landslides were too few. However, in oblique photogrammetric 3D models and shipborne photogrammetric 3D models with higher resolution, the signature of landslides was very obvious, and landslides could be directly identified without field validation.

For the landslides in steep terrain areas (e.g., landslides in Wu Gorge where the slope is nearly 90°), as displayed in Figure 15, it was impossible to obtain valuable information from either the GF-2 satellite image or the aerial orthophotographs because of the vertical view angle. In the oblique photogrammetric 3D model, only partial features were visible. However, in the shipborne photogrammetric 3D model, a full picture of the landslide could be obtained.

4. Discussion

4.1. Discussion on the Accuracy

In this paper, four different configurations of the BBA were adopted in the shipborne image processing workflow. The accuracies of the resultant shipborne photogrammetric 3D models were assessed using the 10 CPs extracted from the shipborne LiDAR point cloud. Table 4 shows the planimetric errors and elevation errors of the 10 CPs under different data processing configurations, and the statistics (MEAN, RMSE, MAX) of the planimetric errors as well as the elevation errors. Overall, the absolute georeferencing accuracies of the four 3D models indicated by the RMSE gradually improved.

Since the self-designed shipborne mobile photogrammetry system used a low-cost single-frequency GNSS receiver, the georeferencing accuracy was rather low in Configuration 1, in which the raw GNSS data were directly used to assist the BBA without GCPs. The RMSE and the MAX of the planimetric errors were 7.89 m and 13.42 m, respectively, and the RMSE and the MAX of the elevation errors were 24.30 m and 31.69, respectively. This accuracy level was insufficient for locating landslides.

However, if the elevation component of the GNSS data is fixed according to the known water level and the distance between the camera and the water surface (Configuration 2), the georeferencing accuracy improved greatly. The RMSE and the MAX of the planimetric errors were 3.95 m and 4.17 m, respectively, and the RMSE and the MAX of the elevation errors were 1.35 m and −1.70 m. The comparison results between Configuration 1 and Configuration 2 showed that the accuracy of the GNSS data had a large influence on the accuracy of the shipborne photogrammetric 3D models when no GCPs were used. Recently, real-time kinematic (RTK) GNSS has been widely used in SfM photogrammetry. The RTK GNSS can provide more accurate initial camera positions and lead to more accurate photogrammetric products [73,74,75]. In the future, we will replace the currently used GNSS receiver with a pair of dual-frequency RTK GNSS receivers to update the hardware of our shipborne photogrammetry system.

When extracting the GCPs from the historical aerial images (Configuration 3), the accuracy further improved. The RMSE and the MAX of the planimetric errors were 2.00 m and 2.26 m, respectively, and the RMSE and the MAX of the elevation errors were 0.46 m and 0.84 m, respectively. Compared with the results reported in other literature related to GCP-assisted SfM photogrammetry (the RMSE was at approximately the centimeter level) [76,77,78], the accuracy level of Configuration 3 was still low. The main reason was that the accuracy of the GCPs extracted from the historical oblique photogrammetric 3D models was not high enough. The RMSEs of the planimetric errors and elevation errors of the GCPs were 2.07 m and 0.46 m, respectively. As a result, we can see from Table 4 that the planimetric and elevation errors of the CPs in Configuration 3 showed the characteristics of a systematic shift. Comparing the accuracy of Configuration 3 with GCPs extracted from the historical aerial photogrammetric 3D models, it could be seen that the shipborne photogrammetric 3D models generated by Configuration 3 could achieve georeferencing accuracy comparable to that of the aerial images that were used for extracting GCPs. Because historical aerial images are easy to obtain, Configuration 3 is the most practical configuration and convenient for implementing in large-scale engineering projects. It should be pointed out that obtaining GCPs from the historical satellite images is also a feasible option for BBA. However, the resolution and georeferencing accuracy of the satellite images are generally lower than the aerial images, thus the resultant accuracy of BBA with the GCPs from the satellite images will also be generally lower.

BBA Configuration 4 using the survey-grade mobile LiDAR point cloud as a control achieved the highest accuracy. The RMSE and the MAX of the planimetric errors were 0.59 m and 0.79 m, respectively, and the RMSE and the MAX of the vertical errors were 0.40 m and −0.72 m, respectively. Notably, the error statistics may also have errors. The point density of the shipborne LiDAR data was only approximately 28 pts/m², and the average point distance was approximately 0.19 m. The shipborne photogrammetric 3D models in the validation area had a higher resolution of approximately 0.03 m. Manual identification of CPs from these two data sets will inevitably result in small random errors. This kind of error existed in all four configurations.

Although the absolute georeferencing accuracies of the four configurations indicated by the RMSE varied significantly, as shown in Table 4, the four corresponding measured lengths of the red line between the two endpoints at the bottom of the landslide, as displayed in Figure 12 for the four 3D models, had small differences (199.96 m, 199.78 m, 199.68 m, and 199.45 m, respectively). The main reason for this phenomenon was that the errors of the four 3D models were mostly due to the systematic shift caused by the errors in the raw GNSS data or GCPs. It can be easily seen from Table 4 that each coordinate component of the errors has the same sign (+/−) in each configuration.

4.2. Discussion on the Landslide Detection Ability

Different types of remote sensing data had their own advantages and disadvantages in landslide detection, as illustrated in

Table 6. Advantages and disadvantages of different remote sensing data on landslide detection in the WLFZ.

Data Name	Advantages	Disadvantages
High-resolution satellite images	High data collection efficiency; Periodic repetitive observation; Containing near-infrared band.	Low spatial resolution; Vertical viewing angle.
Aerial orthophotographs	High data collection efficiency; High spatial resolution and accuracy.	Affected by weather conditions, time consuming; Vertical viewing angle.
Oblique photogrammetric 3D models	Ability to obtain the cliff face information of steep terrain areas; 3D visualization effect.	High cost; Very time consuming; Unable to obtain complete information of the cavities.
Shipborne photogrammetric 3D models	Very high resolution; Flexible, unaffected by airspace control, less affected by weather; Horizontal viewing angle; Low cost.	Limited by the river channel navigation conditions; Small field of view of the cameras.

. Shipborne mobile photogrammetry technology was not a substitute but was an effective supplement to traditional remote sensing technology. The integration of spaceborne, airborne, and ground-based multisource remote sensing data sets is an important development direction in the future.

In terms of landslide detection in the WLFZ, the detection rate of the shipborne 3D model was the highest (100%), followed by the oblique aerial photographic 3D models (96.42%), aerial orthophotographs (73.08%), and high-resolution satellite images (46.15%). The main reason for the low detection rate of the high-resolution satellite images and aerial orthophotographs was that most of the landslides in the assessment area were small landslides (e.g., the area was approximately 300 m² or smaller). These small landslides were difficult to recognize from low-resolution images. The oblique aerial photographic 3D models obtained very good landslide detection results at 96.42%. Most of the landslides could be recognized from oblique aerial photographic 3D models, with the exception of one landslide in steep terrain. However, it should be noted that the high detection rate of the oblique aerial photographic 3D models was due to the very few landslides in steep terrain in the assessment area. If the number of landslides in steep terrain increases, the detection rate of the oblique aerial photographic 3D models would become much lower.

The results also reflect the fact that the shipborne photogrammetric 3D model has significant advantages over traditional remote sensing data, especially in the identification of small landslides and landslides in steep terrain because of the horizontal view angle of the camera and the high resolution of the shipborne photogrammetric 3D models. For example, for small landslides, such as a bank collapse with an area of less than 300 m² (Figure 14), it is difficult for even experienced geomorphologists to distinguish the landslides from the GF-2 satellite images and aerial orthophotographs. However, the use of shipborne 3D models at extremely high spatial resolution is effective for the detection of landslides. For the areas with steep terrain, as shown in Figure 15, the landslides can be identified effectively only from the shipborne 3D models, while these landslides are completely invisible on the GF-2 satellite images and aerial orthophotographs and partly invisible on the oblique aerial photogrammetric 3D models.

In the area for landslide detection ability assessment, all 26 landslides were recognizable in the shipborne photogrammetric 3D models. However, if we take into consideration the landslides in other areas of the WLFZ, we find that the shipborne photogrammetric 3D models also have shortcomings in landslide detection, for example, limited by the river channel navigation conditions and the small field of view of the cameras. The ships are able to sail on the mainstream Yangtze River without obstacles, but in many tributaries (e.g., Caotang River, Shennv Stream), the water was very shallow or the river channel was blocked by dams or other artificial structures, so the ship was unable to enter the river channel. In addition, the vertical field of view of the shipborne cameras is small (approximately 53°), and the distance between the cameras and the bank is short in the narrow valley, so the shipborne images can cover only the limited extent of the WLFZ. It should be noted that some large landslides may cover a large area extending to high elevations and invisible to shipborne cameras; thus, it is impossible to analyze the landslides completely based on shipborne images. As illustrated in Figure 16, two landslides in the WLFZ of the Shennv Stream could not be surveyed by the shipborne photogrammetric method due to the shallow water level in the Shennv Stream. Moreover, a landslide in the WLFZ of the Yangtze River could not be completely imaged by shipborne cameras because the elevation in that area was too high.

Furthermore, the shipborne mobile photogrammetry method proposed in this paper was mainly oriented to the 3D mapping and landslide detection of the WLFZ, and it can also be used in the study of other ecological environmental problems, such as those related to vegetation [79] and soil erosion [80], in the WLFZ as a general topographic survey technique.

4.3. Comparison with Results in Other Research

The accuracy and landslide detection ability assessments in this work has proven that the proposed shipborne mobile photogrammetry approach was effective in landslide detection in the WLFZ of the TGRA. However, it should be pointed out that the proposed approach is not the only option for landslide detection in the WLFZ of the TGRA. Many other remote sensing techniques, which have been used in the river-channel mapping and cliff monitoring [81,82], can also be applied in the WLFZ of the TGRA, such as terrestrial SfM photogrammetry [83], portable mobile laser scanning [84], boat-based mobile laser scanning [38], helicopter-based photogrammetry [52], UAV photogrammetry [85,86,87], etc. Among them, the UAV photogrammetry was the most frequently used approach.

However, all these techniques have their own disadvantages compared with the shipborne mobile photogrammetry approach proposed in this paper especially when used in the WLFZ of the TGRA. For example, terrestrial SfM photogrammetry is mainly used for mapping a single landslide, and it will be very time consuming and labor intensive to map the whole WLFZ of the TGRA with this technique. With regard to the portable mobile laser scanning and boat-based mobile laser scanning, the operation cost is very high and the texture quality of the laser scanning is much worse than that of the photogrammetry models, which could be more useful for analysis and interpretation [88]. Furthermore, the helicopter-based photogrammetry is too dangerous to fly in the narrow river gorges and the operation cost is also very high. For UAV photogrammetry, despite the many advantages, the challenges still exist when using this platform in the investigation of the TGRA at regional scale. These challenges include: (a) considerable initial investment in operator training; (b) long time for pre-flight planning of the site [89]; (c) limitations posed by the local legislation and regulations; (d) easily affected by weather conditions such as high winds [83]; and (e) short flight time because of the drone battery life.

Taking into consideration that the TGRA is an important navigation channel of the Yangtze River, the local geological disaster prevention agencies inspect the reservoir bank regularly every year using the dedicated ships which can then be utilized to set up the shipborne platform. Thus, the proposed shipborne mobile photogrammetry approach offers distinct cost and logistical advantages over the abovementioned techniques.

5. Conclusions

In conclusion, aiming at the limitations of traditional remote sensing observation methods in landslide investigations in the WLFZ, such as difficulties in remote sensing data acquisition and lack of facade information in vertical remote sensing images, we proposed a novel shipborne mobile photogrammetry approach for 3D mapping and landslide detection of the WLFZ in the TGRA. In this approach, in-house self-designed hardware and a data acquisition and processing workflow were developed, and, subsequently, comprehensive accuracy and landslide detection ability assessments were carried out to prove the effectiveness and applicability of the proposed approach.

In the accuracy assessment of the resultant 3D models of the WLFZ, we developed and adopted four different configurations of the BBA procedure in the data processing workflow. The assessment results demonstrated that the BBA with the raw GNSS data provided a plane accuracy of RMSE 7.89 m and an elevation accuracy of RMSE 24.30 m, which were insufficient for locating landslides. The remaining three configurations gradually improved the accuracy, with the highest accuracy (e.g., 0.59 m in plane and 0.40 m in height) provided by the BBA with a survey-grade mobile LiDAR point cloud as a control. Moreover, the BBA with GCPs extracted from historical aerial photogrammetric products was the most practical configuration and convenient to implement, especially in large-scale landslide detection projects.

In the landslide detection ability assessment, we compared the shipborne 3D models generated by the proposed approach with high-resolution satellite images, aerial orthophotographs, and oblique photogrammetric 3D models in an area with 26 landslides using a visual interpretation method. The detection rates were 46.15%, 73.08%, 96.42%, and 100% for high-resolution satellite images, aerial orthophotographs, oblique photogrammetric 3D models, and shipborne 3D models, respectively. The comparison showed that the proposed approach provided the highest and unique advantages in small landslide detection as well as landslide detection in steep terrains due to more detailed features of landslides provided by shipborne 3D models; thus, it is an effective and flexible supplement to traditional remote sensing survey methods. However, the workflow was still limited by the river channel navigation conditions and the camera field of view so that the landslide information on the upper part of the WLFZ could not be obtained. Therefore, landslide detection in the WLFZ requires the combined use of shipborne mobile photogrammetry and aerial remote sensing technology.

Although the approach proposed in this paper has been proven to be effective and practical for 3D mapping and landslide detection in the WLFZ, there are still a few places left for improvement in the future. First, since a low-cost GNSS receiver was used in the proposed shipborne mobile photogrammetry system, the accuracy of the BBA with the raw GNSS data was very low. Therefore, the RTK GNSS system at the centimeter level can be used to improve the accuracy of the shipborne camera positioning, which can lead to more accurate 3D models. Second, the accuracy assessment of the shipborne 3D models in this paper was preliminary, and only the Wuxia segment of the WLFZ was selected as the validation area. A larger coverage of validation areas will help quantify the effect of differences in terrain features on the accuracy of the derived 3D models. Finally, a multi-temporal data set may be collected to facilitate the calculation of the volume and frequency of the landslides in the future.