Editorial on Special Issue “Techniques and Applications of UAV-Based Photogrammetric 3D Mapping”

1. Introduction

Recently, 3D mapping has begun to play an increasingly important role in photogrammetric applications. In the last decade, unmanned aerial vehicle (UAV) images have become one of the most critical remote sensing data sources because of the high flexibility of UAV platforms and the extensive usage of low-cost cameras. The techniques and applications of UAV-based photogrammetric 3D mapping are undergoing explosive development, which can be observed from the adopted cutting-edge techniques, including SfM (Structure from Motion) for offline image orientation, SLAM (Simultaneous Localization and Mapping) for online UAV navigation, and the deep learning (DL) embedded 3D reconstruction pipeline.

This Special Issue includes a collection of papers that mainly focus on the techniques and applications of UAV-based 3D mapping. There are a total of 13 papers published in this Special Issue, which range from review papers on recent techniques to research papers for feature detection and matching, false match removal, camera self-calibration, SfM-based image orientation, MVS-based (Multi-view Stereo) dense point cloud generation, building façade model reconstruction, and other related applications in varying fields. The details of each paper will be described in the following section.

2. Overview of Contributions

Yao et al. [1] gave a review of recently reported learning-based methods for wide-baseline image matching, which includes approaches involving feature detection, feature description, and end-to-end image matching. By using benchmark datasets, some typical methods have also been evaluated in this study. The paper reveals that no algorithm can adapt to all wide-baseline images and the generalization ability of learning-based methods should be improved by expanding training data or combining different model design strategies.

For robust and accurate image orientation, Huang et al. [2] proposed a camera self-calibration solution for long-corridor UAV images, such as transmission lines. The proposed solution combines two novel strategies for parameter initialization and high-precision GNSS fusion, in which the former is implemented by an iterative camera parameter optimization algorithm, and the latter is achieved by inequality constrained bundle adjustment. The validation of the proposed solution was verified by using four UAV images that are recorded from transmission corridors. The experimental results demonstrate that the proposed solution can alleviate the “bowl effect” for weakly structured long-corridor UAV images and achieve high precision in absolute orientation when compared with other methods.

In SfM-based image orientation, match pair selection is a key step, which can improve the efficiency of feature matching and decrease the involvement of false matches. In the work of Xiao et al. [3], a progressive structure-from-motion technique was designed to cope with false match pairs retained from repetitive patterns and short baseline images, which iteratively selects initial matches by extracting minimum spanning trees and cycle consistency inference. They verified the validation of the proposed algorithm by using UAV images.

After SfM-based image orientation, multi-view stereo is used to resume dense point clouds. Considering the 3D reconstruction of fine-scale power lines, Huang et al. [4] designed an efficient PatchMatch-based dense matching algorithm, which improves the steps of random red–black checkerboard propagation, matching cost computation, and depth map fusion. When compared with the traditional PatchMatch algorithm, speedup ratios ranging from 4 to 7 were achieved in the tests for transmission corridor UAV images. In addition, the proposed algorithm can improve the completeness of reconstructed power towers and lines.

In contrast to dense matching of normal objects, Zhou et al. [5] proposed a dense matching algorithm, termed DP-MVS, for detail-preserving 3D reconstruction. DP-MVS is achieved by using detail-preserving PatchMatch for the depth estimation of individual images and detail-aware surface meshing to reconstruct final models. The proposed algorithm can cope with the 3D modeling of thin objects, such as communication towers and transmission corridors, and it is 4 times faster than other methods in the dense matching of benchmark datasets.

Zhang et al. [6] presented a newly developed method for automatically generating 3D regular building façade models from the photogrammetric mesh model using the contour as the main cue. The contours tracked on the mesh are grouped into trees and segmented into groups to represent a topological relationship of building components. Then, each component of the mesh is iteratively abstracted into cuboids and the parameters of each cuboid are adjusted to be close to the original mesh model.

Wang et al. [7] proposed a U-Shaped Residual Network for Lightweight Image Super-Resolution (URNet), which applies to low-computing-power or portable devices. Firstly, a more effective feature distillation pyramid residual group (FDPRG) is proposed to extract features from low-resolution images. Then, a step-by-step fusion strategy is utilized to fuse the features of different blocks and further refine the learned features. To capture the global context information, a lightweight asymmetric non-local residual block is introduced. In addition, to alleviate the problem of smoothing image details caused by pixel-wise loss, a simple but effective high-frequency loss function is designed to help optimize the model.

In their study, Wang et al. [8] developed a workflow to extract building 3D information from GF-7 multi-view images. The workflow consists of four main steps, namely building footprint extraction from multi-spectral images, point cloud generation from the stereo image pair with SGM matching, normalized digital surface model (nDSM) generated from the point cloud, and building height calculation. Among the four steps, the main contribution is the multi-stage attention U-Net (MSAU-Net) designed for building footprint extraction. The experiments based on a study area in Beijing show the RMSE between the estimated building height and the reference building height is 5.42 m, and the MAE is 3.39 m.

The study by He et al. [9] proposed a novel approach to achieve CityGML building model texture mapping by multi-view coplanar extraction from UAV or terrestrial images. They first utilized a deep convolutional neural network to filter out object occlusion (e.g., pedestrians, vehicles, and trees) and obtain building-texture distribution. Then, point-line-based features are extracted to characterize multi-view coplanar textures in a 2D space under the constraint of a homography matrix, and geometric topology is subsequently conducted to optimize the boundary of textures by combining Hough-transform and iterative least-squares methods. This approach can map the texture of 2D terrestrial images to building façades without the requirement of exterior orientation information.

To deal with the problem that some existing semantic segmentation networks for 3D point clouds generally have poor performance on small objects, Liu et al. [10] presented a Spatial Eight-Quadrant Kernel Convolution (SEQKC) algorithm to enhance the ability of the network for extracting fine-grained features from 3D point clouds. Based on the SEQKC, they designed a downsampling module for point clouds, and embed it into classical semantic segmentation networks (PointNet++, PointSIFT, and PointConv) for semantic segmentation. As a result, the semantic segmentation accuracy of small objects in indoor scenes can be improved.

Ran et al. [11] presented a building multi-feature fusion refined network (BMFR-Net) to extract buildings accurately and completely. BMFR-Net was based on an encoding and decoding structure, mainly consisting of two parts: the continuous atrous convolution pyramid (CACP) module and the multiscale output fusion constraint (MOFC) structure. The CACP module was positioned at the end of the contracting path and the MOFC structure performed predictive output at each stage of the expanding path and integrated the results into the network.

Hu et al. [12] presented an automated modeling approach that could semantically decompose and reconstruct the complex building light detection and ranging (LiDAR) point clouds into simple parametric structures, and each generated structure was an unambiguous roof semantic unit without overlapping planar primitive. The method begins by extracting roof planes using a multi-label energy minimization solution, followed by constructing a roof connection graph associated with proximity, similarity, and consistency attributes. Then, a progressive decomposition and reconstruction algorithm was introduced to generate explicit semantic subparts and hierarchical representation of an isolated building.

Zheng et al. [13] made a digital subsidence model (DSuM) for deformation detection in coal mining areas based on airborne light detection and ranging (LiDAR). Noise points were removed by multi-scale morphological filtering, and the progressive triangulation filtering classification (PTFC) algorithm was used to obtain the ground point cloud. The DEM was generated from the clean ground point cloud, and an accurate DSuM was obtained through multiple periods of DEM difference calculations. Then, data mining was conducted based on the DSuM to obtain parameters such as the maximum surface subsidence value, a subsidence contour map, the subsidence area, and the subsidence boundary angle.

3. Conclusions

This Special Issue aims to attract a collection of papers that focus on the recent techniques for UAV-based 3D mapping, especially for trajectory planning for data acquisition in complex environments, recent algorithms for feature matching, SfM and SLAM for efficient image orientation, the usage of DL techniques in 3D mapping, and the applications of UAV-based 3D mapping. Furthermore, this Special Issue hopes to promote and inspire further research in the field of UAV-based photogrammetric 3D mapping.