PR-Alignment: Multidimensional Adaptive Registration Algorithm Based on Practical Application Scenarios

Abstract

In the present day, 3D point clouds are considered to be an important form of representing the 3D world. In computer vision, mobile robotics, and computer graphics, point cloud registration is a basic task, and it is widely used in 3D reconstruction, reverse engineering, among other applications. However, the mainstream method of point cloud registration is subject to the problems of a long registration time as well as a poor modeling effect, and these two factors cannot be balanced. To address this issue, we propose an adaptive registration mechanism based on a multi-dimensional analysis of practical application scenarios. Through the use of laser point clouds and RGB images, we are able to obtain geometric and photometric information, thus improving the data dimension. By adding target scene classification information to the RANSAC algorithm, combined with geometric matching and photometric matching, we are able to complete the adaptive estimation of the transformation matrix. We demonstrate via extensive experiments that our method achieves a state-of-the-art performance in terms of point cloud registration accuracy and time compared with other mainstream algorithms, striking a balance between expected performance and time cost.

1. Introduction

Among the many forms in which 3D data can be represented, depth images, polygonal meshes, and point clouds can be used to represent the real physical world [1,2]. In recent years, the maturity and widespread use of 3D point cloud data acquisition equipment, such as Lidar, as well as the improvement in computing capabilities, have led to the widespread use of point cloud data [3,4,5,6]. The point cloud data collected by different stations may have some deviations due to the differing angles and frames used to collect point cloud data, as shown in Figure 1. With point cloud alignment technology, it is possible to unify different frames of point clouds under the same coordinate system, which is an effective solution to this issue [7]. As an unstructured data form with occlusion [8], noise [9,10], and large amounts of data [11,12], point cloud registration has proven to be a challenging research direction in point cloud data processing. It needs to be performed precisely and quickly in order to achieve the desired results. In this paper, the main focus is on rigid registration of point clouds, which is the registration of point clouds by translation and rotation alone, without scaling or deformation. There are many applications of point cloud registration technology, such as simultaneous localization and mapping (SLAM) for mobile robots [13,14], high-quality map construction [15], environmental awareness for autonomous vehicles [16], forest structure parameter evaluation, city modeling, augmented reality (AR) scene detection, image synthesis for medical detection, construction monitoring, and as-built modeling, among others [17,18]. Alignment accuracy is often required in these application scenarios, as well as very fast alignment speeds in some cases.

The performance of these applications can be greatly enhanced by highly accurate and fast alignment algorithms. Currently, point cloud registration methods are categorized as either traditional non-learning alignment methods [19,20] or deep learning-based alignment methods [21,22,23]. The non-learning registration method, however, was found to have many problems during the implementation of the registration scheme, including an easy fall into local minima, poor generalization, time consumption, and the use of manual features for separating correspondences during the implementation phase. Moreover, the actual registration effect may be influenced by the designer’s experience and ability to adjust the parameters. There has been significant progress in deep learning in recent years, and more and more algorithms have improved their performance through deep learning, which has been widely applied to point cloud registration [24,25]. Speed improvements have been achieved by a large number of algorithms, and they have been shown to be robust to noise, low overlap, and occlusions. In the field of point cloud registration, learning-based methods have the potential to improve performance. While the deep learning method represented by PointNet improves the processing speed and robustness of the algorithm to a certain degree, it is often less mature than the non-learning method and is still primarily focused on simple object alignment. Furthermore, a number of issues also need to be considered, including the need for more computational resources, the difficulty of implementing it, the difficulty of interpreting it, and the need for extensive training.

Based on practical application requirements, this paper selects a more mature and easy-to-deploy non-learning approach as the research object. To overcome many of the problems associated with traditional methods, we propose an adaptive registration algorithm, PR-Alignment, that takes into account the actual application environment. By introducing laser point clouds and RGB image information, this method increases the dimensionality of the data. We use the NARF and FPFH methods to detect and describe 3D feature points in laser point clouds, establish a matching relationship between feature point pickup and local feature description, and create the geometric matching point set $M_{\mathrm{g}}$ by ranking the matching points from strong to weak. For RGB images, the SIFT algorithm is used to detect and describe the two-dimensional feature points and to complete the matching between the source and target frames. Following this, the two-dimensional matching relationship is mapped to the three-dimensional space using bilinear interpolation. Based on the ranking of the matches from strong to weak, the photometric matching point set $M_{\mathrm{p}}$ is generated. A mixed matching point set $M_{\mathrm{c}}$ is generated by combining the environmental information by fusing $M_{\mathrm{g}}$ and $M_{\mathrm{p}}$, and then a robust and effective estimation of the transformation matrix is achieved by combining a biased random sampling method and an adaptive evaluation criterion. Lastly, the registration process of 3D point clouds is completed by combining ICP. Through extensive validation experiments on the Semantic 3D dataset and our own collected dataset, we found that our proposed algorithm has significantly improved speed and accuracy.

Briefly, our contributions include:

• Data collection of the scene point cloud is carried out by using the UAV-borne Lidar system, and the point cloud is then preprocessed before being used for analysis. The structure composition and working principle of the point cloud data acquisition system was studied, a calibration experiment for the point cloud data acquisition system was designed and completed, and the internal parameters and relative positional relationship between the color camera and laser radar were determined. An experiment to acquire 3D point clouds of the scene has been completed, and a colored point cloud has been produced. Moreover, for the pre-processing of scene point cloud, the efficient organization of point cloud data is realized based on the data structure of a K–D tree, and an adaptive voxel grid down-sampling method is proposed to ensure a good and stable down-sampling effect. The unreliable outliers are eliminated by statistical filtering, and the point cloud normal vector is estimated based on PCA.
• An algorithm for coarse registration of point clouds based on improved RANSAC is studied. We use NARF and FPFH to detect, describe, and match 3D key points on 3D point clouds, along with the establishment of geometric homonym point pairs. For color images, SIFT is used to detect, describe, and match two-dimensional key points, and their values are obtained through bilinear interpolation, resulting in the establishment of photometric point pairs with the same name. Geometric matching and photometric matching are combined adaptively according to the judgment of the current scene category. Aiming at the deficiency of the traditional RANSAC algorithm, an improved RANSAC algorithm is proposed innovatively, which sets the bias of random sampling and establishes an adaptive hypothesis evaluation method, which shows that the validity and robustness of transformation matrix estimation are improved.
• A scene classification algorithm is studied. A binary validity template is proposed to filter out invalid information from the local description of the luminosity texture features and geometric structure features of the scene by using the LBP and CLBP operators, and the scene feature vector is extracted by computing the normalized statistical histogram of the LBP and CLBP operators. For scene classification, a three-class SVM is used, and the labeled dataset is constructed by using a combination of self-built point cloud data and third-party data.

2. Related Work

In the course of this paper, there are three components that are worth noting: ICP-variants, point-set alignment, and feature point matching.

ICP-variants. The iterative closest point (ICP) is one of the most classical algorithms for registering point clouds. It estimates the transformation matrix by iteratively searching for corresponding points and minimizing their overall distance [26,27,28,29]. As a result of calculating the Euclidean distance between two point clouds, the algorithm determines the corresponding points between them, constructs a rotation and translation matrix based on these points, estimates the error function between the transformed source and target point clouds, and iterates until the error is less than the set threshold. As a result of iterative calculation, the ICP algorithm has relatively good accuracy, but it is relatively slow because of its iterative nature [30,31]. The algorithm is highly dependent on the initial positions of the two point clouds, and it may result in a local optimum for the objective function. Several variants based on the ICP algorithm have been developed in order to address this problem [32,33,34]. The GICP introduces probabilistic information (in the form of a covariance array) and proposes a model of ICP that addresses both point-to-point and point-to-surface ICP, as well as surface-to-surface ICP [35,36]. In addition to constraints on the distance between points, NICP adds constraints on the normal vector and curvature of the surface on which the point cloud is located [37,38]. A comparison of the registration effects between the different methods is shown in Figure 2. In this paper, we present an improved RANSAC in which the sampling bias for geometric matching and photometric matching can be adjusted depending on the environment and in which adaptive evaluation criteria can be applied to the evaluation process.

Point-set alignment. For the alignment of 3D point clouds, several methods are available. ICP, NDT algorithms, and their variants require that the point clouds are already roughly aligned or have a priori poses such as IMU, GNSS, odometry, etc. Otherwise, coarse alignment is required to provide an initial pose for them. In the subsequent work, much attention will be devoted to finding more robust correspondence, such as point-to-point, face-to-face, and feature-to-feature correspondence [39,40]. By utilizing these correspondences, alignment parameters are estimated, thereby improving the accuracy of point cloud alignment and reducing the reliance on the correct initial poses. Another method of coarse alignment is based on the allometric four-point-cloud set (4PCS) under rigid transformation, where the transformation of some points is also the transformation of all points, and the most accurate transformation is obtained by verifying multiple sets of allometric sets. There are, however, several problems associated with these alignment methods, including poor alignment, long alignment times, and manual settings. The use of additional ground real gestures may facilitate the learning of geometric characteristics from point clouds in some learning methods [41,42,43]. The process of collecting these basic and accurate annotations, however, is not straightforward. In recent years, unsupervised learning methods have been developed, such as UR & R [44] and BYOC [45], which strengthen the training of cross-view geometry and visual consistency before hidden supervision registration takes place. However, the features used for correspondence extraction are either directly based on RGB data or are trained by pseudo-correspondence tags in visual correspondence, where visual and geometric cues are usually extracted independently without effective use of their correlations. Moreover, the coarse-matching algorithm for point clouds based on local attributes includes the detection, description, and matching of key points as well as an estimation of the transformation matrix based on random sampling consistency (RANSAC). Generally, it is feasible to combine three-dimensional geometric characteristics with two-dimensional light characteristics, using RANSAC to generate the point clouds across continuous frames based on the richness of both types of information [46,47]. It is noteworthy, however, that traditional RANSAC cannot be applied to the matching relationship between combining different methods due to an unbalanced base of geometric matching and optical matching, thus ensuring that the transformation matrix estimation is stable and effective. Consequently, this problem will become even more acute as the overlapping area decreases. Based on the environment characteristics, we establish geometric and photometric matching point pair sets, which can optimize the alignment effect and enable adaptive parameter adjustment at the same time.

Feature point matching. The use of features for matching is an important method for achieving coarse point cloud registration. The point feature histogram (PFH), developed by Rusu et al.  [48], estimates surface conditions using the normal vector between neighboring point clouds and point cloud features [49,50,51]. As a result of the improved fast point feature histogram (FPFH), the computation time for feature points has been further reduced [52,53]. Guo et al. proposed a three-dimensional shape context (3DSC) [54,55]. A vector value is used to establish a correspondence between different surface points, and its shape characteristics are described based on the value of the vector. In this paper, we use NARF [56,57], FPFH, SIFT, and bilinear interpolation methods for key point detection and description.

3. Methodology

3.1. Overview

Our proposed adaptive registration algorithm is based on practical application scenarios in order to address the problems associated with traditional registration methods, such as poor results and long registration times. We introduce the principle of 3D point cloud alignment by combining theory with pictures to demonstrate that the most important part of this paper is the solution of the optimal transformation matrix T (Section 3.2).

After point cloud data and RGB images have been extracted, geometric matching point pair sets $M_{\mathrm{g}}$, photometric matching point pair sets $M_{\mathrm{p}}$, and mixed matching point pair sets $M_{\mathrm{c}}$ are generated (Section 3.3.1). Then, we classify the scene according to the characteristics of RGB image and point clouds (Section 3.3.2). An adaptive evaluation criterion and a biased random sampling method are then used to obtain an efficient and robust estimation of the transformation matrix (Section 3.3.3). The full pipeline is illustrated in Figure 3.

3.2. 3D Registration Model

As part of the registration, point clouds from different perspectives obtained are merged into the same three-dimensional coordinate system in order to obtain a complete geometric model [58,59,60,61], as shown in Figure 4. It is possible to represent the rigid transformation of the model in the space coordinate system using a rotation–translation matrix, referred to as T in this paper. For the dynamic point cloud $P=\left\{p_i\right\},i=1,2,\cdots ,N_p$ and the static point cloud $Q=\left\{q_j\right\},j=1,2,\cdots ,N_q$ to be registered, the optimal Euclidean transformation matrix T must be obtained between the two point clouds in order to make the corresponding points of the overlapping parts of the two point clouds completely coincide in the spatial coordinate system. There are six undetermined parameters in the transformation matrix, namely, the translation amounts $T_x,T_y,T_z$ along the three coordinate axes and the rotation angles $\alpha ,\beta ,\gamma$ around the three coordinate axes. The transformation matrix T can be expressed as follows without taking into account the size of the model:

$$T=R_X{R}_Y{R}_{Z}S$$

(1)

For Equation

$${\mathit{R}}_X=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& cos\alpha & sin\alpha & 0\\ 0& -sin\alpha & cos\alpha & 1\\ 0& 1& 0& 1\end{array}\right]$$

(2)

$${\mathit{R}}_Y=\left[\begin{array}{cccc}cos\beta & 0& -sin\beta & 0\\ 0& 1& 0& 0\\ sin\beta & 0& cos\beta & 0\\ 0& 0& 0& 1\end{array}\right]$$

(3)

$${\mathit{R}}_Z=\left[\begin{array}{cccc}cos\gamma & sin\gamma & 0& 0\\ -sin\gamma & cos\gamma & 0& 0\\ 0& 0& 1& 0\\ 0& 1& 0& 1\end{array}\right]$$

(4)

$$\mathit{S}=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ T_X& T_Y& T_Z& 1\end{array}\right]$$

(5)

In an ideal situation, the distance between the point $T\left(p_i\right)$ obtained by transforming a point $p_i$ in P and its corresponding point $q_j$ in Q should be 0. As a matter of fact, their distance can be expressed as follows:

$$d_i={∥T\left(P_i\right)-Q_j∥}_2.$$

(6)

Nevertheless, because of measurement errors and noise, the distance between corresponding points cannot reach the ideal value of 0, so the registration problem is transformed into an optimization problem, namely, finding the optimal transformation matrix to minimize the Euclidean distance between $T\left(p_i\right)$ and $q_j$:

$$E_i\left(\mathit{T}\right)=\underset{T}{min}\left[\left|T\left(P_i\right)-Q_j\right|\right].$$

(7)

In conjunction with the adaptive registration algorithm proposed in this paper, the optimal transformation matrix T is subsequently determined.

3.3. Adaptive 3D Registration for PR-Alignment

3.3.1. Feature Correspondences

Using a UAV-borne Lidar or TLS scanning systems, the object can be scanned with high-density 3D point clouds and color RGB images. For the pass-through filtering of point clouds, a reliable depth range should be selected. It is critical to detect and describe 3D key points using normal aligned radial features (NARF) and FPFH, establish the matching relationships between key points in accordance with the local feature description, and sort from strong to weak based on the geometric matching point pair set $M_{\mathrm{g}}$’s geometric matching. RGB images represent the photometric texture information of the scene, while SIFT is used to detect and describe the two-dimensional key points and to match the RGB images of the source frame and the target frame. By means of bilinear interpolation, 3D information is obtained from key points, and the 2D matching relationship is translated into 3D space. Based on the matching relationship, the photometric matching point pair set $M_{\mathrm{p}}$ is sorted from strong to weak, as shown in Figure 5.

In some instances, the geometric information and the photometric information of the scene are not balanced or rich. According to the relative richness of the two types of information, this experiment categorizes the scenes into three categories: scenes with richer photometric information, marked as −1; scenes with relatively balanced geometric information and photometric information, marked as 0; scenes with richer geometric information, marked as +1. Depending on the current scene category, the geometric matching point pair set $M_{\mathrm{g}}$ and the photometric matching point pair set $M_{\mathrm{p}}$ are combined differently. The combined mixed-matching point pair set $M_{\mathrm{c}}$ can be expressed as follows:

$$\left|M_{\mathrm{c}}\right|=\left|M_{\mathrm{cg}}\right|+\left|M_{\mathrm{cp}}\right|=\mathrm{ceiling}\left(\alpha \left|M_{\mathrm{g}}\right|\right)+\mathrm{ceiling}\left[(1-\alpha )\left|M_{\mathrm{p}}\right|\right]$$

(8)

In Equation (8): $|·|$ is the base of the set; ceiling (·) is an up-integer function; $M_{\mathrm{cg}}$ and $M_{\mathrm{cp}}$ are subsets of geometric and photometric matching points in $M_{\mathrm{c}}$, respectively; and $\alpha$ is the combination coefficient corresponding to different scene categories. Based on Equation (8), the first matching point pairs of ceiling $\left(\alpha \left|M_{\mathrm{g}}\right|\right)$ of $M_{\mathrm{g}}$ and ceiling $\left[(1-\alpha )\left|M_{\mathrm{p}}\right|\right]$ of $M_{\mathrm{p}}$ are selected to be merged into $M_{\mathrm{c}}$. Their respective values are 0.2, 0.5, and 0.8 when the scene category is −1, 0, and +1, as suggested by the literature [47,62]. It is recommended to select fewer geometric matches and more photometric matches in a sequence when facing a scene that contains richer photometric information; in situations where geometric information and photometric information are relatively equal, select geometric matching first followed by photometric matching; whenever a scene contains richer geometric information, more geometric matches are selected sequentially over fewer photometric matches. In particular, $\alpha$ = 0 and $\alpha$ = 1 represent two edge cases where $M_{\mathrm{c}}$ can only be formed with full photometric or geometric matching.

As the object environment is often cluttered, occluding, repetitive, etc., and there is a certain non-overlapping area between keyframes, many false matches are encountered. In the event of mismatching in the transformation matrix, the RANSAC algorithm must be used in order to eliminate the mismatch and make an accurate estimation of the matrix. A traditional RANSAC algorithm is employed in the literature [63,64,65,66]. For each iteration, a number of matching point pairs are randomly sampled and used to solve the transformation matrix assumption $T_{\mathrm{h}}$ and calculate the interior point number $\left|p_{\mathrm{h}}\right|$ and interior point error $\left|e_{\mathrm{h}}\right|$. $T_{\mathrm{h}}$ is considered to be an optimal candidate if $p_{\mathrm{h}}$ is greater than $p_{\mathrm{th}}$, which is the threshold for a reasonable assumption of the transformation matrix. As soon as the iteration ends, the optimal candidate with the smallest $e_{\mathrm{h}}$ is selected as the output of the estimation of the transformation matrix.

The $\left|M_{\mathrm{c}}\right|$ subset is formed by merging the $M_{\mathrm{g}}$ and $M_{\mathrm{p}}$ subsets. The bases of $M_{\mathrm{g}}$ and $M_{\mathrm{g}}$ are mainly determined by the number of key points detected. Since NARF and SIFT have different algorithmic design and application objects, $\left|M_{\mathrm{g}}\right|$ is often much smaller than $\left|M_{\mathrm{p}}\right|$. Therefore, for $\left|M_{\mathrm{c}}\right|$ in the random sampling stage, regardless of the scene category, geometric matching is always less likely to be obtained than photometric matching. In other words, when solving $\left|T_{\mathrm{h}}\right|$, the participation of the two matching types is not well adapted to differing classification results. In addition, since $\left|M_{\mathrm{g}}\right|$ is much smaller than $\left|M_{\mathrm{p}}\right|$, $\left|M_{\mathrm{c}}\right|$ will have a large difference when $\alpha$ takes different values. In order to take into account $\left|M_{\mathrm{c}}\right|$ under a different $\alpha$, it is necessary to set $\left|p_{\mathrm{th}}\right|$ to a smaller value according to the situation of $\alpha$ = 0.8. But when $\alpha$ = 0.2 or $\alpha$ = 0.5, due to the significant increase in $\left|M_{\mathrm{c}}\right|$, a reasonable transformation matrix assumption needs the support of more interior points. Thus, the preset smaller $\left|p_{\mathrm{th}}\right|$ does not guarantee the rationality of the optimal candidate, and the final transformation matrix estimation is not highly reliable. The purpose of this paper is to propose an adaptive registration algorithm based on practical application scenarios. On the basis of the original RANSAC algorithm, this method has been improved by adjusting the sampling weight for geometric and photometric matching according to the scene category, as well as by adopting an adaptive evaluation standard to achieve a robust and effective estimation of the transformation matrix.

3.3.2. Scene Classification

As shown in Figure 6, the flow of the algorithm for scene classification is illustrated. To extract the feature quantity of the current scene, a valid template is combined with the local binary pattern (LBP) operator and the curvature local binary pattern (CLBP) operator. The eigenvectors are input into the three-category support vector machine (SVM) to obtain the classification results.

Two-dimensional local texture features can be described using the LBP operator. It encodes the texture features of the local area by comparing the value of the central pixel of the grayscale image with its neighboring pixels. The improved ${\mathrm{LBP}}_{8.1}^{\left(\mathrm{riu}2\right)}$(the number of sampling points is 8, and the sampling radius is 1 pixel) is encoded in a uniform mode, which is invariant to rotation, descriptive, efficient, and robust. By taking $g_c$ as the value of the central pixel and $g_i(i=0,\cdots ,7)$ as the value of neighboring pixels, the coded value v of ${LBP}_{8.1}^{(\mathrm{riu}2)}$ can be expressed as follows:

$$v=\left\{\begin{array}{c}\sum _{i=0}^{7}s\left(g_i-g_c\right),\mathrm{if}u⩽2\hfill \\ 9,\mathrm{otherwise}\hfill \end{array}\right.$$

(9)

$$u=\left|s\left(g_7-g_{\mathrm{c}}\right)-s\left(g_0-g_{\mathrm{c}}\right)\right|+\sum _{i=1}^7\left|s\left(g_i-g_{\mathrm{c}}\right)-s\left(g_{i-1}-g_{\mathrm{c}}\right)\right|$$

(10)

$$s\left(x\right)=\left\{\begin{array}{cc}1,\hfill & x⩾0\hfill \\ 0,\hfill & x<0\hfill \end{array}\right.$$

(11)

In essence, CLBP approximates the local surface curvature using a second order differential to describe three-dimensional local structural features. The only difference between CLBP and LBP is that CLBP is a gradient magnitude map applied to the point cloud image. Consequently, ${\mathrm{CLBP}}_{8.1}^{(\mathrm{riv}2)}$ can be obtained easily from ${\mathrm{LBP}}_{8.1}^{(\mathrm{riu}2)}$.

The scene classification algorithm uses LBP and CLBP to extract scene feature vectors. In the following description, only LBP and CLBP are referenced. In order to obtain the LBP image from the RGB image, the grayscale image must be converted, and each pixel must be encoded with LBP; the gradient magnitude image of the point cloud image is obtained using the Sobel operator, and the CLBP image is obtained by encoding each pixel with CLBP. As pixels in the dataset that are invalid or beyond the reliable depth range are considered unavailable, it is necessary to filter the LBP and CLBP images. This study constructs a binary validity template: first, the original template $S_{init}$ with the same size as the point cloud image is established. The $S_{init}(x,y)$ value should be 0 if the pixel point $(x,y)$ of the point cloud image is not available, otherwise $S_{init}(x,y)$ should be 1. Since the calculation of LBP and CLBP is related to the neighboring pixels and CLBP acts on the gradient magnitude map of the point cloud image, $S_{init}$ is corroded with a $5\times 5$ convolution kernel to obtain the validity template S. The pixels of the LBP and CLBP maps must be scanned to generate the LBP and CLBP statistical histograms, and the pixels with $S(x,y)=0$ must not be counted. As a result, the statistical histograms of both LBP and CLBP are normalized and simply connected, and the resulting 20-dimensional vector is used as the scene’s feature vector. Considering that there is still a large overlap between the source frame and the target frame, only the feature vector extracted from the source frame can be used to represent the scene features.

The scenes are classified using a three-class SVM in this study. The SVM is a supervised learning algorithm that must be trained on a labeled dataset in order to be used for classification purposes. Consequently, this paper employs category labeling for several frames in the self-constructed dataset and the Semantic 3D dataset to construct a dataset for training the classifier. For the constructed dataset, feature vectors are extracted using LBP and CLBP for SVM training and testing.

3.3.3. Random Sampling and Adaptive Evaluation

The method proposed in this paper employs a biased random sampling method and an adaptive evaluation criterion for estimating the transformation matrix for the mixed-matching point pair set $M_{\mathrm{c}}$.

At the random sampling stage, a random variable X is introduced that obeys a Bernoulli distribution whose parameter is $\alpha$. Whenever X = 1, randomly select a pair of geometrically matched points from $M_{\mathrm{cg}}$; when X = 0, randomly select a photometric matching point pair from $M_{\mathrm{cp}}$. In this case, the probability of selecting a geometric match is $\alpha$ and the probability of selecting a photometric match is 1−$\alpha$. This means that the sampling bias of the two types of matching is appropriate for the scene category: when the scene category is −1, the luminosity matching will be more involved in the calculation of the transformation matrix assumption $T_{\mathrm{h}}$; when the scene category is 0, the luminosity matching will be more involved in the calculation of the transformation matrix assumption $T_{\mathrm{h}}$, whereas geometric matching and photometric matching take part in the calculation of $T_{\mathrm{h}}$ with the same probability, while geometric matching takes part in calculation more often when the scene category is +1.

In the hypothesis evaluation stage, the internal point number $p_{\mathrm{best}}$ and the internal point error $e_{\mathrm{best}}$ of the current optimal transformation matrix hypothesis $T_{\mathrm{best}}$ are used as references, and the upper limit ratio parameter ${\beta }_{\mathrm{ub}}$ and the lower limit ratio parameter ${\beta }_{\mathrm{lb}}$ are introduced to form an adaptive evaluation standard $\left(0⩽{\beta }_{\mathrm{ub}}⩽1,0⩽{\beta }_{\mathrm{lb}}⩽1\right)$. The investigation of $T_{\mathrm{h}}$ is divided into three levels based on the internal point number $p_{\mathrm{h}}$ and the internal point error $e_{\mathrm{h}}$ of $T_{\mathrm{h}}$. As $p_{\mathrm{h}}>\left(1+{\beta }_{\mathrm{ub}}\right)p_{\mathrm{best}}$ is the acceptance domain, it is considered that there are quite a few $p_{\mathrm{h}}$, and accepting $T_{\mathrm{h}}$ directly is therefore the current optimal solution. The $\left(1-{\beta }_{\mathrm{lb}}\right)p_{\mathrm{best}}<p_{\mathrm{h}}⩽\left(1+{\beta }_{\mathrm{ub}}\right)p_{\mathrm{best}}$ field is the confirmation field, and it is considered that there are enough $p_{\mathrm{h}}$ and that $T_{\mathrm{h}}$ represents a reasonable assumption regarding the transformation matrix. However, further comparison between $e_{\mathrm{h}}$ and $e_{\mathrm{best}}$ is required to verify whether $T_{\mathrm{h}}$ represents the current optimal transformation matrix. Using $p_{\mathrm{h}}⩽\left(1-{\beta }_{\mathrm{lb}}\right)p_{\mathrm{best}}$ as the rejection domain, we will think that $p_{\mathrm{h}}$ is too small and reject $T_{\mathrm{h}}$ directly. In accordance with experience, this study will require ${\beta }_{\mathrm{ub}}={\beta }_{\mathrm{lb}}=0.1$. Considering the fact that $p_{\mathrm{best}}$ and $e_{\mathrm{best}}$ are used as reference standards, the evaluation standard becomes self-adaptive when faced with $M_{\mathrm{c}}$ formed from different combinations of variables. The introduction of ${\beta }_{\mathrm{ub}}$ and ${\beta }_{\mathrm{lb}}$ adds the following features. The investigation of any possible $T_{\mathrm{h}}$ tends to compare the number of interior points. In order to increase the number of interior points of $T_{\mathrm{best}}$, when $p_{\mathrm{best}}$ is small, the scope of confirmation domain is small. The scope of the investigation of $T_{\mathrm{h}}$ tends to compare the number of interior points when $p_{\mathrm{best}}$ is small. The confirmation domain is broader when $p_{\mathrm{best}}$ is larger, and the investigation of $T_{\mathrm{h}}$ is generally a composite comparison of the number of interior points and the number of interior point errors in order to reduce the interior point error of $T_{\mathrm{best}}$ as long as sufficient interior point support is provided. As a result of such characteristics, $T_{\mathrm{best}}$ retained at the end of the iteration has more interior points, which is a reasonable assumption of the transformation matrix and produces a smaller error in the interior points. In addition to being universal, this evaluation criterion also applies to the boundary cases of $\alpha$ = 0 and $\alpha$ = 1.

The pseudocode for the adaptive 3D registration method for PR-Alignment can be seen in Algorithm 1, and the process of iteration should begin after the initialization has been completed. A biased random sampling is performed on $M_{\mathrm{c}}$ in one iteration, from steps 3 to 12, in order to generate a random pair of point pairs $M_{\mathrm{r}}$. Steps 13 to 15 involve the use of the singular value decomposition (SVD) algorithm to solve $T_{\mathrm{h}}$ determined by ${\mathrm{M}}_{\mathrm{r}}$, and to determine ${\mathrm{p}}_{\mathrm{h}}$ and ${\mathrm{e}}_{\mathrm{h}}$. The aim of steps 16 to 22 is to determine whether ${\mathrm{T}}_{\mathrm{h}}$ is the current optimal transformation matrix with adaptive evaluation criteria. Iterations are terminated when the number of iterations reaches the preset upper limit or when $e_{\mathrm{best}}$ is small enough. The final transformation matrix estimate is calculated by recalculating the transformation matrix determined by all interior points under $T_{\mathrm{best}}$ in steps 25 to 26.

Algorithm 1 The pseudocode for the adaptive 3D registration method for PR-Alignment.

Require: Input the mix-and-match point-pair set $M_{\mathrm{c}}$ Ensure: Output the rotation–translation matrix T. 1: Initialize ${\mathit{T}}_{\mathrm{best}}\leftarrow \mathit{I},p_{\mathrm{best}}\leftarrow 0,e_{\mathrm{best}}\leftarrow \infty$, it $\leftarrow 0$; 2: while  $it<i{t}_{max}$  and  $e_{\mathrm{best}}\ge e_{\mathrm{th}}$do 3:     $X\sim BE\left(\alpha \right)$; 4:     $M_{\mathrm{r}}\leftarrow \varphi$; 5:     while $\left|M_{\mathrm{r}}\right|<3$ do 6:         if $X=1$ then 7:            Make a random selection from $M_{\mathrm{cg}}$ of correspondences; 8:         else 9:            Make a random selection from $M_{\mathrm{cp}}$ of correspondences; 10:         end if 11:         put the correspondence into $M_{\mathrm{r}}$; 12:     end while 13:     calculate $T_{\mathrm{h}}$ for $M_{\mathrm{r}}$; 14:     transform the source keypoints in $M_{\mathrm{c}}$ by $T_{\mathrm{h}}$; 15:     traverse $M_{\mathrm{c}}$ and calculate $p_{\mathrm{h}}$ and $e_{\mathrm{h}}$; 16:     if $p_{\mathrm{h}}>\left(1+{\beta }_{\mathrm{ub}}\right)p_{\mathrm{best}}$ then 17:         ${\mathit{T}}_{\mathrm{best}}\leftarrow {\mathit{T}}_{\mathrm{h}},p_{\mathrm{best}}\leftarrow p_{\mathrm{h}},e_{\mathrm{best}}\leftarrow e_{\mathrm{h}}$ 18:     else if $\left(1-{\beta }_{\mathrm{lb}}\right)p_{\mathrm{best}}<p_{\mathrm{h}}\le \left(1+{\beta }_{\mathrm{ub}}\right)p_{\mathrm{best}}$ then 19:         if $e_{\mathrm{h}}<e_{\mathrm{best}}$ then 20:            ${\mathit{T}}_{\mathrm{best}}\leftarrow {\mathbf{T}}_{\mathrm{h}},p_{\mathrm{best}}\leftarrow p_{\mathrm{h}},e_{\mathrm{best}}\leftarrow e_{\mathrm{h}}$ 21:         end if 22:     end if 23:     $it\leftarrow it+1$; 24: end while 25: calculate T for all the inliers of $T_{\mathrm{best}}$; 26: return  T

In contrast to the traditional RANSAC algorithm, the adaptive 3D registration method presented in this paper considers the particular composition of $M_{\mathrm{c}}$, provides two types of matching corresponding sampling weights based on scene category, and determines the optimal transformation matrix assumption using an adaptive evaluation standard to improve the robustness and effectiveness of the transformation matrix estimation.

As a result of the above initial registration of point clouds, the target and source point clouds have roughly coincided, but there is still a certain distance error between the two. The aim of this paper is to further improve the registration accuracy of point clouds by employing an iterative optimization algorithm, ICP, based on the principle of optimal matching. In order to minimize the overall distance between the matching points of the two sets of point clouds, the rigid body transformation matrix is calculated, and the relative pose is adjusted, thereby allowing the two sets of point clouds to be continuously estimated in terms of their corresponding point sets. Whenever the registration error is less than the preset value or the number of iterations reaches the preset value, the iteration ends, and the specific principle is as follows:

• Perform coarse registration on the source point cloud P in order to obtain a new point cloud $P^{\mathrm{T}}$ that needs to be registered. For each point $p_i^{\mathrm{T}}$ in $P^{\mathrm{T}}$, find the point qj closest to it in the target point cloud Q as the corresponding point pair $\left(p_i^{\mathrm{T}},q_{\mathrm{j}}\right)$.
• Estimate the correspondence between corresponding point pairs in order to calculate the rigid body transformation matrix. Using the transformation matrix, we can obtain the average distance error D between the point in the transformed point cloud $P^2\mathrm{T}$ and the corresponding point in the target point cloud Q.

  $$\left\{\begin{array}{c}{P}^{2T}=T\times P^T\hfill \\ D=\frac{1}{n}\sum _{i=1}^n{∥p_i^{2T}-q_i∥}^2\hfill \end{array}\right.$$

  (12)
• In order to obtain the final rigid body transformation matrix, repeat the above steps until the set error threshold $D_{\mathrm{th}}$ is reached or the maximum number of iterations $I_{\mathrm{th}}$ is reached.

4. Experiments

The purpose of this section is to provide additional details regarding the implementation of the experiments and the datasets used in the experiments. An overview of the sources and information about the datasets used in the experiments is presented (Section 4.1), as well as metrics for evaluating the experimental results (Section 4.2). In addition, tables and images are presented as part of the presentation and discussion of the experimental results (Section 4.3).

4.1. Datasets and Experimental Conditions

4.1.1. Datasets

• Self-built datasets. Three-dimensional scanning was conducted at the National Ski Jumping Center and Biathlon Ski Center for the 2022 Beijing Winter Olympics using existing airborne Lidar systems and ground-based Lidar systems (as shown in Figure 7). Laser point cloud data and RGB image sets were included in the collected data. According to the characteristics of the data and the collection environment, we have organized these data into four major categories and eight subcategories, with a total data volume of 43GB. In

  Table 1. Lidar system parameters. In the process of acquisition, various parameters of the Lidar system are taken into consideration.

  Category	ULS	TLS
  Model	DJI M600 pro+UAV-1 series	Faro Focus 3D X130
  Measuring Distance	3–1050 m	0.6–120 m
  Scan Angle	360°	360° (horizontal), 300° (vertical)
  Measurement Rate	10–200 lines/s	976,000 points/s
  Angular Resolution	0.006°	0.009°
  Accuracy	±5 mm	±2 mm
  Ambient Temperature	−10–+40 °C	−5–+40 °C
  Wavelength	1550 nm	1550 nm

  , we present the configuration parameters of the acquisition equipment that we used.
• Third-party datasets. Additionally, in order to verify the universality of the method proposed in this paper, we selected the third-party database Semantic 3D for algorithm verification experiments. There are eight semantic classes that cover a wide range of urban outdoor scenes, such as churches, streets, railway tracks, squares, villages, football stadiums, and castles. Approximately four billion hand-labeled points have been evaluated, and the subversions are constantly being updated.

4.1.2. Experimental Conditions

The detailed software and hardware configurations for the performed verification experiment are shown in

Table 2. Platform configuration. A suitable test platform is selected whose software and hardware meet the requirements of the experiment.

Category	Configuration
Software	Operating System	Win 11
	Point Cloud Processing Library	PCL 1.11.1
	Support Platform	Visual Studio 2019
Hardware	CPU	Intel(R)Core(TM)i5-9400F
	Memory	DDR4 32 GB
	Graphics Card	Nvidia GTX 1080 Ti 11 GB

.

4.2. Evaluation Metrics

Methods are based on measuring the mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) between the true and predicted values. In an ideal situation, all of these error metrics should be zero if the rigid alignment is flawless. Additionally, all angles in our results are measured in degrees.

4.3. Results and Analysis

The experiment consisted of selecting three sets of data from the third-party dataset and our self-built dataset as the experimental objects. Accordingly, we selected birdfountain-station1-xyz-intensity-rgb (Bird), domfountain-station1-xyz-intensity-rgb (Dom), and sg27-station1-intensity-rgb (Sg27) from the third-party dataset. In our self-constructed dataset, we selected three-dimensional data from China’s National Ski Jumping Center (SJC), National Cross-Country Skiing Center (CSC), and National Biathlon Skiing Center (BSC). Several small samples of data from each group were selected from various perspectives as experimental objects due to the large amount of data in each group. A representation of the complete raw data, laser point cloud, and RGB information for each of the six sample sets are shown in Figure 8.

Our next step was to perform a registration test on Bird, Dom, and Sg27 in sequence in the third-party database. As part of the testing process, we used RANSAC [67], FGR [68], NDT [69], GICP [70], NICP [70], and our proposed method to test and verify the registration effect. As can be seen in Figure 9, a specific registration effect can be seen.

As shown in Figure 9, when using Bird data to test, FGR has the worst registration effect, followed by RANSAC, and our proposed method and NICP have excellent results. During testing with Dom data, RANSAC has the poorest registration performance, while the proposed method and NICP are still effective. Using Sg27 data, the overall registration effect is poor, and the NICP registration effect is the best, followed by the proposed method. To quantitatively assess the registration effect of each group, we use MSE, RMSE, and MAE as measurement standards, and

Table 3. Evaluation of results from third-party databases. Based on third-party test samples, we evaluated the performance of our method and other similar methods. (The ↓ indicates that the smaller the value, the better; bold indicates the best performance; underline indicates the second-best performance.)

Methods	MSE(R)↓			RMSE(R)↓			MAE(R)↓			MSE(t) (×${10}^{-2}$)↓			RMSE(t) (×${10}^{-2}$)↓			MAE(t) (×${10}^{-2}$)↓
	Bird	Dom	Sg27	Bird	Dom	Sg27	Bird	Dom	Sg27	Bird	Dom	Sg27	Bird	Dom	Sg27	Bird	Dom	Sg27
RANSAC	91.867	174.339	95.075	9.584	13.203	9.750	6.483	8.586	6.580	0.006	0.037	0.007	0.077	0.192	0.083	0.062	0.116	0.065
FGR	231.879	74.285	145.049	15.227	8.618	12.043	9.75	5.919	7.913	0.002	0.008	0.002	0.048	0.093	0.048	0.032	0.071	0.036
NDT	15.923	6.379	208.834	3.990	2.525	14.451	3.179	2.253	9.310	0.002	0.003	0.032	0.040	0.051	0.179	0.026	0.038	0.122
GICP	4.706	21.739	118.729	2.169	4.662	10.896	2.006	3.586	7.246	0.001	0.002	0.002	0.035	0.041	0.042	0.025	0.032	0.032
NICP	2.844	2.699	2.782	1.686	1.642	1.668	1.626	1.586	1.609	0.031	0.006	0.003	0.176	0.079	0.054	0.105	0.059	0.041
OURS	1.774	1.859	6.556	1.332	1.363	2.561	1.189	1.253	2.276	0.001	0.001	0.001	0.023	0.037	0.023	0.018	0.030	0.015

and Figure 10 provide evaluation results for Bird, Dom, and Sg27. We can conclude that RANSAC and FGR perform the worst, while the proposed method and NICP perform better. In light of the fact that these groups of point clouds selected as test data represent a small range of scenes, NICP demonstrates a stronger registration effect. As a result of various specific data, our method has always been able to maintain a high level of registration accuracy. According to the experimental results of this group, the method proposed in this paper has an optimal rate of 83.33% compared to other methods included in the study.

Additionally, we performed sequential registration tests on the self-built databases for SJC, CSC, and BSC. We tested and verified the registration effect using RANSAC, FGR, NDT, GICP, NICP, and our proposed method during the testing process. As shown in Figure 11, the specific registration effect is illustrated.

According to Figure 11, RANSAC performed the worst in the testing process on SJC data, while GICP and the proposed method performed relatively well. Our testing of the CSC data has shown that FGR and RANSAC have poor registration effects, while our method has the best results. As a result of testing on BSC data, the NICP algorithm performs the worst, while our method and GICP achieve good results. Together, our method achieves excellent performance when performed on SJC, CSC, and BSC datasets. The experimental results of each group were quantified and compared using quantitative statistics, as shown in

Table 4. Evaluation of the results of the self-built database. On the basis of our self-built test samples, we evaluated the performance of our method and other similar methods. (A smaller value indicates better performance, a bolded value indicates the best performance, and an underlined value indicates the second-best performance.)

Methods	MSE(R)↓			RMSE(R)↓			MAE(R)↓			MSE(t) (×${10}^{-2}$)↓			RMSE(t) (×${10}^{-2}$)↓			MAE(t) (×${10}^{-2}$)↓
	SJC	CSC	BSC	SJC	CSC	BSC	SJC	CSC	BSC	SJC	CSC	BSC	SJC	CSC	BSC	SJC	CSC	BSC
RANSAC	148.107	76.566	47.062	12.169	8.750	6.860	7.986	5.996	4.890	0.195	0.066	0.006	4.413	2.567	0.08	1.642	0.709	0.054
FGR	41.707	134.866	32.836	6.458	11.613	5.730	4.653	7.663	4.223	0.039	8.464	0.002	1.989	29.093	0.044	0.445	2.487	0.031
NDT	17.813	2.986	16.496	4.221	1.728	4.061	3.319	1.677	3.223	0.032	0.019	0.001	1.798	1.394	0.026	0.202	0.284	0.018
GICP	2.947	34.933	2.596	1.716	5.910	1.611	1.653	4.330	1.556	0.004	0.049	0.001	0.636	2.207	0.032	0.759	0.540	0.027
NICP	9.867	1.993	63.956	3.141	1.411	7.997	2.673	1.328	5.568	0.012	0.004	0.034	1.097	0.176	0.185	0.815	0.145	0.123
OURS	1.667	1.930	2.436	1.291	1.389	1.561	1.013	1.296	1.506	0.001	0.003	0.001	0.018	0.179	0.023	0.004	0.120	0.015

and Figure 12. As a result of our comparative analysis, we can conclude that the test results are similar to those of the third-party database. Among the six methods that participated in the test, the registration effect achieved by our proposed method was the best. The large range of scenes in these test sets of point clouds makes NICP’s registration performance unpredictable, whereas GICP is more stable. When comparing the experimental results of this group with other methods participating in the experiment, the method presented in this paper has a comprehensive optimal rate of accuracy close to 100%.

In addition to the registration effect, we also measured the effectiveness of each group of methods and calculated statistics and comparisons of the measurement results, as shown in

Table 5. Time consumption comparison. Various methods are used sequentially to register selected samples, record the respective time, and analyze the results. (There is an indication of the best performance in bold, while there is an indication of the second-best performance in underline.)

Methods	Time(s)
	Bird	Dom	Sg27	SJC	CSC	BSC
RANSAC	46.293	31.584	65.791	43.626	55.418	30.084
FGR	76.575	60.527	78.614	64.571	50.008	54.225
NDT	89.402	124.051	166.569	103.366	88.629	63.748
GICP	94.035	84.504	108.608	77.547	80.644	66.313
NICP	80.224	35.271	70.547	48.514	61.522	38.693
OURS	48.669	28.053	62.181	46.296	48.158	25.466

and Figure 13. From this, we can see that NDT and GICP have the longest processing times, which greatly reduces the efficiency of point cloud registration. RANSAC and our method have the shortest registration times. In the testing process of six groups of samples, our method takes the shortest time in 66.67% of the cases, compared to other methods.

In conclusion, compared with methods such as RANSAC, FGR, NDT, GICP, and NICP, our method balances the contradiction between registration effectiveness and processing time while maintaining good alignment performance when dealing with third-party datasets such as Bird, Dom, and Sg27, or self-built datasets such as SJC, CSC, and BSC.

5. Conclusions

Based on the type of application scenario, we propose an adaptive registration algorithm that addresses the problems of long registration times and poor modeling accuracy in the mainstream point cloud registration method. As a result of analyzing the laser point cloud, we are able to extract the geometric matching relationship of the target object, and we are also able to extract the photometric matching relationship of the target by processing the RGB image, thus increasing the dimension of the feature point data. In addition, we also added scene classification information, combined with geometric matching and photometric matching, to improve the RANSAC algorithm and complete the adaptive estimation of the transformation matrix.

We selected three samples from Semantic 3D and the self-built dataset (Bird, Dom, Sg27, SJC, CSC, BSC) for testing experiments in order to verify the effectiveness and robustness of our method. As well as comparing the accuracy of each group of algorithms in terms of MSE, RMSE, and MAE, we also compared the processing time for each group of algorithms for six samples. The results of our experiments indicate that the transformation matrix estimate calculated by our method can provide a useful initial value for the fine registration algorithm, allowing the algorithm to maintain high processing efficiency while achieving high precision.