Affine Iterative Closest Point Algorithm Based on Color Information and Correntropy for Precise Point Set Registration

In this paper, we propose a novel affine iterative closest point algorithm based on color information and correntropy, which can effectively deal with the registration problems with a large number of noise and outliers and small deformations in RGB-D datasets. Firstly, to alleviate the problem of low registration accuracy for data with weak geometric structures, we consider introducing color features into traditional affine algorithms to establish more accurate and reliable correspondences. Secondly, we introduce the correntropy measurement to overcome the influence of a large amount of noise and outliers in the RGB-D datasets, thereby further improving the registration accuracy. Experimental results demonstrate that the proposed registration algorithm has higher registration accuracy, with error reduction of almost 10 times, and achieves more stable robustness than other advanced algorithms.


Introduction
With the rapid development of 3D reconstruction in various fields, such as image processing [1][2][3][4][5][6], machine vision [7][8][9][10][11], and simultaneous localization and mapping (SLAM) [12][13][14], point set registration becomes more and more important, which is a key technique in image registration. The purpose is to match point sets from two or more images obtained at different times, different environments, or in different sensors. After geometric transformations, such as rotation and translation, the transformed point set is consistent in spatial expression. However, in practical applications, there are still several problems in point set registration, including (1) the correspondence between the two sets of points is unknown; and (2) there is insufficient information for point set matching when the local structures are unsalient. Besl and McKay [15] proposed an iterative closest point (ICP) algorithm, which is an efficient algorithm for solving the problem of rigid registration with two point sets.
In recent years, with the widespread application of the ICP algorithm, many scholars have carried out a lot of research around the ICP algorithm to improve it from the aspects of speed, convergence, and robustness. Among them, Blais et al. [16] adopted the method of random sampling of the original point set to boost the registration speed, but could not guarantee the accuracy of registration. Fitzgibbon [17] used the Levenberg-Marquardt algorithm to make the ICP registration algorithm converge faster. Sharp et al. [18] studied the ICP registration algorithm based on invariant features to decrease the probability of the algorithm falling into a local minimum and improve the robustness of the algorithm. However, the algorithm cannot solve the problem of a lot of noise interference in the dataset. For the registration problem of datasets with noise and outliers, Chetverikov et al. [19] proposed a TriICP algorithm to suppress noise and outliers. Phillip et al. [20] improved the trimmed ICP algorithm, which further improved the accuracy of registration. Myronenko et al. [21] proposed the Coherent Point Drift (CPD) algorithm, which used a probabilistic method to register the point sets. The model point set was represented by a Gaussian Mixed Model (GMM), and the target point set was regarded as the observation of the model point set. When these two point sets are registered, the correspondence is the maximum GMM posterior probability, which overcomes noise and outliers. Yang et al. [22] proposed a branch and bound solution based on searching the entire space SE (3), which solved the problem that the ICP algorithm was easily trapped in local extreme values. Although the accuracy of registration is improved, its calculation speed becomes slower. Du et al. [23] established a probabilistic model of the ICP algorithm and proposed the PICP algorithm to improve the robustness of the ICP algorithm.
In addition, some learning-based approaches have been proposed to address partial overlap registration [24][25][26][27]. For example, Zeng et al. [28] proposed a fast rigid registration method based on deep learning for 3D reconstruction, which can also be extended to other registration. RPNet [29] is less sensitive to initialization and more robust for point cloud registration. The network of this method obtains the corresponding soft allocation of points to solve the local overlap of point clouds problem. Deep closest point [30] uses dynamic graph neural network for feature extraction, and uses an attention module to generate features to establish correspondence. OMNet [31] estimates the overlapping masks to reject non-overlapping regions for partial-to-partial point cloud registration. Similarly, FINet [32] leverages the dual branches feature interaction and achieves better registration results. Yu et al. [24] presented a rotation invariant transformer to better address low overlap registration with large rotations. Gu et al. [25] proposed a recurrent framework for the 3D motion estimation to avoid the irregular data structure. Hu et al. [26] proposed a novel deep learning framework to generate omnidirectional 3D point clouds of human bodies by registering the front-and back-facing partial scans captured by a single-depth camera. Sun et al. [27] designed an end-to-end 3D graph deep learning framework of point cloud registration, which can simultaneously learn the detector (graph attention expression) and the descriptor (graph deep feature) for point cloud registration in a weakly supervised way, so that the learned detector and descriptor promote each other in the process of model optimization. Qin et al. [33] proposed Geometric Transformer to learn geometric feature for robust superpoint matching, which encodes pair-wise distances and triplet-wise angles, making it robust in low-overlap cases and invariant to rigid transformation.
However, in the actual application scene, the images to be registered often have some deformation. Therefore, the non-rigid registration problem has become another hot issue for many scholars. For affine registration problems, Feldmar et al. [34] tried to find the best global affine transformation by introducing differential information. Linear combinations of neighbor and similar rigid transformations are used to obtain more accurate local affine transformations between point sets. Amberg et al. [35] constrained the objective function by adding rigid regular terms and labeled terms to solve the affine transformation. Du et al. [36] put forward an affine registration algorithm based on Independent Components Analysis (ICA) to achieve accurate point set registration.
With the development of RGB-D imaging technology, many researchers pay more attention to the color information of the collected data [37]. Men et al. [38] added color information to the spatial information, which improved the registration accuracy and accelerated the convergence speed of the algorithm. Korn et al. [39] proposed an ICP algorithm based on Lab color information based on the Generalized-ICP algorithm, which improved the robustness of registration.
Although the above-mentioned algorithms improve the traditional registration algorithm from various aspects, they cannot match RGB-D data with a lot of noise and slight deformation well. In real scenes, the RGB-D data often contain a large number of moving objects, and the moving objects have a slight deformation during data collection, and the RGB-D data often have some noise and outlier interference, which reduces the registration accuracy. To solve these problems, this paper proposes a precise affine registration with color information and correntropy. First, this paper introduces color information into the algorithm as a feature to further improve the more accurate and reliable correspondence.
Secondly, to solve the interference of noise and outliers in the dataset, this paper introduces the correntropy as a distance measurement method, which improves the registration accuracy. Experimental results demonstrate that our registration algorithm has higher registration accuracy and more stable robustness than other registration algorithms.
Overall, the contributions of this paper are summarized as: • We introduce color information into affine point cloud registration, which can increase the robustness of the algorithm. • we introduce the robust correntropy metric to address outliers and noises in the point clouds for more accurate registration.
The rest of this paper is organized as follows. Among them, in Section 2, a review of the traditional affine ICP algorithm. In Section 3, the precise affine registration with color information and correntropy is proposed. In Section 4, we use simulated data and real data to conduct experiments and analyze the experimental results. At last, the conclusion is summarized in the last section.

A Review of the Traditional Affine ICP Algorithm
The traditional affine ICP algorithm is a very widely used registration algorithm [40][41][42]. We usually define two point sets Then, we can establish the objective function of affine registration between the two sets of points to be matched that we define, and give the objective function as the following least squares (LS) problem [43]: where A is the affine matrix, t is the translation vector.
The method of solving the algorithm is divided into two steps. Firstly, for any point m i in the point set M, the affine ICP algorithm finds the closest point in the point set D as the corresponding point; secondly, we can calculate the affine matrix A and translation vector t of the current loop; thirdly, the corresponding point information is updated according to the above steps, and the transformation matrix is continuously calculated until the registration error is minimum or the iteration time exceed the maximum number, then the cycle is terminated.

Precise Affine Registration with Correntropy and Color Information
In this section, we propose a precise affine registration algorithm with color information and correntropy to achieve accurate registration.

Problem Statement
In real-world scenarios, collected data from cameras and other equipment may have slight deformations, and some noise data may be mixed in at the same time. Although traditional affine registration can solve the problem of deformation registration of objects, the accuracy of matching weak geometric data with noise and outliers is not high, and Figure 1 verifies this problem. It can be seen from the figure that the affine registration algorithm is not accurate for solving such problems. For weak geometric data, the color information in the image is more obvious, so more reliable correspondences can be found, thereby improving the registration accuracy. Then, to improve the registration accuracy, we can introduce color information as a feature, thereby increasing the registration accuracy for weak geometric data. Therefore, we can get the following formula where r m i , r d j , g m i , g d j , b m i , and b d j are the color information of two points, and α represents the weight ratio of color information, which is set to be 0.1. Color information is easily affected by changes in illumination. Therefore, when light changes are more obvious, the parameter α can be adjusted to a smaller value, thus reducing the effect of color errors. The objective Equation (2) enables a more accurate and reliable correspondence between these two point sets. However, real datasets often contain a lot of noise and outliers, which leads to a reduction in registration accuracy. To solve this problem, this paper introduces a measure of correntropy, which replaces the measure of Euclidean distance measurement in the traditional registration method. The measurement method is as follows where σ is a free variable that is set to 0.1. It can be known from the above formula that if the distance is larger, the correntropy distance is smaller and the registration result is worse; otherwise, the correntropy distance is larger, and the registration accuracy is higher. In other words, the correntropy metric falls into the robust kernel functions which assign large errors with bigger weights and assign small errors with smaller weights for alleviating the influence of inliers and outliers. According to the above equation, we can build a new objective function:

Precise Affine Registration with Color Information and Correntropy
To solve the registration problem described by Equation (4), this algorithm uses an iterative solution algorithm similar to the affine ICP algorithm. In each loop, the nearest neighbor search algorithm is used to establish the correspondence between these two point sets, and then the affine transformation is solved according to the correspondence. This algorithm is mainly divided into two steps, which are expressed as follows: (1) According to the transformation in step (k − 1), the corresponding relationship is established between these two point sets: (2) Calculating (A k , t k ) by the correntropy method for step k: The algorithm iteratively calculates the affine matrix A and translation vector t to make it approach the optimal solution until the error converges to a preset value or exceeds the maximum number k of cycles.
For the problem of correspondence search described in Equation (5), we will use the k-d tree-based method [44][45][46] to search the nearest point in three dimensions. For the optimization problem described in Equation (6), we will give detailed algorithm solution steps and closed-form solution of the algorithm.
First, we calculate the approximate solution of t. Then, Equation (6) is rewritten as follows: Assuming By calculating the derivative of t for Equation (7), let ∂F(A, t) ∂ t = 0, we can get: The algorithm is solved iteratively, and the final loop termination is conditional on registration accuracy. Although the value of each iteration is an approximate solution, the final registration result will not be affected.
Second, we calculate the approximate solution of A k . Substituting Equation (9) into (7), we can get a constraint optimization problem about A k . Then, we can get: Let , we can get: The process of solving A k is similar to [47,48], so we can solve A k in the same way, let U and V as N m × (m + 1) matrices where each row represents the point m i and d c k (i) , respectively, then we can get where 1 T N d represents a row vector of length N m , and G(V − A k U) = b represents a column vector with entries . Then, we can get A k by calculation where D(b) = diag(b). The above algorithm is summarized in Table 1. Since the proposed algorithm is similar to the traditional affine ICP algorithm, the time complexity is also similar to the traditional affine ICP algorithm. The calculation time of the proposed algorithm is mainly in the search for the closest point of the first step. In this step, the search strategy of the k-d tree is used in this step, and step two hardly affects the calculation time of the algorithm. Therefore, the time complexity of the algorithm proposed in this paper is O(N m ln N d ). The code of our algorithm is implemented in MATLAB 2018. The PC environment is Intel (R) Core (TM) i5 3.00GHz, 4GB RAM. The precise a f f ine registration algorithm with color in f ormation and correntropy Input : Two point sets M and D Initialization : The initial a f f ine trans f ormation A 0 , t 0 .

Repeat two steps :
Step1 : According to the trans f ormation in step (k − 1), the corresponding relationship is established between the two point sets via Equation(5).
Step2 : Calculating (A k , t k ) via Equation (6). End Output : A k , t k

Experimental Results
This section further verifies the accuracy and robustness of the proposed algorithm through experiments. First, the proposed algorithm is verified by simulation experiments, and we compare it with the ICP algorithm [15], the ICP algorithm with color information (CICP) [36], the scaling ICP algorithm (SICP) [49], the affine ICP algorithm (AICP) [50], and the affine algorithm with correntropy (ACICP) [51,52], GeoTransformer(Geo) [33], respectively. Among them, the simulation experiments include single objects and indoor scenes. In addition, we further test with real data.

Simulation Experiment
In this section, we use simulation experiments to verify the accuracy and robustness of the algorithm in this paper. First, we use RGB-D data collected by Kinect [53][54][55], termed AffineMatch. Second, we add affine deformation and some noise to the point cloud data, and the data before and after the transformation are used for registration. Finally, when the ground-truth is known, the relative errors are computed. In the experiments, {A * , t * } is the ground-truth, and the relative errors are defined as ε A = ||A − A * || 2 /||A * || 2 , and ε t = || t − t * || 2 /|| t * || 2 . The experimental results are shown as Table 2 and Figures 2-4. Table 2, comparing the proposed algorithm with the other six algorithms, it is verified that the proposed algorithm is better than other algorithms. For the ACICP algorithm, although the correntropy criterion ensures the robustness of the algorithm to noise and outliers, it still cannot effectively align the two point sets in the absence of the correct correspondence between the point sets. Although the Geo [33] method is currently an excellent deep learning registration method, the registration accuracy for this affine deformation is low. The proposed algorithm performs well on simulated datasets with weak geometric structures with small errors. Figures 2-5 are the result of simulation experiments on single objects, including books, bags, pillows, and airplanes. Obviously, the results of our algorithm are better than the registration results of the ICP algorithm, the ICP algorithm with correntropy, the affine algorithm with color information in terms of registration accuracy.

Indoor Scenes Experiment
In this section, we use indoor scenes experiments to verify the accuracy and robustness of the algorithm in this paper. The data are captured by a Microsoft Kinect sensor, which can simultaneously obtain RGB and depth images. We capture the real data in indoor scenarios and four point cloud pairs are used to perform registration experiments. The experimental results are shown as Table 3. As shown in Table 3, comparing the proposed algorithm with the other six algorithms, it is verified that the proposed algorithm is better than other algorithms. Figure 6 is the results with indoor data, where the red ones represent the source point set and the green ones represent the target point set. It can be seen from experiments that our algorithm is better than the other algorithms and can match more points, so it achieves a more accurate registration. Moreover, to further verify that the algorithm proposed in this paper may converge faster and obtain more accurate and robust registration results, we use indoor data to compare the convergence of the algorithm proposed in this paper with the other five algorithms, as shown in Figure 7. We define our algorithm as Affine HCICP. It can be seen from the figure that our algorithm has a faster convergence speed and stronger accuracy and robustness, which further proves that the proposed algorithm performs better.

Experiments with Real Data
In this section, we test on real data and further prove that the proposed algorithm is better than the other five algorithms. Since the real data do not have accurate ground-truth, and the real data contain noise and outliers, we use the root mean square error (RMS) method to analyze the accuracy of the proposed algorithm and other algorithms under the real data. First of all, because after the registration algorithm is matched, most of the points can find their corresponding points. After a lot of experiments, we select the first 60% of the points after registration to measure the registration accuracy of the algorithm. Secondly, we use the corresponding point pair as the root mean square error ε to measure the registration accuracy of the algorithm. As shown in Table 4, we compare different registration algorithms on two sets of real data. It can be obtained that our algorithm has higher matching accuracy and robustness compared to the other five algorithms.
In addition, as shown in Figure 8, when tested on a real dataset, we can see that the proposed algorithm can match two point sets well, while the registration accuracy of other algorithms is not as good as the proposed algorithm. Thus, the accuracy and robustness of the proposed algorithm are verified. Moreover, to further verify that the proposed algorithm converges faster on real datasets and obtains more accurate and robust registration results, we use real data to conduct the convergence of the proposed algorithm with the other five algorithms. As shown in Figure 9, it can be seen from the test on real data that our algorithm has faster convergence speed, stronger accuracy, and robustness, which further proves that the algorithm performs better.

Conclusions
This paper proposes a precise affine registration algorithm with correntropy and color information. This algorithm can effectively handle the large noise and outliers and small deformation in the dataset in the registration algorithm. First, the algorithm introduces color features to establish more accurate and reliable correspondence. Second, this paper introduces the measurement of correntropy, which overcomes the influence of noise and outliers in the dataset, and further improves the registration accuracy. Experimental results demonstrate that our registration algorithm has higher registration accuracy and more stable robustness than other advanced algorithms. In the future, the robustness against illumination changes of the algorithm can be further studied.

Conflicts of Interest:
The authors declare no conflict of interest.