Multi-View Laser Point Cloud Global Registration for a Single Object

Global registration is an important step in the three-dimensional reconstruction of multi-view laser point clouds for moving objects, but the severe noise, density variation, and overlap ratio between multi-view laser point clouds present significant challenges to global registration. In this paper, a multi-view laser point cloud global registration method based on low-rank sparse decomposition is proposed. Firstly, the spatial distribution features of point clouds were extracted by spatial rasterization to realize loop-closure detection, and the corresponding weight matrix was established according to the similarities of spatial distribution features. The accuracy of adjacent registration transformation was evaluated, and the robustness of low-rank sparse matrix decomposition was enhanced. Then, the objective function that satisfies the global optimization condition was constructed, which prevented the solution space compression generated by the column-orthogonal hypothesis of the matrix. The objective function was solved by the Augmented Lagrange method, and the iterative termination condition was designed according to the prior conditions of single-object global registration. The simulation analysis shows that the proposed method was robust with a wide range of parameters, and the accuracy of loop-closure detection was over 90%. When the pairwise registration error was below 0.1 rad, the proposed method performed better than the three compared methods, and the global registration accuracy was better than 0.05 rad. Finally, the global registration results of real point cloud experiments further proved the validity and stability of the proposed method.


Introduction
Flash laser three-dimensional imaging technology [1,2] can obtain an object's three-dimensional information by transmitting a single pulse, which is an effective means to realize three-dimensional imaging of moving objects. With the advantages of a long imaging distance and freedom from illumination effects, it has high military and civilian value. For example, in the application of space-based non-cooperative object three-dimensional imaging, the object and the imaging system are always in relative motion, and the positions of the two can change at any time. Due to the occlusion of perspective, reconstructing the complete 3D model of the object requires collecting a variety of different perspective point clouds in combination with the object's motion, and unifying them to the same coordinate system through point cloud registration. Multi-view point cloud registration can be divided into two steps: Pairwise registration and global registration. Pairwise registration refers to the registration between two point-clouds collected by adjacent perspectives, and there are currently only relies on the pairwise registration relationship to separate the true pairwise registration from the pairwise registration with errors. LRS is more suitable for global registration of multi-view laser point clouds. However, LRS is rarely used in laser point cloud global registration, and there is still room for improvement of loop-closure detection and robustness to the noise of laser three-dimensional imaging.
In this study, loop-closure detection was realized according to the spatial distribution characteristics of the point cloud. Then, the corresponding weight matrix and transformation relation matrix of the multi-view point cloud were established, and the global registration objective function was constructed. The optimal solution of global registration was obtained by LRS, and orthogonalization projection of the optimal solution, as well as the registration relationships between the point clouds, were obtained. The following sections are arranged as follows: The second section explains the proposed method in detail. In the third section, the design of the corresponding simulated analyses is presented, the experiments are described, and the results are analyzed. In the fourth section, the algorithm is summarized. In the fourth section, the proposed method is summarized and prospects for areas that can be improved are discussed.

Study Method
In moving object multi-view point cloud global registration, pairwise registration relationships between several point clouds are known, and the coordinates of any point cloud, P i , (where i ∈ [1, n], n is the total number of point clouds) are considered the reference coordinate system. Then, the transformation relationship of other arbitrary point clouds, P j , (where j ∈ [1, n]) to the reference coordinates must be found, as shown in Figure 1. Global registration can be divided into four steps: Pre-processing, loop-closure detection, post-preprocessing, and relation calculation. The green line in Figure 1b means the corresponding loop-closure point clouds founded in loop-closure detection. The assumption of global registration is that any P j can obtain a transformation matrix between P i and P j directly or indirectly; that is, there are no isolated point clouds in the multi-view point cloud. As shown in Figure 1, assuming that the change relationship of P i to P j is M ij , since there is a difference in the viewing angle between the cloud points of each viewpoint, the area of overlap is affected. In most cases, P i cannot be directly registered with any other P j ; that is, only some M ij values are known.
The unknown transformation relationship can also be obtained by continuous transformation of the intermediate point clouds from P i to P j , but M ij generally has outliers and errors. Outliers can be rejected by use of robust estimators, but the noise will become more severe as the number of point clouds increases; a simple continuous transformation accumulates these outliers and errors. Global registration is a solution to the accumulation of outliers and errors. With these known transformation relations, combined with loop-closure detection, a global transformation relation matrix of point clouds is constructed, and the transformation relationship between each point cloud is considered globally. LRS decomposition of the transformation relation matrix obtains the estimated relationship,M ij , which minimizes the global registration total error. Equation (1) shows the generalized global registration objective function:

Point Cloud Spatial Distribution Feature Extraction and Loop-Closure Detection
Loop-closure detection refers to finding point clouds that can be registered with more than one other dataset, yielding in different transformation parameters. Then, a new transformation parameter can be obtained with the pairwise registration of the point cloud obtained by loop-closure detection. The new transformation parameter forms a new constraint, which can effectively eliminate the error caused cumulatively by the continuous pairwise registration. Therefore, loop-closure detection is a necessary step in global registration. Moving object multi-view point clouds are in relative motion, and positions change between the system and the object at the time of acquisition. The point cloud of each viewpoint has severe laser noise, and there is a density difference between point clouds. It is necessary to evaluate the similarity of each point cloud.
In this work, we constructed the spatial distributions for each point cloud, and the relationship between each point cloud was extracted by spatial distribution features to achieve closed-loop detection. As shown in Figure 2, the first point cloud, P 1 , in the multi-view point cloud was used as a reference, and all the other point clouds were transformed by the known transformation, M ij , to the coordinate system of P 1 . The minimum bounding box, B mmb_all , which surrounded all point clouds, was extracted and rasterized into n b × n b × n b grids, and the grids were numbered from 1 to n b × n b × n b . The quantity of points where P i falls in each grid was normalized, and the distribution histogram, h hist (i), of P i in space was obtained, where h hist (i) describes the spatial distribution characteristics of P i . The distribution features of each point cloud in the grid can be regarded as the probability distribution characteristics of the point cloud in space. The points with similar spatial distribution features can be considered acquired when the viewpoint is similar, and the distribution feature relations between the point clouds are obtained. By correlating the distribution features, h hist (i), between the point clouds and removing the point cloud adjacent to P i , the point cloud with distribution features similar to P i is the point cloud collected again from a viewpoint similar to P i , and loop-closure detection is realized accordingly. The point cloud distribution features obtained after spatial rasterization are not sensitive to point cloud density changes, and they showed robust performance when there are outliers and noise. The point clouds obtained by loop-closure detection were acquired from similar viewpoints; they have a high overlap rate and can be pairwise-registered to form a new constraint. For the point clouds of similar viewpoints, the similarity can be evaluated by their distribution features in space. The higher the similarity of distribution characteristics, the greater the similarity of the viewpoint, and the more accurate the pairwise registration. Equation (2) defines the similarity, Θ ij , between P i and P j :

Transformation Relation Matrix and Weight Matrix
Assuming P i+k is a point cloud from a viewpoint similar to P i obtained in loop-closure detection, then the transformation, M i,i+k , from P i to P i+k forms a new constraint of the global registration. M i,i−j . . . M i,i+j and M i,i+k−j . . . M i,i+k+j can be obtained by pairwise registration for P i , in which j is the maximum quantity of adjacent point clouds that can be registered. The value of j is determined by the adjacent point cloud overlap rate and the pairwise registration method. The higher the overlapping rate of adjacent point clouds, and the more robust the adjacent registration method, the larger the value of j.
According to the known transformation relationship, M ij , between point clouds, the transformation relation matrix, T global , is constructed as shown in Equation (3): When M ij is unknown, M ij in T global is an all-zero matrix. The more constraints obtained by closed-loop detection, the denser the matrix, T global , and the better the performance of the global registration. Considering the existence of outliers in pairwise registration, combined with the known condition that M ij and M ji are mutually reversible matrices, M ij and M −1 ji are considered to be correct only when the difference between M ij and M ji is below a threshold. Otherwise, they are considered to be inaccurate registrations, and all values in M ij and M ji are set to zero.
The weight matrix, W weight , is the same size as T global , and the purpose of constructing W weight is to make the objective function only work on valid values in T global , which can reduce the effect of the sparsity caused by element deletion in T global on global registration. Equation (4) shows the value of element w ij in W weight :

Objective Function Construction
Outliers and errors are inevitable in pairwise registrations, and the transformation relationships in T global are not always accurate. T global can be decomposed into the following equation: whereT global is the estimated value matrix of the accurate registration, and it is also the expected target in global registration; N noise is the noise matrix, i.e., the outliers and errors of registration existing in T global . The element in T global is only partially known, and T global is a sparse matrix; the block element, M ij , in T global is a rigid body transformation with a rank of 4, thus, the rank of T global is also 4, so T global is a sparse low-rank matrix. The purpose of global registration is to complement the missing elements in T global under the constraints of known noisy observations. From the perspective of matrix analysis-that is, when the rank of the target matrix is known-the unknown matrix information is complemented by the known rank constraint and the limited known matrix elements. This is a low-rank sparse matrix completion problem, as shown in Figure 3. The pairwise registration of adjacent point clouds in the preprocessing usually does not cause errors because their overlap is enough; but it cannot guarantee that the point cloud obtained by closed loop detection are all completely correct, as there may be false results in the loop-closure detections and the pairwise registration between them may be wrong, resulting in the outliers in T global . Considering that the L 1 norm is more robust than L 2 norm when there are missing values and outliers, we used the L 1 norm to evaluate the difference between T global andT global . To solve this,T global is usually bilinearly decomposed into two matrices, U and V. Equation (6) shows the objective function and constraints.
where rank(T global ) is the rank ofT global .
However, the minimization of Equation (6) is a non-convex discontinuous problem. It is necessary to introduce a relaxation term to normalize it, and the introduction of the relaxation term cannot have a great impact on the value of the original objective function. We used the matrix completion method in [19] to normalize the objective function, as shown in Equation (7).
where λ is the weight parameter of the relaxation term, which is used to control the relationship between the observed value and the estimated value. When λ is small, the obtained estimated value, T global , describes the observed value more accurately, but its prediction effect is not good and it over-optimizes; when λ is large, the obtained estimated value,T global , is inaccurately described for the observed value, and it is under-optimized. The empirical value of λ is generally The constructed objective function has no orthogonality constraints on U and V, which exceeds the limit of six degrees of freedom, so it does not compress the solution space. Compared with existing methods, the proposed method is more in line with the actual situation of single-object multi-view global registration, and it has the advantage of robustness.

LRS Decomposition by Augmented Lagrange Method
Equation (7) is equivalent to Equation (8), which we solved using the Augmented Lagrange method.
The Augmented Lagrange method adds a penalty term based on the Lagrange method to obtain the solution. The corresponding Augmented Lagrange function of Equation (8) is Equation (9).
where L is a Lagrange multiplier matrix for the iterative solution; µ is a penalty factor; and A, B is defined as a trace of A T B. The minimization of f (T global , U, V) generally uses the Gauss-Seidel method to iteratively solveT global , U, and V. The solution of Equation (9) is decomposed into three minimization subproblems; that is, each iteration is divided into three steps that are mutually constrained. For example, the mth iteration is as follows: (0) Parameter: where ρ is a constant parameter used to adjust the convergence speed. The larger the value of ρ, the closer the value ofT global to UV, which speeds up the convergence of the algorithm, but it simultaneously affects the accuracy of the algorithm to a certain extent. In this paper, ρ = 1.05 and µ 0 = 1 × 10 −6 , and all elements in L 0 are 1 × 10 −12 .
(1) With fixedT global and V, solve U. Then, Equation (9) can be expressed as a function of U, as shown in Equation (11).
When the derivative of Equation (11) on U is 0, ζ(U) takes the minimum value, and U is as shown in Equation (12).
(2) With fixedT global and U, solve V. Next, Equation (9) can be expressed as a function of V, as shown in Equation (13).
When the derivative of Equation (11) on V is 0, ζ(U) takes the minimum value, and V is as shown in Equation (14).
Equation (15) can be solved by an element-by-element shrink operation [20]. Then, Equation (16) shows the equation forT global .
where W weight is the complement of W weight . The contraction operator, S ε (x), is as shown in Equation (17).
When the difference betweenT global and UV is sufficiently small, the iteration is terminated, and the registration relationship matrix,T global , can be obtained. Considering that the diagonal block matrix, M ii , of the matrix, UV, is a transformation relationship between the point cloud and itself, in the ideal case, all of the M ii values should be equivalent to I 4 . The final iteration termination condition is as shown in Equation (18).
where tr(UV − I 4n ) is the trace of UV − I 4n , and e threshold_iteration and e threshold_optimal are determined by the accuracy of the pairwise registration. In this paper, e threshold_iteration = 1 × 10 −9 and e threshold_optimal = 1 × 10 −8 , which can serve as a reference.

Block Matrix Orthogonalization
In the ideal case, where there are no influences, such as outliers and errors, each of the 4 × 4 block matrices,M ij , in the matrix,T global , obtained in Section 2.4 is a registration matrix of P i to P i+j . However, considering the noise caused by outliers and errors,M ij obtained by low-rank sparse decomposition generally does not satisfy the orthogonal constraints required for rigid body transformation, and it needs to be projected into orthogonal space.
The first step is to normalizeM ij , assuming that the element inM ij ism ij . Equation (19) shows the normalization ofM ij .M ij =M ij /m 4,4 Then, the 0th, 1st, and 3rd row elements of the 4th column are set to 0 to satisfy the constraint of the rigid body transformation, as shown in Equation (20).
wherer ij is an element in the 3 × 3 matrix,R ij , which is defined in Equation (21).
where Q is a 3 × 3 diagonal matrix, and the diagonal elements are 1, 1, and UM 33 V TM 33 , respectively. Equation (22) shows the definition of UM 33 and VM 33 .
[UM 33 , SM 33 , The obtained 4 × 4 block matrix,M ij , inT global is the registration matrix of P i to P j . The registration matrix of any P i to P j under the global optimization condition can be obtained, and the global registration is completed.

Experiments and Analysis
The experiments were divided into a simulation point cloud analysis and a real point cloud test. In Sections 3.1 and 3.2, the simulation point clouds were used to control the variables to analyze the performance of the proposed method. In Section 3.3, the real point clouds were used to comprehensively test the proposed method's practical performance. The simulation point clouds were simulated by an array plane 3D camera based on the time-of-flight (TOF), as described in [21]. During the process of generating a simulated point cloud with continuous adjacent angles of view, there was a relative position transformation between the object and the system to simulate the motion of the object. The ratio of the maximum and minimum distance between the system and the object in adjacent point clouds was 2:1, and the angle differences between adjacent point clouds was fixed at 20 • . Outliers and Gaussian noise were added to simulate laser three-dimensional imaging noise. The quantity of outliers was 0.25 times the quantity of points in each point cloud; the mean of the Gaussian noise was 0, and the variance was 0.01 times the length of the minimum bounding box diagonal. Noise and outliers were added primarily to analyze the robustness of the loop-closure detection, and we assumed that the pairwise registrations between continuous point clouds in the preprocessing were achievable. Six models were selected as objects: Bunny [22], Armadillo [23], and Dragon [24] from Stanford; MRO (Mars reconnaissance orbiter), Skylab, and Voyager from NASA (National Aeronautics and Space Administration) [25]. Bunny, Armadillo, and Dragon had rich structural information, while space targets, MRO, Skylab, and Voyager, had relatively simple structural information, such as numerous repetitions and similar structures. Figure 4 shows the rendered point clouds based on shading; the first row is the front view of the point clouds of the six objects, and the second row was the corresponding point cloud after adding the outliers and Gaussian noise to the point cloud of the first line, which is also the simulation point cloud used in our simulation analysis.

Loop-Closure Detection Accuracy Analysis
The point cloud distribution feature histogram is related to the quantity of grids. At the same time, the pairwise registration error of the point cloud also affects the point cloud distribution feature, which directly affects the loop-closure detection accuracy. To test the robustness of the proposed loop-closure detection method, the correlation between the loop-closure detection accuracy, the quantity of grids, and the error of pairwise registration was analyzed. Figure 5 shows the weight matrix under three different adjacent registration errors. In the simulation, the object rotated 20 • continuously around the fixed axis. Therefore, under ideal conditions, the point cloud of the ith view should form a closed loop with the point cloud of the i ± 17th view. Figure 5 shows that when the pairwise registration error was 0.01 rad, the cumulative error was relatively small. The proposed method could complete loop-closure detection, and the similarity of the point cloud obtained by loop-closure detection was relatively high, so the weight of the transformation relationship was also relatively high. When the pairwise registration errors were 0.05 rad and 0.1 rad, the cumulative error became severe as the pairwise registration error increased, and the point cloud weights obtained by loop-closure detection decreased. There were four detection errors at 0.1 rad.  Figure 6a shows the relationship between the loop-closure detection accuracy and the pairwise registration error. When the pairwise registration error was below 0.055 rad, the loop-closure detection accuracy of the six point-clouds was over 90%. However, as the registration error increased further, the accuracy began to decline. Because of the repetition or similar structures of the three space targets, the accuracy was more susceptible to the adjacent registration error. This shows that the proposed loop-closure detection method has certain requirements for the registration error of adjacent frames, one of them being that it should not be too large.  Figure 6b shows the relationship between the quantity of grids and the loop-closure detection accuracy. When the quantity of grids was too small, the loop-closure detection accuracy was correspondingly low because it could not express the detailed distribution feature of each point cloud. When the quantity of grids was too large, the loop-closure detection accuracy also decreased because it expressed the point cloud distribution feature with too much detail. When the quantity of grids was between 5 3 and 12 3 , the loop-closure detection accuracy of the six point-clouds reached more than 90%, which indicates that the proposed loop-closure detection method can achieve good results within a wide range of the quantity of grids. Figure 6 also shows that MRO, Skylab, and Voyager were more sensitive to changes in the quantity of grids, which is due to the fact that there were more repeats or similar structures in the three space targets. When the quantity of grids was not appropriate, the impact was more obvious.

Simulation Point Cloud Analysis
With the same simulation point cloud as in Section 3.1, the proposed method was used in its entirety to analyze the relationship between global registration, pairwise registration outliers, and errors. The incremental registration, the LUM [11] in the PCL (Point Cloud Library) library [26], and the LRS-based RegL1 algorithm in [18] were selected as the methods to compare with the proposed method. Considering that the loop-closure detection method was not given in [13], the proposed loop-closure detection method described in this paper was used with the RegL1 method, but the assignment of elements in the weight matrix still adopted the method in [18]. Additionally, both of the four kinds of global registration method were based on the same pairwise registration method and the same termination in the consecutive point clouds pairwise registration to minimize the influence of the pairwise registration result.

Global Registration Accuracy of a Single Object
When the pairwise registration error was 0.01 rad, 37 consecutive point clouds of a single object were globally registered by the four different methods, and Figure 7 shows the global registration errors of the respective viewpoints. Figure 7 also shows that, with the increase of the point cloud index, the incremental registration method formed an obvious cumulative registration error, and LUM averaged the cumulative error to a certain extent. RegL1 and the proposed method had better performances.

Relationship between Global Registration and Pairwise Registration Outliers
When there were errors during loop-closure detection, the detected false neighboring point cloud overlap rate was low. A low overlap rate might lead to a wrong pairwise registration, which might result in an outlier in T global . The proposed loop-closure detection method had an accuracy of over 90% under most parameter conditions. Therefore, the performance of the proposed method with 1-10% outliers was tested. We replaced the correct transformation matrix, M ij , of the random position with the corresponding proportion of a 4 × 4 unit matrix, and Figure 8 shows the experimental results. When the percentage of outliers was 1-10%, the performance of the proposed method was relatively robust, the global registration error was lower than 0.005 rad, and the overall performance was better than RegL1.

Relationship between Global Registration and Pairwise Registration Errors
The results of global registration were also affected by pairwise registration errors. Figure 9 shows the performance of the four methods with different pairwise registration errors. The global registration error represented by the curve in Figure 9 was the average of the registration errors of all point clouds after global registration with the current pairwise registration error. Figure 9 also shows that the incremental registration method tended to increase the error as the pairwise registration error increased. When the pairwise registration error reached a certain level, the incremental registration error tended to be stable. LUM averaged the error caused by pairwise registration to a certain extent, but the overall performance was not optimal. RegL1 and the proposed method had better performances on Bunny, Armadillo, and Dragon, but the performance of the proposed method was more robust. RegL1 and the proposed method had some serious global registration errors with MRO, Skylab, and Voyager. This is because when the pairwise registration error increased, the probability of the error during loop-closure detection increased, which led to more outliers and errors in T global at the same time. The three space-point clouds were more severely affected due to their simple structures, which resulted in a more rigorous test for the stability of the global registration method. However, RegL1 had a larger number of severe global registration errors than the proposed method in general, which we presume to be caused by the compression of the solution space by RegL1. The proposed method was more robust due to the more reasonable objective function and the corresponding weight matrix. When the pairwise registration error was 0.01-0.1 rad, the global registration error of the proposed method was less than 0.05 rad in most cases.  Table 1 shows the global registration results of each method applied to six kinds of point clouds when the pairwise registration error was 0.03 rad. We chose the point clouds at 0 • and 360 • because, in the ideal condition, they were completely overlapping, and the Euclidean distance between the corresponding points should be equal to 0. We filtered out the outliers in the point clouds, calculated the distance between the corresponding points after registration, and tinted them according to the distance value. The corresponding residuals are below the results. The four global registration methods could optimize the pairwise registration results to a certain extent, but in the case of severe noise and outliers in these point clouds, the LUM had difficulty in finding corresponding points. The LRS-based RegL1 and our method were relatively robust, and our method performs best overall.

Experimental Test and Result
The proposed method was further tested with a real point cloud. The point clouds were obtained by an array plane 3D imaging device based on the TOF principle. The models were diffuse reflection space models, such as the Apollo, Skylab, and Tian-gong models. The models were placed on a rotary table with a controllable rotation angle, the object was rotated after each acquisition, and the rotation angle was fixed at 20 • . A total of 37 point-clouds were acquired, which is consistent with the simulation point cloud parameter settings. The point clouds with an odd index were twice the imaging distance of the adjacent point clouds with an even index. Thirty-seven point-clouds of an object were transformed in the same coordinate system, since the global registration relationship between real point clouds is not exactly clear, the point clouds after transformation were sliced at the same position. Table 2 shows the global registration results and the slice results of the four methods. Table 2 shows that there were serious outliers and noise in the three real point clouds, and the four methods were affected to different degrees. There were certain differences between the global registration results of the real point clouds and the simulated point cloud. The reason for these differences is that the outliers and noise affected the results of the pairwise registration, which in turn affected the results of the global registration. In this case, Table 1 shows that LUM did not obtain the optimal solution for Apollo and Skylab, but the results for Tian-gong were good. RegL1 failed to register with Skylab, but the results for Apollo and Tian-gong were good. The proposed method obtained fine edges of the three kinds of point clouds, and the contour information of the object was clear, which shows the robustness of the proposed method when applied to a real point cloud.

Conclusions
In this paper, a multi-view laser point cloud global registration method based on low-rank sparse decomposition was proposed. The spatial distribution features of each point cloud were extracted by spatial rasterization, and loop-closure detection of multi-view point clouds was realized according to the similarity of spatial distribution features. According to the similarity of point clouds obtained by loop-closure detection, the corresponding weight matrix was designed, which enhanced the robustness of the global registration to sparseness and errors in the transformation relation matrix. The objective function that satisfied the global optimization condition was constructed. In the regularization process, the compression solution space problem caused by the column-orthogonal hypothesis of the matrix was avoided, which made the solution more realistic and more robust. The objective function was solved by the Augmented Lagrange method, and the iterative termination condition was designed by considering the iterative error and the a priori condition of global registration. The simulation analysis proved that the proposed loop-closure detection method was robust with a wide range of preset parameters, and the accuracy of loop-closure detection was over 90%. When there were outliers in the pairwise registration, the performance of the proposed method was better than that of RegL1. When the pairwise registration error was below 0.1 rad, the proposed method performed better than the three other methods, and the global registration accuracy was 0.05 rad. Finally, further experiments were carried out with real point clouds. The validity and robustness of the proposed method were further proved by the global registration results and slice results.
In general, the proposed method is robust with a wide range of preset parameters, and is less affected by density changes and noise. These advantages make it suitable for global registration of moving object multi-view laser point clouds, and it has a relatively high application value. However, there are some shortcomings in the proposed method. For example, for the three kinds of space target point clouds with repeated structures, the accuracy of the proposed method declined to some extent, which indicates that the loop-closure detection method has certain requirements for the object structure. At the same time, the proposed method has certain requirements for the error of pairwise registration. The proposed method can also be used by multiple iterations to further improve the global registration accuracy.