GraM: Geometric Structure Embedding into Attention Mechanisms for 3D Point Cloud Registration

Abstract

3D point cloud registration is a crucial technology for 3D scene reconstruction and has been successfully applied in various domains, such as smart healthcare and intelligent transportation. With theoretical analysis, we find that geometric structural relationships are essential for 3D point cloud registration. The 3D point cloud registration method achieves excellent performance only when fusing local and global features with geometric structure information. Based on these discoveries, we propose a 3D point cloud registration method based on geometric structure embedding into the attention mechanism (GraM), which can extract the local features of the non-critical point and global features of the corresponding point containing geometric structure information. According to the local and global features, the simple regression operation can obtain the transformation matrix of point cloud pairs, thereby eliminating the semantics that ignores the geometric structure relationship. GraM surpasses the state-of-the-art results by 0.548° and 0.915° regarding the relative rotation error on ModelNet40 and LowModelNet40, respectively.

1. Introduction

Three-dimensional reconstruction provides digital 3D models by presenting a real-world scene, promoting the development of augmented reality, such as autonomous driving and digital twins [1,2,3,4,5]. On the other hand, point cloud data is fundamental for building a digital 3D model because 3D point cloud registration is the core technology for achieving 3D reconstruction by providing stereoscopic models [6,7]. In recent years, the rapid development of sensor technologies enables 3D point cloud data to visualize the world, further promoting technology innovation in practice.

Three-dimensional point cloud registration is a crucial step in the 3D reconstruction process. It aims to learn the local information of an object from multiple perspectives and integrate it into a unified perspective to obtain objects with rich global information. Specifically, it evaluates the correlation of each corresponding point in the point cloud sample by learning the characteristics of each point cloud sample under multiple visual angles.

Then, it estimates the rigid transformation parameters that ensure all the corresponding points can be directly transformed in 3D space. Subsequently, it obtains the converting matrix of the point cloud sample, thereby achieving the goal of 3D point cloud registration.

In the latest research, many methods utilize attention mechanisms for learning point cloud features [8,9]. For instance, the Transformer can extract the local and global features of the corresponding points between the clouds to obtain the transformation matrix for point clouds [10,11,12]. However, the features extracted by the traditional Transformer do not contain the critical geometric information for point cloud registration, making it impossible to achieve better performance. Therefore, for 3D point cloud registration technology, extracting features containing geometric structure information becomes a solid challenge.

To solve the above challenges, we propose a 3D point cloud registration method that embeds geometric structure into the attention mechanism, forming an end-to-end registration framework. More specifically, REGTR [13] is a classic 3D point cloud registration model. Its famous innovation is to adopt the attention mechanism in the Transformer, which can effectively obtain the global and local features, to replace traditional feature matching.

Nonetheless, the extracted features do not include information on geometric structure, which is extremely important for registering point cloud data. To this end, we take REGTR as the primary architecture and introduce two embedded modules to extract geometric structures. The two structures are bound with the original two self-attention structures of REGTR, respectively, to learn more rich features that contain information on geometric structure. The fusion of global and local features that include geometric structure information can improve the accuracy of point cloud registration. Extensive experimental validation demonstrates that the proposed method can significantly outperform the state-of-the-art mechanisms.

The main contributions of this paper are summarized as follows:

• As far as authors know, this is the first proposal to embed the geometric structure into an improved REGTR network. The proposed GraM effectively promotes the local features integrated with information on geometric structure and global features.
• We introduce the attention mechanism to the point cloud registration task and optimize the feature extraction on the REGTR network, significantly improving the accuracy and efficiency of the low-overlap point cloud registration task.
• Comprehensive experiments on the reconstructed ModelNet40 and KITTI datasets show that GraM obtains better accuracy than state-of-the-art methods.

The remainder of this paper is organized as follows. Section 2 illustrates the related work on 3D point cloud registration technology. Descriptions of the problem definition and the core technology used are in Section 3. Our research methodology and specific implementation steps are introduced in detail in Section 4. Section 5 evaluates the performance of our proposed 3D point cloud registration method. Finally, we summarize the paper and present future research in Section 6.

2. Related Work

Extensive research has been conducted on 3D point cloud registration technologies, which include optimization-based registration, feature learning-based registration, and end-to-end learning-based registration.

Optimization-based point cloud registration. Besl et al. [14] proposed the classic Iterative Closest Point (ICP) algorithm, which iteratively estimates corresponding points between two point clouds and their transformation matrix to achieve registration. IMLP was proposed in [15] to improve the corresponding point estimation of ICP by incorporating measurement noise into the transformation estimation. Segal et al. [16] proposed a generalized version of ICP that allows for the inclusion of arbitrary covariance matrices in ICP variants using point-to-plane metrics. Zhu et al. [17] proposed a graph registration method that simultaneously considers vertices and edges to find point-to-point correspondences between two graphs. Huang et al. [18] introduced a novel pruning module to enhance deep learning-based point cloud registration in low overlap scenarios (Predator), resulting in significant performance improvements. However, the computational efficiency and registration accuracy were significantly decreased when these methods were used to deal with large-scale datasets and low-overlap scenarios.

Feature learning-based point cloud registration. Zeng et al. [19] introduced 3DMatch, a parallel network trained from RGBD images, to extract features by combining the local structure around critical points and further capture the local characteristics of the 3D point cloud. The network 3DFeatNet [20] uses a weakly supervised approach to learn feature correspondences from 3D point clouds. RPMNet [21] can obtain soft correspondences of points in partially overlapping point clouds from a mixture of features learned from spatial coordinates and local geometry. A dynamic graph convolutional neural network is employed in Deep Closest Point (DCP) [22] for feature extraction, which then uses an attention module to learn the correspondence between two point clouds. It still utilizes an SVD module to calculate the rotation matrix and translation vector required for the transformation. These algorithms cannot optimize post-processing operations through learning methods during training, resulting in significant limitations in performance. Wang et al. [23] proposed a novel local descriptor-based framework(YOHO). It employs rotation-equivariant descriptors to achieve robust and efficient point cloud registration with superior performance compared to conventional methods. Recently, Zhang et al. [24] presented a novel approach utilizing rotation-invariant features and spatial geometric consistency for robust partial-to-partial point cloud registration, outperforming existing methods, particularly in handling large rotations. Liu et al. [25] proposed a group-wise contrastive learning (GCL) scheme to extract density-invariant geometric features.

End-to-end learning-based point cloud registration. The core idea of these methods is to add the transformation matrix to the learning network to avoid the impact of post-processing operations on the algorithm’s performance. Deng et al. [26] proposed a relative pose regression network that can directly estimate the relative pose of point clouds based on features learned from local descriptors. Yasuhiro et al. [27] proposed PointNetLK by combining PointNet with the Lucas–Kanade (LK) algorithm [28]. DGR [29] utilized fully convolutional geometric features (FCGFs) [30] for feature extraction from point clouds. The six-dimensional convolutional network structure was employed to estimate point correspondences, and a weighted Procrustes model was used to estimate the transformation. Yu et al. [31] employed rotation-invariant and globally aware descriptors for robust point cloud registration (RIGA), surpassing state-of-the-art methods, especially in managing large rotations across diverse datasets. Recently, Yew et al. [13] applied Transformer for the first time in the point cloud registration. It extracted global and local features through a multi-headed attention mechanism (REGTR), alleviating the problem of point cloud registration at low overlap. These methods can achieve higher accuracy but require more complex network structures and greater computational power.

Although the above methods can complete the point cloud data registration at a certain level, they ignore the most critical geometric structure information. Only the extracted features include geometric structure information, and the 3D coordinates obtained will be more accurate, further improving the registration accuracy.

3. Preliminary

3.1. Problem Definition

Three-dimensional point cloud registration task can be described as follows: there are two point clouds to be registered, $\mathbf{X}\in {\mathbb{R}}^{M\times 3}$ and $\mathbf{Y}\in {\mathbb{R}}^{N\times 3}$. $\mathbf{X}$ represents the source point cloud. $\mathbf{Y}$ represents the target point cloud. M and N are the number of points in the source and target point clouds, respectively. The task of 3D point cloud registration involves utilizing a rigid transformation composed of a rotation matrix $\mathit{R}\in SO\left(3\right)$ and a translation vector $\mathit{t}\in {\mathbb{R}}^3$ to align the source point cloud $\mathbf{X}$ with the target point cloud $\mathbf{Y}$. Therefore, the process of 3D point cloud registration is the process of finding the optimal rotation matrix $\mathit{R}$ and translation vector $\mathit{t}$.

3.2. Transformer Model

The Transformer represents a paradigm shift in sequence modeling within the domain of deep learning. Its core concept lies in utilizing self-attention mechanisms for processing sequence data. This mechanism enables the model to dynamically allocate attention weights to various elements within the sequence without needing fixed window sizes or recurrent structures. Such flexibility allows the Transformer model to better capture global and local data information. Additionally, its effectiveness has been demonstrated in the field of computer vision. The self-attention mechanism comprises scaled dot-product attention and multi-head attention.

Scaled dot-product attention. Within the self-attention mechanism, attention weights are computed by scaling the dot product of the query and key vectors and applying the result to the value vector.

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V$$

(1)

where Q, K, and V represent the query, key, and value vectors, respectively, with $d_k$ denoting the dimensionality of the keys.

Multi-head attention. Multi-head attention augments the model’s representative capacity by parallelly applying multiple queries, key, and value projection sets. The results from multiple attention heads are concatenated and linearly transformed to obtain the final output.

$$\mathrm{MultiHead}(Q,K,V)=\mathrm{Concat}({\mathrm{head}}_1,\dots ,{\mathrm{head}}_h)W^O$$

(2)

where each attention head ${\mathrm{head}}_i$ is computed as $\mathrm{Attention}(Q{W}_i^Q,K{W}_i^K,V{W}_i^V)$ and $W_i^Q$, $W_i^K$, $W_i^V$, and $W^O$ represent the weight matrices for linear transformations.

4. The Proposed Method

Transformation estimation from the corresponding points in two point cloud samples is crucial in point cloud registration. Corresponding point estimation involves identifying the correspondence between points in two point cloud frames corresponding to the same object or scene in the same location. Transformation estimation determines the rotation and translation operations required to align the corresponding positions of two point cloud frames seamlessly. The problem definition, the network architecture, and the loss function are presented and discussed in detail accordingly.

4.1. The Overall Network Architecture

REGTR is an end-to-end network based on Transformer, which can predict the probability of being in the overlapping region for each point in the source point cloud and their corresponding positions in the target point cloud cloud [13]. It could effectively extract the global and local features and perform well in predicting the rigid transformation. To overcome the problem of REGTR not obtaining the essential geometric structure, we propose GraM’s overall network architecture that optimizes the REGTR. Specifically, we bind two geometric structure embedded modules on the two self-attention layers in the Transformer cross-coding, respectively. Consequently, they can learn about local features, including information on geometric structure.

The architecture of GraM is devised by embedding the end-to-end point cloud registration network with a geometric structure, as shown in Figure 1. The architecture contains three core modules: (1) Feature extraction module. We utilize the kernel point convolution backbone [32] to extract critical points’ features from the source and target point clouds while downsampling the input point cloud. (2) Cross-encoder module. A cross-encoder embedded with a geometric structure receives the features. It utilizes a multi-head self-attention layer to learn features of non-critical points within the point cloud itself and a multi-head cross-attention layer to learn features corresponding to points to be registered. (3) Output module. The output decoder obtains the predicted corresponding critical point positions and transformation matrices between the two point clouds using simple regression operations.

4.1.1. Feature Extraction

KPConv (Kernel Point Convolution) exhibits excellent spatial preservation in feature extraction in 3D point cloud data. Based on the ability of spatial preservation, it can handle point cloud data of varying densities and shapes, showcasing outstanding performance across multiple tasks. With the features of preprocessed point cloud data, we optimize the feature extraction process using KPConv networks. Specifically, we build a double-input KPConv network with shared weights to extract the same homogeneous characteristics from the original and target point cloud data. It can meet the requirements that they come from the same dataset and can allow swaps. The network iteratively applies a series of residual blocks, including convolution, kernel point convolution, stridden convolution, normalization modules, and LeakyReLU activation functions. Specifically, multiple times of feature extraction and downsampling are firstly performed on the input source point cloud $\mathbf{X}\in {\mathbb{R}}^{M\times 3}$ and target point cloud $\mathbf{Y}\in {\mathbb{R}}^{N\times 3}$, and then, they are transformed into critical point sets $\tilde{\mathbf{X}}\in {\mathbb{R}}^{M^{\prime }\times 3}$ and $\tilde{\mathbf{Y}}\in {\mathbb{R}}^{N^{\prime }\times 3}$, along with their features $F_{\tilde{\mathbf{X}}}\in {\mathbb{R}}^{M^{\prime }\times D}$ and $F_{\tilde{\mathbf{Y}}}\in {\mathbb{R}}^{N^{\prime }\times D}$.

With the above thought, we apply this feature extraction network separately to the source and target point clouds, obtaining critical points’ features for both the source and target point clouds. This process supports the subsequent learning of features in the cross-encoder.

4.1.2. Cross-Encoder

We employ the transformer cross-encoder network to learn features of points in the point cloud and their correspondences with points in the target point cloud. To solve the problem that the output dimensions of different depths from the feature extraction network are diverse, we introduce linear feature projection functions to reduce the dimensions of the outputs before passing them into the cross-encoder. Figure 2 shows the cross-encoder structure, which can extract local and global features. One is the local feature of points outside the key point of a self-point cloud, and the other is the global features that describe the correlation of the two point clouds.

Although the classical Transformer performs sine positional encoding to embed the coordinate information, the coordinate-based encoding is unfixed or invariant. As a result, when executing point cloud registration, the point cloud coordinates change accordingly if different initial poses are used for the same point cloud pair. In this case, coordinate-based coding does not work [33].

In this paper, we replace the sine positional encoding of point clouds in the cross-encoder with geometric structure position encoding. This modification enables the cross-encoder to learn the geometric structure features between critical points before learning self-features and relevant features. It can further improve point cloud registration accuracy. Geometric structure position encoding includes pairwise distance embedding and triangular embedding. The former represents the distance of the pair of critical points, and the latter is the angle of the triple critical points.

Pairwise distance embedding. We assume that $\widehat{P_i}$ and $\widehat{P_j}$ are given points and the distance between the two points $d_{i,j}={||\widehat{P_i}-\widehat{P_j}\parallel }_2$. The distance formula for pairwise distance embedding $r_{i,j}^D$ should satisfy

$$\left\{\begin{array}{c}\hfill r_{i,j,2k}^D=sin\left(\frac{d_{i,j}/{\sigma }_d}{{10000}^{2k/d_t}}\right)\hfill \\ \hfill r_{i,j,2k+1}^D=cos\left(\frac{d_{i,j}/{\sigma }_d}{{10000}^{2k/d_t}}\right)\hfill \end{array}\right.$$

(3)

where $d_t$ is the feature dimension and ${\sigma }_d$ is the temperature coefficient controlling the sensitivity to distance changes.

Triangular embedding. The triangular embedding can be calculated using the same method. Assuming that the given angles is ${\alpha }_{i,j}^k$, the triangular embedding $r_{i,j,k}^A$ can be calculated based on

$$\left\{\begin{array}{c}\hfill r_{i,j,k,2x}^A=sin\left(\frac{{\alpha }_{i,j}^k/{\sigma }_a}{{10000}^{2x/d_t}}\right)\hfill \\ \hfill r_{i,j,k,2x+1}^D=cos\left(\frac{{\alpha }_{i,j}^k/{\sigma }_a}{{10000}^{2x/d_t}}\right)\hfill \end{array}\right.$$

(4)

where ${\sigma }_a$ is another temperature coefficient controlling the sensitivity to angle variations.

4.1.3. Output Decoder

This paper’s output decoder differs from the original Transformer’s decoder in architecture. Since the cross-encoder has already learned the local and global features, there is no need to use attention mechanisms for decoding. Instead, simple regression operations are sufficient to estimate the corresponding positional coordinates and transformation matrices.

In estimating corresponding positional coordinates, we use a two-layer MLP to regress the required coordinates. So, the corresponding position of the critical point $\widehat{\mathbf{X}}$ of the source point cloud in the target point cloud $\widehat{\mathbf{Y}}\in R^{M^{\prime }\times 3}$ is

$$\begin{array}{c}\hfill \widehat{\mathbf{Y}}=ReLU({\overline{F}}_{\tilde{\mathbf{X}}}{W}_1+b_1)W_2+b_2\hfill \end{array}$$

(5)

where $W_1$, $W_2$, $b_1$, and $b_2$ are learnable weights and biases, respectively. Similar methods can be employed to obtain the predicted positions $\widehat{\mathbf{X}}\in {\mathbb{R}}^{N^{\prime }\times 3}$ after receiving the critical points $\widehat{\mathbf{Y}}$ of the target point cloud. Simultaneously, we utilize a fully connected layer with the sigmoid activation function to predict overlap confidences ${\widehat{O}}_{\mathbf{X}}\in {\mathbb{R}}^{M^{\prime }\times 1}$ and ${\widehat{O}}_{\mathbf{Y}}\in {\mathbb{R}}^{N^{\prime }\times 1}$. This design eliminates interference from points outside the overlapping region that cannot accurately predict corresponding relationships, significantly improving the accuracy of the transformation matrix estimation. After obtaining the predicted transformation coordinates, the estimation of the transformation matrix can be performed. Connecting the predicted transformation positions for the two point clouds yields a $M^{\prime }+N^{\prime }$ dimensional corresponding point set, as shown in Equation (6):

$${\widehat{\mathbf{X}}}_{corr}=\left[\begin{array}{c}\tilde{\mathbf{X}}\\ \widehat{\mathbf{X}}\end{array}\right],{\widehat{\mathbf{Y}}}_{corr}=\left[\begin{array}{c}\tilde{\mathbf{Y}}\\ \widehat{\mathbf{Y}}\end{array}\right],{\widehat{O}}_{corr}=\left[\begin{array}{c}{\widehat{O}}_{\mathbf{X}}\\ {\widehat{O}}_{\mathbf{Y}}\end{array}\right]$$

(6)

where $\tilde{\mathbf{X}}$ and $\widehat{\mathbf{Y}}$ represent sets of critical points and $\widehat{\mathbf{X}}$ is the predicted value for the critical points of the target point cloud $\mathbf{Y}$ corresponding to the source point cloud $\mathbf{X}$, while $\widehat{\mathbf{Y}}$ is the predicted value for the critical points of the source point cloud $\mathbf{X}$ corresponding to the target point cloud $\mathbf{Y}$.

$$\widehat{R},\widehat{t}=\underset{R,t}{argmin}\underset{i}{\stackrel{M^{\prime }+N^{\prime }}{\mathsf{\Sigma }}}\widehat{o_i}{\left|\right|R{\widehat{x}}_i+t-{\widehat{y}}_i\left|\right|}^2$$

(7)

where ${\widehat{x}}_i$, ${\widehat{y}}_i$, ${\widehat{o}}_i$ represent i-th rows of matrices ${\widehat{\mathbf{X}}}_{corr}$, ${\widehat{\mathbf{Y}}}_{corr}$, and ${\widehat{o}}_{corr}$, respectively. R is the rotation transformation matrix, t is the translation transformation vector, and $\widehat{R}$ and $\widehat{t}$ are the minimum predicted values for R and t satisfying the Equation (7). In this paper, we follow the approach proposed in [21,34] by using a differentiable weighted Kabsch–Umeyama [35,36] algorithm to solve the equation and obtain the rotation matrix and translation vector.

4.2. The Loss Function

Three loss functions for the supervised training of an end-to-end network incorporating attention mechanisms are used in this paper: the feature loss function, the overlap loss function, and the correspondence loss function.

The feature loss. To obtain the geometric structure when calculating the correspondence of the critical point, we apply InfoNCE loss [37] on the features related to both the current point cloud and another point cloud. Considering the correspondence between the critical point set $x\in \tilde{\mathbf{X}}$ of the source point cloud and the critical point set $\tilde{\mathbf{Y}}$ of the target point cloud, the InfoNCE loss for the source point cloud can be described as follows:

$${\mathcal{L}}_f^{\mathbf{X}}=-{\mathbb{E}}_{x\in \tilde{X}}\left[log\frac{f(x,p_x)}{f(x,p_x)+{\mathsf{\Sigma }}_{n_x}f(x,n_x)}\right]$$

(8)

We follow the work of Oord et al. [37], where the function $f(·)$ in the Equation (8) is a log-linear model, expressed as follows:

$$f(x,c)=exp\left({\overline{f}}_x^T{W}_f{\overline{f}}_c\right)$$

(9)

where ${\overline{f}}_x$ denotes the conditional feature of point x. $p_x$ and $n_x$ denote the sets of critical points in the target point cloud critical point set $\tilde{\mathbf{Y}}$ that match and do not match with x, respective, that is, the positive and negative sample sets. These two sets are determined by the margin values of positive and negative samples $(r_p,r_n)$, where the values of $(r_p,r_n)$ are set to $(m,2m)$ and m is the voxel distance used in the final downsampling layer of KPConv. All negative sample points falling outside the negative margin are included in the set $n_x$.

The overlap loss. To calculate the overlap rate of the point cloud and predict the corresponding point relationships, avoiding some redundant work of critical point extraction, we use the binary cross-entropy loss to calculate overlap loss. The expression for the overlap loss function of the source point cloud $\mathbf{X}$ is depicted as follows:

$${\mathcal{L}}_o^{\mathbf{X}}=-\frac{1}{M^{\prime }}\underset{i}{\stackrel{M^{\prime }}{\mathsf{\Sigma }}}{o}_{{\tilde{x}}_i}^{*}·log{\widehat{o}}_{{\tilde{x}}_i}+(1-o_{{\tilde{x}}_i}^{*})·log(1-{\widehat{o}}_{{\tilde{x}}_i})$$

(10)

To obtain the true value for the overlap labels $o_{{\tilde{x}}_i}^{*}$, we employ the approach proposed by Huang et al. [18] to calculate the truth labels for the original dense point cloud. Thus, the truth label for point ${\mathbf{X}}_i\in X$ is defined as follows:

$$o_{{\tilde{x}}_i}^{*}=\left\{\begin{array}{c}1,\left|\right|{\mathcal{T}}^{*}\left(x_i\right)-NN({\mathcal{T}}^{*}\left(x_i\right),\mathbf{Y})\left|\right|<r_o\hfill \\ 0,otherwise\hfill \end{array}\right.$$

(11)

where ${\mathcal{T}}^{*}\left(x_i\right)$ represents the truth transformation matrix $\{R^{*},t^{*}\}$, $NN(·)$ denotes spatial nearest neighbors, and $r_o$ is a predefined overlap threshold. Subsequently, average pooling is employed to obtain the truth overlap labels $o_{{\tilde{x}}_i}^{*}$ for the downsampled critical points by using the same parameters as the pooling operation in the downsampling step of KPConv.

The loss ${\mathcal{L}}_o^{\mathbf{Y}}$ for the target point cloud Y can be obtained in a similar manner. Thus, the total overlap loss is given by ${\mathcal{L}}_o={\mathcal{L}}_o^{\mathbf{X}}+{\mathcal{L}}_o^{\mathbf{Y}}$.

The correspondence loss. Matching the main points in the overlapping area is used to calculate the overlapping rate. Therefore, we apply an ${\mathcal{L}}_c^{\mathbf{X}}$ loss on the predicted transformation matrix for critical points in the overlapping region. The ${\mathcal{L}}_c^{\mathbf{X}}$ loss for the source point cloud $\mathbf{X}$ is defined as follows:

$${\mathcal{L}}_c^{\mathbf{X}}=\frac{1}{{\mathsf{\Sigma }}_i{o}_{{\tilde{x}}_i}^{*}}\underset{i}{\stackrel{M^{\prime }}{\mathsf{\Sigma }}}{o}_{{\tilde{x}}_i}^{*}|{\mathcal{T}}^{*}\left({\tilde{x}}_i\right)-{\widehat{y}}_i|$$

(12)

The ${\mathcal{L}}_c^{\mathbf{Y}}$ loss on the target point cloud is similar to ${\mathcal{L}}^{\mathbf{X}}$, and the overall loss for the correspondence is ${\mathcal{L}}_c={\mathcal{L}}_c^{\mathbf{X}}+{\mathcal{L}}_c^{\mathbf{Y}}$. Therefore, the final loss in this paper is a weighted sum of these three components: $\mathcal{L}={\mathcal{L}}_c+{\lambda }_o{\mathcal{L}}_o+{\lambda }_f{\mathcal{L}}_f$, where ${\lambda }_o=1.0$ and ${\lambda }_f=0.1$.

5. Experiments

This section presents the dataset, metrics, baselines, experimental setup, main results, and ablation studies. The code is available at https://github.com/liupin-source/CSR-RegTR (accessed on 11 May 2024).

5.1. Dataset

We conducted extensive experiments on the representative ModelNet40 and KITTI datasets. To address issues such as insufficient data volume and information in the ModelNet40 dataset and inaccuracies in some truth labels in the KITTI dataset, we reconstructed a dataset more suitable for 3D point cloud registration tasks.

ModelNet40. ModelNet40 is a subset of the ModelNet dataset built by Princeton University and includes 40 types of point cloud data. We directly sampled the initial ModelNet40 point cloud dataset [38] twice at complete random to generate the source and target point clouds, which do not have precisely corresponding points. Then, we selected 4096 points in each sampling. Finally, we applied segmentation operations to the point cloud data, which can generate datasets with overlap rates of 70% and 50%. These two datasets are named ModelNet40 and LowModelNet40, respectively, in which sample point cloud data are shown in Figure 3 and Figure 4.

KITTI. KIT (Karlsruhe Institute of Technology) and TTI-C (Toyota Technological Institute at Chicago) jointly founded the dataset and obtained data from the collection vehicle equipped with a Velodyne lidar with 0.09 degrees resolution. KITTI contains multiple datasets, such as 3D object detection and visual ranging. We only use point cloud data for 3D registration tasks. To address the problem that some truth labels in the KITTI dataset [39] are not accurate, we perform a manual matching to calibrate the truth labels. Because of the large scale of the KITTI dataset, we employed voxel filtering with a grid size of 0.3 m for downsampling to preserve the density of the 3D point cloud after subsampling. An example of the preprocessed KITTI point cloud dataset is shown in Figure 5.

5.2. Evaluation Metrics

Relative rotation error (RRE). RRE is the degree difference between the predicted rotation matrix and the actual rotation matrix used to measure the error of the rotation matrix.

$$\begin{array}{c}\hfill RRE=co{s}^{-1}\left(\frac{1}{2}trace(R^T\overline{R}-1)\right)\hfill \end{array}$$

(13)

where R represents the true rotation matrix and $\overline{R}$ represents the predicted rotation matrix.

Relative translation error (RTE). RTE refers to the Euclidean distance between the predicted translation vector and the actual translation vector, serving to quantify the error in the translation vector.

$$\begin{array}{c}\hfill RTE={\left||t-\overline{t}|\right|}_2\hfill \end{array}$$

(14)

where t represents the true translation matrix and $\overline{t}$ represents the predicted translation matrix.

Registration recall (RR). RR measures the accuracy of point cloud registration algorithms predicting the transformation matrix. The larger the value, the higher the accuracy of the transformation matrix. RR refers to the average ratio of correspondences correctly matched in the overlapping region to the total ones. This correct match occurs when the source point cloud is registered with the target point cloud using the predicted transformation matrix.

$$RR=\frac{1}{M}\underset{i=1}{\stackrel{M}{\mathsf{\Sigma }}}\left[\sqrt{\frac{1}{\left|e^{*}\right|}\underset{(p_{x_i}^{*},q_{y_i}^{*})\in e^{*}}{\mathsf{\Sigma }}{\left||\widehat{T}{p}_{x_i}^{*}-q_{y_i}^{*}|\right|}_2^2}<{\tau }_3\right]$$

(15)

where $e^{*}$ represents point pairs in the true labels with corresponding relationships and $(p_{x_i}^{*},q_{y_i}^{*})$ denotes a pair of true corresponding points. $\widehat{T}\in SE\left(3\right)$ represents the predicted transformation matrix. Additionally, ${\tau }_3$ represents the error threshold between the predicted and true values.

5.3. Baselines

3DFeatNet [20]: A network for learning feature correspondences from 3D point clouds using weak supervision methods.

RPMNet [21]: A deep learning-based point cloud registration method that is less sensitive to initialization and more robust.

DCP [22]: A learning-based approach that includes a point cloud feature extraction network, point cloud matching prediction based on attention mechanisms, and a differentiable singular value decomposition layer.

PointNetLK [27]: A 3D point cloud registration method that combines PointNet with the LK algorithm.

REGTR [13]: An end-to-end 3D point cloud registration network that utilizes attention mechanisms.

DGR [29]: A differentiable network architecture designed for actual point cloud data.

Predator [18]: A point cloud registration method explicitly designed to handle low overlap scenarios.

5.4. Experimental Setup

For the ModelNet40 and LowModelNet40 datasets, their training sets included 6316 pairs of point cloud data. The former’s test set contained 5995 pairs of point cloud data, and the latter’s test set contained 12,311 pairs of point cloud data. The convolution radius of KPConv is 2.75, and the initial sampling radius is 0.0375. During the training process, we trained the network using the AdamW [40] optimizer with an initial learning rate of 0.0001 and weight decay of 0.0001. The training epochs were 80, and each training iteration was verified in the validation set. For the KITTI dataset, the training and test sets contained 1358 and 555 pairs of point cloud data, respectively. The convolution radius of KPConv was 4.5, and the initial sampling radius was 0.3. The optimizer settings were consistent with the ModelNet40 dataset. The training consisted of 200 epochs, with a learning rate decay of 0.5 every 50 epochs. After each training iteration, the network was tested over the validation set.

5.5. Convergence Analysis

The core idea of GraM is to extract the global and local features containing geometric structure information to improve the importance of the 3D point cloud. To analyze whether it can achieve the above goal, we recorded the losses, creating feature-loss, overlap-loss, and correspondence-loss curves on the ModelNet40 and KITTI datasets, as shown in Figure 6. On the one hand, all loss curves quickly converge (i.e., 10 epochs on the ModelNet40 and 40 epochs on the KITTI), which shows that GraM can quickly position and learn the features related to geometric structure information. On the other hand, all the loss curves are relatively stable, without oscillation. This indicates that the loss function we designed can accurately restrict the limited conditions for each feature learning.

5.6. Comparison with State-of-the-Art Methods

To verify the superiority of GraM’s performance, we compare it with those of feature learning-based methods RPMNet and DCP, as well as end-to-end methods PointNetLK, basic REGTR, etc.

Table 1. The comparison of GraM to the state-of-the-art approaches, where the best results are in bold.

Method	ModelNet40		LowModelNet40		KITTI
	RRE (°)	RTE (m)	RRE (°)	RTE (m)	RRE (°)	RTE (m)
RPMNet	1.712	0.018	7.342	0.124	1.021	0.633
DCP	11.975	0.171	16.501	0.300	0.965	0.583
PointNetLK	29.725	0.297	48.567	0.507	2.352	0.936
REGTR	1.473	0.014	3.930	0.087	0.482	0.425
3DFeatNet	2.057	0.039	4.026	0.073	0.254	0.259
Predator	1.948	0.026	3.568	0.072	0.277	0.068
DGR	2.004	0.024	3.627	0.069	0.373	0.320
GraM	0.925	0.010	2.653	0.049	0.270	0.110

shows the experimental results of the ModelNet40, LowModelNet40, and KITTI datasets on RRE, RTE, and RR evaluation metrics. The results in Table 1 indicate that our method, GraM, is slightly advantageous compared to basic REGTR. However, GraM has a significant advantage over DCP and PointNetLK. The main reason is that processing the ModelNet40 and LowModelNet40 datasets involves only quantitative changes. Significantly, the number of point clouds increases, allowing the network to capture as much sufficient information as possible. In conclusion, our algorithm shows a certain degree of performance improvement effect, as indicated by the registration results shown in Figure 7, Figure 8 and Figure 9.

5.7. Analysis Sensitivity of Sampling Radius

Downsampling is a critical procedure of 3D point cloud data processing in KPConV. The sampling radius is an essential parameter of the process. The appropriate sampling radius can effectively reduce the scale of the point cloud data to facilitate feature learning of subsequent network structures. To analyze the sensitivity and effectiveness of the sampling radius, we conducted experiments using GraM with different sampling radii.

Table 2. Experiment results using GraM with different sampling radii on the KITTI dataset. TC represents the time consumption (hours) of model training. The best results are in bold.

Method	KITTI
	RRE (°)	RTE (m)	RR (%)	TC (h)
radius-0.4	0.352	0.214	97.3	4.27
radius-0.5	0.413	0.325	96.1	6.24
radius-0.3	0.270	0.110	99.8	3.51

displays three metrics’ performance results and the model training’s time consumption on the KITTI dataset. From the performance results in Table 2, we can observe that a large sampling radius does not achieve the high accuracy of our geometrically embedded 3D point cloud registration algorithm because a massive sampling radius may overlook critical point information, preventing the cross-encoding network from learning sufficient features. In particular, the model training time decreases as the sampling radius increases. The core reason is that the small sampling radius can make the data too large, which causes many network parameters and eventually increases model training time.

5.8. Ablation Studies

The following subsections discuss the ablation studies, including the effectiveness of each component of GraM and performances with different loss functions.

5.8.1. Effectiveness of GraM’s Each Component

GraM takes the architecture of the REGTR as a carrier and introduces the feature extraction module (KPConv) and geometric structure embedding (GSE) module of the shared weight. To analyze the effectiveness of these two modules on GraM’s performance, we carried out an ablation study and recorded the results on ModelNet40 and KITTI datasets, as shown in

Table 3. Registration result using different components on ModelNet40 and KITTI datasets. GSE indicates the geometric structure embedding network used to extract geometric structure information. The best results are in bold. *, †, and ‡ refer to the baseline, GraM, and our final GraM. ↓ means the RRE and RTE are reduced.

Method	ModelNet40		KITTI
	RRE (°)	RTE (m)	RRE (°)	RTE (m)
Baseline (REGTR) *	1.473	0.014	0.482	0.425
Our GraM (REGTR+KPConv) †	1.248	0.013	0.324	0.301
Our Final GraM (REGTR+KPConv+GSE) ‡	0.925	0.010	0.270	0.110
† relative to *	0.225↓	0.001↓	0.158↓	0.124↓
‡ relative to †	0.323↓	0.003↓	0.054↓	0.191↓

. Overall, our final GraM (REGTR+KPConv+GSE) achieved optimal performance on all metrics. The table also shows that the two modules we introduce can effectively improve the performance of 3D point cloud registration. From the perspective of individual modules on performance effects, compared with GSE-based (i.e., ‡ relative to †) performance improvement and KPConV-based (i.e., † relative to *) performance improvement, three of the four metrics achieved the optimal result. This illustrates that the contribution of geometric structure embedding rather than KPConv to performance improvement is more significant. The results further verify that geometric structure embedding can effectively extract valuable geometric structure information to 3D points cloud registration.

5.8.2. Effectiveness of GraM with Different Loss Functions

We conducted experiments using GraM with different combinations of loss functions and recorded the results in

Table 4. Registration result of GraM using different loss functions on ModelNet40 and KITTI dataset. The best results are in bold.

Method	ModelNet40		KITTI
	RRE (°)	RTE (m)	RRE (°)	RTE (m)	RR(%)
Baseline (${\mathcal{L}}_c$ loss in Equation (12))	2.442	0.020	0.302	0.174	98.9
Our GraM (${\mathcal{L}}_o+{\mathcal{L}}_f$)	2.206	0.016	0.345	0.142	98.6
Our GraM (${\mathcal{L}}_c+{\mathcal{L}}_f$)	2.125	0.015	0.342	0.139	99.1
Our GraM (${\mathcal{L}}_c+{\mathcal{L}}_o$)	2.241	0.017	0.351	0.153	98.0
Our Final GraM (${\mathcal{L}}_c+{\mathcal{L}}_o+{\mathcal{L}}_f$)	1.623	0.013	0.270	0.110	99.8

. We can observe from Table 4 that any one or any two of the three loss functions cannot achieve satisfactory accuracy. It is worth noting that the algorithm can achieve the best performance only when all three loss functions are used simultaneously for network training. Judging from the analysis of a single loss function, comparing GraM with (${\mathcal{L}}_c+{\mathcal{L}}_f$) to GraM with (${\mathcal{L}}_c+{\mathcal{L}}_o$), the former achieves four maximum values in the four values, which shows that A is more advantageous than performance improvement (i.e., ${\mathcal{L}}_f>{\mathcal{L}}_o$). Similarly, ${\mathcal{L}}_c>{\mathcal{L}}_o$ and ${\mathcal{L}}_c>{\mathcal{L}}_o$. Therefore, we can sort the contribution of the three loss functions to the point cloud distribution performance as follows: ${\mathcal{L}}_f>{\mathcal{L}}_c>{\mathcal{L}}_o$.

6. Conclusions

This paper proposes a 3D point cloud registration method, GraM, which embeds the geometric structure into the attention mechanism to form an end-to-end registration framework. The framework can effectively extract local and global features containing the geometric structure information. With this feature, simple regression is enough to obtain the corresponding position coordinates and transformation matrix, thereby improving the registration accuracy of the point cloud. Extensive experiments show that the proposed method is far superior to existing state-of-the-art methods. In the future, we will explore techniques to achieve better registration accuracy for large-scale point cloud datasets with low overlap rates using lightweight models.