Cyclic Global Guiding Network for Point Cloud Completion

Abstract

The application of 3D scenes has gradually expanded in recent years. A 3D point cloud is unreliable when it is acquired because of the performance of the sensor. Therefore, it causes difficulties in utilization. Point cloud completion can reconstruct and restore sparse and incomplete point clouds to a more realistic shape. We propose a cyclic global guiding network structure and apply it to point cloud completion tasks. While learning the local details of the whole cloud, our network structure can play a guiding role and will not ignore the overall characteristics of the whole cloud. Based on global guidance, we propose a variety of fitting planes and layered folding attention modules to strengthen the local effect. We use the relationship between the point and the plane to increase the compatibility between the network learning and the original sparse point cloud. We use the attention mechanism of the layer overlay to strengthen the local effect between the encode and decode. Therefore, point clouds are more accurate. Our experiments indicate the effectiveness of our method on the ShapeNet, KITTI, and MVP datasets and are superior to other networks.

1. Introduction

Point cloud data obtained directly by radar are the most direct, effective, and convenient representation of 3D data. Each point in the point cloud represents a measured value in the 3D space, including the coordinate value x, y, and depth value z of the ordinary 2D image [1,2,3]. It can represent a collection of massive points in the same spatial reference system. It expresses the spatial distribution and surface features of the target. However, the sparsity of point clouds has always been a serious problem for radar perception [4,5]. How to make sparse point clouds dense and complete is the focus of research [6,7]. Scene-based completion is difficult because of the extreme uncertainty of object and position. Therefore, researchers commence with the individual object and conduct direct completion studies on the point clouds of targets. In addition, current point cloud completion studies only focus on sparse and incomplete point clouds without occlusion interference.

Through the research work GR-Net [8], 3D grids are introduced as intermediate representations to regularize unordered point clouds and to supply the loss of details. It uses a 3D convolution in the point cloud completion network as a feature extraction and has achieved good results. This leads to inadequate learning of structure and context. The regularization of disordered point clouds leads to a deviation of each point [9], which is low but not negligible in accumulation. It reduces some of the unevenness and, in some shapes, causes the overall effect to deviate [10]. However, the accuracy of the overall structure is also vital.

Sometimes, we request high requirements for detail point clouds. However, there are other times when the overall structure of a point cloud is more vital. For example, for outdoor driving, intelligent vehicles request to judge the object and position of surrounding objects according to Lidar. At this point, the tightness and structural accuracy of the point clouds are vital for safe driving.

To solve this pivotal problem and ensure the accuracy of local details and the global structure, we implement cyclic global guidance for the whole network structure in this paper. As we all know, low resolution blurs details. However, at the same time, it costs a degree of global construction at a relatively low cost. On the one hand, we use a double-branch structure in the mesh part, which will retain a complete low-resolution global shape, and reduce the loss of resolution reduction in guiding 3D convolution continuous sampling. On the other hand, we still use the overall global structure to guide the process of gradual up-sampling. These features maintain structural features to enhance the effectiveness of the network.

In addition, Folding-Net [11] proposed a new decoding operation called folding and showed that it is theoretically universal in point cloud reconstruction. However, the 2D grid points of the folding operation are randomly generated and are usually independent of the actual point clouds. Therefore, to reflect the overall features of the point clouds, we use the multiple fitting planes to generate a 2D grid with global features. To improve the quality of the reconstructed point clouds effectively, we fold them into point cloud features between up-sampling and down-sampling. The attention mechanism can select the focusing position to produce a more discriminative feature representation. We integrate folding operation and attention module and use grid points to conduct feature analysis of point cloud information layer by layer to achieve a better point cloud completion effect. It works better through cyclic guidance of up-sampling and down-sampling.

In addition, 3D convolution and 2D convolution have similarities and universalities. With the continuous development of convolution compared with other point cloud completion methods, our network is more promising for the application. Point cloud completion methods mainly include traditional methods and methods of deep learning [12]. Traditional methods include geometry-based methods and allocation-based methods. However, they both require the use of geometric attributes and shapes of objects or complete databases. Due to the high requirements for original information, it does not apply to realistic scenes and cannot complete most of the missing point clouds. The geometric order of the points does not affect its representation of the overall shape in space because the point clouds are disorderly. Therefore, using a neural network to process point clouds directly faces serious challenges. Many basic frameworks for point cloud processing networks have emerged in recent years.

MLP-based: Point-Net [13,14] overcame the influence of the disorder and rotation of point clouds on neural network input and constructed the backbone framework of the point cloud processing network based on MLP (multilayer perceptron), which makes it possible to deal directly with point clouds and avoids the intermediate processing of original data. Since then, people have begun to study it. Folding-Net [11] proposed a novel folding-based decoder and deformed a canonical 2D grid onto the underlying 3D object surface of a point cloud. It can achieve low reconstruction errors, even for objects with delicate structures. PCN [15] operated on the original point cloud directly without any structural assumptions or comments on the underlying shapes. Through the design of the decoder, dense and complete point clouds are generated in the missing area of the input with varying degrees of incompleteness and noise to complete the realistic construction. 3D point capsule networks [16] designed an auto-encoder to deploy the dynamic routing scheme and the peculiar 2D latent space. MSN [17] merged coarse-grained prediction with input point clouds using a novel sampling algorithm. Top-Net [18] generated the structured point cloud without assuming any specific construction or topology on the underlying point set. PMP-Net [19] mimicked the behavior of an earthmover and predicted a unique point moving path for each point according to the constraint of point moving distances to improve the quality of the predicted complete shape. MLP has a poor understanding of context. It requests the addition of additional content to the network that enriches semantic information.

GAN-based: The two networks of GAN (generator and discriminator) are trained and compete in minimization maxima algorithms [20]. Achlioptas et al. [21] trained the deep Auto Encoder (AE) to learn a latent representation first and then train a generative model in that fixed latent space to distinguish synthesized from realistic samples. RL-GAN-Net [22] converted noisy partial point clouds into a high-fidelity completed shape by controlling the GAN. Based on PCN [15], PF-Net [23] estimated the missing point cloud hierarchically by utilizing a feature-points-based multi-scale generating network and used the reinforcement learning agent to control GAN to reduce the prediction time of point cloud completion. Spare-Net [24] regarded the shape feature as a style code that modulates the normalization layers during folding, which considerably enhances its capability. However, compared with the other models, GAN exists in two different networks, not in a single end-to-end network. Sometimes, the training process is erratic.

Graph-based: The point clouds inherently lack topological information. Designing a recover topology-based graph CNN (convolutional neural networks) can enrich the representation power of point clouds [25]. DGCNN [26] proposed a new neural network module, dubbed Edge-Conv, which can be stacked and applied to learn global shape properties. Edge-Conv constructed a local graph explicitly and learned the embeddings for the edges. In multi-layer systems, affinity in feature space captures semantic features over potentially long distances in the original embedding. Song et al. [27] processed the category nodes through a graph convolutional network to generate the global priors adapted to point clouds, and used them for point cloud segmentation. LDGCNN [28] eliminated the transformation network from DGCNN [26] and connected hierarchical features from different dynamic graphs to calculate informative edge vectors and avoid vanishing gradient problems. However, the feature extraction effect of the graph convolution structure may not be as good as that of convolution, and the learning ability of the point cloud may be insufficient. It is possible to increase feasibility by adding branching and deep optimization.

3D Conv-based: 3D convolution is developing in various fields related to point clouds. There will be broader room for progress in the future. 3D convolution can extract point cloud features after regularization. Point-Cnov [29] proposed a density convolution that can fully approximate 3D continuous convolution on any set of 3D points. Hua et al. [30] presented a convolutional neural network for semantic segmentation and object recognition with 3D point clouds. Lei et al. [31] proposed an octree-guided neural network architecture and spherical convolutional kernel for machine learning from arbitrary 3D point clouds. The network architecture capitalizes on the sparse nature of irregular point clouds and hierarchically coarsens the data representation with space partitioning. GR-Net [8] introduced 3D grids as intermediate representations to regularize unordered point clouds and convert between point clouds and 3D grids without losing structural information. Although the method achieves relatively accurate results, the detailed features are still lost after quantization. Therefore, it is necessary to have a good grasp of the details and overall features by learning about global and local features circularly.

Transformer-based: With the transformer becoming more widely used in natural language processing and vision [32], PT [33] and PCT [34] have applied the transformer to point clouds. It is inherently permutation invariant for processing a sequence of points; therefore, it is well suited for point cloud learning. By representing the point cloud as a set of unordered groups of points with position embeddings, PoinTr [35] converted the point cloud to a sequence of point proxies and adopted a transformer encoder–decoder architecture for point cloud completion. However, the transformer is costly in construction and poor in universality. It may be improved through lightweight research of transformers in the future.

Others: Sun et al. [36] proposed a new representation method for point clouds that learns a set of quadratic terms based on static and global reference surfaces to describe 3D shapes. However, it is not applicable to point cloud completion because the point clouds are incomplete.

In summary, frameworks for point cloud processing are branching out. However, with the improvement of 2D information processing, many innovative ideas about convolution have been put forward. 3D convolution is similar to 2D convolution, and many methods are applicable. Therefore, we believe that a framework based on 3D convolution has rosy prospects for development. In general, we designed corresponding modules specifically to improve the effect of local details and global structure and to strengthen the final completion effect.

We propose a network with cyclic global guidance for point cloud completion. Our main contributions to the paper are summarized as follows:

• We designed the global guided down-sampling and up-sampling constructions. The complete and dense point clouds are reconstructed by combining overall construction with contextual semantic information.
• We integrated the traditional fitting plane of point clouds adapted to point cloud features into the deep learning network, which uses the original features of point clouds to reduce uncertainty.
• We combine a folding operation with an attention mechanism to complete the point cloud by stratification for focusing position creatively.

2. Methods

In this section, we introduce the overall structure of our CGG-Net. As shown in Figure 1, our network consists of one main route and several sub-routes. We use 3D convolution as a feature extraction framework, which refers to the idea of GR-Net [11]. The overall shape is an innovatively guided optimization that considers global influence from various aspects and perspectives.

From the perspective of the cascade, our circular construction network includes a global guided down-sampling module, layered folded attention module, multiple fitting plane module, and global guided up-sampling module. They are interlinked and distributed in various parts of the network construction to work together cyclically to guide the learning of the whole in the correct direction. In addition, the modules also work on the network in parallel. For the basic framework, we carried out global guided sampling to improve the learning effect of the network on the overall construction in the process of convolution of meshed point clouds. At the same time, we always pay attention to global structural integrity in our network architecture through folding and attention modules based on overall construction.

2.1. Global Guided Sampling

The global guided sampling module is the backbone framework of our network. By considering the whole and the details of the point clouds, the network can produce better results. Zhang et al. [37] designed a feature-expansion module. It can learn local and global point features via a down-feature operator and an up-feature operator. Furthermore, we do not handle both the global and the local in one way. We propose integrating global-based guidance into two samplings. A new global guided branch connects to the main route. It plays a role in reducing the loss of the overall construction and gathering dispersed points. After each convolution, the whole will be aggregated through this branch to prevent the loss of global construction. In the process of convolution, the branch will constantly supervise and correct the learning direction of convolution and integrate the original point cloud information to obtain a compact internal representation.

In addition, the original 3D convolution sampling will result in a loss due to the scale drop. We use the parallel structure of the main and branch in convolution. The global guided sampling network can not only guarantee the effect of the global structure in the process of convolution but also reduce the loss of most resolution because of the dense fusion of the main path and the branch path. As the main network, the guided network is also multi-resolution, which can be targeted to compensate for the loss of different resolutions. We still use a global bootstrap to reduce losses once again when upsampling.

The output of the layer $\omega$ can be defined as

$${({\Gamma }^{\omega })}_{\frac{1}{2}\theta }=\Psi (F{({\Gamma }^{\omega -1})}_{\theta },F{({\Gamma }_{j_i}^{\omega -1})}_{\theta })$$

(1)

$F(\cdot )$ is the convolution. ${\Gamma }^{\omega -1}$ is the main output of the $\omega -1$ layer. The global guide branch ${\Gamma }_{j_i}^{\omega -1}\in \Re (i=1,\dots ,n)$ is the lead range of each branch. $j$ is the number of branches and $i$ is the serial number of the branches. $\theta$ is the resolution of the feature. $\Psi (\cdot )$ is the way of connecting branches. We discovered that connecting worked better than superposing. Connections can preserve more differences between branches instead of merging information, which would lose the original features. In addition, the connection parameter is for the number of channels, not the resolution size.

Hourglass 3D convolution stacked for feature extraction. There is a loss of global features. We designed an auxiliary global branch to guide sampling at each step. As depicted in Figure 2, our network gradually refines and filters shapes through the overall structure. At the same time, the resolution of our overall construction also changes to accommodate different stages of sampling. The aircraft point clouds are refined gradually by cyclic sampling. Step by step, it becomes denser and has less edge noise.

Then, we interpolate the low-resolution features back to the high resolution. The branching construction is similar to the subsampling mentioned above. Features are extracted gradually through stacking of 3D convolutions, which is necessary for details.

The output of the layer $\omega$ can be defined as

$${({\Gamma }^{\prime }{}^{\omega })}_{2\theta }=\Psi (F{({\Gamma }^{\prime }{}^{\omega -1})}_{\theta },F{({\Gamma }^{\prime }{}_{j_i}^{\omega -1})}_{\theta })$$

(2)

The construction can fabricate the absence of global features in the sampling process through a cyclic global guide. The network grasps the overall direction of feature learning.

2.2. Multiple Fitting Planes

Folding 2D grids into incomplete points has been proven to work [10]. However, the randomly generated grid loses its connection to its construction. A multiple fitting plane module is proposed to establish this global relation. Based on the idea of the virtual point method, VR-NET [38] learns a new virtual point to form a pair of consistent point clouds. In point cloud completion, we can form corresponding virtual relationships for each or multiple point clouds. The corresponding relationship is then used as the basic parameter of point cloud completion to generate features.

Point clouds have disorder and rotation. Most feature representation methods commonly used by other data forms are not applicable to point clouds [39]. However, the normal vector, the distance to the center of gravity point, and the average distance of the neighbor node parameters are the best choices for the representative features of point cloud information. Because of this information, point cloud translation, rotation, and noise points cannot cost a lot of change. The fitting plane is the projection of the 3D point cloud to 2D and represents the traditional features of the sub-point set. We proposed multiple fitting planes to integrate the traditional point cloud representation method into a deep learning network. It can learn the overall construction of the point clouds better.

The fitting plane is precomputed before learning. For a point cloud $P=\left\{p_i(x_i,y_i,z_i)|i=1,\dots ,N\right\}$, the nearest neighbor point of the point $p_i$ can be defined as

$$P^{\prime }=\left\{p_{ij}(x_{ij},y_{ij},z_{ij})|j=1,\dots ,k\right\}$$

(3)

where $N$ is the size of the point cloud, $k$ is the number of neighbor points.

Neighborhood centers $O_i$ can be obtained as

$$O_i=\frac{1}{k}{\displaystyle \sum _{j=1}^k{p}_{ij}}$$

(4)

Then, the normal vector of the point can be obtained by solving the covariance matrix of the point $T_i$. ${\lambda }_3$ is the minimum eigenvalue of the solution.

$$T_i={\left[\begin{array}{c}\begin{array}{c}\begin{array}{c}{p}_{i1}-O_i\\ p_{i2}-O_i\end{array}\\ \cdots \end{array}\\ p_{ik}-O_i\end{array}\right]}^T\left[\begin{array}{c}\begin{array}{c}\begin{array}{c}{p}_{i1}-O_i\\ p_{i2}-O_i\end{array}\\ \cdots \end{array}\\ p_{ik}-O_i\end{array}\right]$$

(5)

Therefore, ${\lambda }_3$ is the normal vector in which the point fits into the plane. In the representation of vectors, the normal vector of the point is obtained after uniformization as

$$N({\lambda }_3)=(a,b,c)$$

(6)

In the representation of a plane, the fitting plane of $p_i(x_i,y_i,z_i)$ is calculated as

$$C_i:a(X-x_i)+b(Y-y_i)+c(Z-z_i)=0$$

(7)

Folding-Net [11] used two consecutive three-layer perceptions to wrap a constant 2D mesh into the shape of an input point cloud. Because the network incorporates mapping from lower dimensions, it increases the possibility of more points. However, 2D meshes are always random. The randomness of point clouds for different objects is the same. Therefore, we propose that the fitting plane be used as a feature to construct the mesh to guide the folding effect. For different sparse point clouds, the constructed grid is closely related to itself and irreplaceable, which can improve the folding effect.

The set of fitting planes of each point in the point cloud ${\rm T}=\left\{C_i|i=1,\dots ,N\right\}$ can represent the features of the whole point clouds. The planes computed by different sub-points are discontinuous. The normal vectors are discrete. 2D grids are constructed by discrete normal vectors according to the quadrants of the coordinate system, with the normal vectors of the surface making the 2D features correspond to 3D. In this way, the global features can make the most of guiding the folding.

As shown in Figure 3, we have fully integrated the traditional optimal plane of the point cloud with the folding method of deep learning. This helps to improve the density and compactness of the point clouds. Learning is in close contact with the original construction. It can output better reconstruction point clouds.

2.3. Layered Folding Attention

We still pay special attention to global construction in the layered folding attention module. We integrated the attention mechanism into the network in layers. Every layer must be combined with the last result to ensure the accuracy of the learning direction.

Zhang et al. [40] proposed a network combining multi-scale hierarchical feature fusion to gradually and adaptively improve network performance in image defogging. The layered approach can reduce feature redundancy and learn from compact internal representations. The attention mechanism is widely used in neural network architecture. Its essence is to assign different attention to the input weights to maximize context and better results. However, it operates on the features themselves or on residual constructions [41]. We put forward the idea of adding folding points to the network through attention in Figure 1. The attention is no longer limited to its feature extraction but becomes a better way to fusion and obtain more effective information. We designed a layered attention mechanism and used it for further context fusion of features and enhanced information association layer by layer.

The outlier noise point of the point cloud is also a problem for the point cloud completion task. We believe that outlier noise points are caused by insufficient context processing of the local details of point clouds. Therefore, our layered folding attention module is used in the intermediate step with feature extraction. It can perform contextual fusion and processing of features in convolution layer by layer, which can eliminate deviation points and error points.

As shown in Figure 1, point cloud features are layered first. Then, they are fused layer by layer in our network, starting with the features of the grid and the top layer and working down. We integrate the grid into the network rather than fold it directly.

The output ${\Omega }_n$ can be regarded as follows:

$$\{\begin{array}{cc}{\Omega }_n=\Psi (F(G),{\Omega }_{n-1})& n=1\\ {\Omega }_n=\Psi (P_{n-1},F({\Omega }_{n-1}))& n>2\end{array}$$

(8)

where $n$ is the number of layers, $G$ is the grid to be folded, $F(\cdot )$ is the variable convolution, $P$ is the slice of the input feature, and $\Psi (\cdot )$ is a connection method of computing.

2.4. Others

Our network references the GR-Net [8] backbone framework. 3D convolution is applied to feature extraction of the point cloud by gridding and gridding reverse. We obtained the final point clouds after feature sampling. In this section, we introduce the basic modules from GR-Net [8].

2.4.1. Gridding and Gridding Reverse

To preserve the spatial layout and further processing, the point cloud $p_i(x_i,y_i,z_i),i=1,\dots ,n$ with $n$ points is represented as a regular 3D grid $T$ with $N$ resolution consisting of two parts $V$ and $W$.

$${\rm T}=\langle V,W\rangle$$

(9)

where $V=\left\{v_i\right\},i=1,\dots ,n$ represents eight neighbor coordinates vertices in each cell. $W=\left\{w_i\right\},i=1,\dots ,n$ is the point of the cell.

$v_i$ is defined as:

$$v_i\in \left\{(-\frac{N}{2},-\frac{N}{2},-\frac{N}{2}),\dots ,(\frac{N}{2}-1,\frac{N}{2}-1,\frac{N}{2}-1)\right\}$$

(10)

According to GR-Net [8], $w_i$ is defined as

$$\{\begin{array}{cc}{w}_i={\displaystyle \sum _{p\in {\rm N}(v_i)}\frac{\omega (v_i,p)}{\left|{\rm N}(v_i)\right|}}& \left|{\rm N}(v_i)\right|\ne 0\\ w_i=0& \left|{\rm N}(v_i)\right|=0\end{array}$$

(11)

$$\omega (v_i,p)=(1-\left|x_i-x\right|)(1-\left|y_i-y\right|)(1-\left|z_i-z\right|)$$

(12)

where $\left|{\rm N}(v_i)\right|$ is the number of neighboring points of $v_i$.

Conversely, a grid inverse operation is required to recover each of the eight neighborhood points ${\Theta }_i,i=1,\dots ,8$ of each cell grid at the end of the network, which is defined as

$$\{\begin{array}{cc}{p}_i^c=\frac{{\displaystyle \sum w_{\theta }{}^{\prime }{v}_{\theta }}}{{\displaystyle \sum w_{\theta }{}^{\prime }}}& {\displaystyle \sum w_{\theta }{}^{\prime }}\ne 0\\ ignore& {\displaystyle \sum w_{\theta }{}^{\prime }}=0\end{array},\theta \in {\Theta }_i$$

(13)

$w_{\theta }{}^{\prime }$ and $v_{\theta }$ still represent the coordinates and the values of the eight vertices.

2.4.2. Feature Sampling

Max-pooling may cause the loss of local context information and affect the recovery effect of details. Therefore, feature sampling replaces it according to GR-Net [8]. We connected eight neighbor points in the cell grid in series as a feature of this point. Points are extracted from the coarse point cloud to form a complete point cloud.

2.5. Loss Function

We take the Chamfer Distance (CD) as the loss function [2]. For point clouds $S_1$ and $S_2$, the CD is defined as

$$d_{CD}(S_1,S_2)={\displaystyle \sum _{x\in S_1}\underset{y\in S_2}{\min }{\Vert x-y\Vert }_2^2}+{\displaystyle \sum _{y\in S_2}\underset{x\in S_1}{\min }{\Vert x-y\Vert }_2^2}$$

(14)

CD measures the average distance between each point in a point and the nearest point in another point clouds [42]. It is also the most extensive way to judge the completion effect.

3. Results and Discussion

We implemented the network using Pytorch and used the Adam optimizer with $\beta 1=0.9$ and $\beta 2=0.999$ during training. The initial learning rate is set to ${10}^{-4}$ and is decayed by 0.1 at 100 and 150 epochs. We train the network for 200 epochs with 32 batch sizes on one GeForce RTX 2080 Ti GPU. It takes about 8 min per epoch on ShapeNet.

We use the CD and the F-Score (FS) to measure the effectiveness of the network [43]. The CD is defined in the loss function section and is more computationally efficient and widely used. The FS is defined as follows:

$$FS=\frac{2P(d)R(d)}{P(d)+R(d)}$$

(15)

where $P$ is precision. It calculated the percentage of corresponding points in the reconstructed point clouds within a certain threshold distance from the ground truth value. $R$ is the recall rate. It calculated the percentage of the ground-truth value corresponding to the correct point within a certain threshold with the reconstructed point cloud distance.

We use Uniformity from PU-GAN [20] to test the integrity of the point clouds. This can better show the effect of the global completion we pursue. Uniformity includes global and local measures. It is defined as follows:

$$Uniformity(P)=\frac{1}{N}{\displaystyle \sum _{i=1}^n{U}_{imbalance}(S_i)U_{clutter}(S_i)}$$

(16)

$$U_{imbalance}(S_i)=\frac{{(|S_i|-\widehat{n})}^2}{\widehat{n}}$$

(17)

$$U_{clutter}(S_i)=\frac{1}{\left|S_i\right|}{\displaystyle \sum _{j=1}^{\left|S_i\right|}\frac{{(d_{i,j}-\widehat{d})}^2}{\widehat{d}}}$$

(18)

where $S_i$ is a subset of point clouds $P$. It was calculated using the farthest sampling and the ball query of radius. And $\widehat{n}$ is the expected number of points in $S_i$, $d_{i,j}$ is the distance to the nearest neighbor for the $j-th$ point.

If $S_i$ has a uniform distribution, $\widehat{d}$ is defined as

$$\widehat{d}=\sqrt{\frac{2\pi P}{\left|S_i\right|\sqrt{3}}}$$

(19)

3.1. Results of Comparative Experiments

3.1.1. ShapeNet

The ShapeNet from PCN [14] includes eight categories, such as aircraft, automobiles, etc. There are 30,974 models of ground truth extracted from the categories. This makes the input distribution more similar to the sensor data in the real world. The 16,384 sampling points of the object were back-projected to 3D through 2.5D depth images to generate partial point clouds rather than a subset of complete point clouds [44]. The number of sparse point clouds for each object is not fixed. Most of them are in the range of 500 to 2000. Through a deep learning network, we complete the number of a point cloud of the object to 16,384 points and compare them with the ground truth.

A comparison of our network effects is shown in

Table 1. The results of the experiment using chamfer distance ×10³ with L2 norm on ShapeNet (lower is better). The bold numbers are the best.

Model	F-Net	Top-Net	MSN	GR-Net	Spare-Net	CGG-Net
Airplane	0.62	0.22	0.25	0.29	0.18	0.27
Cabinet	1.61	0.56	0.97	0.63	0.66	0.59
Car	0.62	0.35	0.45	0.32	0.36	0.31
Chair	1.55	0.63	0.77	0.55	0.62	0.54
Lamp	2.03	0.75	0.93	0.58	0.63	0.38
Sofa	1.54	0.69	1.15	0.69	0.79	0.76
Table	1.53	0.48	0.67	0.48	0.50	0.41
Watercraft	0.91	0.44	0.49	0.31	0.38	0.29
Overall	1.30	0.52	0.71	0.48	0.52	0.44

. The CD of most categories is significantly lower than that of other networks. The overall CD of our network is approximately 0.04 less than that of GR-Net [8] (using the same loss function and the same training parameters). At the same time, our CD results are much better than those of other networks.

Our point cloud completion results on ShapeNet [24] are shown in Figure 4. Our CGG-Net completes the missing shape well and makes it close to the ground truth with less noise.

The time of the network is also vital. It can ensure real-time performance and wide application. As shown in

Table 2. Results of comparative experiments on ShapeNet. The bold numbers are the best.

Model	GAN	CD	FS	Time
Spare-Net	√	0.52	0.6607	1.055
GR-Net	×	0.48	0.6179	0.020
CGG-Net	×	0.44	0.6266	0.022

, our network has the best CD value. Although Spare-Net [24] has the best FS value, it uses GAN construction. In addition, although the FS of our network is only about 0.035 less than Spare-Net [24], the time is almost 100 times better. Although our network time is only about 0.002s longer than GR-Net [8], the FS improves by about 0.01. This proves that our network has an advantage in effect and time.

3.1.2. KITTI

KITTI without ground truth is also from PCN [15]. PCN [15] extracts the point clouds according to the points in the object surrounding the box of the car, which generates 2483 partial point clouds. Finally, each point cloud is converted to a world coordinate system [45]. Each car has a smaller number of points, with most having fewer than 500 points. For unity, we still reconstruct it to 16,384 points.

We show the completion effect in Figure 5. The shape of the car is completed nicely through our network. By training the vehicle model on ShapeNet, the effective network parameters for point clouds are obtained to reconstruct KITTI. As shown in

Table 3. The uniformity results of comparative experiments on ShapeNet-car and KITTI (lower is better). The bold numbers are the best.

Datasets	Percentage	Atlas-Net	PCN	F-Net	Top-Net	MSN	GR-Net	CGG-Net
ShapeNet-Car	0.6%	-	-	-	-	-	0.23	0.12
	0.8%	-	-	-	-	-	0.14	0.11
	1.0%	-	-	-	-	-	0.08	0.07
	1.2%	-	-	-	-	-	0.05	0.04
KITTI	0.6%	1.01	5.81	1.30	1.32	0.68	0.27	0.27
	0.8%	0.87	7.71	1.26	1.22	0.52	0.20	0.19
	1.0%	0.76	9.33	1.16	1.07	0.46	0.15	0.13
	1.2%	0.69	10.82	1.06	0.95	0.38	0.12	0.10

, we use Uniformity to measure the effect of point clouds. Our network is significantly better at recreating the car. In all the percentages calculated, our vehicle construction as a whole receives the best completion.

3.1.3. MVP

The MVP is a high-quality, multi-view partial point cloud dataset [6]. Besides the conventional 8 categories in existing datasets on ShapeNet, MVP allows the evaluation of 8 additional categories. It consists of over 100,000 high-quality incomplete and complete point clouds. The number of sparse point clouds is about 1000 points. Similarly, we completed 16,384 points for comparison.

Top-Net [18] selection tree-shaped mlp focuses on the overall structure. GR-Net [8] uses the same 3D convolution structure as our network. Therefore, we chose them for comparison with our network experiment on MVP to display global performance. Our results on the MVP using CD × 102 and FS are shown in

Table 4. Results of comparative experiments on MVP. The bold numbers are the best.

	Model	CD_t	CD_p	FS
MLP tree-based	TOP-Net	1.915	0.120	0.299
3D Conv-based	GR-Net	1.871	0.172	0.377
	CGG-Net	1.718	0.143	0.398

. The category of MVP is similar to ShapeNet. Therefore, we chose the overall comparison results. Our network is built based on 3D convolution, so we compared it with GR-Net [8] based on the same foundation. In addition, we chose two other MLP-based networks for the experiment. CD_t and CD_p are two different CD calculation methods. Even with a little more time, our network achieved better results.

$$CD\_t=mean(C{D}_1)+mean(C{D}_2)$$

(20)

$$CD\_p=0.5(mean(\sqrt{C{D}_1})+mean(\sqrt{C{D}_2}))$$

(21)

where $C{D}_1$ is the chamfering distance from the completion point cloud to the ground truth. In contrast, $C{D}_2$ is the chamfering distance from the ground truth to the completion point cloud.

3.2. Results of Ablation Experiments

3.2.1. Global Guided Sampling

As shown in

Table 5. Ablation experiment of global guided sampling. The bold numbers are the best.

Categories/ Model		Air-Plane	Cabinet	Car	Chair	Lamp	Sofa	Table	Water-Craft	Overall
CD (Lower is better)	None	0.28	0.60	0.33	0.49	0.51	0.98	0.45	0.32	0.50
	Add	0.27	0.58	0.35	0.56	0.41	0.84	0.43	0.30	0.47
FS (Higher is better)	None	0.77	0.51	0.60	0.58	0.69	0.48	0.64	0.69	0.61
	Add	0.78	0.51	0.60	0.58	0.69	0.49	0.65	0.70	0.62

, the CD is about 0.03 lower after global guided sampling. The FS is about 0.01 higher than before. Overall, more categories receive better results using our module.

It is vital to control the overall shape of the point clouds continuously. As shown in Figure 6 and Figure 7, the point cloud completion effect is improved significantly through the global sampling module. The wing and fuselage sections of the aircraft have better shape completion and structural details, as shown in Figure 6. In Figure 7, global guided sampling makes the overall surface of the lamp smooth. The shape of the lamp cap was complete. At the same time, the surface of the lamp post becomes smooth after using the module because of the reduction of noise.

3.2.2. Multiple Fitting Planes

As shown in

Table 6. Ablation experiment of multiple fitting planes. The bold numbers are the best.

Categories	CD (Lower Is Better)
	None	Add
Airplane	0.27	0.29
Cabinet	0.58	0.63
Car	0.35	0.32
Chair	0.56	0.53
Lamp	0.42	0.42
Sofa	0.84	0.65
Table	0.43	0.46
Watercraft	0.30	0.30
Overall	0.47	0.46

, the multiple fitting planes module has a significant impact on reducing CD. We reduce more than 0.04 in CD after adding the module. The advantages of the modules are also shown in Figure 8. The point clouds are denser than before with the use of modules. At the same time, the shape and contour of the point cloud object are better completed.

3.2.3. Layered Folding Attention

We conduct grouping experiments with different parameters for layered folding attention modules. The experimental results are shown in

Table 7. Ablation experiment of layered folding attention. The bold numbers are the best.

Categories/ Model		Airplane	Cabinet	Car	Chair	Lamp	Sofa	Table	Watercraft	Overall
CD (Lower is better)	None	0.29	0.63	0.32	0.53	0.42	0.65	0.46	0.30	0.46
	16	0.29	0.56	0.35	0.51	0.41	0.78	0.48	0.29	0.46
	32	0.27	0.59	0.31	0.54	0.38	0.76	0.41	0.29	0.44
	64	0.24	0.55	0.32	0.55	0.39	0.78	0.57	0.30	0.46
FS (Higher is better)	None	0.78	0.52	0.60	0.59	0.70	0.49	0.65	0.70	0.62
	16	0.77	0.51	0.60	0.58	0.69	0.48	0.64	0.69	0.62
	32	0.79	0.53	0.61	0.59	0.70	0.50	0.66	0.70	0.64
	64	0.78	0.52	0.60	0.58	0.70	0.49	0.65	0.69	0.63

. The best effect can be obtained using a 2D grid with a length of 32 sides. The CD is more than 0.05 lower than before. The FS is about 0.02 higher. Figure 9 shows the point cloud completion effect visually after layered folding.

3.2.4. Summary of the Experiment

The results of our progressive ablation experiment are shown in

Table 8. Ablation experiment of cyclic global guiding network. The bold numbers are the best.

Model	CD	FS	Weights (M)	Time (s)
Original	0.50	0.61	306.8	0.019
Original+GGS	0.47	0.62	307.0	0.020
Original+GGS+MFP	0.46	0.62	321.5	0.020
Original+GGS+MFP+LFA-16	0.46	0.62	338.3	0.022
Original+GGS+MFP+LFA-32	0.44	0.64	338.3	0.022
Original+GGS+MFP+LFA-64	0.46	0.63	338.3	0.022

. Our final selections are displayed in bold. The weights and times of our network only slightly increase. However, the CD and FS are gradually better than in the previous step. At the same time, we found that the size of the grid dot was not the larger the better. For this, we think that the folded grid needs to maintain a certain kind of fitting and the suitability of the point cloud object.

The CD of the test dataset varies with the number of training epochs as shown in Figure 10. We took a multiple of 25 from 25 to 200 in the experiment. All of our ideas have had beneficial effects. Our CGG-Net training curve is gentler and more difficult to overfit. The expression ability is better than that of others.

4. Conclusions

In this paper, we propose a new CGG-Net for sparse and partial point cloud completion that pays more attention to the control of the whole construction completion. In network construction, the whole construction of the point clouds is effectively taken into account through cyclic global guidance. In addition, we propose a folding method of multiple fitting planes to integrate traditional feature representations into deep learning. Finally, the point cloud is carefully completed step-by-step through layered folding attention. We proved the effectiveness of the network in the experiment. The superiority of the network is shown by comparison with other methods on ShapeNet, KITTI, and MVP.