A LiDAR Multi-Object Detection Algorithm for Autonomous Driving

Abstract

Three-dimensional object detection is the core of an autonomous driving perception system, which detects and analyzes targets around the vehicle to obtain their sizes, shapes, and categories to provide reliable operational decisions for achieving autonomous driving. To improve the detection and localization accuracy of multi-object targets such as surrounding vehicles and pedestrians in autonomous driving scenarios, based on PointPillars fast object detection network, a three-dimensional object detection algorithm based on the channel attention mechanism, ECA Modules-PointPillars, is proposed. Firstly, the improved algorithm uses point cloud columnarization features to convert a three-dimensional point cloud into a two-dimensional pseudo-image. Then, combining the 2D backbone network for feature extraction with the Efficient Channel Attention (ECA) modules to achieve the enhancement of the positional feature information in the pseudo-image and the weakening of the irrelevant feature information such as background noise. Finally, the single-shot multibox detector (SSD) algorithm was used to complete the 3D object detection task. The experimental results show that the improved algorithm improves the mAP by 3.84% and 4.04% in BEV mode and 3D mode, respectively, compared to PointPillars, which improves the mAP by 4.64% and 5.89% in BEV mode and 3D mode, respectively, compared to F-PointNet, improves the mAP by 11.78% and 14.19% in BEV mode and 3D mode, respectively, compared to VoxelNet, and improves the mAP by 9.47% and 6.55% in BEV mode and 3D mode, respectively, compared to SECOND, demonstrating the effectiveness and reliability of the improved algorithms in autonomous driving scenarios.

1. Introduction

Three-dimensional object detection is the core of an autonomous driving perception system, which detects and analyzes targets around the vehicle to obtain their size, shape, category, and motion characteristics, providing reliable operational decisions for realizing autonomous driving [1,2,3]. In recent years, scholars at home and abroad have proposed a variety of effective and feasible 3D object detection algorithms according to the categories of sensors used in the visual perception system, among which the method based on LIDAR point cloud has been deeply studied due to its excellent detection performance [4].

Depending on how the point cloud is processed, it is further categorized into multi-view based, point based, and voxel based methods [5]. Multi-view based methods first convert the point cloud into a front view or bird’s eye view (BEV), and then use mature 2D object detection algorithms for feature extraction [6]. For example, VeloFCN [7] first converts the point cloud into a front view 2D feature map and then uses a 2D object detection algorithm for object detection, which combined with VoxelNet [8] and Fully Convolutional Networks (FCNs) performs well in multi-category and multi-object detection tasks, and can detect pedestrians, vehicles, bicycles, and other multiple objects simultaneously. However, the algorithm consumes a large amount of memory and computational resources, resulting in slower network training and inference, which requires high performance computing devices and longer training and testing time. PIXOR [9] converts the point cloud height information to color channels to achieve a bird’s eye view representation, and then uses a 2D detection network for detection, which is applicable to various roadside light environments for cars, solves the occlusion problem of the front view, and has high accuracy and robustness. However, the algorithm has high computational complexity and large data volume, and requires the use of GPU for accelerated computation.

Point-based methods directly use sparse point clouds for processing, which enhances the feature extraction and feature characterization of point clouds. For example, PointNet [10] proposed by Qi C R et al. from Stanford University directly uses the original point cloud to learn spatial information without conversion to other forms, which can be applied to various point cloud processing tasks. Aiming at the problem of PointNet’s insensitivity to local features and poor extraction effect, the PointNet++ [11] algorithm optimizes the processing of global features in the PointNet algorithm, and improves the algorithm’s ability to perceive both local and global structures through hierarchical aggregation and local feature extraction, and also has good stability for data with noise and sampling density changes. However, the algorithm has difficulty capturing the local feature information of non-uniformly distributed or sparse point clouds in some complex and diverse 3D scene tasks. F-PointNet [12] introduces fine-grained feature representations and multi-scale contextual information, further improves the PointNet algorithm, and can effectively classify, segment, and estimate the point cloud by adding a branching network and extracting the fine-grained feature representations of each point in the point cloud, with better performance [13]. However, for point clouds with different scales and densities, the performance of this algorithmic model may be degraded, and further optimization and improvement is needed. The output of the model may vary greatly with different arrangements of the input point cloud, and the interpretability is poor.

Voxel based methods usually convert irregular point clouds into regular voxel representations [14]. For example, ZHOU Y et al. proposed an end-to-end deep neural network, VoxelNet [8], for 3D object detection in unmanned scenarios, which preserves the original features of the point cloud, but the traditional 3D convolutional network of VoxelNet is computationally cumbersome and slow in operation. In order to improve the 3D convolutional operation speed of the point cloud, Yan Y et al. proposed the SECOND [15] method, which has good adaptability to complex scenes, and is able to deal with scenes such as vehicles, pedestrians, and buildings almost in real time and efficiently, and at the same time, it is applicable to all kinds of LIDAR data, and it has a better versatility. However, the algorithm is based on a sliding window, which will waste some computational resources and reduce the processing efficiency and speed. In order to speed up the operation of the algorithm, Lang A H et al. proposed the PointPillars [16] method, which adopts the point cloud columnar structured features and then transforms them into a two-dimensional pseudo-image, and feature extraction by two-dimensional convolutional neural network, which solves the problem of three-dimensional object detection that cannot be practically applied due to the slow computing speed, and its discarding of the traditional three-dimensional convolutional [17] layer to achieve a detection speed of 62FPS, which have become the more practical algorithms at present. Although all the above algorithms have achieved some success in the field of point cloud target detection, there are still some limitations and challenges. The sparsity and non-regularity of point cloud data is one of the common challenges of these algorithms, which requires better feature extraction and representation methods, introduction of more powerful models and deep learning frameworks, and optimization of computational algorithms.

In summary, the 3D object detection algorithm can detect and analyze the size, shape, and category of targets around the vehicles, provide accurate spatial perception to improve the accuracy and reliability of detection, and plays an important role in the recognition and localization of objects in the automatic driving scene. Therefore, improving the 3D object detection algorithm is of great significance to further enhance the performance and safety of autonomous driving technology. In this paper, for the problem of information loss and inaccurate localization due to the column division of the point cloud in the PointPillars algorithm, the Efficient Channel Attention module [18] is introduced to improve the original algorithm’s 2D backbone network. And the contributions of our work are:

(1)

Based on the PointPillars algorithm, ECA Modules–PointPillars are proposed by reconstructing the backbone network by connecting the ECA module and the Conv of the down-sampling module in series.

(2)

The improved algorithm is evaluated on the KITTI datasets, and the visual detection results of the algorithm are also shown for three scenarios: highways, rural roads, and urban roads.

(3)

The optimization effect of the proposed improved algorithm is analyzed and a preliminary solution is proposed for its insignificant accuracy in terms of pedestrian category performance.

2. Methods

The PointPillars algorithm uses the Pillar feature extraction method, which is able to efficiently extract information such as shape, position, and orientation from a point cloud and convert it into an efficient 3D feature representation. This representation allows the algorithm to maintain a low computational cost while maintaining high detection accuracy, improving the real-time performance of the algorithm.

2.1. PointPillars Algorithm

The PointPillars algorithm has a fast processing speed and real-time results even above the radar scanning frequency. It consists of three main stages: (1) a pseudo-image of a point cloud through Pillar Feature Net; (2) feature extraction by 2D Backbone while preserving the 3D features of the point cloud; (3) output of the detection results by Detection Head, and acquisition of 3D bounding box position and attitude of the 3D bounding box [19]. The network architecture is shown in Figure 1.

2.1.1. Point Cloud Pseudo-Image

Phase 1: The point cloud is processed. Point a is used to represent one of the points of cloud data, and its coordinates are $\left(x,y,z,r\right)$, x, y, and z are the position coordinates of point a, and r is the reflectance. Firstly, ignore the z-axis information, in the top view x-y plane for the $\mathrm{H}\times \mathrm{W}$ dimensional point cloud is gridded, and all points falling into the same grid are considered to constitute a column P. The points in each pillar are gridded using $x_{\mathrm{c}}$, $y_{\mathrm{c}}$, $z_{\mathrm{c}}$, $x_{\mathrm{p}}$, and $y_{\mathrm{p}}$ for enhancement, i.e., the enhanced point a has coordinates $\left(x,y,z,r,x_{\mathrm{c}},y_{\mathrm{c}},z_{\mathrm{c}},x_{\mathrm{p}},y_{\mathrm{p}}\right)$, and a is now D = 9 dimensional. $x_{\mathrm{c}}$, $y_{\mathrm{c}}$, $z_{\mathrm{c}}$ is the arithmetic mean distance from point a to all points in the column, $x_{\mathrm{p}}$ and $y_{\mathrm{p}}$ are the deviation of point a relative to the center of the column in the x and y directions. The sparsity of the point cloud causes the generated column P to be mostly empty, and there are only a few points in the non-empty column P. So, if the number of points in the non-empty column P is greater than N, the random sampling is taken to N. Otherwise, the tensor is filled with zeros to N. A tensor of size $\left(D,P,N\right)$ can be created. N is the number of points sampled and P is the number of grids.

Phase 2: A simplified PointNet network is used to learn from the original D-dimensional data of each point to obtain C-dimensional features, to generate a tensor of size $\left(C,P,N\right)$. Then the maximum feature in N is extracted on the channel using the maximum pooling layer and calculated by forward propagation to obtain a tensor of $\left(C,P\right)$ size. The feature vector output from the network is decoded into pictures [20,21,22] and the information such as the position, size, and orientation of the target object is restored in the two-dimensional image plane, thus realizing the conversion of 3D data to 2D pseudo-images [23]. The processing flow is shown in Figure 2.

2.1.2. Two-Dimensional Backbone Network

The 2D backbone network is a pyramid structure containing two sub-networks as shown in Figure 3: (1) top-down extraction of features at smaller and smaller spatial resolutions, and (2) up-sampling and concatenation of the extracted features.

The top-down subnetwork consists of a series of convolutional blocks, Block $\left(S,L,F\right)$ where S represents the step size, L and F represent that a block has L 3 $\times$ 3 size two-dimensional convolutional kernels, and F the output channels. Each block is followed by a BatchNorm [24] and a ReLU [25] to accomplish pseudo-image feature extraction by convolution. The first convolutional step size in the layer is $S/S_{in}$, to ensure that the block handles on stride S after receiving a step size of $S_{in}$ input, keeping the size of the input and output consistent. The subsequent convolution step in the block is 1, which avoids over-squeezing the input signal and losing important details due to the large first convolution step in the network layer.

The up-sampling network combines feature maps of different scales by up-sampling them to the same size through deconvolution and then connecting them. First, the feature maps at different scales are up-sampled-$\mathrm{U}\mathrm{P}\left(S_{in},S_{out},F\right)$ where $S_{in}$ is the initial step size, and $S_{out}$ is the final step size. Then all the modules from different steps are connected. The up-sampling network can improve the receptive field of the model through the deconvolution operation and compensate for the information loss generated during the down-sampling process.

2.1.3. SSD Detection Head

The detection module adopts the classical single-step detection algorithm SSD [26] to complete the classification and regression of the 2D bounding box, which has a fast detection speed and high accuracy. To meet the characteristics of point cloud with large-scale transformations, the SSD network adapts to the multi-scale object detection task by introducing the idea of anchors. It contains three main parts:

(1)

Backbone: SSD uses VGG16 as the base network, and after the first five layers of the network Conv1-Conv5, the third last and the second last fully connected layers FC6 and FC7 of the VGG16 are changed to a 3 × 3 convolutional layer Conv6 and 1 × 1 convolutional layer Conv7; this part is used for the image preliminary feature extraction.

(2)

Extra Layers: Conv8, Conv9, Conv10, and Conv11 convolutional layers are added on top of the backbone to obtain more feature maps for detection.

(3)

Multi-box Layers: final object classification detection and non-greatly suppressed regression location operations.

Through the SSD algorithm, the 3D point cloud feature tensor after feature extraction and voxel segmentation is transformed into a series of bounding box predictions. In practice, many of the generated large numbers of bounding box predictions are redundant or inaccurate, followed by classification and non-extreme suppression to determine the final object detection results. PointPillars matches the output 2D detection frame with the ground truth frame for Intersection over Union (IoU) [27] and the height alone as an additional regression object.

2.1.4. Loss Function

The output of the PointPillars network mainly includes the object class and the 3D bounding box parameters. The total loss function mainly contains the 3D bounding box positioning loss, classification loss, and orientation loss, as shown in Equation (1).

$$L=\frac{1}{N_{pos}}\left({\beta }_{loc}{L}_{loc}+{\beta }_{cls}{L}_{cls}+{\beta }_{dir}{L}_{dir}\right)$$

(1)

$N_{pos}$ denotes the number of positive probability anchors, i.e., the number of generated boxes greater than the specified IoU threshold; $L_{loc}$ represents the regression loss function of the 3D bounding box; $L_{cls}$ represents the classification loss function; $L_{dir}$ represents the object orientation loss function. ${\beta }_{loc}$ = 2; ${\beta }_{cls}$ = 1; ${\beta }_{dir}$ = 0.2, which are the corresponding weight coefficients of the three types of loss functions, respectively.

The positioning loss function of the 3D bounding box mainly contains the position regression loss function, the size regression loss function, and the orientation loss function of the 3D bounding box.

$$L_{loc}=\sum _{b\in \left(x,y,z,w,l,h,\theta \right)}{\mathrm{S}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}}_{\mathrm{L}1}\left(∆b\right)$$

(2)

$$∆x=\frac{x_{gt}-x_p}{d_p},∆y=\frac{y_{gt}-y_p}{d_p},∆\mathrm{z}=\frac{z_{gt}-z_p}{d_p}$$

(3)

$$∆w=\mathrm{l}\mathrm{o}\mathrm{g}\frac{w_{gt}}{w_p},∆l=\mathrm{l}\mathrm{o}\mathrm{g}\frac{l_{gt}}{l_p},∆h=\mathrm{l}\mathrm{o}\mathrm{g}\frac{h_{gt}}{h_p}$$

(4)

$$∆\theta =\mathrm{s}\mathrm{i}\mathrm{n}\left({\theta }_{gt}-{\theta }_p\right)$$

(5)

where $\left(x,y,z,w,l,h,\theta \right)$ are used to characterize the 3D bounding box, where $\left(x,y,z\right)$ denotes the coordinates of the center position of the 3D frame in the LiDAR coordinate system; $\left(w,l,h\right)$ denotes the width, length, and height of the 3D frame, respectively; $\theta$ denotes the yaw rotation angle of the 3D frame around the z-axis. $∆b$ denotes the difference between the predicted 3D bounding box and the true bounding box, i.e., the coordinate deviation of the predicted bounding box. ${\mathrm{S}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}}_{\mathrm{L}1}$ is the L1 smoothing function, which is mainly used to reduce the effect of the coordinate error of the prediction bounding box on the loss function to minimize the effect of extreme values. The subscript $\mathrm{g}\mathrm{t}$ is the variable that is the true value of the object; the scalar in the following table is the predicted or anchor value of the object for p; $d_p=\sqrt{{w_p}^2+{l_p}^2}$.

The classification loss function of the 3D bounding box uses the focal loss function [28], as shown in Equation (6).

$$L_{cls}={-{\alpha }_a\left(1-P^a\right)}^{\gamma }\mathrm{l}\mathrm{o}\mathrm{g}{P}^a$$

(6)

where $P^a$ is the probability value of a category, i.e., the anchor point; $\alpha$ = 0.25; $\gamma$ = 2.

The localization loss cannot distinguish whether the 3D frame is flipped or not, so the $L_{dir}$ uses the softmax function to learn the orientation of the 3D frame in the discrete direction.

The loss function is optimized using the Adam optimizer, which updates the parameters using an exponentially weighted average of the gradient, along with a momentum mechanism that can effectively handle non-smooth objective functions and large-scale datasets. The initial learning rate is $2\times {10}^{-4}$ and the learning rate decay strategy is adopted to decay to 0.8 times of the current learning rate every 15 cycles.

2.2. The PointPillars Algorithm Based on the ECA Module

The attention mechanism is a concept inspired by human behavioral characteristics to selectively use important parts of the data when making decisions rather than treating all information equally. In the attention mechanism, the representation vectors of data points are weighted and summed with their corresponding weight vectors to obtain a weighted representation vector. In this way, the attention mechanism enables the network to learn the important information in the pseudo-image and suppress the unimportant information. Advantages such as small number of parameters, high interpretability, and high robustness make it popular.

2.2.1. ECA Module

The ECA attention module is a type of channel attention that is often used in visual models because of its plug-and-play support, which allows the input feature map to be enhanced in the channel direction, and the final output does not affect the size of the input feature map. Compared with SE-Net [29], it replaces the original learning of channel attention information by the fully connected layer with $1\times 1$ convolution to capture information between the different channels, which reduces the number of parameters, eliminates the negative impact of the will-be operation on the prediction of channel attention adopted by SE-Net, and avoids the inefficiency and unnecessary of obtaining the relationship between all channels at the same time. The structure of the ECA module is shown in Figure 4.

The size of the convolution kernel k in convolution represents the range of action to capture information between different channels, which is proportional to the channel dimension size, i.e., there is a mapping relationship between the two:

$$C=f\left(k\right)$$

The simplest mapping relationship is a linear function, i.e., $f\left(k\right)=p\times k-q$. Since the channel dimension is usually expressed as several times the square of 2, the expression (7) is obtained as:

$$C=f\left(k\right)=2^{p\times k-q}$$

(7)

When the number of channels is known, derive Equation (8):

$$k=g\left(C\right)={\left|\frac{{\mathrm{l}\mathrm{o}\mathrm{g}}_2\left(C\right)+q}{p}\right|}_{odd}$$

(8)

where k denotes the convolutional kernel size; C denotes the number of channels; ${\left|A\right|}_{odd}$ denotes the odd number closest to A; p = 2 and q = 1 used to change the ratio between the number of channels and the convolution kernel size.

2.2.2. ECA Modules–PointPillars

The use of ECA modules can improve the expressiveness of the model by enabling the network to focus on the important information in the pseudo-image features and ignore the irrelevant information without imposing too much extra burden on the training, operation, computation, and storage properties of the algorithm. For the PointPillars algorithm, due to the column division of the point cloud and random sampling, there is a loss of information problem, combined with the two-dimensional backbone network of the PointPillars model. The original two-dimensional backbone network from the top to bottom sub-network consists of a series of convolutional blocks. The ECA modules are sequentially connected behind the original down-sampling modules Block1, Block2, and Block3 to reconstruct the backbone network, and the ECA Modules–PointPillars algorithm is proposed to improve the algorithm to address the practical challenges of the point cloud spatial loss due to the division of the columns of PointPillars, thus affecting the accuracy of the target detection. The ECA Attention Modules adaptively strengthen the important features and suppress the irrelevant features inside the column channels to compensate for the loss of spatial information. The network structure is shown in Figure 5.

The output features of the network after point cloud pseudo-imaging are input to the ECA1 module $\left(C,K\right)$, where C denotes the channel dimension of the input features and K denotes the size of the convolution kernel. The ECA modules connected after the convolution block of each top-down 2D backbone sub-network can be denoted as ECA2 module $\left(C,K\right)$, ECA3 module $\left(2C,K\right)$, and ECA4 module $\left(4C,K\right)$. The input features are fed into Block1 $\left(S,4,F\right)$ after the ECA1 module, where S denotes the step size, Block1 has 4 $3\times 3$ size 2D convolutional layers, and F denotes output channels. Each block is followed by a BatchNorm and a ReLU, and to ensure that the network layer can keep as much information as possible when processing the input, the first convolutional step size in the layer is $S/S_{in}$, and the subsequent convolutional steps are all $1$ to ensure that the network layer still has a step size of $S$ after receiving an input with a step size of $S_{in}$. This avoids over-compression of the input signal due to the large first convolutional step in the network layer and avoids the loss of information to the greatest extent possible.

As can be seen in Figure 5, the output characteristics of Block1 are sequentially passed through ECA2 module, Block2 $\left(2S,\mathrm{6,2}F\right)$, ECA3 module, Block3 $\left(4S,\mathrm{6,4}F\right)$, ECA4 module, and the output characteristic sampling process used by ECA2 module for the up-sampling part is denoted as $\mathrm{U}\mathrm{P}1\left(S,S,2C\right)$, where the parameters are initial step size S, final step size S, and 2C output channels are obtained through the inverse convolution. The ECA3 module and ECA4 module are used for the up-sampling part of the output characteristic sampling process denoted as $\mathrm{U}\mathrm{P}2\left(2S,S,2C\right)$ and $\mathrm{U}\mathrm{P}3\left(4S,S,2C\right)$. The up-sampling network is followed by a BatchNorm and a ReLU, and then all the modules from different step sizes are connected to generate the final output characteristics, which are fed to the SSD detection header.

In summary, the network output features after point cloud pseudo-imaging are inputted to the ECA1 module, and the output characteristics of Block1 are sequentially passed through ECA2, Block2, ECA3, Block3, and ECA4; each time the features are extracted, they undergo the process of enhancement of the positional feature information and the weakening of the irrelevant feature information, such as the background noise, by the ECA module. ECA2, ECA3, and ECA4 output features are used for up-sampling and connectivity, through which the final output features are generated and output to the SSD detection head.

Compared with the algorithm proposed in references [8,12,15,16,22], the improved algorithm first divides the point cloud data into different voxels, and extracts features for the point cloud data in each voxel, such as the location and reflective intensity of the point cloud, which can help the algorithm better understand the geometric and semantic information of the point cloud data. Then the ECA attention module is introduced, the algorithm maps the global averaged pooled input features into channel attention weights through a 1 × 1 convolutional layer in the feature extraction stage, and then after feature weighting, the resulting channel attention weights are multiplied with the input features, where a larger weight will amplify the important information, while a smaller weight will suppress the irrelevant information. The attention to important information such as shape, size, position, and orientation of the target object is strengthened, and the interference of redundant information such as background noise and sensor noise is reduced. By learning the geometric and semantic information in the point cloud data, the improved algorithm can better understand the contextual relationships and shape features of the target, helping the network to better focus on useful features and reduce the influence of occlusions on the prediction. Also, due to the global average pooling operation in the ECA module, it helps the network to focus on the statistical information of the overall features instead of just focusing on the local area or individual channel features. By considering the global distribution of features, the network can capture more comprehensive and diverse feature information in order to better cope with the appearance of unexpected obstacles that can affect the prediction. Through validation on the KITTI datasets, the improved algorithm shows good performance indicating that the algorithm is more responsive in dealing with challenges such as occlusions, sensor noise, and unexpected obstacles, which improves the reliability and safety of the autonomous driving system. However, in extreme occlusion scenarios, a completely occluded target may not be detected and localized correctly. Therefore, further research and improvements are still necessary, and combination with other sensors may be required to meet the demand for high-performance target detection and localization in autonomous driving systems [30].

3. Results and Discussion

3.1. Experimental Setup

The experiments were conducted using the OpenPCDet 0.6.0 object detection framework with an environment of the Ubuntu 18.04 operating system, Python 3.8, CUDA 11.3, Pytorch 1.11.0 [31], cuDNN 8, NVCC, torchvision 0.12.0, and torchaudio 0.11.0. The processor was an Intel (R) Xeon (R) CPU E5-2686 v4 @ 2.30 GHz, the graphics card was NVIDIA GeForce RTX 3080 Ti, graphics memory was 12 GB, GPU bandwidth was 912.10 GB/s, number of CPUs available was 8, and hard disk bandwidth was 377.42 MB/S. The KITTI datasets used in this experiment are some of the most important datasets in the field of autonomous driving. It focuses on image processing techniques in the field of automated driving and applications in automated driving perception and prediction. It also involves localization and SLAM techniques, and was jointly established by the Karlsruhe Institute of Technology (KIT), Germany, and the Toyota Technological University Chicago (TTIC). The datasets contains point cloud and image data acquired by a 64-line 3D LIDAR, two grayscale cameras, two color cameras, and four optical lenses in scenes such as urban roads, countryside, and highways, and is used to evaluate the performance of computer vision technologies such as stereo imagery, optical flow, visual ranging, 3D object detection, and 3D tracking in in-vehicle environments [32]. Each image in the datasets contains up to 15 vehicles, 30 pedestrians, and obstacles with different levels of occlusion, and only point clouds are used in the training process. In order to evaluate the model performance and prevent overfitting, the datasets are divided into 7481 training sets and 7518 test sets, and the training set is divided into 3712 training samples and 3769 validation samples in training [33] in order to monitor the model performance in a timely manner, and to make adjustments to the model to make it better adapted to the new data. The samples in datasets can be categorized into three grades: simple, medium, and difficult according to the truncation value and masking scale, and the specific indexes are shown in

Table 1. Classification of difficulty levels.

Difficulty Level	Easy	Moderate	Hard
Shelter	Fully visible	Partial shade, mostly sheltered	Hard see
Truncation value	≤15%	15~50%	≥50%
Number of pixels	≥40	≥25

. Default parameters: x and y resolution of the Pillar columns are set to 0.16 m, the maximum number of columns P is 12,000, and the maximum number of point clouds inside the columns N is 100. The 3D object detection frame for each object category consists of a 0 degree and 90-degree direction, the width, length, and height of the car anchor point are 1.60 m, 3.90 m, and 1.56 m, z center is −1.00 m, and the matching positive and negative thresholds are 0.60 and 0.45. The width, length, and height of the pedestrian anchor point are 0.60 m, 0.80 m, and 1.73 m, and that of the cyclist anchor point are 0.60 m, 1.76 m, 1.73 m, all with a z-center of −0.60 m and matching positive and negative thresholds of 0.5 and 0.35 [34].

3.2. Experimental Results and Analysis

Average precision (AP) was used as an evaluation metric for 3D object detection algorithms in order to evaluate 2D, bird’s eye view, 3D, and Average Orientation Similarity (AOS) [35]. Based on the degree of occlusion of the target, the KITTI datasets were categorized into three levels of difficulty: easy, moderate, and hard. The easy, moderate, and hard overlap thresholds were set to 0.7, 0.7, and 0.7 times IoU for the car category, and for the cyclist and pedestrian category the easy, moderate, and hard overlap thresholds are set to 0.5, 0.5, and 0.5 times IoU, respectively.

Table 2. Experimental results of the BEV model.

Method		Cars			Pedestrians			Cyclists
	mAP	Easy	Moderate	Hard	Easy	Moderate	Hard	Easy	Moderate	Hard
F-PointNet [12]	65.39	88.70	84.00	75.33	58.75	51.05	47.20	68.09	57.48	50.77
VoxelNet [8]	58.25	89.35	79.26	77.39	46.13	40.74	38.11	66.70	54.76	50.55
SECOND [15]	60.56	88.07	79.37	77.95	55.10	46.27	44.76	73.67	56.04	48.78
PointPillars [16]	66.19	88.35	86.10	79.83	58.66	50.23	47.19	79.14	62.25	56.00
Complex-YOLO [22]	62.84	85.89	79.26	77.39	46.08	45.90	44.20	72.37	63.36	60.27
This paper	70.03	89.91	87.39	83.32	58.73	54.86	49.95	83.76	67.84	63.62

shows the comparison of the detection performance results with the literature [8,12,15,16,22] algorithms based on the KITTI datasets cars, pedestrians, and cyclists in BEV mode at three difficulty levels.

Table 3. Experimental results of the 3D model.

Method		Cars			Pedestrians			Cyclists
	mAP	Easy	Moderate	Hard	Easy	Moderate	Hard	Easy	Moderate	Hard
F-PointNet [12]	57.35	81.20	70.39	62.19	51.21	44.89	40.23	71.96	56.77	50.39
VoxelNet [8]	49.05	77.47	65.11	57.73	39.48	33.69	31.50	61.22	48.36	44.37
SECOND [15]	56.69	83.13	73.66	66.20	51.07	42.56	37.29	70.51	53.85	46.90
PointPillars [16]	59.20	79.05	74.99	68.30	52.08	43.53	41.49	75.78	59.07	52.92
Complex-YOLO [22]	54.01	67.72	64.00	63.01	41.79	39.70	35.92	68.17	58.32	54.30
Voxel R-CNN [36]	-	89.41	84.52	78.78	-	-	-	-	-	-
3DSSD [37]	-	89.71	79.45	78.67	-	-	-	-	-	-
This Paper	63.24	86.76	76.96	74.07	53.83	48.60	45.21	81.14	64.15	60.21

shows the comparison of the detection performance results with literature [8,12,15,16,22,36,37] algorithms based on the KITTI datasets cars, pedestrians, and cyclists in 3D mode at three difficulty levels.

Table 4. Experimental results of the AOS model.

Method		Cars			Pedestrians			Cyclists
	mAP	Easy	Moderate	Hard	Easy	Moderate	Hard	Easy	Moderate	Hard
SECOND [15]	54.53	87.84	81.31	71.95	51.56	43.51	38.78	80.97	57.20	55.14
PointPillars [16]	68.86	90.19	88.76	86.38	58.05	49.66	47.88	82.43	68.16	61.96
This Paper	66.84	90.76	89.39	88.20	42.07	39.73	37.55	84.47	71.39	66.98

demonstrates the comparison of detection performance results with literature [15,16] algorithms based on the KITTI datasets cars, pedestrians, and cyclists in AOS mode at three difficulty levels.

The experimental results show that compared with the PointPillars algorithm in moderate difficulty, the mAP of the improved algorithm is improved by 3.84% in the BEV mode, 4.04% in the 3D mode, and the mAOS is reduced by 2.02% in the AOS mode. In addition, the AP of cars in the BEV mode improves to 87.39% in the moderate difficulty, 76.96% in the 3D mode, and 89.39% in the AOS mode; the AP of cyclists in the BEV mode improves to 67.84% in the moderate difficulty, 64.15% in the 3D mode, and 71.39% in the AOS mode; except for the pedestrian category, the improved algorithm outperforms the other detection models in the table under the other two categories. From Table 2 and Table 4, it can be seen that the detection results under the AOS mode for pedestrians do not present optimal results. The attention mechanism can help the network to focus on the key information in the input data and suppress the unimportant information. However, there are some issues that may have an impact on the detection results of small objects, for example, pedestrians may be confused with other slender objects pedestrians such as poles, tree trunks, and the like, and the presence of certain vertical objects may also affect the number of point clouds. Real-world environments with different lighting, shooting angles, and complex backgrounds also affect detection effects, etc. In order to solve these problems, a combination of image-based methods [38] may be a viable solution.

In addition to list the official KITTI evaluation protocols, i.e., an IoU threshold of 0.7 for cars and 0.5 for pedestrians and cyclists, it also lists the IoU thresholds of 0.7, 0.5, and 0.5 for the bbox AP, BEV AP, 3D AP, and AOS values for car detection and the IoU thresholds of 0.5, 0.25, and 0.25 for bbox AP, BEV AP, 3D AP, and AOS values for pedestrian and cyclist detection. According to the different methods of taking the equal interval of the recall rate, the results of the interpolation precision of AP11 (indicating the interpolation precision of 11 points, taking the equal interval of the recall rate, calculating the average value of the relative exact rate, and obtaining the average precision; R11 = {0, 0.1, 0.2, ……, 1}) and AP40 (indicating the interpolation precision of 40 points; R40 = {1/40, 2/40, 3/40, ……, 1}) are shown in

Table 5. Car detection results by ECA Modules–PointPillars on the KITTI datasets.

Detection Type	Cars–Easy	Cars–Moderate	Cars–Hard
bbox AP11	90.82	89.62	88.60
bbox AP40	95.52	92.03	90.99
BEV AP11	90.82	90.03	89.37
BEV AP40	95.59	94.62	93.75
3D AP11	90.82	89.95	89.22
3D AP40	95.56	94.43	93.30
AOS AP11	90.76	89.39	88.20
AOS AP40	95.45	91.78	90.56

,

Table 6. Pedestrian detection results by ECA Modules–PointPillars on the KITTI datasets.

Detection Type	Pedestrians–Easy	Pedestrians–Moderate	Pedestrians–Hard
bbox AP11	64.78	60.99	57.57
bbox AP40	64.75	60.72	57.06
BEV AP11	69.74	66.77	63.48
BEV AP40	71.06	67.33	63.56
3D AP11	69.68	66.59	63.12
3D AP40	70.99	66.95	63.18
AOS AP11	42.07	39.73	37.55
AOS AP40	41.69	39.23	36.75

and

Table 7. Cyclist detection results by ECA Modules–PointPillars on the KITTI datasets.

Detection Type	Cyclists–Easy	Cyclists–Moderate	Cyclists–Hard
bbox AP11	85.49	73.92	69.51
bbox AP40	88.74	74.71	70.74
BEV AP11	84.88	70.85	66.63
BEV AP40	87.95	71.55	67.57
3D AP11	84.88	70.85	66.62
3D AP40	87.95	71.54	67.47
AOS AP11	84.47	71.39	66.98
AOS AP40	87.49	71.94	67.88

.

Theoretically, the ECA attention module has the property of supporting plug-and-play and does not bring too much extra burden to the training, running, computation, and storage of the algorithm. However, in order to verify whether the complexity it brings is reasonable or not, and whether it will limit real-time processing in autonomous driving scenarios and affect the scalability of the algorithm, in this paper, the speed comparison of the algorithm before and after the improvement is conducted on NVIDIA GeForce RTX 3080 Ti graphics card, and the comparison results are shown in

Table 8. Algorithmic inference speed comparison.

Method	Average Inference Time	Average Inference Speed	Bash_Size
PointPillars	0.116 s	34.483 FPS	4
This paper	0.133 s	30.075 FPS

.

Where FPS denotes the number of detectable scenes per second, it can be seen that the inference speed of the improved algorithm is basically kept at the same level as that of the PointPillars algorithm. It is shown that the additional introduction of the ECA module does not affect the real-time performance of the algorithm too much, and the improved algorithm is able to improve the detection accuracy while still maintaining the computational efficiency, which preliminarily verifies the effectiveness and scalability of the algorithm. However, the actual scalability is still affected by factors such as hardware platform, datasets size, and computational resources. Therefore, further tuning and adaptation are still necessary to meet the needs of specific application scenarios.

3.3. Object Detection Results and Analysis

In order to facilitate the study, observation and explanation of the illustration, the RGB image and 3D frame prediction results in BEV mode are shown, and the object detection results are shown in Figure 6 (green for cars, yellow for cyclists, and blue for pedestrians).

Figure 6 shows the RGB original images and their corresponding point cloud detection result maps for three different road scenes. Among them, the left side, Figure 6a,c,e, show the RGB original images under the highway driving scenario, the rural road driving scenario, and the urban road driving scenario, respectively, and the right side, Figure 6b,d,f, show their corresponding point cloud prediction result maps. Through the point cloud detection, the shape and position information of target objects such as cars, pedestrians, cyclists, etc. are obtained. The point cloud detection result graphs demonstrate the accurate detection and recognition capability of the target objects of these original maps. As shown in the original RGB image and the point cloud detection image, the point cloud data are acquired by the LiDAR sensor, which contains the 3D information of the whole environment, including the target, the ground, the background, etc., covering the spatial area from the sensor to the farthest reachable distance in the scene. However, only the point cloud within the camera’s field of view has significance for the vision task, so filtering out the point cloud that is not within the camera’s field of view and then detecting is the next step in the endeavor.

4. Conclusions

To address the problem of information loss in the column segmentation of the PointPillars algorithm, the ECA module is introduced, which is connected in series with the Conv of the down-sampling module to reconstruct the backbone network. The improved algorithm improved the mAP in 3D mode by 4.04% compared to PointPillars, 5.89% compared to F-PointNet, 14.19% compared to VoxelNet, and 6.55% compared to SECOND. It shows that the improved algorithm is effective and reliable in multi-target detection in autonomous driving scenarios. For the problem that the improved algorithm performs ordinary in the pedestrian category, a point cloud and image fusion based approach may be a feasible solution, with the following two specific solutions: building a complete semantic understanding framework by carrying cameras around the vehicle, fusing the image information of other targets and the point cloud information; and incorporating a de-fogging and de-raining model to process the data before fusion. In addition to multi-sensor collaboration and multi-modal data fusion, the lightweighting of network models is also a direction for further exploration to achieve a balance between accuracy and inference speed, and to improve the real-time performance of the algorithms, thus enabling the models to be deployed to mobile.