Laser Radar and Micro-Light Polarization Image Matching and Fusion Research

Abstract

Aiming at addressing the defect of the data blindness of a LiDAR point cloud in transparent media such as glass in low illumination environments, a new method is proposed to realize covert target reconnaissance, identification and ranging using the fusion of a shimmering polarized image and a laser LiDAR point cloud, and the corresponding system is constructed. Based on the extraction of pixel coordinates from the 3D LiDAR point cloud, the method adds information on the polarization degree and polarization angle of the micro-light polarization image, as well as on the reflective intensity of each point of the LiDAR. The mapping matrix of the radar point cloud to the pixel coordinates is made to contain depth offset information and show better fitting, thus optimizing the 3D point cloud converted from the micro-light polarization image. On this basis, algorithms such as 3D point cloud fusion and pseudo-color mapping are used to further optimize the matching and fusion procedures for the micro-light polarization image and the radar point cloud, so as to successfully realize the alignment and fusion of the 2D micro-light polarization image and the 3D LiDAR point cloud. The experimental results show that the alignment rate between the 2D micro-light polarization image and the 3D LiDAR point cloud reaches 74.82%, which can effectively detect the target hidden behind the glass under the low illumination condition and fill the blind area of the LiDAR point cloud data acquisition. This study verifies the feasibility and advantages of “polarization + LiDAR” fusion in low-light glass scene reconnaissance, and it provides a new technological means of covert target detection in complex environments.

1. Introduction

Night vision imaging technology is similar to normal imaging technology in that it involves imaging received targets and scenes through a detector. However, night vision imaging techniques have a more challenging low-light operating environment, resulting in relatively poor image quality. On the other hand, the micro-light imaging technique utilizes the weak nighttime light reflected from the target object to perform imaging, with the resulting images usually having low contrast and requiring further processing [1]. With the gradual recognition and utilization of polarized imaging properties, polarization imaging technology was introduced into night vision imaging technology, which resulted in night vision polarization imaging technology [2]. Polarization imaging technology can not only obtain information from traditional images but also obtain the polarization information of the target scene on that basis. Polarization information enhances the contour information and detailed texture of an object, thus improving the contrast between the target and the scene. In addition, information such as the texture, material, and surface roughness of the target object will be reflected in polarization images in different ways. However, micro-light polarization images can only provide two-dimensional planar information, and the spatial position and depth perception of the target object are weak.

In recent years, LiDAR has been widely used in many fields [3,4,5]. LiDAR has the advantages of strong 3D perception, high measurement accuracy, and strong immunity against interference, which are unrivaled by other imaging technologies [6]. With the continuous advancement of laser and electronic technologies, significant breakthroughs have been made in laser radar imaging technology. Various new types of LiDAR systems have emerged, such as scanning LiDAR and imaging LiDAR [7]. Scanning LiDAR scans across the target area mechanically or optically to obtain 2D or 3D information about the target [8]. Imaging LiDAR adopts more advanced detectors and signal processing techniques to directly acquire target image information, and its imaging resolution and accuracy have been significantly improved. Since then, the fusion of LiDAR and 2D images has been explored [9]. Hans Moravec (Carnegie Mellon University) conducted early research in the field of robotic navigation and proposed a sensor-based approach to environmental awareness [10]. However, LiDAR point cloud data lacks detailed features such as the surface texture and material of the target object. Paul Besl and Neil McKay proposed the iterative closest point (ICP) algorithm, which laid the foundation for point cloud alignment [11]. In 2005, Sebastian Thrun, the “father of driverlessness”, led the team that developed an autonomous driving system based on LiDAR and camera fusion. In 2012, Andreas Geiger (creator of the KITTI dataset) released the KITTI dataset, which pushed forward the algorithm for the fusion of LiDAR and image research [12,13]. Wei Kai has achieved remarkable results in terms of laser phased array technology and multi-band image matching and fusion technology [14]. Zhiyan Dong (Fudan University) made a breakthrough in relation to the SLAM map-building method for the fusion of LiDAR and binocular vision during UAV level flight [15]. Youth Zhang (Huaiyin Institute of Technology) developed an intelligent mobile robot system based on the ROS platform, which realized the efficient fusion of camera and LiDAR [16]. In 2017, Charles Qi (Stanford University) proposed PointNet, which is the first application of deep learning in point cloud data processing [17].

Although the above fusion methods can improve the comprehensive information available to optical measurement systems [18], they cannot effectively enhance the description of tail features such as the surface texture and material of target objects through point cloud data. In 2024, Li M and team pointed out that the polarization intensity can serve as an additional characteristic channel for geometric completion [19]. In 2024, Liu H proposed the fusion of the visible light polarized texture and a portable LiDAR point cloud, which can achieve millimeter-level defect detection of wooden components, verifying the advantages of polarization LiDAR fusion in weak texture structures [20]. In 2024, Dong Q et al. used visible light polarization images to assist with LiDAR point cloud denoising and surface detail restoration in the reconstruction of ancient rockeries, proving that polarization information can effectively improve the reconstruction integrity of complex geometries [21]. In 2024, Zhu Q pointed out that strategies based on the polarization degree, such as point cloud color transfer and geometric consistency checking, can be directly used to improve the quality of training samples in low-light scenes [22]. Although there is a lot of literature on image fusion with LiDAR point clouds [23,24,25,26], no studies have been reported on the use of micro-light polarization images to compensate for the blind spots of LiDAR point cloud data. The aim of this study is to construct a micro-light polarization and LiDAR acquisition system to collect and process data from different sensors, fully exploit the advantages of each sensor, and compensate for their respective shortcomings, so as to obtain more comprehensive, accurate, and reliable target information. To this end, an optimal scheme for matching and fusing micro-light polarization images with radar point cloud data is proposed.

This paper first elaborates on the principle and optimization algorithm for matching and fusing LiDAR with low-light polarization images. Then, the construction methods for the low-light polarization imaging system and the LiDAR point cloud acquisition system, along with related image processing techniques, are introduced. Finally, using the optimized matching–fusion algorithm, two-dimensional low-light polarization images and LiDAR point clouds are fused and optimized, filling the blind spots in LiDAR data acquisition in low-light environments.

2. Preliminary

2.1. Matching of LiDAR and Low-Light Polarization Images

Because of the strong laser reflection by glass, LiDAR can acquire point clouds in ordinary scenes but fails to detect targets behind optical glass. Therefore, the position of the glass surface is first derived from the LiDAR point cloud. Then, micro-light polarization images of the same scene are acquired. A MATLAB program is used to process these images to generate micro-light polarization images [23]. The LiDAR point cloud is a three-dimensional image, while the microlight polarization image is a two-dimensional image. Thus, the matching–fusion of the LiDAR and micro-light polarization image is the matching–fusion of three-dimensional point cloud and two-dimensional image.

Image matching involves co-calibrating the LiDAR camera coordinate system, extracting and matching feature points, and establishing correspondences between them. For LiDAR and camera systems, the same target is targeted within a common field of view due to the fact that the LiDAR resolution is lower than the camera resolution. Therefore, each 3D point cloud has a unique image pixel point corresponding to it. In other words, the external parameter calibration of the LiDAR and camera system can be understood as solving the mapping matrix between the 3D LiDAR point cloud and the image pixel points. First, the camera’s intrinsic matrix is obtained. Next, the corresponding feature point coordinates are extracted from both the LiDAR point cloud and the optical image, and the extrinsic calibration is formulated as a perspective-n-point (PnP) problem to compute the mapping matrix.

After obtaining the feature points corresponding to the LiDAR and 2D images, the coordinates of the feature points are (X_L, Y_L, Z_L) in the LiDAR coordinate system, and (u, v) in the pixel coordinate system; their relationship is established as follows:

$$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\mathit{K}\left[\begin{array}{cc}\mathit{R}& \mathit{T}\\ 0& 1\end{array}\right]\left[\begin{array}{c}{X}_{\mathrm{L}}\\ Y_{\mathrm{L}}\\ Z_{\mathrm{L}}\\ 1\end{array}\right]=\mathit{M}\left[\begin{array}{c}{X}_{\mathrm{L}}\\ Y_{\mathrm{L}}\\ Z_{\mathrm{L}}\\ 1\end{array}\right]=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]$$

(1)

where K denotes the camera internal reference; R denotes the rotation matrix from the LiDAR coordinate system to the camera coordinate system; and T denotes the translation vector. By extracting a number of sets of LiDAR and image corresponding feature points, a system of constraint equations is established to be solved for the 12 elements in the matrix M. Expanding Equation (1), we get

$$\left\{\begin{array}{c}u=m_{11}{X}_{\mathrm{L}}+m_{12}{Y}_{\mathrm{L}}+m_{13}{Z}_{\mathrm{L}}+m_{14}\\ v=m_{21}{X}_{\mathrm{L}}+m_{22}{Y}_{\mathrm{L}}+m_{23}{Z}_{\mathrm{L}}+m_{24}\\ 1=m_{31}{X}_{\mathrm{L}}+m_{32}{Y}_{\mathrm{L}}+m_{33}{Z}_{\mathrm{L}}+m_{34}\end{array}\right.$$

(2)

At least six sets of feature points are required. In practice, additional feature points are collected to reduce the errors. The mapping matrix is first initialized via the least squares method and then refined to obtain the final result.

When the mapping matrix M is solved by the least squares method, errors may arise in both the localization and the depth of the feature points. This stems from the mismatch between the pixel-level positioning error of low-light polarization images and that of LiDAR point clouds after projection. Consequently, the per-point error increases and is further amplified by the distance. In response to the above errors, this article first uses the least squares method to assign weights w_i = 1/(σ_i²) to each pair of feature points, and then weights the residuals twice to further reduce the average error.

2.2. Fusion Algorithm of LiDAR and Micro-Light Polarization Images

The fusion of point cloud and image is based on the premise of the alignment of the two, through the alignment of the point cloud and image data, to establish the correspondence between each 3D point and its corresponding image pixel. First, the 3D point cloud is projected onto the optical image to obtain the corresponding pixel coordinates (x, y). Then, the R, G, B values at pixel (x, y) are assigned to the point cloud, thereby coloring the 3D data with true-color texture. After obtaining the optimized extrinsic matrix, the relationship between image and point cloud coordinates is established via the covariance equation, and the R, G, B values of each image pixel are transferred to the corresponding point in the point cloud model, completing the fusion [27].

$$\left\{\begin{array}{c}x-x_0=-f{\displaystyle \frac{a_1\left(X_{\mathrm{L}}-X_{\mathrm{C}}\right)+b_1\left(Y_{\mathrm{L}}-Y_{\mathrm{C}}\right)+c_1\left(Z_{\mathrm{L}}-Z_{\mathrm{C}}\right)}{a_3\left(X_{\mathrm{L}}-X_{\mathrm{C}}\right)+b_3\left(Y_{\mathrm{L}}-Y_{\mathrm{C}}\right)+c_3\left(Z_{\mathrm{L}}-Z_{\mathrm{C}}\right)}}\\ y-y_0=-f{\displaystyle \frac{a_2\left(X_{\mathrm{L}}-X_{\mathrm{C}}\right)+b_2\left(Y_{\mathrm{L}}-Y_{\mathrm{C}}\right)+c_2\left(Z_{\mathrm{L}}-Z_{\mathrm{C}}\right)}{a_3\left(X_{\mathrm{L}}-X_{\mathrm{C}}\right)+b_3\left(Y_{\mathrm{L}}-Y_{\mathrm{C}}\right)+c_3\left(Z_{\mathrm{L}}-Z_{\mathrm{C}}\right)}}\end{array}\right.$$

(3)

where (x, y) are the pixel coordinates corresponding to the point cloud (X_L, Y_L, Z_L) in the image plane coordinate system; (X_C, Y_C, Z_C) are the center coordinates of the camera in the image space coordinate system; f is the focal length; and (a₁, a₂, a₃, b₁, b₂, b₃, c₁, c₂, c₃) are the coefficients of the rotation matrix. The corresponding R, G, B color values at (x, y) are assigned to the corresponding point cloud data using Equation (3). The fused point cloud data has the color texture information of the optical image.

2.3. Optimization of Fusion Between LiDAR and Low-Light Polarization Images

To further improve the fusion of LiDAR and low-light polarization images, the system’s feature extraction algorithm has been optimized.

Firstly, calculate the pixel coordinates of each laser point cloud, such as the pixel coordinates of the i-th laser point cloud (u_i, v_i).

Then, based on the 4 time-division low-light polarization images, calculate the polarization degree (DoP) and polarization angle (AoP) of each point cloud at the pixel coordinates:

DoP_i = DoP[v_i, u_i]

(4)

AoP_i = AoP[v_i, u_i]

(5)

Next, extract the reflection intensity I_i of each point cloud in each LiDAR point cloud datum.

Furthermore, by searching for the reflection intensity of each point cloud point and the corresponding pixel polarization degree and polarization angle information on the polarization image, a 3D feature vector combining the polarization intensity is constructed for each LiDAR point:

$$f_i=\left[\begin{array}{c}{DoP}_i\\ {AoP}_i\\ I_i\end{array}\right]$$

(6)

In this way, each three-dimensional laser point (X_i, Y_i, Z_i) is associated with its corresponding DoP_i (polarization degree), AoP_i (polarization angle), and echo intensity I_i on the polarization image, forming a seven-dimensional joint feature vector for all the cross-modal information of each laser point by point [28,29]:

$$F\left(\begin{array}{ccc}X& Y& Z\end{array}\right)=\left[\begin{array}{c}\begin{array}{c}\begin{array}{c}\begin{array}{c}{X}_i\\ Y_i\\ Z_i\end{array}\\ {DOP}_i\end{array}\\ {AOP}_i\end{array}\\ I_i\\ 1\end{array}\right]=M_{polar}\left[\begin{array}{c}\begin{array}{c}{X}_i\\ Y_i\\ Z_i\end{array}\\ 1\end{array}\right]$$

(7)

Among them, the first three dimensions are the 3D coordinates of the LiDAR point; the fourth and fifth dimensions represent the degree and angle of polarization of the corresponding pixel; the sixth dimension holds the LiDAR echo intensity; and the seventh dimension is set to one for the subsequent least squares or homogeneous transformations. Simply put, the F-matrix jointly represents “3D LiDAR points + polarization features”, thereby injecting polarization information into the existing point cloud. Using the F-matrix $M_{polar}$ and least squares method to calculate the mapping matrix in Formula (7). This mapping matrix contains depth shifts related to polarization information. By substituting $M_{polar}$ into Formula (1) and performing the inverse operation, the low-light polarization image can be converted into a 3D point cloud without laser point cloud positions. Finally, coordinate matching and fusion are performed between the LiDAR point cloud and the low-light polarization image point cloud, and Canny’s edge detection is used for correction.

On the basis of extracting pixel coordinates, this optimization algorithm adds the extraction of vectors such as the polarization degree and polarization angle of polarized image pixels, as well as the extraction of the reflection intensity of laser radar point clouds, which improves the fitting of the mapping matrix and effectively improves the fusion effect.

3. The Proposed Fusion System

3.1. Imaging System Design and Construction

3.1.1. Micro-Light Polarization Imaging System

This research focuses on the polarization of static objects in low-light environments, so a time-sequential polarization imaging system is employed. Compared with real-time systems, the time-sequential system offers lower cost and simpler operation. In the time-sequential setup, the polarizer is rotated manually or by a motor in front of the imaging system. When the polarizer is rotated continuously, the detector obtains all the polarized components of the scene light wave through light intensity changes. Mainly based on the selection of experimental instruments to meet the system design requirements for imaging, the specific hardware is as follows:

(1) Micro-light camera: This experiment used the Watec Corporation WAT-902H2 black and white camera, as shown in Figure 1, and

Table 1. Specific parameters of micro-light camera.

Minimum Illumination	Signal-to-Noise Ratio	Gain Mode
0.0003 LuxF1.4 (AGC:High)	46 dB	Automatic

shows the parameters. In the low illumination range, with the imaging lens, a clear image can be obtained.

(2) Polarizer: Through early-stage comparative experiments on low-light polarization images across various bands, it was found that a polarizer film with high transmittance in the 400–1000 nm range yields the best low-light polarization imaging performance. Consequently, a polarizer with appropriate filtering properties was selected, as shown in Figure 2.

(3) USB video capture card: Figure 3 shows the actual product.

This experiment employed VideoCap4.5 software together with a Watec WAT-902H2 monochrome camera (Watec Co., Ltd., Tsuruoka, Japan) and its imaging lens, which allowed the micro-light polarization image to be clearly displayed on the computer after focusing.

A time-division low-light polarization imaging system was constructed using the aforementioned laboratory equipment. Experimental equipment: microlight cameras, lenses, experimental platforms, computers, polarizers, LiDAR, natural objects, man-made targets. When the illuminance of the target surface met 1 × 10⁻³ lx or was lower than 1 × 10⁻³ lx, the microlight polarizer was driven to rotate according to the four angles of 0 degree, 45 degree, 90 degree and 135 degree for the target scene. The polarized image was acquired through four exposures. After obtaining the low-light polarization image, the Stokes vector imaging formula was used to solve the polarization state of the target scene, and the polarization degree and polarization angle images were obtained. Figure 3 shows the micro-light polarization system.

Rotating actuator: Adopted the TECO DST42-03A two-phase stepper motor from Dongwei Company (Wuxi, China) (step angle 1.8°, rated 1.5A, static torque 0.3 N·m), controlled by a YKA2204MA subdivision driver (Shenzhen YAKO Automation Technology Co., Ltd., Shenzhen, China) (up to 64 subdivisions), with an equivalent step distance of 0.028°/step.

Angle calibration: Used Photon Precision’s PDL-25 laser displacement sensor (Phoskey (Shenzhen) Precision Technology Co., Ltd., Shenzhen, China) (range 25 mm, repeatability 0.02 mm) for closed-loop calibration at four positions: 0°, 45°, 90°, and 135°, with zero drift < 0.05°.

Control core: Used the Qinheng CH32V203C8T6 development board (RISC-V 72 MHz, Nanjing Qinheng Microelectronics Co., Ltd., Nanjing, China), where the angle was reported in real time to the PC through USB-CDC.

Multiple exposure time: Based on the ambient illumination, used 40 ms (maximum SNR without exposure) as the final exposure.

The micro-light polarization imaging system was connected to the computer through the data cable, and the MATLAB2019 software was used to drive the micro-light polarization imaging system and acquire the images in the field of view. For the time-sharing micro-light polarization imaging system, during the debugging process, attention was paid to the time interval between image acquisitions and to avoiding the influence of personnel walking on the ambient light, which in turn affected the experimental conclusions.

3.1.2. LiDAR Imaging System

This research focuses on the multi-sensor matching and fusion of LiDAR and micro-optical camera. In order to improve the matching rate of the subsequently collected data, the data collection for the same target at the same location point should be selected. During the acquisition process, the data quality of the acquired point cloud is observed in real time by ZvisionView V2.3.3 software to ensure the subsequent image processing. Mainly based on the selection of experimental instruments to meet the system design requirements for imaging, the specific hardware is as follows:

(1) LiDAR: ZVISION ML-30S LiDAR (ZVISION, Beijing, China), which has a wavelength of 905nm. With the imaging lens, a clear point cloud image can be obtained. Figure 4 shows the actual product, and

Table 2. Specific parameters of ML-30S LiDAR.

Machine version		ML-30s B1
FOV (°)		Horizontal FOV: 140° (70°~+70°) Vertical FOV: 70° (FOV1: −10°~+20°) (fov2: −50°~−10°)
Detection distance (m)	100 k Lux	Resolution	HXV: 320 × 160
Maximum detection distance (m)	40 m	Minimum detection distance (m)	20 cm
Frame rate (FPS)	10	Data interface	100Base-T1 (data: UDP; control: TCP)
Data Type	Distance, reflectivity, azimuth and altitude Angle	Points per second	512,000 points per second

shows the parameters.

(2) Electrical adapter: Figure 5 shows the actual product.

(3) Ethernet data cable: Standard Ethernet cable.

This experiment uses the ML-30S LiDAR supported by the use of ZvisionView V2.3.3 software with the imaging lens, which can display a clear point cloud image on the computer.

The point cloud information collected in this experiment needs to be preprocessed to remove the clutter. This experiment uses the cloud compare software to preprocess the point cloud in the form of PCD.

For the IP address configuration of the specific steps to open the control panel interface, click to enter the Network and Sharing Center, the “Ethernet” option to jump to the “Ethernet status” interface, click the “Properties” button to enter. Click the “Properties” button. Double-click to select “Internet Protocol Version 4 (TCP/IPv4)” to set the IP address as 192.168.10.1, adjust the subnet mask to 255.255.255.0, and click “Confirm” after all the settings are complete. Click “Confirm” after all the settings are chosen, and then finish setting the IP address of the computer’s network port. To close the firewall, open the computer control panel, enter the System and Security interface, select and open Windows Defender Firewall, select Enable or Disable Windows Defender Firewall, and click Close Windows Defender Firewall. Construct a laser radar imaging system using the existing equipment in the laboratory mentioned above.

Experimental equipment: LiDAR, electrical adapter, Ethernet data cable, computer, natural objects, man-made targets, etc. Set up the computer network environment configuration, connect LiDAR, computer, start ZvisionView V2.3.3 software, add LiDAR device, play back collected point cloud, and save point cloud data to computer. Figure 6 shows the LiDAR system construction.

System trial acquisition and debugging: Use ZvisionView V2.3.3 software to drive the LiDAR system and acquire data within the field of view. Care is taken to check the LiDAR measurement range to ensure that both the point cloud and the micro-light polarization image fall within this range, preventing mismatches or blind-zone effects that could compromise the subsequent fusion.

3.2. Matching and Fusion Processing

3.2.1. Data Acquisition and Preprocessing

The imaging system must reach a stable state before data acquisition. As the polarizer rotates to different angles, the light intensity varies, causing brightness changes. To eliminate acquisition errors, the system gain is set to 0, ensuring no additional error is introduced. The lens aperture and focal length are then adjusted to obtain a clear image of the target scene, and recording begins.

It is essential to note that the time-sequential micro-light polarization system requires careful polarizer adjustment; any vibration introduced during this process can degrade the image quality, obscuring object contours and impairing target recognition. Therefore, a rigid and stable optical platform must be used to avoid experimental errors. Figure 7 shows images acquired at different polarization angles, while Figure 8 presents the pre-acquired point clouds projected onto each plane.

The acquired images at different polarization angles are processed in MATLAB to compute and visualize the Stokes parameters and polarization characteristics, yielding the micro-light polarization image. Figure 9 shows the resulting micro-light polarization image. The micro-light environment and high-gain settings introduce significant grain noise. Stokes vector averaging followed by Gaussian filtering further reduces the noise while preserving the polarization edge information.

3.2.2. LiDAR and Micro-Light Polarization Image Matching

After obtaining the micro-light polarization image, suitable feature points must be matched with those in the point cloud. Because of the large volume of collected point-cloud data, feature point matching faces a heavy computational burden. Therefore, “cloud compare” is employed to remove stray points from the point cloud. The correspondences between the point cloud coordinates and the pixel positions in the micro-light polarization image are then established, and the least squares method is used to solve the 12 elements of the M matrix.

Table 3. Calibrated M-parameter matrix elements during pre-acquisition.

−0.04235	−3.456867	0.0916	9.1191
0	2.670461	0	5.673622
0	6.916017	0	6.666525

shows the calibrated M-parameter matrix elements during pre-acquisition. Among them, SIFT (Scale Independent Feature Transform) is used for feature points, with the main parameters as follows: MaxReginon = 1,320,000, MaxReginon = 80,000, Aspect Ratio = 1~10. Here, a threshold exceeding the threshold is used to automatically remove matching outliers.

3.2.3. LiDAR and Micro-Light Polarization Image Fusion

Because the micro-light polarization image is binary, the fused point cloud appears as black and white points that are difficult to interpret and lack surface texture or material cues. Therefore, the Canny operator in MATLAB is applied to extract edge details from the micro-light polarization image, and a gradient color map is assigned to the edges. Figure 10 shows the resulting color-mapped polarization image.

The fusion of point cloud and image is based on the premise of two alignments, where through the alignment of the point cloud and image data, the correspondence between each coordinate point of the point cloud and each pixel of the image has been established. The 3D point cloud is projected onto the optical image to obtain the corresponding pixel coordinates (x, y), and then the corresponding color R, G, and B information at the pixel point (x, y) is assigned to the point cloud, which in turn realizes the coloring of the 3D point cloud data to make it have true color texture information. The MATLAB20192019 software program is used to assign the color information of the micro-light polarization image to the point cloud, so that the pixel position information of the point cloud in the micro-light polarization image can be judged according to the different color information of the point cloud, and then both the texture information of the micro-light polarization and the coordinate information of the point cloud can be displayed on the point cloud image. Figure 11 shows the point cloud image after fusion. Figure 12 shows a flowchart of the matching and fusion algorithm of the LiDAR point cloud and micro-light polarization image.

The fusion effect is shown in Figure 13 by assigning different colors to the micro-light polarization image from top to bottom, so that its color has a mapping relationship with the image pixel coordinates. After the data fusion between the LiDAR point cloud data and the micro-light polarization image, the method of extracting the micro-light polarization image and LiDAR feature points to establish the coordinate matching matrix in Section 2 of this article is used to make the color information in the micro-light polarization image correspond to the corresponding point cloud in the same one-to-one way. Accordingly, it is possible to find the coordinates of the point cloud corresponding to any point of the micro-light polarization image and its spatial position information by searching for the position of any point of the micro-light polarization image and then finding the point cloud coordinates corresponding to any point of the micro-light polarization image according to its corresponding color information. Figure 14 shows a flowchart of the whole system of matching and fusion of the LiDAR and micro-light polarization image. This article requires manual migration from MATLAB2019 to Python.

4. Experiment and Analysis

4.1. Fusion Data

In practical applications, LiDAR is unable to accurately measure a specific target due to its working mechanism. For example, when measuring the target to be measured behind the optical glass, the point cloud cannot show the target to be measured, and the radar point cloud is missing in the area of the target to be measured. In contrast, the micro--light polarization technique can obtain the micro-light polarization image behind the glass. The aim of this research is to display the micro-light polarization image information in the same position radar point cloud coordinate system through the fusion of the LiDAR point cloud and micro-light polarization image. It can make up for the missing point cloud in LiDAR and provide data support for surveyors.

Figure 15 shows the LiDAR point cloud image with optical glass in the scene, and Figure 16 shows the optical glass scene under white light conditions. The point cloud data shows that LiDAR can scan the point cloud data in general scenes, but it cannot detect the target for the back side of the optical glass. In this regard, we can obtain the position information of the glass surface based on the LiDAR point cloud data. Then, we can take the micro-light polarization image of the same scene. After that, we can use the MATLAB2019 software program in Section 1 of this article to process the collected polarization image to obtain the micro-light polarization image. Finally, we use the matching and fusion methods in Section 2 and Section 3 of this article to fuse the LiDAR point cloud image with the micro-light polarization image, so as to obtain the laser point cloud image and micro-light polarization image under the optical glass scene. Finally, using the proposed matching and fusion method, the LiDAR point cloud image is fused with the low-light polarization image to obtain a fused stereo image of the LiDAR and low-light polarization images in optical glass scenes. From Figure 17 and Figure 18, it can be seen that in the fused radar point cloud, both the information of the cart can be displayed and the approximate positional coordinates of the cart as well as the 3D spatial layout can be determined. Of course, since the relative positions of the LiDAR and the low-light camera may change during the acquisition process, the micro-light polarization image and the 3D point cloud need to be re-matched before fusion. The grainy noise apparent in Figure 19 is dominated by sensor read-out noise. Slight mechanical vibrations during the time-division rotation of the polarizer, combined with the high AGC gain, amplify the random noise, resulting in consistent bright and dark speckles across all four polarization angle images.

Polarization image processing is performed on images collected at different polarization angles, the Stokes parameters and polarization characteristics are calculated and visualized, and the processed low-light polarization images are obtained. Figure 19 shows polarization angle images for each polarization angle in the optical glass scene. Figure 20 shows the processed low-light polarization image.

From the LiDAR and micro-light polarization images, it is easy to find out that the LiDAR cannot measure the object behind the optical glass, while the micro-light polarization image can display the information of the object behind. Using the matching and fusion methods, which need to be re-matched due to changes in the relative positions of the LiDAR and the low-light camera during the acquisition process, the fused point cloud displays information about the cart and can determine the approximate positional coordinates of the cart as well as the 3D spatial layout.

4.2. Matching Effect Evaluation

For the evaluation of the micro-light polarization image and LiDAR point cloud matching, it is necessary to consider both the matching rate and the error of the micro-light polarization image and LiDAR point cloud matching results. Due to the reflection of laser by glass, LiDAR cannot collect point cloud data behind the glass, which means that the radar point cloud is missing the data behind the glass. Micro-light polarization images are designed to compensate for the missing data in low-light environments. When filling in missing data, we are more concerned about the spatial or structural consistency between the image data being filled and the missing data in the radar point cloud. Therefore, it is better to select the “geometric consistency” evaluation metric—the matching rate—as well as the maximum and average position errors as the evaluation parameters to analyze the data quality after matching.

(1) The matching rate reflects the accuracy of the cross-modal feature alignment, that is, the ratio of the number of pairs of feature points successfully matched to the total number of pairs of detected feature points. The more feature points that are successfully matched, the larger the matching rate is, and the better the matching quality is.

$$R={\displaystyle \frac{N_{\mathrm{correct}}}{N_{\mathrm{total}}}}\times 100\%$$

(8)

where N_correct denotes the number of feature point pairs correctly matched and N_total denotes the number of feature point pairs of the total matching attempts.

(2) The maximum error reflects the limit value of the degree of inaccurate matching, that is, the maximum value of single point deviation among all the corresponding point pairs (or feature pairs) in the matching or fusion result.

$$E_{\max }=\max \left(\sqrt{{\left(u_i-u_i\prime \right)}^2+{\left(v_i-v_i\prime \right)}^2}\right)$$

(9)

where (u_i, v_i) are the pixel coordinates of the point cloud mapped on the micro-light polarization image, and (u_i′, v_i′) are the corresponding 2D coordinates manually labeled in the image.

(3) The average error reflects the average degree of matching inaccuracy.

$$E_{\mathrm{average}}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N\sqrt{{\left(u_i-u_i\prime \right)}^2+{\left(v_i-v_i\prime \right)}^2}}$$

(10)

where (u_i, v_i) are the pixel coordinates of the point cloud mapped on the micro-light polarization image, (u_i′, v_i′) are the corresponding 2D coordinates manually labeled in the image, and N is the number of matching points.

According to Equations (8)–(10), the evaluation of the effect of matching the LiDAR point cloud with the micro-light polarization in Figure 16, respectively. The Statistical data is showed in Figure 21.

As can be seen from Figure 21, the matching rate between the micro-light polarization image and the LiDAR point cloud in the experiment shown in Figure 19 is 52.14%, the maximum error is 72.313 mm, and the average error is 39.659 mm, which basically achieves the matching effect.

4.3. System Optimization

As can be seen from Figure 21, the mapping matrix of the micro-light polarization image and LiDAR point cloud is established based on the feature points. Although the matching purpose is initially realized, the matching rate is not high and the error is relatively large. Accordingly, several system optimization schemes are proposed.

4.3.1. System Design Optimization

The system can be divided into multiple modules, such as data acquisition, preprocessing, matching and fusion, to facilitate independent optimization and maintenance. In the micro-light polarization imaging system, the different illumination of the scene affects the imaging quality of each polarization degree, which in turn affects the micro-light polarization image. Therefore, the imaging quality of the micro-light polarization image under different illumination conditions can be analyzed experimentally to find out the corresponding illumination when the micro-light polarization imaging effect is optimal, which can be used to guide the subsequent experimental operation.

4.3.2. Optimization of the System Feature Extraction Algorithm

The point cloud matching in this study is based on geometric features, which have been disturbed in the low-light environment. For example, the introduction of polarization features fused with geometric features or the construction of new feature vectors using the polarization degree and polarization angle can significantly improve the accuracy of feature extraction. Meanwhile, only local feature points are selected in the feature matching in this study, which are limited in number, and there is a matching error when matching the micro-light polarization with LiDAR.

Accordingly, the optimized system feature extraction algorithm is designed. Firstly, for the flickering polarized image, the Stokes parameters are calculated to obtain the polarization degree and polarization angle. Secondly, the reflective intensity of each point cloud in the LiDAR point cloud data is extracted. Finally, a mapping relationship is established by finding the polarization degree and polarization angle information of each point cloud point reflection intensity with the corresponding pixel polarization degree and polarization angle information on the polarization image to correlate the polarization degree and polarization angle information with the point cloud reflection intensity.

After the optimization of the algorithm, on the basis of the original extraction of pixel coordinates, the extraction of vectors such as the polarization degree and polarization angle of the pixels of the polarized image, as well as the extraction of the reflection intensity of the points of each point cloud of the LiDAR, is also added. Figure 22 and Figure 23 show the fusion of the LiDAR and micro-light polarization images after the optimization of the mapping relationship between the polarization information vectors and the reflection intensity of the point cloud.

Firstly, as can be seen from Figure 22, the front view of the micro-light polarization image in the new fusion map remains unchanged after system optimization, and the detail information of its micro-light polarization image is retained. Secondly, in Figure 23 and Figure 24, the curved part of the micro-light polarization image is highly matched with the point cloud data in the radar point cloud, and the fusion effect of the radar point cloud and the micro-light polarization image is better. The matching effect is also tested. From Figure 25, it can be seen that the matching rate of the feature points is improved and the system error is reduced. At the same time, we also use standard point-to-point ICP as a control, and the experimental conditions are completely consistent with the method proposed in this paper (same point cloud, same feature point set). The comparison results of the matching effects are shown in

Table 4. Comparison of matching effects before and after optimization.

Method	Matching Rate (%)	Average Error (mm)	Maximum Error (mm)
Original method	52.14	39.659	72.313
Optimizing method	74.82	21.893	63.256
Standard ICP	60.37	30.880	68.934

. Compared with standard ICP, the method proposed in this paper exhibits a higher matching rate and lower reprojection error in low-light and low-texture scenes, verifying the effectiveness of using polarization information to improve cross-modal registration.

However, systematic errors are introduced during data processing and matching optimization. During mapping and fusion based on the polarization degree, polarization angle, and LiDAR reflectance intensity, the algorithm may employ approximations or simplifications, leading to image deformation (e.g., curved poles in Figure 23). The LiDAR scene is assembled from overlapping partial point clouds. Overlapping partial clouds inevitably alter the original reflectance values. Using these altered reflectance values for polarization mapping causes excessive bending and deformation.

4.3.3. System Response Optimization

Due to the limitations of the time-sequential micro-light polarization system, we can leverage GPU acceleration and multithreading to improve the system response speed. For example, the GPU hardware can use the Moore Thread MTT S4000 (Moore Threads Technology Co., Ltd., Beijing, China). The parallel framework can use Moore’s Thread MUSA Toolkit 2.3 to compile the core operators (Stokes vector computation, point cloud projection) into PTX through mucc and call them using MATLAB R2023b’s gpuArray+parallel.gpu.CUDANel. In terms of the thread allocation strategy, on the CPU side, MATLAB parpool (“local”, 12) can be used for outlier removal; on the GPU side, each 640 × 480 image block is processed in parallel with 1024 × 1 × 1 thread block.

5. Conclusions

In this paper, for the problem of LiDAR being unable to generate point clouds for the area behind the glass, a method of fusing the micro-light polarization image with the laser LiDAR point cloud is proposed to realize the reconnaissance, identification and ranging method for the blind area of the point cloud caused by transparent media such as glass in low illumination environments. The method optimizes the 3D point cloud generated from the micro-light polarization image by introducing the polarization degree, polarization angle, and reflection intensity of each point of the LiDAR and constructing a point cloud–pixel mapping matrix that contains a depth offset and a better fit on the basis of extracting the pixel coordinates of the 3D LiDAR point cloud. Subsequently, with the help of 3D point cloud fusion and the pseudo-color mapping algorithm, the matching and fusion algorithm of the micro-light polarization map and the radar point cloud are further optimized, and the alignment and fusion of the 2D micro-light polarization map and the 3D LiDAR point cloud are successfully completed. Experiments show that the alignment rate of the two reaches 74.82%, which can effectively detect hidden targets behind the glass under low illumination conditions while compensating for the blind spot of the LiDAR point cloud data, verifying the feasibility of “Polarization + LiDAR” reconnaissance in low illumination glass scenes and providing new ways of detecting hidden targets in complex environments. It provides a new means of hidden target detection in complex environment.

This study validates the effectiveness of micro-light-polarization–LiDAR fusion for glass-occlusion detection in static, low-light scenes, but it still has the following limitations:

First, feature point selection relies on a limited sample size, introducing randomness and leading to substantial errors and low accuracy when matching low-light polarization images with LiDAR point clouds.

Second, dynamic scene adaptability is limited: the current time-division polarizer requires four sequential exposures, causing motion blur in moving targets and increasing feature point mismatches.

Third, remote detection accuracy degrades with distance: the ML-30S LiDAR point density declines sharply, while the SNR of micro-light polarization images falls with the square of the distance, thereby increasing the matching errors.

Future work will be carried out from the following three aspects. Matching optimization: increase the number of feature points and optimize the point cloud matching process. Dynamic perception: develop a snapshot polarization camera based on a liquid crystal adjustable polarizer (μs-level switching), combined with an event camera to achieve blur free imaging of moving targets. Of course, a split aperture acquisition system can also be used to attempt real-time acquisition of polarization images using four channels simultaneously. Remote enhancement: introduce high-power fiber lasers and dual axis MEMS scanning to effectively expand the detection range; simultaneously using multiple frames of polarized image super-resolution reconstruction to improve SNR.