Super-Resolution Reconstruction of LiDAR Images Based on an Adaptive Contour Closure Algorithm over 10 km

Abstract

Reflective Tomography LiDAR (RTL) imaging, an innovative LiDAR technology, offers the significant advantage of an imaging resolution independent of detection distance and receiving optical aperture, evolving from Computed Tomography (CT) principles. However, distinct from transmissive imaging, RTL requires precise alignment of multi-angle echo data around the target’s rotation center before image reconstruction. This paper presents an adaptive contour closure algorithm for automated multi-angle echo data registration in RTL. A 10.38 km remote RTL imaging experiment validates the algorithm’s efficacy, showing that it improves the quality factor of reconstructed images by over 23% and effectively suppresses interference from target/detector jitter, laser pulse transmission/reception fluctuations, and atmospheric turbulence. These results support the development of advanced space target perception capabilities and drive the transition of space-based LiDAR from “point” measurements to “volumetric” perception, marking a crucial advancement in space exploration and surveillance.

1. Introduction

With the rapid advancement of space technology and its expanding applications, long-range high-resolution imaging of space debris and other small, faint dark space targets has garnered growing attention from aerospace and national defense agencies worldwide. This focus aims to ensure safer and more efficient space utilization [1,2,3]. However, passive photoelectric detection imaging technologies are inherently constrained by their operational principles and the physical limitations of receiving optical apertures, which poses significant challenges for achieving high-resolution detection and imaging of distant space targets [4,5,6]. Reflective Tomography LiDAR (RTL) imaging represents a cutting-edge active photoelectric detection technology [7]. Theoretically, its imaging resolution is independent of both detection range and system aperture. By capturing the envelope information of laser echo pulses reflected from targets, RTL circumvents the strict requirements for platform stability and laser coherence that characterize synthetic aperture LiDAR [1]. As such, it offers a promising new paradigm for high-resolution imaging of space debris and other small, dim celestial objects in dark space environments.

RTL imaging technology is essentially a method for restoring image contours in distance-distinguishable and angle-indistinguishable scenes by introducing multi-angle echo information [1]. The RTL imaging technique was first introduced by Parker et al. in 1988, who employed a filtered back-projection (FBP) algorithm to reconstruct a cross-sectional image of a conical target at 10 m with a resolution of 12.6 cm [7]. One year later, Knight et al. enhanced the experimental setup by substituting the photomultiplier with an EG&G stripe camera, achieving a fourfold improvement in imaging resolution to 4 cm [8]. In 2001, researchers from the U.S. Air Force Research Laboratory, led by Matson et al., successfully reconstructed the relative positions of a LACE satellite (orbital altitude: 514 km) and two germanium angle mirrors on its telescope arm (separation: 6 m), demonstrating a spacing resolution of ~0.2 m [9]. In 2010, Murray et al. from Areté Associates Optical Systems Research (Longmont) conducted a range-resolution imaging experiment using pulse compression, enabling the reconstruction of three uniformly rotating angle mirrors (1 m spacing, 30 cm size) at 22.4 km with a 15 cm spacing resolution [10]. Most recently, in 2021, Hu et al. performed an RTL imaging experiment at 0.98 km, achieving image reconstruction of a composite planar target—three 5 cm square panels arranged at 135° angles—with an imaging resolution of ~2 cm [3].

Projection registration, a critical step in RTL imaging, aims to align multi-angle echo data with the target’s rotation center [11]. The reference screen method, the most widely adopted projection registration technique, is frequently used in ground-based verification experiments [12]. This approach requires the reference screen to remain stationary relative to the laser beam, with registration performed by leveraging the constant range characteristic between multi-angle echo data and the rotation center. However, the inherent challenge of locating an ideal reference screen in the vicinity of actual space targets limits its applicability to real-world detection scenarios [13].

In 2010, Jin et al. proposed the feature point tracking method, which is based on the point where the reflection coefficient of the target surface changes suddenly [14]. However, the target surface often lacks such feature points, and to distinguish the feature points, the extremely narrow laser pulse width required does not align with the practical design of the system. In 2020, Chen et al. proposed a new projection registration idea according to the high correlation between adjacent projections of the same target, taking the comparison of the maximum average envelope of the total projection as the alignment criterion [15]. However, this method only estimates the rotation center position of the target roughly; it ignores the signal drift in time dimension owing to the interference of target or detector jitter, laser pulse emission, and detector receiving random jitter during the operation of the RTL imaging system. In 2021, Hu et al. applied the spectrophotometry method to RTL imaging, which lets the laser beam pass through a beam splitter and split into two paths: one path shines through and completely covers the target, while the other path enters the detector as the reference [3]. This method solves the interference of the above factors well, and the registration accuracy is mainly affected by the stability of laser energy. Moreover, influences of atmospheric turbulence and other factors exist in remote RTL imaging, which also lead to signal drift in the time dimension. In 2022, Hu et al. proposed a projection registration algorithm based on automatic target contour correction based on the spectrophotometry method; they conducted an RTL imaging experiment with a distance of 0.98 km to verify the algorithm’s effectiveness [16].

In this study, addressing the challenge of aligning multi-angle echo data with the target’s rotation center, we propose an adaptive contour closure algorithm for automatic projection registration of multi-angle echo data in RTL imaging, leveraging the spectrophotometry method. To validate the algorithm, we conducted a long-range RTL imaging experiment at a distance of 10.38 km in the Zipeng Mountain area, Hefei. Experimental results demonstrate that the proposed algorithm effectively resolves the projection registration issue for closed-surface targets with regular contours, such as cubes and triangular prisms. Specifically, the quality factor of the reconstructed images after registration is enhanced by over 23%. Moreover, the algorithm significantly mitigates the adverse effects of various interferences, including target or detector vibrations, random fluctuations during laser pulse transmission and detector reception, and atmospheric turbulence, in the processed multi-angle echo data.

2. Methodology

The reconstruction of a target image using multi-angle projection data was first used in computed tomography (CT) imaging [17], which essentially detects the intensity of X-rays transmitted and then uses the attenuation information to reconstruct to obtain the tomography image of the target [18]. The projection data $p$ of CT imaging has directional reversibility:

$$p(r,\varphi +\pi )=p(2d-r,\varphi )$$

(1)

where $\varphi$ is the incident angle, $d$ is the range from the center of rotation, and line $r=x\cos \varphi +y\sin \varphi$ is perpendicular to the incident direction. This indicates that the collinear projection data of CT imaging is the same and that only 180 degrees of projection data need to be collected for imaging. Developed from CT imaging but unique, the RTL imaging technique needs to collect 360 degrees of echo data of the target. The echo data $p(r,\varphi )$ at angle $\varphi$ and the echo data $p(r,\varphi +\pi )$ at angle $(\varphi +\pi )$ after symmetry transformation are superimposed together to form 180 degrees of projection data [17]. The projection data formed after superposition can be expressed as follows:

$$p^{*}(r,\varphi )=p(r,\varphi )+p(2d-r,\varphi +\pi )$$

(2)

The projection data $p^{*}(r,\varphi )$ of each angle is back smeared in accordance with angle $\varphi$, and the back-projection images of each angle are superimposed. Direct back-projection produces many artifacts, and the FBP algorithm is often used to remove artifacts. The reconstructed cross-sectional image of the target using the FBP algorithm can be expressed as follows [19]:

$$g(x,y)={\displaystyle {\sum }_{i=1}^n{F}_1^{-1}\left\{\left|\omega \right|F_1\left[p^{*}(r,\varphi )\right]\right\}}$$

(3)

where $n$ is the amount of projection data; $F_1$ and $F_1^{-1}$ represent the operator of 1-D Fourier transform and inverse 1-D Fourier transform contravariant converter, respectively; and $\left|\omega \right|$ is the Jacobi factor from cartesian to polar coordinates, which acts as a ramp filter. Considering that an ideal ramp filter does not exist, this paper uses the Ram-Lak filter function, most commonly used in RTL imaging, to set the cut-off frequency and realize frequency domain windowing.

As deduced from Equation (2), the error in determining the rotation center range d causes the error in the projection data $p^{*}(r,\varphi )$ of all angles. According to Equation (3), the error directly affects the accuracy of the target contour in the reconstructed image. After the rotation center range d has been determined, the echo data $p(r,\varphi )$ at angle $\varphi$ and the echo data $p(r,\varphi +\pi )$ at angle $(\varphi +\pi )$ drift in the time dimension owing to the interference of atmospheric turbulence and other factors in remote RTL imaging. That is, the range between $p(r,\varphi )$ and $p(r,\varphi +\pi )$ relative to the rotation center fluctuates near the true range, so the range between $p(r,\varphi )$ and $p(r,\varphi +\pi )$ before the symmetry transform and the rotation center needs to be calibrated; otherwise, the reconstructed image will be fuzzy and the accuracy of the reconstructed image will be insufficient.

The flow chart of the automatic registration of multi-angle echo data algorithm proposed in this paper is shown in Figure 1. This algorithm exploits the fact that space targets are mostly volume targets, while general volume targets have closed contours, so this algorithm is proposed for closed surface targets with regular contours, such as cubes and triangular prisms.

First, multi-angle echo data were used to reconstruct the image reconstructed by FBP algorithm, and then the line segment detector (LSD) algorithm was used to extract the line segment in the target reconstruction image. The LSD algorithm is a line segment detection algorithm that can achieve sub-pixel accuracy in linear time and can be used in any digital image without parameter adjustment [20]. The line segment $L_1,L_2,\dots ,L_n$ was extracted by the LSD algorithm, and then classification was used to fit the line segment. The goal was to group line segments whose directions are close to each other and whose range is less than the range resolution into one class. The piecewise linear fitting of all points in each class was carried out to obtain intersecting lines, and the coordinates of the intersection points of each line were obtained. Then, the closed patterning was connected successively to obtain the closed target contour initial estimation. Combined with the main parameters, such as the laser pulse width and detecting circuit bandwidth, the estimated image was generated. The normalized average absolute distance measurement value was selected as the error function and expressed as follows:

$$r={\displaystyle \frac{{\displaystyle {\sum }_{i,j\in ROI}\left|t_{i,j}-r_{i,j}\right|}}{{\displaystyle {\sum }_{i,j\in ROI}\left|t_{i,j}\right|}}}$$

(4)

where $t_{i,j}$ and $r_{i,j}$ represent gray values at pixel $(i,j)$ in the original image and the estimated image, respectively.

$ROI$ represents the region of the image by changing the line parameters until the error function has become minimal. The error function of the estimated image and the original image after iterative threshold segmentation was compared, and then the target contour optimal estimation with minimum error function was obtained by continuous automatic target contour correction. The position of the first peak point of the projection data $p^{*}(r,\varphi )$ was taken as the peak position of the echo data under the angle $\varphi$. The multi-angle laser echo data after correction were obtained by correcting the deviation of the peak value of the echo relative to the echo data under each angle, and then the target image was reconstructed again.

3. Experiment Result and Analysis

Figure 2 shows a bird’s eye view of the long-range active-imaging experiment, where the RTL system was placed on the stairs of a South China City community (31°46′20″ N, 117°5′35″ E), and the target was placed on the test tower of Zipeng mountain (31°43′28″ N, 116°59′55″ E). The optical receiver system features a Cassegrain telescope with a 260 mm aperture and a high-precision two-axis rotating stage to allow manual control to aim at the far-field target. The optical components were assembled on a custom-built aluminum platform with excellent shock absorption. The entire optical hardware system is compact and suitable for mobile applications [see Figure 2d].

Specifically, as shown in Figure 2c, the actively mode-locked picosecond laser (1064 ± 0.1 nm wavelength, 73 ps pulse width, 15 Hz repetition rate) served as the light source for illumination. The maximal average laser power transmitted was 1.1 W, or 73 mJ per pulse. Using a near-infrared wavelength is advantageous for several reasons, such as reduced solar background, low atmospheric absorption loss, and a higher eye safety threshold compared with the visible band. After being reflected through R1, the laser entered the Pin detector through NPBS and was collected by channel 1 of high-speed data acquisition. After passing through NPBS, the other laser was reflected by R2 into the beam expander, illuminated the far-field target, and covered it. The echo modulated by the target surface was coupled with the optical receiver system into the multi-mode fiber and input into the APD detector, and then it was collected by channel 2 of high-speed data acquisition. The transmitted signal collected in channel 1 was used as a reference, and the delay triggered a certain time to start collecting the signal of channel 2.

The bandwidth of the APD detector was 7.5 GHz, and the bandwidth of the Pin bandwidth was 15 GHz. The data were collected by the high-speed data acquisition with a sampling rate of 50 GSPS, and the industrial computer controlled the whole system and completed the real-time data processing. A photograph of the typical space target model is shown in Figure 2b, which is a cube with sides of 40 cm. To achieve a high-efficiency, low-noise RTL system, we established several optimized optical designs, most of which differ from previous RTL systems [7,8,9,10]. With these new technologies, the imaging resolution and the signal-noise ratio (SNR) can be greatly improved.

The multi-angle echo data of the cube target with relatively stable time obtained by the spectrophotometry method are shown in Figure 3a, where the sampling angle ranges from 1° to 360°, and the sampling angle interval is 1°. In Figure 3, the horizontal axis represents the angle, and the vertical axis represents the range. The echo intensity at different ranges is marked by colors in the image. Considering that the echo signal value is not less than zero, all echo data are preprocessed with a negative return to zero.

The range $d$ from the rotation center was determined by spectrophotometry method and marked in Figure 3a. The echo data from 181° to 360° were symmetrically transformed with the rotation center as the axis of symmetry and then superimposed with the echo data from 1° to 180°. The projection data formed after superposition were obtained, as shown in Figure 3b. According to these projection data, the target reconstruction image is obtained by using FBP algorithm, as shown in Figure 4a, and the image processed by iterative threshold segmentation algorithm is shown in Figure 4b.

To objectively compare the segmentation quality of the reconstructed image before and after the automatic registration of multi-angle echo data, based on the target surface shape, we used the main parameters, such as laser pulse width and detection circuit bandwidth, and obtained an ideal reconstructed image of the target. We used the following evaluation metrics:

(a) False negative rate (FNR): This is the ratio of the number of false pixels extracted to the number of true pixels. False pixels are referred to as negative samples; true pixels are referred to as positive samples. FNR is calculated by the following:

$$FNR={\displaystyle \frac{FN}{TP+FN}}={\displaystyle \frac{FN}{P}}$$

(5)

where FN is the number of positive samples extracted as negative samples, TP is the number of positive samples extracted as positive samples, and P is the total number of positive samples. The perfect FNR value is zero, at which point all the pixels have been correctly extracted.

(b) False positive rate (FPR): This is the ratio of the number of false pixels extracted to the total number of pixels extracted. FPR is calculated by the following:

$$FPR={\displaystyle \frac{FP}{FP+TN}}={\displaystyle \frac{FP}{N}}$$

(6)

where FP is the number of negative samples extracted as positive samples, TN is the number of negative samples extracted as negative samples, and N is the total number of samples. The perfect FPR value is zero, at which point all the pixels have been correctly extracted.

(c) Quality factor (QF): This is used to evaluate overall image segmentation quality, considering both the false positive and false negative rates. QF is calculated by the following:

$$QF={\displaystyle \frac{TP}{P+FP}}$$

(7)

The perfect QF value is one, at which point all the pixels have been extracted correctly.

The images reconstructed by the FBP algorithm using the multi-angle echo data registration by the algorithm proposed in Reference [15] and that proposed in this paper are shown in Figure 5a,c, respectively. To further compare the effect of the registration algorithms, we had to compare the reconstructed images before and after registration with the ideal reconstructed images. Considering the existence of a large number of artifacts and noise in a gray image, the image had to be segmented. The target extraction and segmentation of RTL image has been widely studied, and the most commonly used segmentation algorithm is the iterative threshold method [20]. This algorithm is based on the idea of approximation and is advantageous because it requires little computation and is unaffected by the change in the gray image under certain conditions. The specific process was to first obtain the maximum gray value $T_{\max }$ and minimum gray value $T_{\min }$ of the image; then, we took the mean value as the initial threshold:

$$T_0={\displaystyle \frac{T_{\max }+T_{\min }}{2}}$$

(8)

According to the initial threshold $T_0$, the image was divided into two parts. The average gray values $Z_0$ and $Z_1$ of the two parts could be calculated as follows:

$$Z={\displaystyle \frac{{\displaystyle {\sum }_{i=1}^{255}\omega (i)\times i}}{n}}$$

(9)

where $n$ is the total number of pixels, $i$ is the gray value, and $\omega (i)$ is the weight of the gray value $i$. The new threshold value could be calculated as follows:

$$T_1={\displaystyle \frac{Z_0+Z_1}{2}}$$

(10)

$T_1$ was assigned to $T_0$, then the image was re-segmented according to the assigned $T_0$, and the average gray value of the two parts was recalculated. The process was repeated until the end of $T_0=T_1$. The images of Figure 5a,c after processing by the iterative threshold segmentation algorithm are shown in Figure 5b,d, respectively.

Next, the segmentation quality of reconstructed images before and after automatic registration of multi-angle echo data was objectively compared. The compared FNR, FPR, and QF values of reconstructed images before and after the automatic registration of multi-angle echo data are shown in

Table 1. Comparison of reconstructed images before and after automatic registration of multi-angle echo data.

Algorithm	FNR	FPR	QF
Only spectrophotometry method is used	0.5535	0.4985	0.3093
The algorithm proposed in Reference [15]	0.5925	0.3683	0.3293
The algorithm proposed in this paper	0.5971	0.1221	0.3815

.

The above table presents a comprehensive comparison of image reconstruction results, offering valuable insights into the effectiveness of different registration algorithms. For the algorithm proposed in Reference [15], the False Positive Rate (FPR) of the target reconstructed image post-registration is lower than that of the pre-registration image, while the Quality Factor (QF) of the registered image demonstrates a 6.5% increase. These findings are in close alignment with the conclusions drawn in Reference [15], validating the algorithm’s efficacy in enhancing image quality to a certain extent.

Notably, the algorithm proposed in this paper exhibits even more remarkable performance. The FPR of the target reconstructed image after applying our algorithm is significantly reduced compared to the pre-registration state. More importantly, the QF value of the registered target reconstruction image experiences a substantial boost of over 23.3%, marking an improvement of 11.6% relative to the algorithm in Reference [15]. This substantial enhancement in the QF value indicates that our algorithm can achieve a higher-quality image reconstruction, capturing more accurate details of the target. A lower FPR implies a decreased likelihood of misidentifying non-target elements as part of the target, thereby improving the reliability and precision of the reconstructed images. The superior performance of our algorithm can be attributed to its unique adaptive contour closure mechanism, which better addresses the challenges in multi-angle echo data alignment and projection registration, making it more suitable for dealing with complex target structures and real-world interference factors.

4. Conclusions

In summary, the algorithm developed in this study represents a significant advancement in RTL imaging, offering robust solutions for projection registration of closed-surface targets with regular contours, including cubes and triangular prisms. The experimental results demonstrate its exceptional efficacy, with the quality factor of registered reconstructed images enhanced by over 23%. This performance gain is accompanied by the algorithm’s remarkable ability to mitigate multiple sources of interference, such as target or detector vibrations, fluctuations during laser pulse transmission and reception, and atmospheric turbulence, thereby ensuring high-fidelity multi-angle echo data processing.

Looking ahead, the implications of this research extend far beyond the current study. In the field of space debris monitoring, where accurate identification and tracking of small, irregularly shaped objects are crucial for safeguarding operational satellites, this algorithm can facilitate high-resolution imaging, enabling more precise orbit prediction and collision risk assessment. For aerospace exploration, it paves the way for detailed imaging of distant celestial bodies or space structures during flybys or rendezvous missions, providing essential data for scientific analysis and mission planning [21]. In the realm of industrial non-destructive testing, particularly for inspecting complex mechanical components with regular geometric features, the algorithm can enhance the accuracy of internal defect detection, contributing to improved product quality control.

Notably, while the iterative threshold method employed in this paper for image segmentation achieved satisfactory results, it left room for improvement, especially regarding the residual artifacts in target contours of registered reconstructed images. Future research efforts could focus on developing more advanced image segmentation techniques, such as deep-learning-based methods, which have demonstrated superior performance in handling complex edge detection and noise suppression tasks. By integrating these refined segmentation algorithms with the existing registration approach, we anticipate a further substantial boost in the quality factor of reconstructed images, potentially unlocking new applications in high-precision 3D modeling, autonomous robotic inspection, and augmented reality-based navigation systems. Overall, this study not only addresses an existing technical challenge but also serves as a springboard for future innovations in active photoelectric detection and imaging technologies.