Underwater Target 3D Reconstruction via Integrated Laser Triangulation and Multispectral Photometric Stereo

Abstract

With the gradual application of 3D reconstruction technology in underwater scenes, the design of vision-based reconstruction models has become an important research direction for human ocean exploration and development. The underwater laser triangulation method is the most commonly used approach, yet it misses details during the reconstruction of sparse point clouds, which do not meet the requirements of practical applications. On the other hand, existing underwater photometric stereo methods can accurately reconstruct local details of target objects, but they require relative stillness to be maintained between the camera and the target, which is practically difficult to achieve in underwater imaging environments. In this paper, we propose an underwater target reconstruction algorithm that combines laser triangulation and multispectral photometric stereo (MPS) to address the aforementioned practical problems in underwater 3D reconstruction.This algorithm can obtain more comprehensive 3D surface data of underwater objects through mobile measurement. At the same time, we propose to optimize the laser place calibration and laser line separation processes, further improving the reconstruction performance of underwater laser triangulation and multispectral photometric stereo. The experimental results show that our method achieves higher-precision and higher-density 3D reconstruction than current state-of-the-art methods.

1. Introduction

In the field of marine exploration research, acoustic imaging and optical imaging are the two primary detection technologies [1,2,3]. Acoustic imaging, leveraging the strong penetration capability of sound waves in water bodies, far exceeds the detection range of optical imaging. However, limited by its wavelength characteristics, acoustic imaging systems suffer from low resolution, making them inadequate for high-precision applications such as target identification and biological feature analysis. In contrast, optical imaging technology, with its advantage of submillimeter-level high-resolution reconstruction, has gradually become a dominant method in high-precision detection scenarios, including seabed topographic mapping, underwater archaeology, and ecological monitoring.

Laser triangulation and photometric stereo are currently the most widely used 3D reconstruction techniques based on optical imaging [4,5]. The traditional laser triangulation method can reconstruct high-precision sparse point clouds, while the traditional photometric stereo method can reconstruct scale-free dense point clouds. The lack of details in laser triangulation and the relative stillness requirement between the camera and the target in photometric stereo methods greatly limit the application of these two methods underwater.

In order to alleviate the relative stillness requirement between the camera and the target in photometric stereo, the multispectral photometric stereo (MPS) technique has been developed, which only requires one image to obtain the 3D information of the target object [6]. More precisely, multispectral photometric stereo requires an image captured under the illumination of light sources of three primary colors (red, green, and blue), and can calculate the surface information of the target through three color channel separation. MPS has the advantage of dense reconstruction in photometric stereo and limits the influence of the relative stillness constraint between photometric stereo cameras and targets. Multispectral photometric stereo is essentially a photometric stereo with three primary color light sources, which changes the traditional time-division multiplexing (multiple single light sources) of photometric stereo to frequency-division multiplexing (single multiple light sources) [7]. Due to the fact that multispectral photometric stereo can be reconstructed from a single image without relying on lighting transformations between multiple images, this method can be applied to the processing of dynamic videos. However, the multispectral photometric stereo algorithm has some limitations in reconstruction [8], as the results obtained are pixel-based and lack true scale information. Since the laser triangulation method can obtain high-precision scale information, this paper investigates a method that could combine effectively linear laser and multispectral photometric stereo to reconstruct underwater targets.

In linear laser triangulation, the laser device emits a laser plane that intersects with the surface of the object to be measured, generating laser lines on the surface of the object. By calculating the changes in the laser lines on the image, the scale information of the object can be obtained [9,10,11]. In laser triangulation, the extraction of laser lines is crucial and directly affects its reconstruction accuracy.Although some studies have optimized the extraction of laser lines [12,13,14,15,16], their development did not alleviate the urgency and necessity of performing high-precision extraction based on laser lines.

In order to achieve high-precision and high-resolution 3D reconstruction of the target object only using a single underwater image, we leverage multispectral images containing laser lines. More specifically, our approach consists of separating an input image into a multispectral image and a laser image, and using the true scale information of the laser image to provide prior information for the 3D reconstruction of the multispectral image. The focus of this step is on the separation of laser lines in the image. Due to the fact that laser lines have the same color information as the multispectral images, traditional morphological processing is difficult to achieve. Methods such as the traditional algorithm for hole filling via waveform information proposed by Crimnisi et al. [17], the tensor nuclear norm minimization algorithm introduced by Lu et al. [18], and the iterative correction algorithm with feedback mechanism developed by Zeng et al. [19] have primarily been applied to image completion and have not been utilized for laser line separation. Therefore, the separation process of laser lines from multispectral information remains the main task of investigation in this paper.

We select images with the same time interval as key frames, and after completing the 3D reconstruction of a given underwater image, we use the different camera poses to fuse the point clouds obtained from multiple images together. Due to the continuity of the captured video sequence, the intervals between the selected key frames will inevitably lead to multiple values at the same position on the object’s surface during the reconstruction process.

To address the issues of laser line extraction, laser line separation, and data redundancy, we propose the following solutions:

• The use of a laser plane calibration method and line fitting to optimize the laser line extraction process.
• The design of a keyframe restoration method using image sequences to solve the separation of laser lines.
• The development of a weighted method to achieve efficient data fusion, leveraging the distance between points from point clouds and laser line information.

2. Related Work

2.1. Linear Laser Triangulation

Calibrating the camera and laser plane is the first step in linear laser triangulation reconstruction. Over the years, the camera calibration technology has been well developed. The calibration method proposed by Zhang [20] overcomes the calibration problem of internal and external camera parameters and has continually been in use. Hong et al. [21] proposed a non-iterative real-time camera calibration method that solves the problem of local minima without requiring an initial estimation. Niu et al. [22] achieved the calibration of the relative direction of the camera fixed on the rotation axis by using a constrained global optimization algorithm with two chessboards and a wide-angle camera. The laser plane calibration process is relatively simple, in which the normal vector of the laser plane is calculated by the cross product of two non-parallel laser lines [9,23]. Similar methods such as converting pixel coordinate systems to world coordinate systems through the cross-ratio invariance property [10,24] have also been designed.

Some underwater laser scanning methods can compensate for the refraction of incident and emitted light at the interface [25,26]. Laser line scanning has a stable anti-interference ability and can be applied to complex underwater environments [27]. Therefore, this paper proposed to exploit underwater laser scans with a direct calibration method.

As for linear laser 3D point cloud registration, Li et al. [28] proposed a 3D point cloud registration method based on linear laser scanner and data processing. The deviation between the known set of a point cloud and the laser-scanned point cloud is evaluated through feature extraction, and the point cloud data are obtained by a registration algorithm. Feng et al. [29] proposed to perform point cloud registration using the Grey Wolf Optimizer (GWO). This GWO algorithm can extract various parameters of the rotation matrix to optimize local and global optimal solutions in a relatively short time. Li et al. [30] proposed a point cloud registration method based on feature quadratic distance, which optimizes the distance between two Gaussian mixture models (GMMs) in order to obtain a rigid transformation between two sets of points. Zou et al. [11] proposed a new Local Angle Statistical Histogram (LASH), which achieves point cloud registration through geometric description of local shapes and matching points of the same similarity. In addition, there have been many global applications and studies for 3D reconstruction of line lasers [31,32,33,34]. Hong et al. [35] present a probabilistic normal distributions transform (NDT) representation which improves the accuracy of point cloud registration by using the probabilities of point samples. Some studies have optimized NDT to improve the registration of point clouds [36,37].

2.2. Multispectral Photometric Stereo

Narasimhan and Nayar [38] proposed to recover albedo, normal, and depth maps from scattered data to address scattering and refraction issues in underwater environments. To this end, they designed a physical model of the target object surface surrounded by the scattering media. Tsiotsios et al. [39] mitigated light interference through pixel-adaptive backscattering modeling, enabling 3D data reconstruction without requiring prior medium/scene parameter calibration. Subsequently, Jiao et al. [40] developed a hybrid framework integrating depth-sensing devices with multispectral photometric analysis, achieving micron-level surface reconstruction for submerged objects. Building upon this, Fan et al. [41] introduced a multimodal imaging paradigm that synergizes structured laser illumination with photometric stereo principles, significantly enhancing measurement accuracy in turbid aquatic environments. In the near-range underwater photometric stereo, considering a close enough range to the target, the scattering, attenuation, and refraction effects can be neglected.

The multispectral photometric stereo algorithm only requires one RGB image to achieve 3D reconstruction of the target’s object. Since MPS was proposed, it has received extensive attention and research. Zhou et al. [42] proposed a design of a concentric multispectral light field (CMSLF), which can extract the shape and reflectivity of any material surface in a single shot through multispectral ring lamps and spectral multiplexing. Ju et al. [43] reconstructed dense and high-accuracy surface normals from a single color image without prior or additional information. To this end, they designed a deep neural network framework as a “demultiplexer”. Hamaen et al. [44] proposed to convert multi-color objects into monochrome objects. Their approach generates intrinsic images unaffected by reflectance through decomposition and obtains surface normals through conventional photometric stereo algorithms. Guo et al. [45] proposed solutions to some restrictive assumptions regarding multispectral photometric stereochemistry. Miyazaki et al. [46] used a measurement device that can achieve multispectral photometric stereo with sixteen colors to address mathematical and color constraints. Yu et al. [8] proposed to use a high-speed rotating light source to induce radiation changes and trigger event signals, which can estimate surface normals of targets by converting continuous events into zero space vectors. Lu et al. [47] reconstructed object surfaces by separating multispectral images with laser lines into laser images and spectral images.

3. Theory and Method

In this section, we first propose a laser (laser plane) calibration method that fits straight lines. Secondly, we provide a laser line separation solution for multispectral images containing laser lines. Finally, we use weighted data fusion in mobile reconstruction to solve the problem of data redundancy. The overall 3D reconstruction flowchart is shown in Figure 1, where the steps marked in red denote our three different innovations.

3.1. Calibration of Camera and Laser Plane

The calibration of a camera is mainly aimed at eliminating the impact of radial and tangential distortions of the lens on camera imaging. For nonlinear imaging models, the distortion coefficient, as an important component of camera internal parameters, can be calculated through camera calibration. This calculated distortion coefficient can be used to eliminate distortion in the original image. In this paper, we utilize the traditional camera calibration approach [20]: we first capture standard checkerboard patterns underwater at different positions and directions (shown in Figure 2), and then use the calibration toolbox provided in MATLAB to complete the camera calibration.

After completing the calibration of the camera, its internal parameter matrix can be expressed as:

$$K=\left(\begin{array}{ccc}{f}_x& s& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right)$$

(1)

where $f_x={\displaystyle \frac{f}{dx}}$, $f_y={\displaystyle \frac{f}{dy}}$ are focal lengths, and $f_x$ and $f_y$ are the partial derivatives of focal length f in the x and y directions. $u_0$, $v_0$ represent principal point coordinates, and s is the coordinate axis tilt parameter, ideally 0;

Before solving the laser plane, we define the coordinate system:

• In the world coordinate system $(Xw,Yw,Zw)$, the origin is $Ow$, which can be any point in the 3D world;
• In the image coordinate system $(Ui,Vi,1)$, the origin is $Oi$, located in the top left corner of the image;
• In the camera coordinate system $(Xc,Yc,Zc)$, the origin is $Oc$, located in the optical center of the camera.

To solve the laser plane equation, the first step is to convert the image coordinates of the pixels on the laser line to the camera coordinate system. Due to the fact that the world coordinate system defined during camera calibration is based on the plane where the calibration plate is located as the X−O−Y plane, i.e., the plane equation of the calibration plate plane in the world coordinate system is Z = 0, the next step is to calculate the equation of the calibration plate plane in the camera coordinate system. By calibrating the camera, the rotation and translation $(R,t)$ from the world coordinate system to the camera coordinate system are obtained. R and t are transformations from the world coordinate system to the camera coordinate system, and their inverses ($R^{-1}$ and $t^{-1}$) from the camera coordinate system to the world coordinate system are calculated as follows:

$$\left(\begin{array}{c}Xw\\ Yw\\ Zw\end{array}\right)=R^{-1}\left(\begin{array}{c}Xc\\ Yc\\ Zc\end{array}\right)+t^{-1}$$

(2)

The following equation represents the correspondence between any point on the imaging plane and its coordinates in the normalized camera coordinate system:

$$\left(\begin{array}{c}Ui\\ Vi\\ 1\end{array}\right)=K\left(\begin{array}{c}Xc\\ Yc\\ 1\end{array}\right)$$

(3)

Assuming the laser plane equation is $AX+BY+CZ+D=0$ in the camera coordinate system, the centroid positions of N 3D points on the laser plane are

$$\overline{X}={\displaystyle \frac{\sum Xi}{N}},\overline{Y}={\displaystyle \frac{\sum Yi}{N}},\overline{Z}={\displaystyle \frac{\sum Zi}{N}}$$

(4)

We then reset the center of mass to zero:

$$\left(Xi-\overline{X},Yi-\overline{Y},Zi-\overline{Z}\right)$$

(5)

Finally, the covariance matrix can be expressed as:

$$\left[\begin{array}{ccc}\sum (Xi-\overline{X})(Xi-\overline{X})& \sum (Xi-\overline{X})(Yi-\overline{Y})& \sum (Xi-\overline{X})(Zi-\overline{Z})\\ \sum (Yi-\overline{Y})(Xi-\overline{X})& \sum (Yi-\overline{Y})(Yi-\overline{Y})& \sum (Yi-\overline{Y})(Zi-\overline{Z})\\ \sum (Zi-\overline{Z})(Xi-\overline{X})& \sum (Zi-\overline{Z})(Yi-\overline{Y})& \sum (Zi-\overline{Z})(Zi-\overline{Z})\end{array}\right]$$

(6)

After that, we perform singular value decomposition (SVD) on the covariance matrix, and the eigenvector corresponding to the minimum eigenvalue is the normal vector of the laser plane.

We propose a sub-pixel laser line extraction method based on line fitting, whose specific workflow is as follows:

Firstly, we preprocess the obtained image by selecting the image containing laser lines and the calibration checkerboard, cropping the original image ($1920\times 1080$) to a resolution of $800\times 800$ that only contains the laser lines and calibration board, and positioning the laser lines in the middle of the cropped image.

Next, we perform separation and binarization of the three color channels. For the separation step, we use the image $(:,:,1)$, image $(:,:,2)$, and image $(:,:,3)$ functions in Matlab to separate the red, green, and blue component images, generating three grayscale value images, and traverse all pixels of the image for binarization; that is, the pixels of the image are either black or white, containing only two values: 0 or 1.

Then, we perform image morphology judgment, remove the influence of noise such as overexposed areas, and highlight areas based on the connected area and circularity. We mark and number the connected regions in the image, and calculate the area and perimeter of each connected region. Based on experience, we label connected regions with over 40,000 pixels as overexposed areas, which are set to 0. As for the connected regions whose area is less than 40,000 pixels, we determine their circularity using $e=(4\times \pi \times area)/\left(perimete{r}^2\right)$. When the circularity is greater than 0.5, the connected region is labeled as a highlight area, which is also set to 0. After that, we record the coordinates of candidate points for each channel according to the set threshold.

Finally, we fit a straight line over these candidate points and remove candidate points with low confidence based on their dispersion from the line until the optimal laser line and laser plane are obtained. More specifically, we traverse all non−zero pixels in the image, record the coordinates of each pixel, perform linear fitting, detect deviations using the rmoutliers function from Matlab, then remove these outliers. This step is repeated until the fitted line no longer changes, and the optimal laser line is obtained. The above steps are repeated to obtain the laser plane equation through multiple non−parallel optimal lines.

We provide the visualization results of these four steps and a comprehensive flowchart of our proposed algorithm in Figure 3a and Figure 3b, respectively.

3.2. Multispectral Photometric Stereo

Ideally, only the Lambertian model can perform traditional photometric stereo reconstruction, so we assume a Lambertian reflection model without the influence of ambient light. Given three distant light sources illuminating the surface of an object, the brightness information of the surface is given by the following equation:

$$c=\rho L·N$$

(7)

where c represents the brightness of the surface, and $\rho$ represents the albedo or the reflectivity of the surface of the object to be measured, which is a unique fixed constant for the same object. L is the direction vector of three distant light sources relative to the surface of the object being measured, which is a known quantity, and N is the normal vector of the surface. Therefore, given a set of input images, this equation can be used to reconstruct the normal direction of the object under test.

The multispectral photometric stereo (MPS) algorithm is a variation of the traditional photometric stereo algorithm, which replaces the three white lights in the traditional photometric stereo with far-point light sources of three primary colors: red, green, and blue. Multispectral photometric stereo is essentially a form of photometric stereo with three primary color (R, G, B) light sources. It replaces the time-division multiplexing (multiple single light illumination) used in traditional photometric stereo with frequency-division multiplexing (single multi-light illumination). The difference between PS and MPS is shown in Figure 4. Compared with traditional photometric stereo algorithms, the dynamic and real-time performance of single image reconstruction using multispectral photometric stereo can be used in the reconstruction of video sequences. Assuming that the object conforms to the Lambertian model, the brightness value of any point on the object in the i-th (i = 1, 2, 3, representing red, green, and blue, respectively) channel is expressed as:

$$Ci=V·L·N$$

(8)

In this equation, the vector V represents a combination matrix that simulates surface chromaticity, spectral distribution, camera spectral sensitivity, etc., which will be different for areas with different chromaticities. L is the direction vector of the three far-point light sources relative to the surface of the object to be measured, which is a known quantity. N is the normal vector of the surface. The decomposition of V can be expressed as:

$$V=\int col(\lambda )E(\lambda )S(\lambda )\rho d\lambda$$

(9)

where $col\left(\lambda \right)$ is the chromaticity information of the object’s surface, $E\left(\lambda \right)$ is the distribution information of the light source spectrum, $S\left(\lambda \right)$ represents the sensitivity of the camera to light, and $\rho$ is the albedo of the object’s surface at this point.

Multispectral photometric stereo only uses one image to calculate the surface normal direction when the overall surface chromaticity and albedo of the target object are the same. In Equation (9), $E\left(\lambda \right)$ and $S\left(\lambda \right)$ are determined by the light source and camera, which results in the image uncertainty in the mapping relationship between the grayscale values of the R, G, and B channels and the surface normal direction. In order to obtain a unique mapping value, additional restrictions need to be added. In our paper, we use the superpixel segmentation method, which divides the object’s surface into multiple areas of the same surface color and calculates a unique value of the object’s normal in each area. As this approach requires the initial depth as a prior, we add laser information to the multispectral image to provide it with a prior of the initial depth and a criterion of the true scale.

3.3. Laser Line Separation

In the process of using images with laser lines and multispectral features for 3D reconstruction, we found that some primary light sources contain the same or similar spectra as the laser lines in the image, which makes it difficult to extract laser lines through region-based image segmentation and morphological operations. In addition, the area illuminated by the laser line on the object under test will only display the color of the laser line, losing the pixel information of the corresponding color and brightness of the multispectral light source in this area. When performing direct (uncorrected) 3D reconstruction with a three-channel separation, it will cause segmentation phenomena (Figure 5), where the area marked by a red ellipse shows the unevenness of the texture or pattern caused by the reconstruction under the influence of the laser line. The mutual influence between laser and multispectral light sources greatly reduces the reconstruction performance, which puts forward another serious challenge to consider. In order to complete the reconstruction of multispectral photometric stereo, it is necessary to remove the influence of laser lines on multispectral images while retaining the laser line information.

We propose a method for restoring keyframe images through image sequences, and the implementation process is as follows (Figure 6):

• Firstly, we select an image containing the laser line and the object to be measured, and then crop the original image according to the object’s size so that the image only contains the laser line and the object to be measured. Assuming that, in a given frame, the laser line is partially bent, then the frame is used as a keyframe to perform the three-channel separation. Threshold segmentation is performed on each channel, and their intersection is exploited to find the area where the pixel grayscale value is distorted.
• Next, we pick the adjacent image pairs ($j-1$, $j+1$) of the previously selected keyframe image j. After separating these two frames into three channels, perform threshold segmentation to determine the areas where pixel grayscale values are distorted. We perform intersection processing on the distorted areas of images $j-1$ and $j+1$ to determine if there is an intersection. If there is an intersection, pick the $j-2$ frame and the $j+2$ frame to form a pair of images with larger intervals ($j-2$, $j+2$). After separating the three channels, determine whether there is an intersection in the distorted area in this new pair. If there is an intersection once again, this process of searching for increasing large pairs from frame j is repeated until no intersection is found.
• Finally, the pixel values of the corresponding area (distortion area of the keyframe j) before image binarization are overlapped and evenly divided, then replaced with the distorted area of the keyframe j that has not been binarized, making it a new keyframe without a distortion area. The restored keyframe image is directly subtracted from the original keyframe image j in terms of pixel values to obtain the laser line image. The above steps are repeated until the laser line disappears from the object to be measured.

3.4. Mobile Reconstruction and Data Fusion

COLMAP is a highly integrated Structure from Motion (SfM) and Multi-View Stereo (MVS) pipeline tool that can restore the geometric structure (point cloud) of a 3D scene and the camera pose corresponding to each image from an unordered or ordered collection of 2D photos [48]. We utilize the characteristic of COLMAP to process the video sequence; we collect and record the camera pose corresponding to each keyframe during data collection. We separated the keyframes into multispectral images without laser lines and pure laser images, and combined them with the photometric stereo algorithm based on laser line correction proposed by Fan [49] for 3D reconstruction. After completing the 3D reconstruction of all keyframes, we use the camera pose to form the point cloud. Due to the continuous nature of the video sequence, although there are intervals in the selection of keyframes, the intervals between the selected key frames will inevitably lead to multiple values at the same position on the object’s surface during the reconstruction process. To address this issue, we propose a weighted data fusion method:

$$M=d_1^2+d_2^2+\cdots +d_i^2$$

(10)

where $d_1$ is the distance between the point with multiple values and the laser line of the first image containing this point. $d_2$ is the distance between the point and the laser line of the second image containing this point. $d_i$ is the distance between the point and the laser line of the i-th image containing this point. It can be summarized as:

$$D=[D_1(1-{\displaystyle \frac{d_1^2}{M}})+D_2(1-{\displaystyle \frac{d_2^2}{M}})+\cdots +D_i(1-{\displaystyle \frac{d_i^2}{M}})]/(i-1)$$

(11)

where D represents the final value of the point. $D_1$ represents the value of the point in the first image, and $(1-{\displaystyle \frac{d_1^2}{M}})$ represents the weight of the point in the first image value. $D_i$ represents the value of the point in the i-th image, and $(1-{\displaystyle \frac{d_i^2}{M}})$ represents the weight of the point in the i-th image value.

We apply a weight based on how close a point on the surface is to the location of the laser line in each image. Since the deviation in the reconstruction of the laser line position is the smallest, the closer the reconstructed data to the laser line are, the more accurate they are and the greater the weight.

4. Experiment and Comparison

4.1. Experimental Setup

In our self-made mobile equipment, the camera model is IDS UI-5240CP-C-HQ (IDS Imaging Development Systems GmbH, Obersulm, Germany), the light source model is Aozipudun AZ-SDD03-01 (Zhongshan Minfeng Lighting Co., Ltd., Zhongshan, China), and the linear laser model is Desheng DS2090 (Changshu Desheng Optoelectronics Co., Ltd., Changshu, China). We also built a pool for underwater experiments and laid sediment on the bottom of the pool to simulate the underwater environment under real conditions. The self-made equipment and experimental water tank are shown in Figure 7.

4.2. Comparison of Laser Plane Calibration Effects

The key to laser plane calibration lies in the extraction of laser lines, and the effectiveness of laser line extraction algorithms is commonly measured in terms of accuracy and efficiency. Due to the unknown true position of the laser center point, this paper proposes to measure standard blocks (the standard block is a calibration gauge block used for calibrating Vernier calipers and digital calipers). After using laser triangulation to measure standard blocks of different sizes underwater at close range, the accuracy is evaluated in terms of the mean absolute error and root mean square error between the measured height and the true height. We calculated the efficiency of different methods by extracting the time used for laser lines, and conducted efficiency experiments on an Intel (R) Core (TM) i3-7100H CPU, using a computer with a frequency of 3.90 GHz and 16 GB RAM, and using MATLAB R1016a as the software platform. The results of different methods are shown in Figure 8 and

Table 1. Comparison of root mean square error (RMSE) and mean absolute error (MAE) of six methods (the result unit is mm).

Methods	Grayscale Centroid		Steger		Improved Thinning [14]		Unilateral Tracing [15]		LBDM [16]		Ours
Error	RSME	MAE	RSME	MAE	RSME	MAE	RSME	MAE	RSME	MAE	RSME	MAE
35 × 9 × 70	0.528	0.301	0.472	0.208	0.220	0.298	0.220	0.168	0.177	0.146	0.105	0.139
35 × 9 × 60	0.486	0.304	0.444	0.218	0.483	0.301	0.210	0.170	0.166	0.153	0.100	0.140
35 × 9 × 50	0.446	0.310	0.408	0.290	0.449	0.300	0.200	0.185	0.153	0.149	0.095	0.145
35 × 9 × 40	0.402	0.305	0.375	0.286	0.413	0.296	0.194	0.188	0.140	0.146	0.090	0.142
35 × 9 × 30	0.362	0.308	0.338	0.288	0.380	0.296	0.188	0.186	0.126	0.143	0.086	0.144

.

We compare our proposed method with two traditional methods and three newly improved methods. As can be seen from Figure 8, the grayscale centroid method still has laser scattering points outside the break point; the Steger method shows laser point clustering at the break point; the improved thinning method exhibits misalignment at one end of the break point; the unilateral tracing method presents drift at both ends of the break point; and the LBDM method displays incorrect connections in the laser line. These methods result in significant errors in laser line extraction. Our method remains unaffected at the edges and can obtain more accurate results through the relative distance between two straight lines. It can be seen from Table 1 that the MAE result of our method is at least 53% lower than the traditional grayscale centroid method, 33% lower than the Steger method, 51% lower than the improved thinning method [14], and 17% lower than the unilateral tracing method [15]. Our method is at least 2.76% lower than the LBDM method [16] in other size standard blocks, but 0.69% higher in the 30 mm standard block. The best result is displayed in bold.

In order to verify the efficiency of the algorithm, we compared the time required for each algorithm to extract laser lines from from a single image, and the results are shown in

Table 2. Comparison of running time of six methods.

Methods	Grayscale Centroid	Steger	Improved Thinning [14]	Unilateral Tracing [15]	LBDM [16]	Ours
time (s)	0.689	6.459	3.972	0.167	5.220	0.175

.

The running time of our proposed algorithm is about 4 times faster than the grayscale centroid method, 37 times faster than the Steger method, 23 times faster than the improved thinning method, and 30 times faster than the LBDM method. Although the running time of our proposed algorithm is not the shortest, it is almost the same as the running time of the unilateral tracking method. Our method yielded a fair combination of accuracy and efficiency; the difference in running time with the unilateral tracking method is significantly small. The best result is displayed in bold.

4.3. Comparison of Laser Line Separation Effects

We selected two real objects to verify the feasibility of our proposed method. In order to obtain the ground truth for the multispectral images of the two objects in their real environment, we need to take three images separately with the device stationary. A multispectral image with the laser (input image), a multispectral image without the laser (ground truth), and a laser image (ground truth).

The results of processing images on two real objects, a shell and an airplane model, are shown in Figure 9. The lower left corner of each image is labeled. Among them, a is a multispectral image (collected image) with laser lines; b is a multispectral image without laser lines (collected image), which is also the ground truth (GT) corresponding to the multispectral image a; c is the multispectral image (predicted image) obtained by our proposed method; d is the laser line image (collected image), which is also the ground truth (GT) corresponding to the laser line we aim to obtain; e is the laser image (predicted image) obtained after taking the difference between the input image a and the output image c; and b-c is the difference map between the predicted multispectral image and its corresponding ground truth. The black area indicates no difference, and the brighter the difference, the greater the error; d-e is the difference between the predicted image of the laser line image and the ground truth. The black area means there is no difference, and the whiter the area, the greater the error.

After processing the difference between the two real objects, it can be concluded that there is almost no difference between the predicted image obtained by our method and its corresponding ground truth. Especially for the multispectral images of the shell, the processing effect of laser lines is very obvious. The difference b-c in the multispectral images is almost black everywhere, making it difficult to distinguish the position and color of the colored areas. Moreover, the difference in laser lines d-e only shows some pixel shifts. Although the estimation performance of the laser line in the multispectral image of the airplane object is not as obvious as that of the shell, the difference b-c in its multispectral image is not significant either, only a few vertical green lines appear, and the difference d-e in the laser line only exists in the outer contour of the laser line (the central area is black), indicating that the predicted laser line and its ground truth only differ in thickness, where the central area overlaps. Therefore, our proposed method is feasible and can efficiently exploit a multispectral image with laser lines to a multispectral image without laser lines and an accurate laser line.

To verify the accuracy of our method, we compared the processing results of existing commonly used algorithms with ours. The traditional algorithm for filling holes through waveform information proposed by Criminisi et al. [17] is expressed as Cr. The tensor core norm minimization algorithm proposed by Lu et al. [18] is expressed as Lu. The iterative correction algorithm of the feedback mechanism proposed by Zeng et al. [19] is expressed as Ze. The effect comparison is shown in Figure 10.

Similarly to Figure 9, the label is set in the lower right corner of the predicted image results, where a is a multispectral image with laser lines, b is a multispectral image without laser lines (GT), and c is the multispectral image predicted by our algorithm. Cr is the effect image of filling holes with redundant information, Lu is the effect image obtained by minimizing the tensor kernel norm, and Ze is the effect image of feedback iterative repair. The default image resolution is $256\times 256$ pixels.

Table 3. Comparison of three commonly used algorithms and their accuracy results with our algorithm.

Methods	MAE ($\times {10}^2$) (Pixels)		RMSE (Pixels)
	Shell	Airplane Model	Shell	Airplane Model
Cr	0.0752	0.9937	0.0917	1.2124
Lu	0.0776	1.2463	0.0970	1.5199
Ze	0.0760	0.9733	0.0911	1.1872
c (Ours)	0.0491	0.7031	0.0608	0.9006

shows the mean absolute error (MAE) and root mean square error (RMSE) of the multispectral images restored by different methods for the shell and airplane model, compared to those of our proposed method, where the MAE results are multiplied by ${10}^2$ so that both the MAE and RMSE results are expressed with four decimal digits for better interpretation. Our method surpasses the compared methods in terms of MAE and RMSE on both the shell and airplane model, where the best prediction result is obtained on the shell. The best result is displayed in bold.

4.4. Comparison of 3D Reconstruction Performance

We conducted reconstruction and verification of the carved stone slab and shell collection plate in water, and analyzed their reconstruction results.

Traditional underwater photometric stereo cannot perform mobile reconstruction, so it is impossible to reconstruct the side information of the stone slab and may result in voids, as shown in Figure 11 and Figure 12. The angle of light source illumination and the obstruction of surface undulations of objects are inherent problems in static photometric solids. The advantage of our proposed method is that it can dynamically scan and solve the problems of side reconstruction and reconstruction with holes.

We conducted reconstruction verification on sea snails and shells in water, and analyzed their reconstruction results. The results obtained using the high-precision 3D reconstruction instrument (EinScan Pro, Shining 3D Tech Co., Ltd., Hangzhou, China) are represented as GT. EinScan Pro achieves high reconstruction accuracy, with a maximum resolution of up to 0.02 mm, making it suitable for precision part inspection and complex surface modeling. However, it may struggle to fully capture small holes, sharp edges, or deep groove structures, and its implementation involves relatively high costs. The reconstruction method using slow translation to simulate conveyor belt translation [50] is represented as Liu. By rotating around the camera to simulate multi-angle illumination of the event camera, multiple illumination angles will be obtained, and the reconstruction method with added laser lines [8] is represented as Yu. The method we propose is represented as Ours.

We compared the reconstruction effects of three methods including ours for shells and conches separately, and the reconstruction results are produced in PLY (Polygon) format, as shown in Figure 13. From left to right are Shell 1, Shell 2, Conch 1, Conch 2, and Conch 3. From top to bottom are GT, Liu, Yu and Ours.

From

Table 4. Comparison of mean absolute error (MAE) among methods Liu, Yu and ours.

Methods	Shell1	Shell2	Conch1	Conch2	Conch3
Liu	0.78 mm	0.68 mm	3.44 mm	5.12 mm	4.81 mm
Yu	0.37 mm	0.32 mm	1.86 mm	2.54 mm	2.00 mm
Ours	0.34 mm	0.29 mm	1.72 mm	2.06 mm	1.90 mm

, it can be seen that our method achieved the best results on all objects. Among the shells, our methods achieved the smallest mean absolute error (MAE) of 0.29 mm on Shell 2. Our method achieved the smallest mean absolute error (MAE) of 1.72 mm. The best result is displayed in bold.

5. Conclusions

This paper proposed a 3D reconstruction method for underwater objects that combines laser triangulation and mobile multispectral photometric stereo (MPS). This method uses a mobile MPS device to achieve dynamic reconstruction of the target object. The significance of this paper lies in the fact that our proposed method not only solves the problems of lack of scale, occlusion of shadows, and difficulty in dynamic measurement in photometric stereo in an underwater photometric stereo environment, but also the problem of sparse point clouds in an underwater laser environment. We adopted a weighted data fusion method to optimize the 3D data obtained from keyframes, and proposed an optimization method for laser plane calibration and laser line separation, further improving the reconstruction performance of laser triangulation and multispectral photometric stereo. The final experimental results indicate that our method has achieved remarkable underwater reconstruction results.

The dynamic 3D MPS reconstruction technology proposed in this paper is ultimately aimed at 3D reconstruction of objects in complex underwater environments. Specifically, based on multispectral photometric stereo, this paper utilizes a traditional laser triangulation approach to achieve high-precision 3D reconstruction of motion. Obviously, in addition to these existing hardware and methods, there will definitely be some new hardware and methods in the future to adapt to more complex reconstruction environments, such as recent inertial measurement devices and event cameras [51,52], which can further promote the development of dynamic underwater 3D reconstruction technology.