Research on Calibration Method of Line-Structured Light Based on Multiple Geometric Constraints

Abstract

Line-structured light calibration is an important step in the process of the visual measurement of structured light, and in the calibration process, the light plane equation is fitted by the camera coordinates of the data points on the light plane. There is error in solving the spatial coordinates of the data points, and the existing methods generally cannot effectively find the deviation points in the fitted data points, which leads to deviation when solving the parameters of the light plane equation. To correct the error caused by the deviation point in the calibration, this paper established a calibration method based on geometric constraints. In the method, the first constraint was that all fit points involved in the fitting had a certain consistency, and a part of the error points could be filtered by the distance from the data point to the light plane. In order to further improve the calibration accuracy, the laser was moved on a linear guide, the optimized objective function was established by the second geometric constraint, and the constraint was the relationship that the light plane at each position was parallel. In the experiment, a high-precision gauge block was used as the measured object, and the measurement results could be obtained. The calibration method can improve the calibration accuracy according to the experimental results.

1. Introduction

Structured light vision measurement is an active vision measurement technology, and the profile information of the measured object can be obtained by the characteristic light beam projected onto the surface. Compared with binocular vision technology, structured light vision measurement technology [1,2,3,4] is often used for non-contact measurement in complex environments with better robustness.

Visual measurement based on linear structured light involves projecting a plane beam onto the surface of the measured object, and the measurement results can be obtained by the coordinate of the characteristic point on the transversal line between the plane beam and the surface of the measured object. In order to solve the three-dimensional coordinates of the characteristic points, the space plane parameters corresponding to the light plane need to be calculated. This process is known as structured light calibration, which establishes the spatial connection between the light plane and the camera. In the active vision measurement system based on structured light, camera parameters from the camera calibration [5,6,7,8] and light plane parameters determine the spatial location of feature points on the measured object. Therefore, structured light calibration has been widely studied.

There are various calibration methods for different types of line-structured light projection systems. For multi-line structured light projection, Lu [9] proposed a calibration method based on ray tracing, which simplified the calibration process by tracing horizontal light rays. Yang [10] proposed a calibration method using a one-way multi-line displacement profile measurement method that established a lookup table by correcting polynomial mapping coefficients to avoid lens distortion. This method also had the advantage of flexible operation. The hardware cost of multi-line structured light vision is high, and it is easily affected by the external lighting environment. In order to solve the above problems, structured light vision technology based on laser projection has been extensively studied.

In the early days, light plane equations were obtained by the invariant cross ratio method. Huynh [11] proposed a calibration method using two planes perpendicular to each other. Calibration parameters were computed according to three non-collinear reference points in the target. In this method, the calibration accuracy was limited by the number of data points. Wei [12] used double cross ratio invariance to solve visual system parameters. Compared with the above method, the solution accuracy of the parameters can be improved by having enough data points. Both methods have the disadvantage that it is difficult to manufacture the target.

In order to reduce the manufacturing difficulty and cost of the target, Zhou [13,14] made a plane target with the co-plane of the calibration plate plane and the target plane. In this method, the coordinates of the points were obtained by the parameters of the target, and the spatial feature points were obtained by moving the plane target several times to solve the optical plane equation. This method has simple operation, low hardware cost, and a high practical value. Zhou [15] designed a special plane target containing several calibration areas and obtained the parameters of the light plane equation of different regions. This method reduced the influence of lens distortion on calibration accuracy. However, the production of the calibration object and the algorithm were complicated. Liu [16] used a ball to calculate the light plane parameters. Due to the geometric characteristics of the spherical calibration volume, image acquisition was independent of the placement position of the calibration volume. It was easy to obtain a better-quality target, and the calibration accuracy was improved. However, the manufacturing cost of the spherical calibration target in this method was much higher than that of the plane calibration board. Although the number of data points met the calibration requirements in the above method, the error points could not be eliminated, and the accuracy of the results of calibration could be affected. Li [17] proposed a method for multi-line structured light calibration. This method used the geometric relationship of multiple structured line lights to screen the data points, and the plane parameter was obtained by the filtered data points. However, this method requires a high manufacturing precision of the laser, and the application range of multi-line structured light is relatively small. Bleier [18] proposed a self-calibrating line laser scanning system for underwater environments, which could recover depth information using geometric constraints, such as coplanarity. This method is affordable and can be performed using a single camera and a freely movable cross-line laser projector. Chen [19] used a single circle as the calibration body for the calibration plane. This method could solve the camera parameters, distortion coefficients, and parameters of the light plane equation in a single unit, significantly reducing the calibration cost of the line-structured light system and simplifying the calibration process. However, this method did not optimize the process of solving the parameters of the light plane equation, and there were errors in the data points used to calculate the light plane. The calibration accuracy obtained by this method was 0.0451 mm. Li [20] used a cross-line structured light method to assist in the calibration process of line-structured light, optimizing the equation of the light plane using the perpendicular relationship. However, this method requires the use of a pair of perpendicular light planes to optimize the line-structured light, and the vertical relationship between the two light planes needs to be ensured by the laser’s manufacturing accuracy, which increases the hardware cost of the line-structured light measurement.

Although line-structured light visual measurement technology is a measurement algorithm with good noise immunity, the calibration of line-structured is the key limitation of measurement accuracy. This is because the light plane equation needed to be obtained through the center points of the light stripe on the image. The coordinates of the center points could be obtained by the center point detection algorithm of the light stripe and the camera calibration algorithm. In the process of detecting the center point of the light stripe, the detection algorithm could be influenced by the grayscale of the image and the quality of the light stripe, and the center points of the light stripe also had errors. Due to the influence of the algorithm and the calibration environment, the camera’s internal parameters and the distortion coefficients obtained by the calibration also had errors. The above two factors may cause errors in the solution of the spatial coordinates of the center point of the light bar.

Many existing line-structured light calibration algorithms mainly simplify the calibration process and reduce the calibration cost of line-structured light by using different calibration bodies. However, they do not solve the error problem at the center point of the light stripe, which is a key factor affecting calibration accuracy. In this paper, two geometrical constraints were used to optimize the line-structured light calibration process. The error points in the center point of the light stripe were filtered on the basis of the distance between the center point and the light plane. The method of solving the light plane was further optimized by moving the light plane to obtain multiple parallel light planes. The sum of the algebraic error of the light plane equation was used as the objective function to optimize the parameters of the light plane.

In this paper, Section 2 presents a muti-geometrical linear structure light calibration model and includes the solution of the initial value of the plane equation coefficients, the selection of error data points, and the optimization of the light plane equation parameters through multiple parallel relationships. Section 3 uses experiments to verify the calibration of the algorithm and discusses the influence of the number of parallel constraints on the calibration accuracy. Section 4 presents the conclusion of the paper.

2. The Light Calibration Method for Line Structure based on Multiple Geometric Constraints

On the basis of the original calibration model, a light plane calibration method based on multiple geometric constraints is proposed in the paper, and the constraints included the subordination relationship of the point and surface and the parallel relationship between space planes. First, the initial values of the equation parameters were solved by the calibration model of the light plane. Second, the data points were filtered by the characteristics that the data points should simultaneously be on the light plane and the plane target. Finally, several parallel light planes were obtained by moving the laser line, and the parameters of the plane equation were optimized using the parallel relationship.

2.1. Preliminary Solution of Light Plane Equation Parameters

In this paper, the calibration plane target is shown in Figure 1; a checkerboard calibration plate and a black target plane were clamped by the fixture. The limit plane was used to ensure the coplanar relationship between the checkerboard calibration plate and the black target. The camera’s internal parameters, distortion coefficients, and external parameters of the checkerboard surface could be calibrated under different poses by Zhang’s camera method [6]. According to the characteristics of the target, the parameters of the checkerboard surface were the same as the external parameters of the target plane.

When the positions of the camera and the line laser were fixed, the light was projected onto the target plane by the laser, the positions of the light stripe center points in the pixel plane were obtained using Steger’s algorithm [21], and the camera coordinates of these points were obtained by the camera’s internal parameters. The initial value solution model for laser plane equation parameters is shown in Figure 2; P is the data point used for the calibration of the light plane, and the point was obtained by the center point detection algorithm of the light stripe. P′ is a projection of a point on the imaging plane, and l is the intersection of the laser plane and the target. During the calibration process, images of the calibration target in different positions were obtained, and the spatial coordinates of the data points in each position were obtained using the external parameters of the target surface. The spatial coordinates of the i-th data point on the j-th image were set as $P_{Ci}^j=\left(X_{Ci}^j,Y_{Ci}^j,Z_{Ci}^j\right)$.

O_W-X_WY_WZ_W is the world coordinate system, and O_C-X_CY_CZ_C is the camera coordinate system.

O-xy is the image coordinate system, and Op-uv is the pixel coordinate system.

The plane equation [22] of the light plane is:

$$A{X}_C+B{Y}_C+C{Z}_C-1000=0$$

(1)

The camera coordinates of the data points were replaced by the plane equation, and the new equation is shown in Equation (2):

$$\left[\begin{array}{ccc}{X}_{C1}^1& Y_{C1}^1& Z_{C1}^1\\ ⋮& ⋮& ⋮\\ X_{Ci}^j& Y_{Ci}^j& Z_{Ci}^j\\ ⋮& ⋮& ⋮\\ X_{CK}^N& Y_{CK}^N& Z_{CK}^N\end{array}\right]\left[\begin{array}{c}A\\ B\\ C\end{array}\right]=\left[\begin{array}{c}1000\\ ⋮\\ 1000\\ ⋮\\ 1000\end{array}\right]$$

(2)

In Equation (2), the two variables N and K are the target rotation times and the number of data points at each position. According to Equation (2), the initial parameters can be obtained by the least square method. According to mathematical knowledge, the value of the constant term in the light plane equation does not affect the position of the light plane. In the experiment described in this paper, the parameters of the light plane equation were very small when the constant of the light plane equation was −1. In this case, the light plane equation was inconvenient for expression, and the constant of the light plane equation was set to −1000 in the paper.

2.2. Screening of Error Points According to Calibration Point Dependency

According to the center point detection algorithm of the light strip, the pixels of the points were obtained from the grayscale distribution of the stripe in the image; these center points may not be on the same light plane because they are affected by the external environment and detection algorithm [16]. Line-structured light calibration is the process of obtaining model parameters using a statistical sample. During calibration, the sample should have enough data points, and the data points involved in the calculation need to conform to the statistical model. That is, all data points should be on the calibration plane. Since the coordinates of the data points are obtained by the external parameters of the calibration board, the geometric constraint is not used in the calculation process, and the center points are on the light plane. In the calibration process, the least square method cannot effectively eliminate error points. The geometric relationship of each point on the plane can be transformed into the distance between the point and the space plane as 0, and the distance [22] is shown in Equation (3):

$$d_i^j=\frac{\left|A{X}_{Ci}^j+B{Y}_{Ci}^j+C{Z}_{Ci}^j-1000\right|}{\sqrt{A_{}^2+B_{}^2+C_{}^2}}$$

(3)

where $d_i^j$ is the distance from the i-th data point on the j-th image to the light plane, and it sets the array of distances from all data points to the light plane to E.

According to the error model of data points, the discriminant parameter is set δ₁. When $d_i^j\ge {\delta }_1$, the data points are identified as error points and eliminated. Since the coordinates of the strip center have random error, the data characteristics of array E conform to the normal distribution, and the error points in data points involved in the light plane calibration can be eliminated by the 3σ rule of normal distribution [23]. The variance of array E can be obtained by Equation (4), and the value of the discriminant parameter δ₁ is shown in Equation (5), where μ is the mean of the array E:

$${\sigma }^2=\frac{1}{n-1}{\displaystyle \sum _{i=1}^n{(d_i^j-\overline{d})}^2=}\frac{1}{n-1}[{\displaystyle \sum _{i=1}^n{d}_i^j{}^2-\frac{1}{n}{({\displaystyle \sum _{i=1}^n{d}_i^j})}^2}]$$

(4)

$${\delta }_1=\mu +3\sigma$$

(5)

2.3. The Optimization of Light Plane Equation Parameters with Multiple Parallel Geometric Constraints

To further improve the calibration accuracy, the objective function was created by the multi-parallel relationship. Two parallel constraints are used as examples to introduce the light plane method proposed in this paper, and the relationship between the number of parallel constraints and calibration accuracy is discussed in the experimental part.

The calibration model with multiple geometric constraints is shown in Figure 3. During the calibration process, the camera position does not change, and the laser line moves linearly along the slide rail. In Figure 3, π₁ and π₂ correspond to the initial position and the second position, respectively. The space equations of the π₁, π₂ plane are:

$$A_1{X}_C+B_1{Y}_C+C_1{Z}_C-1000=0$$

(6)

$$A_2{X}_C+B_2{Y}_C+C_2{Z}_C-1000=0$$

(7)

According to the calibration optimization model, the light planes π₁ and π₂ are a pair of parallel planes. Therefore, an objective function is shown in Equations (8) and (9).

$$F(A,B,C,D_1,D_2)=\min {\displaystyle \sum _{i=1}^N(F_1+F_2)}$$

(8)

$$\left\{\begin{array}{c}{F}_1(A,B,C,D_1)=\min {\displaystyle \sum _{i=1}^N\left|A{X}_{Ci}^1+B{Y}_{Ci}^1+C{Z}_{Ci}^1+D_1\right|}\\ F_2(A,B,C,D_2)=\min {\displaystyle \sum _{i=1}^N\left|A{X}_{Ci}^2+B{Y}_{Ci}^2+C{Z}_{Ci}^2+D_2\right|}\end{array}\right.$$

(9)

In the optimization objective function, the optimization parameters are the direction parameters A, B, and C of the parallel light planes and the constant terms D₁ and D₂ of the two plane equations. The initial parameters are the mean values of the corresponding parameters obtained by the least multiplication method at each position. The flow chart of the calibration method proposed in this paper based on the steps above is shown in Figure 4.

The process of the optimization method with multiple parallel constraints is basically the same as the process of the optimization method with two parallel relationships. To improve the calibration accuracy, multiple parallel constraints can be obtained by moving the laser line multiple times. However, increasing the number of parallel geometric constraints leads to an increase in the running time of the algorithm, and locally optimal solutions can be achieved. The paper discusses the number of parallel geometric constraints through experimental methods and determines the number of constraints.

3. The Evaluation Experiment and Analysis

In the experiment, the line-structured light calibration method with multiple geometric constraints was evaluated, and the number of parallel constraints is discussed. The thickness of the gauge block was used as an index to evaluate the calibration accuracy. The images and the stripe center point detection method used in the measurement process of the gauge block were completely the same; only the light plane parameters obtained by different calibration methods were different. Therefore, it is reasonable to evaluate the calibration method by measuring the accuracy of the gauge block. The schematic diagram of the experimental model is shown in Figure 5, and the equipment parameters are shown in

Table 1. Main parameters of equipment.

Equipment Name	Model	Main Parameters
Camera	MER-125-30UM	Resolution: 1292 × 964 pixel
Lens	Computer M2514-MP	Focal length: 25 mm
Laser	LH650-80-3	Power: 0~20 mW
Light source	CCS LFL-200	Luminous area: 200 × 180 mm
Checkerboard calibration board	NANO CBC 25mm-2.0	Manufacturing accuracy: ±1.0 μm

.

3.1. Camera Calibration and Initial Parameter Solution

Before calibrating the structure light, the camera’s internal parameters and distortion coefficients were obtained by Zhang’s method [6], and the camera could no longer be moved in the experiment. In the process, nine images of checkerboard calibration plates were used as calibration images, and the checkerboard calibration plates were needed to cover the visual field of the camera. In the camera calibration, matrix A contained the camera’s internal parameters, which are shown in Equation (10). In the distortion coefficients, k₁, k₂, p₁, and p₂ were the radial distortion and tangential distortion, respectively.

$$\mathit{A}=\left[\begin{array}{ccc}\alpha & \gamma & u_0\\ 0& \beta & v_0\\ 0& 0& 1\end{array}\right]$$

(10)

The camera calibration results are shown in

Table 2. Camera internal parameters and distortion coefficients.

α	β	γ	u0	v0	k1	k2	p1	p2
6884.70	6885.044	0.835078	628.1419	470.1564	0.146356	1.580898	0.000322	0.000299

. The spatial positions of the calibration board in the experiment are shown in Figure 6, and the calibration board images used to calibrate the internal parameters and distortion coefficients are shown in Attached Figure A1.

In structured light calibration, the clamp was used to ensure that the calibration plate surface and target were coplanar. The laser was fixed on the straight slide, and the projected light plane intersected the plane targets. The coordinates of the light stripe center were calculated, and the pixel coordinates of the center were achieved by the detection algorithm [21]. The images of the plane targets are shown in Figure 7, and the red points were the light stripe center.

When the laser was in the initial position, six plane target images with different postures were collected, and the coordinates of the center point were obtained; the center points of the light stripe in the images are shown in Figure 8. The equation of the plane was obtained by the least square method, and the data points were all light stripe centers, which was named the basic light plane calibration model in the experiment. The equation was calculated on the basis of the basic light plane calibration model at the initial position, and the equation is shown:

$$3.3185X_C-0.4073Y_C+1.6432Z_C-1000=0$$

(11)

The laser was moved along the linear guide, and the light plane equations were obtained using the basic calibration model; the plane equations at each position are shown in

Table 3. Calibration results at ten positions.

Corresponding Position of the Laser	Light Plane Equation
0	$3.3187X_C-0.40810Y_C+1.6432Z_C-1000=0$
1	$3.2571X_C-0.43128Y_C+1.6545Z_C-1000=0$
2	$3.0877X_C-0.4158Y_C+1.6719Z_C-1000=0$
3	$3.2051X_C-0.4305Y_C+1.6535Z_C-1000=0$
4	$3.0016X_C-0.4111Y_C+1.6638Z_C-1000=0$
5	$3.0221X_C-0.4156Y_C+1.6539Z_C-1000=0$
6	$3.3346X_C-0.4220Y_C+1.6316Z_C-1000=0$
7	$3.1252X_C-0.4215Y_C+1.6445Z_C-1000=0$
8	$3.1449X_C-0.4024Y_C+1.6326Z_C-1000=0$
9	$3.3445X_C-0.4135Y_C+1.6936Z_C-1000=0$

.

3.2. Evaluation Experiment of Calibration Method with Multiple Geometric Constraints and Result Analysis

The experiment is shown in Figure 9a. The gauge blocks of different thicknesses were placed on the plane target at the initial position of the laser. The laser was projected onto each surface of the gauge block; the pixel positions of the strip center point, which were on the surface of the gauge block, were obtained by the detection algorithm [21].

The coordinates of the light stripe center on the surface of the standard gauge block were solved by the plane equation parameters, and the thickness of the gauge block was the spatial distance from the center to the target plane. After increasing the number of parallel constraints each time, the parameters of the light equation were optimized, and the thickness of the gauge block was recalculated using the new light plane equation after each optimization. The error between the measured thickness and the actual thickness of each gauge block was used to characterize the accuracy of the light plane calibration. In the experiment, the laser was moved nine times, and six kinds of gauge blocks with different thicknesses were selected as the measured objects. The light plane equations corresponding to the initial position after each optimization are shown in

Table 4. Light plane equations after each optimization.

Corresponding Position of the Laser	Light Plane Equation
0	$3.3187X_C-0.40810Y_C+1.6432Z_C-1000=0$
1	$3.3173X_C-0.4089Y_C+1.6389Z_C-1000.0005=0$
2	$3.3194X_C-0.4072Y_C+1.6412Z_C-999.9993=0$
3	$3.3169X_C-0.4096Y_C+1.6454Z_C-1000.0033=0$
4	$3.3173X_C-0.4077Y_C+1.6394Z_C-999.9996=0$
5	$3.3167X_C-0.4069Y_C+1.6413Z_C-1000.0022=0$
6	$3.3183X_C-0.4104Y_C+1.6391Z_C-1000.0031=0$
7	$3.3188X_C-0.4087Y_C+1.6363Z_C-999.9972=0$
8	$3.3196X_C-0.4076Y_C+1.6411Z_C-999.9989=0$
9	$3.3183X_C-0.4096Y_C+1.6407Z_C-999.9980=0$

, and the measurement errors of the gauge block are shown in

Table 5. Measurement error of gauge block (mm).

Standard Gauge Block Thickness	0	1	2	3	4	5	6	7	8	9
3	0.043	0.031	0.029	0.028	0.023	0.024	0.027	0.025	0.031	0.029
4	0.040	0.035	0.034	0.030	0.027	0.029	0.029	0.029	0.027	0.029
5	0.034	0.032	0.026	0.025	0.025	0.029	0.027	0.030	0.029	0.024
6	0.042	0.038	0.025	0.029	0.021	0.023	0.030	0.033	0.030	0.029
8	0.039	0.033	0.029	0.021	0.023	0.021	0.029	0.025	0.027	0.025
10	0.046	0.039	0.027	0.029	0.027	0.030	0.032	0.030	0.028	0.028
Mean error	0.041	0.035	0.028	0.027	0.024	0.026	0.029	0.029	0.029	0.027

.

According to the measurement results in Table 5, the measurement accuracy of the gauge block was significantly promoted by the optimized light plane equation. The measurement results showed that the structured light calibration method based on multiple geometric constraints was effective in improving the calibration accuracy.

In order to discuss the relationship between the number of parallel constraints and calibration accuracy, mean errors corresponding to different numbers of parallel constraints were compared, which were the averages of the thickness measurement errors of the gauge block shown in Table 5. The comparison results are shown in Figure 10. According to the comparison results, the calibration accuracy did not change much and was slightly reduced when the number of geometric constraints exceeded five. The reason for this result was that the error points involved in the calculation also increased with the increase in the number of parallel constraints.

In recent research, the methods proposed in references [19,20] are similar to the methods in this paper. The calibration accuracy obtained by reference [19] was 0.0451 mm, which was lower than the calibration accuracy of this paper (the measurement accuracy of the gauge block was 0.024 mm when using four parallel planes in the paper).

The measurement accuracy of the three methods for gauge blocks was compared. Method A was the basic structured light calibration algorithm, which directly substituted all calibration light stripe center points into the light plane equation, and the light plane equation parameters were obtained by least square fitting. Method B performed the measurement with structured crosshair light, and the measurement data were from the literature [20]. Method C was the light plane calibration method proposed in this paper, and the number of parallel planes was four. The measurement results of the three methods are shown in

Table 6. Comparison of calibration accuracy (mm).

Gauge Block Thickness	Method A	Method B	Method C
3	0.043	0.028	0.023
4	0.040	0.020	0.027
5	0.034	0.021	0.025
6	0.042	0.015	0.021
8	0.039	0.025	0.023
10	0.046	0.015	0.027
Mean error	0.041	0.021	0.024

.

As shown in Table 6, the calibration accuracy of the line structure proposed in this paper was close to that of Method B. However, this method required the use of a pair of perpendicular light planes to optimize the line-structured light, and the perpendicular relationship between the two light planes needed to be ensured by the manufacturing accuracy of the laser, which increased the hardware cost of line-structured light measurement. Secondly, this method required projecting a pair of perpendicular line-structured lights onto the object being measured at the same time. However, for some measurement environments, the auxiliary calibrated light plane may affect the measurement process. For example, when the surface of the measured object was a highly reflective mirror-like object, noise caused by stray light could appear in the acquired image, which affected the measurement process.

4. Conclusions

Improving calibration accuracy is an important means to improving the structured light vision measurement model. The paper proposes a line-structured light calibration method with multiple geometric constraints. On the basis of the structured line light vision measurement process, there are errors in the detection of the light stripe center and the solution of the camera coordinate of these points, which cause errors in the data points involved in the calculation. In theory, the data points are on the light plane, and the error points can be eliminated by the geometric constraint. To further optimize the parameters of the light plane equation, the laser was moved along a line, and the laser planes corresponding to each position were parallel to each other. The objective function for the parameter optimization of the light plane equation was established through the multiple parallel relationships and was solved by the nonlinear optimization algorithm. In the experiment, the thickness of the gauge block was obtained from the light plane equation, and the measured thickness of the standard gauge block was used to evaluate the calibration accuracy of the optical plane. The measurement errors of the basic calibration model and the calibration method proposed in this paper were compared, and the number of parallel constraints was discussed. The results of the experiment showed that the measurement error of the method proposed in this paper was significantly smaller than that of the basic measurement model, and when the number of parallel constraints was five, the calibration accuracy was the highest.