^{1}

^{1}

^{2}

^{2}

^{*}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

In this paper, we propose a simultaneous intrinsic and extrinsic parameter identification of a hand-mounted laser-vision sensor (HMLVS). A laser-vision sensor (LVS), consisting of a camera and a laser stripe projector, is used as a sensor component of the robotic measurement system, and it measures the range data with respect to the robot base frame using the robot forward kinematics and the optical triangulation principle. For the optimal estimation of the model parameters, we applied two optimization techniques: a nonlinear least square optimizer and a particle swarm optimizer. Best-fit parameters, including both the intrinsic and extrinsic parameters of the HMLVS, are simultaneously obtained based on the least-squares criterion. From the simulation and experimental results, it is shown that the parameter identification problem considered was characterized by a highly multimodal landscape; thus, the global optimization technique such as a particle swarm optimization can be a promising tool to identify the model parameters for a HMLVS, while the nonlinear least square optimizer often failed to find an optimal solution even when the initial candidate solutions were selected close to the true optimum. The proposed optimization method does not require good initial guesses of the system parameters to converge at a very stable solution and it could be applied to a kinematically dissimilar robot system without loss of generality.

To measure the range data of objects under an unknown working environment, a range sensing device has been widely applied to various robotic applications [

A laser-vision sensor (LVS) consisting of a CCD camera(s) and a laser stripe projector has been frequently used as an active ranging device [

Most approaches to the calibration of a hand-mounted LVS have made use of the multi-stage technique [

Recently, the online re-calibration of a LVS which is mounted on a Cartesian carriage has been proposed to achieve good resolution and to avoid occlusions when the sensor geometry is modified online [

This paper is organized as follows. In Section 2, we address the hand-mounted laser-vision sensor model and analyze its measurement range and resolution. The parameter identification based on a particle swarm optimization is proposed in Section 3, and its performance is compared with conventional non-linear least square optimization in Section 4. Section 5 illustrates experimental results. The conclusion is given in the final section.

_{u}_{u}_{d}_{d}_{i}_{1}. The coordinate (_{u}_{u}_{f}_{f}_{u}_{v}_{u}_{v}

Next, the 3D position of a point is computed through an optical triangulation principle. As shown in _{C}_{f}_{f}_{f}_{f}

The intrinsic parameters of the LVS model include the intrinsic camera parameters, {_{u}_{v}_{u}_{v}

A kinematic model of a robot manipulator can be modeled by using the Denavit-Hartenberg convention. Let
_{x}_{y}_{z}_{ij}_{C} can be represented as a function of the Euler angle _{x}_{y}_{z}

Since the transformation matrix between the robot base frame and the camera frame,
^{B}^{C}

The baseline distance (_{c}_{c}_{c}_{f}_{f}_{f}

The baseline distance and the projection angle are design parameters of the laser-vision sensor, where they should be selected appropriately based on the requirements of target applications. If we design a sensor with a resolution of less than 1 mm/pixel, we could choose the baseline distance of 100 mm and the projection angle is 25 degree. In this case, measurement ranges about _{c}_{c}_{c}

To investigate the effect of the inexactness of the baseline distance and the projection angle on the measurement errors, we carried out simulations. We assumed 1% inexactness of the baseline distance and the projection angle: the baseline distance is set as 100 mm for its real value of 99 mm, and the projection angle is 25 degree for its real value of 22.5 degree. Measurement errors are shown in

As a first step to identify the model parameters, we should define an objective function to be optimized. Let

For a notational convenience, we rewrite

Searching boundary on parameters is set as:
_{i}_{i}

In this work, we estimated a best-fit parameter vector ^{*} by minimizing the summed squared error of _{i}_{i}_{2} is a Euclidean norm of the error vector

We applied two optimization techniques: nonlinear least squares optimization (NLSO) and particle swarm optimization (PSO). In the following paragraphs, we introduce them in brief. NLSO is a popular algorithm that is frequently used to find the minimum of a multivariate function represented as the sum of squares of the nonlinear functions. Among the various NLSO algorithms, we used the Levenberg-Marquardt algorithm (LMA) which is also referred to as the damped Gauss-Newton Method [_{k}_{k}_{+ 1} by using the following equation:
_{k}

This process is repeated until _{k}_{k}_{k}_{k}

The second identification technique considered is a particle swarm optimization (PSO) [_{1} and _{2} represent the weighting factors that pull each particle toward its previous best position and the global best position; _{1} and _{2} are random numbers uniformly distributed on the interval [0,1].

The inertia weight plays an important role to balance the global and local search abilities; a large inertia weight facilitates a global exploration, while a small one tends to facilitate a local exploration. A preferred weighting function, where the inertial weight is linearly decreased as the iteration proceeds, is described as:
_{max} is the initial weight; _{min} is the final weight;

Once the new velocity of each particle is determined by using

In this way, the algorithm could converge toward a global solution of the given problem. The evolution is continued until the fitness value reaches the preset value or the maximum iterations are reached.

Step 1: Generate swarm and initialize particles in the swarm with random positions and velocities.

Step 2: For each particle, evaluate the fitness function.

Step 3: Memorize best solutions and a global best solution in the swarm.

Step 4: For each particle, update position and velocity by using

Step 5: Repeat Step 2 until predefined conditions are satisfied.

In this section, we perform simulations to compare the performance of the two optimization techniques. Since the exact solution is known in the case of a simulation experiment, it is possible for us to compare the algorithms as to how the found solutions are close to the true optimum. At first, the synthetic data was generated as follows: given pre-specified laser-vision sensor model parameters and a certain robot pose, we compute 100 sets of data using

In order to investigate the influence of the initial candidate solutions on the convergence performance, we used a control parameter, _{i}_{0} is a _{y}

Next, we set the control parameters of the two optimization techniques. In PSO, we used a swarm size of 50, a maximum inertia weight of 0.9, a minimum inertia weight of 0.4, a maximum velocity of 0.1, and a maximum iteration of 5,000. As for the LMA, the maximum iterations are set as 300 times the number of model parameters.

Since the applied optimization techniques set the initial guesses of the parameters in a random manner, they may seek out different minima depending on the initial conditions. Therefore, we performed several runs for each algorithm to evaluate the performance: 1,000 runs for NLSO and 10 runs for PSO. In addition to these algorithms, a random search (RS) was used only as a method to compare other algorithms. In the RS, an evaluation starts with parameters selected by using

The parameter identification results are listed in

The average rms of the PSO with

In order to validate and examine the reliability of the obtained model parameters, we calculate the sum of squared error of randomly generated 50 sets of data which is not used in the optimization by using the model parameters with both best fitness and worst fitness of the PSO. As shown in

As input data for the optimization techniques, we need three kinds of information: joint encoder readings, real coordinates of the reference points, and their corresponding image coordinates. The reference point is a corner point of the checker board pattern, which is given as priori information. The procedures for a data acquisition are as follows:

Adjust the robot so that a horizontal line of the laser stripe overlaps the corner points of the square pattern.

Capture an image and then extract the pixel coordinate of the reference point; record it in the memory of the PC.

Record all the joint angles of the manipulator sent from the joint controller.

Repeat (1)–(3) until we obtain the preset number of data sets.

In order to effectively extract a stripe of laser beam, we used a difference image,

Calibration points are obtained through the matching process between a stripe of laser beam and cross-points of contour lines as shown in

As described in the previous section, the considered parameter identification problem has highly nonconvex and multimodal characteristics such that the LMA often failed to find a good solution, even starting at an initial candidate solution which is near the true solution. Besides, the PSO consistently found a solution close to a true optimum regardless of the searching spaces. Therefore, we only considered the PSO for a parameter identification in the following experiments. As control parameters of the PSO, we used a swarm size of 50, a maximum inertia weight of 0.9, a minimum inertia weight 0.4, a maximum velocity of 0.1, and a maximum iteration of 5,000. The nominal values of the parameters are listed in

Even though the initial particles (candidate solutions) are randomly generated from within the pre-defined range, these two curves are converged to a similar fitness value after approximately 3,000 iterations. This shows the fact that the PSO estimates the parameters with a small error. It takes about 30 seconds to execute the 5,000 generations.

To confirm the applicability of the proposed scheme, we carried out experiments measuring four corner points of the top surface of a gauge block whose size is 30 × 60 × 55 mm. A left-bottom of the block is placed at a point (0, −500, 0) mm with respect to the robot base frame. It is different position from the first experiment where the left-top corner of the pattern is placed at the coordinate (700, 90, 0) mm. We start to move the robot from the robot home position for each trial. We adjust the robot pose so as the laser stripe beam overlap the corner points of the top-side of the object. The measurement results are listed in

In this paper we have proposed a new approach to the problem of a hand-mounted laser-vision sensor system calibration based on a particle swarm optimization. The laser-vision sensor, consisting of a camera with a nonlinear radial lens distortion and a laser stripe generator, was used as a sensor module of the robotic measurement system to measure the range data of an object in the robot base coordinate system; and it was modeled based on the forward kinematics and the optical triangulation principle. The intrinsic and extrinsic parameters of the hand-mounted laser-vision sensor were simultaneously identified through minimizing the overall residual errors between the known reference range data and the estimated data. Simulation and experimental results show that the considered parameter identification problem has highly nonconvex and multimodal characteristics, thus, the nonlinear least square optimizer often failed to find an optimal solution, even when the initial guesses of the model parameters were selected close to the true optimum. On the contrary, the proposed scheme based on the particle swarm optimizer consistently found a stable solution without any need for good initial guesses of the model parameters; thus, it could be a promising tool to identify the model parameters for a hand-mounted laser-vision sensor.

This work was supported by Nuclear Research & Development Program of National Research Foundation of Korea (NRF) funded by Ministry of Education, Science & Technology (MEST) Grant Code: 2011-0002247. This work was also partially supported by the Korea government. MKE under the Grant No. I-2010-1-012 of the ETEP. A preliminary version of this manuscript was presented at the International Conference on Convergence and Hybrid Information Technology (ICHIT 2010), 26 August 2010 in Daejeon, Korea and this work represents an extension of that paper in response to an honorable request by a conference committee member for a journal publication after we obtained some new results.

The laser-vision sensor model. The laser-vision sensor consists of a camera and a laser stripe generator. A 3D range data is obtained by using both the camera projection model and the optical triangulation principle. (

Kinematic model of the hand-mounted laser-vision sensor (HMLVS).

The geometric relation between a 3D illuminated laser point and its 2D image projection.

Measurement range and resolution for different baseline distances, where the projection angle is 25 degree.

Measurement range and resolution for different projection angles, where the baseline H is 100 mm.

Measurement errors of the LVS with 1% inexact value of (

Schematic diagram of the PSO-based parameter identification.

Fitness evaluation results of 50 sets of data not used in the parameter identification.

Hand-mounted laser-vision sensor and the planar calibration pattern.

Detection result of the calibration points.

Convergence of the fitness value.

Calibration results.

3D range measurement of a cylindrical object with holes.

Estimation results with 50 sets of simulation data and with a different number of evaluations: 1,000,000 evaluations for RS, 1,000 runs for NLSO, and 10 runs for PSO.

RS | _{best} |
1.25E-2 | 7.48E-2 | 1.79 | 12.8 | 19.1 |

NLSO | _{best} |
2.01E-5 | 1.70E-4 | 7.50E-4 | 5.81E-3 | 2.49E-2 |

_{worst} |
0.13 | 88.5 | 1.26E3 | 7.24E2 | 1.11E3 | |

_{mean} |
1.01E-1 | 6.75 | 4.67E2 | 6.20E2 | 6.00E2 | |

_{stdev} |
3.13E-2 | 20.5 | 48.6 | 4.43E2 | 4.60E2 | |

PSO | _{best} |
1.69E-5 | 7.78E-5 | 3.79E-4 | 1.41E-3 | 2.55E-3 |

_{worst} |
1.50E-4 | 2.72E-4 | 1.10E-3 | 1.04E-2 | 1.96E-2 | |

_{mean} |
9.34E-5 | 1.89E-4 | 8.65E-4 | 5.55E-3 | 1.16E-2 | |

_{stdev} |
9.11E-5 | 1.58E-4 | 6.05E-4 | 5.76E-3 | 1.13E-3 |

Parameter identification results.

8 | 6 | 10 | 7.70 | |

_{u} |
0.0074 | 0.005 | 0.01 | 0.0074 |

_{v} |
0.0074 | 0.005 | 0.01 | 0.0075 |

_{x} |
320 | 300 | 340 | 323.8 |

_{y} |
240 | 220 | 260 | 238.3 |

0 | 0 | 0.1 | 4.8E-4 | |

140 | 120 | 150 | 137.3 | |

5 |
0 | 0.47 | ||

0 | − |
−1.8E-3 | ||

0 | − |
−0.05 | ||

0 | − |
−6.8E-3 | ||

_{x} |
60 | 40 | 80 | 63.2 |

_{y} |
0 | −10 | 10 | −2.3 |

_{z} |
−45 | −60 | −30 | −46.2 |

Measurement results for verification (mm).

left-top | 60 | −500 | 55 | 59.95 | −499.56 | 55.34 | 0.58 | 0.15 |

right-top | 60 | −530 | 55 | 60.17 | −530.01 | 54.65 | 0.39 | 0.14 |

right-bottom | 0 | −530 | 55 | 0.17 | −529.48 | 55.14 | 0.61 | 0.21 |

left-bottom | 0 | −500 | 55 | −0.51 | −499.97 | 55.05 | 0.71 | 0.18 |