Extrinsic Parameter Calibration Method for a Visual/Inertial Integrated System with a Predefined Mechanical Interface

For a visual/inertial integrated system, the calibration of extrinsic parameters plays a crucial role in ensuring accurate navigation and measurement. In this work, a novel extrinsic parameter calibration method is developed based on the geometrical constraints in the object space and is implemented by manual swing. The camera and IMU frames are aligned to the system body frame, which is predefined by the mechanical interface. With a swinging motion, the fixed checkerboard provides constraints for calibrating the extrinsic parameters of the camera, whereas angular velocity and acceleration provides constraints for calibrating the extrinsic parameters of the IMU. We exploit the complementary nature of both the camera and IMU, of which the latter assists in the checkerboard corner detection and correction while the former suppresses the effects of IMU drift. The results of the calibration experiment reveal that the extrinsic parameter accuracy reaches 0.04° for each Euler angle and 0.15 mm for each position vector component (1σ).


Introduction
Visual/inertial integrated systems have been used across many contexts, including indoor [1][2][3], underwater [4], space environments [5], and taking some measurement tasks [6,7]. To achieve accurate navigation and measurement, camera and inertial sensor frames should be aligned to the carrier frame in a process often called extrinsic parameter calibration or alignment. In this process, a coordinate transforming relationship is established between the sensor frame and the system body frame. Without calibrating the extrinsic parameters, the navigation errors will be coupled with the misalignment errors [8].
Previous studies have developed theoretical models of the camera [9] and the inertial measurement unit (IMU) [10][11][12], based on which the extrinsic parameters of visual/inertial integrated systems have been calibrated. These parameters are generally calibrated in three ways. First, these parameters are calibrated based on the corresponding rotation differences. Given that both the camera and IMU can evaluate their own rotations, a calibration method that minimizes the overall matching error of the rotation matrix between the visual and inertial coordinate systems in multiple rotations has been proposed in [13], whereas You et al. [14] developed a calibration method that relies on the angular velocity differences between the IMU and camera in the subsequent rotations. Second, these parameters are calibrated based on the vertical direction constraint. Given that the gravity and vertical line are measured by an IMU and camera in the same direction (i.e., vertical direction), a calibration method 1. In this study, we exploit the complementary nature of IMU and camera to improve the calibration accuracy. The camera measurements can suppress the inertial propagation drift, whereas the motion parameters evaluated by IMU can accurately extract the feature points under a smearing effect. We exploit the complementary nature of these two components to improve the calibration accuracy.

2.
The method for IMU aided checkerboard corner correction under motion blur is introduced, which is the tip step to improve accuracy of camera calibration. We use the IMU to evaluate the motion parameters, and use these motion parameters to eliminate the effects of motion blur in image. It is described in Section 2.3. The extrinsic parameters are evaluated based on EKF, and the rotation angle is evaluated using camera measurement to suppress IMU's drift. 3.
The calibration process can be a standard calibration step for factory production and user operation. The turntable is the standard equipment for IMU calibration, and the checkerboard is the standard device for camera calibration. So, the cost is low, and the calibration process is simple and convenient. Our experiment results indicate that our proposed method is valid and achieves a fair level of accuracy. Our method can also align the camera frame to the IMU frame. The rest of this paper is organized as follows. Section 2 introduces the mathematical model and the proposed calibration method. Section 3 presents the simulation and the real-world experiment. Section 4 concludes the paper.

Calibration Method
The main idea of the proposed calibration method is shown in Figure 1, which can help understand the part of the calibration. There are four step of this calibration. Firstly, "Preparation", the following preparations should be performed before the system calibration: The checkerboard should be fixed in an appropriate location that can be observed by the cameras while the system rotates along with the turntable; The intrinsic parameters of the camera and IMU should be calibrated; The visual/inertial system should be fixed on the turntable.
Secondly, "data acquisition", the turntable is controlled manually to rotate around two orthogonal axes, such as the x and y axes, for the checkerboard image acquisition by camera and the inertial data acquisition by IMU, respectively.
Thirdly, "blur correction", the motion blur exits in the checkerboard image. The checkerboard corners are detected with the aid of IMU to achieve higher accuracy.
Finally, "estimator", we estimate the extrinsic parameters based on EKF.
Sensors 2019, 19, x FOR PEER REVIEW 3 of 18 The main idea of the proposed calibration method is shown in Figure 1, which can help understand the part of the calibration. There are four step of this calibration. Firstly, "Preparation", the following preparations should be performed before the system calibration: The checkerboard should be fixed in an appropriate location that can be observed by the cameras while the system rotates along with the turntable; The intrinsic parameters of the camera and IMU should be calibrated; The visual/inertial system should be fixed on the turntable. Secondly, "data acquisition", the turntable is controlled manually to rotate around two orthogonal axes, such as the x and y axes, for the checkerboard image acquisition by camera and the inertial data acquisition by IMU, respectively.
Thirdly, "blur correction", the motion blur exits in the checkerboard image. The checkerboard corners are detected with the aid of IMU to achieve higher accuracy.
Finally, "estimator", we estimate the extrinsic parameters based on EKF. The main idea of this calibration method is that establish the equations that provide the extrinsic parameters and solve these equations. To better understand the method, the basic measurement models of the IMU and camera are described in the Section 2.1. Because the calibration operation process is swinging motion, the detailed measurement equations of IMU and camera under swinging motion are described in Section 2.2. We exploit the complementary nature of IMU and camera to improve the calibration accuracy, the method of inertial aided checkerboard corner extraction under motion blur is described in Section 2.3. And the extrinsic parameters are evaluated based on EKF,  Figure 1. Extrinsic calibration process.
The main idea of this calibration method is that establish the equations that provide the extrinsic parameters and solve these equations. To better understand the method, the basic measurement models of the IMU and camera are described in the Section 2.1. Because the calibration operation process is swinging motion, the detailed measurement equations of IMU and camera under swinging motion are described in Section 2.2. We exploit the complementary nature of IMU and camera to improve the calibration accuracy, the method of inertial aided checkerboard corner extraction under motion blur is described in Section 2.3. And the extrinsic parameters are evaluated based on EKF, which is shown in Section 2.4. The rotation angle of turntable is evaluated by camera measurement to suppress the IMU's drift.

Measurement Model
The measurement model for IMU is formulated as follows (The superscripts and subscripts F, B, I, and V represent the frame of checkerboard, system body, IMU, and camera, respectively, while F 0 , B 0 , I 0 , and V 0 represent the frame fixed on the earth and coincident to the frame of checkerboard, system body, IMU, and camera in its initial pose, respectively).
where a I true is the true specific force excluding gravity, ω I true is the true angular velocity, g I is the gravity expressed in the IMU frame, b a and b g are the bias errors of the accelerometer and gyro, respectively, and n a and n g are the process white Gaussian noises of the accelerometer and gyro, respectively. Both b a and b g show minimal changes within a short period.
We use the ideal pinhole model as the measurement model for the camera that maps a 3D point where z c is the camera frame optic axis coordinate of point P, α x , and α y are the scale factors of the u and v axes of the image plane, respectively, γ is the non-orthogonal factor of the image plane axes, u 0 , v 0 is the pixel coordinate of the camera principal point, C V F is the 3 × 3 rotation matrix, t V FV is the 3D translation vector, and M is the intrinsic parameter matrix. In the actual situation, the principal point, focal length deviation, distortion, and other error factors should be considered based on the ideal camera model. These parameters can be acquired via an intrinsic calibration, such as by employing Zhang's method [27].

Extrinsic Parameter Calibration Method for a Visual/Inertial Integrated System
The physical quantities that should be calibrated include the following: The translation matrices C V B and C I B , which occur between the system body frame and the camera frame as well as between the system body frame and the IMU frame, respectively.

2.
The position vectors t B BV and t B BI , which are derived from the camera and IMU principal points to the origin of the system body frame, respectively.
Given that IMU can measure angular velocity and acceleration vectors during rotation, the constraint equations of inertial vectors and the extrinsic parameters of IMU can be established. When the turntable rotates around by one axis, the position vector and Euler angle along the rotation axis cannot be computed. However, when the turntable rotates by around two orthogonal axes, extrinsic parameters of IMU can be computed. Fortunately, IMU calibration is generally performed by using a turntable with two or three orthogonal axes.
Camera calibration generally involves the use of a checkerboard. In the calibration process, both the intrinsic parameters and the translation matrix and position vector between the camera and checkerboard frames can be computed. By turning the turntable, the constraint equation can be established based on the relationship between the extrinsic parameters of camera and the change in the camera position and attitude. The constraint equations of the visual/inertial system are described as follows.
In general, when the Coriolis acceleration is ignored during rotation, the accelerometer measurement is where a B n = Ω B 2 t B BI is the radial acceleration, a B τ = E B t B BI is the tangential acceleration, g I is the gravitational acceleration expressed in the IMU frame, and C I B is the translation matrix from the system body frame to the IMU frame, which can be calculated by the Euler angle ψ I B . Ω B and E B are skew-symmetric matrices defined as where ω B = ω x ω y ω z T is the angular velocity vector, and ε B = ε x ε y ε z T is the angular acceleration vector.
Therefore, the accelerometer measurement can be rewritten as where g I 0 is the gravity accelerometer expressed in the {I 0 } frame and can be written as where f I 0 is the accelerometer measurement in the initial pose. Therefore, b a must be initially evaluated to determine g I 0 . The gyro measurement is formulated as An integrated system rotates with a two-axis turntable. When turntable rotates around the x axis, we have where β x is the rotation angle of the turntable when rotating around the x axis. However, when rotating around the y axis, we have where β y is the rotation angle of the turntable when rotating around the y axis. β k , k = x, y, can be computed by integrating the gyro measurement, but an accumulation error is observed. This error is then evaluated based on the camera measurement in the Kalman process. The camera measurement can be formulated as where subscripts x and y indicate that the turntable rotates around the x and y axes, respectively, u i , v i is the pixel coordinate of each checkerboard corner, and n is the number of checkerboard corners. The value of u i , v i can be computed as and P V 0 (11) can be written as where P F i is the 3D coordinate of the ith checkerboard corner in the checkerboard frame. In Equations (8) and (9), C B B 0 can be computed based on the rotational angle of the turntable and correspond to the rotation of the turntable around the x and y axes, respectively.
Meanwhile, P F i is determined by the user, and u i , v i denotes the pixel coordinate of the checkerboard corner in an image. If the number of checkerboard corners is >8, then C V 0 F and t V 0 V 0 F can be evaluated by using least squares.

Checkerboard Corner Extraction Method under Motion Blur
During the working process of a digital camera, the shutter needs to open for a moment to project light onto the photographic material. This brief moment is called the exposure time. Under highly dynamic conditions, the relative pose between the camera and the object changes evidently during the exposure time, thereby blurring or stretching the generated image [28]. The calibration of the extrinsic parameters requires a rotation, especially for IMU. Therefore, for the camera measurements, we must extract the checkerboard corners during rotation, but a motion blur may be generated in the process. Conventional checkerboard corner detection methods compute the local optimum value [29] with static images. Therefore, when noise and motion blur are present in an image, the errors in the extraction results evidently increase.
We utilize inertial data to eliminate the smearing effect and then extract the checkerboard corners with the aid of the IMU data. The corner detection algorithm is developed as follows. The Lucy-Richardson method is used to rectify the blurred image in step A, and the conventional corner detection method is used to find the corners' pixel coordinates in step B. Given that checkerboard corners are constrained on a few lines, we add a linear constraint to refine the checkerboard corner location. Through the linear constraint, the corners' pixel coordinates are refined in step C. Finally, IMU measurement is used to correct the position of corner detected for movement in step D.

•
A. Pretreatment and smearing effect elimination.
We deblur the image using the Lucy-Richardson method [30][31][32]. The point-spread function (PSF) is a 2D Gaussian model N 0, 0, σ 2 x , σ 2 y , and σ x and σ y denote the standard deviations in the x and y axes, respectively. If the velocity of the checkerboard expressed in the camera frame coincides with the x axis, then a motion blur is only observed in the x axis. Therefore, the PSF can be written as N 0, 0, σ 2 , 0 , and the corresponding covariance matrix is where σ 2 = ηω B , η is the scale factor, and ω B is the angular velocity of rotation. When the cross angle between the velocity of the checkerboard is expressed in the camera frame and the x axis of the camera frame is θ, the covariance matrix of PSF can be written as where R(θ) = cos θ sin θ − sin θ cos θ . θ can be evaluated by camera data. Figure 2 compares the original and deblurred images and shows that the motion blur has been effectively eliminated. We deblur the image using the Lucy-Richardson method [30][31][32].
where 2 = B σ ηω , η is the scale factor, and B ω is the angular velocity of rotation.
When the cross angle between the velocity of the checkerboard is expressed in the camera frame and the x axis of the camera frame is θ , the covariance matrix of PSF can be written as where ( ) cos sin sin cos . θ can be evaluated by camera data. • B. Corner detection.
This step is the same as that in the conventional method. The corners in the image, including the complete checkerboard, are examined. x I and y I denote the x (horizontal) and y (vertical) components of the 2D numerical gradient, and the first and second derivatives are computed as This step is the same as that in the conventional method. The corners in the image, including the complete checkerboard, are examined. I x and I y denote the x (horizontal) and y (vertical) components of the 2D numerical gradient, and the first and second derivatives are computed as The gradient direction is evaluated by where µ is the weight coefficient. C xy and C 45 are computed for each pixel in the picture. Afterward, C xy and C 45 are determined-one of which is valid-by minimizing the energy function proposed in [29]. Based on the threshold set in advance and non-maxima suppression, the detection accuracy reaches the pixel level. The results are presented in Figure 3a.
We can evaluate the lines of checkerboard based on the corners by using least squares. We assume that the line equation is a i x + b j y + c ij = 0 and that i and j represent the row and column numbers, respectively. The cross points of these lines are then computed for the global optimization of the checkerboard corners ( Figure 3b; refer to the yellow "+"). Figure 3c compares the results before and after refinement, respectively. The error of some corners detected before the refinement has remarkably increased due to motion blur. •

D. IMU-aided checkerboard corner modification
At time t, camera and IMU start collecting data simultaneously. The corners extracted have some delay due to the motion blur in the image, with real data at time t + τ/2. τ is the exposure time of camera. We can modify the corner coordinates with the inertial data. The details are presented as follows.
We consider the motion of camera in ∆t, and point P in the camera frame can be written as where where Ω B t is Ω B = ω B × at time t. Equation (19) can be written as t V VR = t x t y t z T is defined, and Equation (21) is expanded as By substituting Equation (22) into Equation (12), we have Given that in this study, we have: Moreover, given that M = When ∆t = −τ/2, we can modify the coordinates of the checkerboard corners, thereby eliminating the smearing effect. The image process algorithm is summarized as Table 1. The corner extraction accuracy can be significantly enhanced by using the above checkerboard corner detection algorithm, especially when a smearing effect exists.

Description of the Estimator
There are many methods that can solve the equations that provide the extrinsic parameters, such as EKF [33,34], genetic algorithm [35], and so on. Here, we utilize an EKF for calibrating the extrinsic parameters. The EKF algorithm is briefly introduced in Table 2  The image process algorithm is summarized as Table 1. The corner extraction accuracy can be significantly enhanced by using the above checkerboard corner detection algorithm, especially when a smearing effect exists.

Description of the Estimator
There are many methods that can solve the equations that provide the extrinsic parameters, such as EKF [33,34], genetic algorithm [35], and so on. Here, we utilize an EKF for calibrating the extrinsic parameters. The EKF algorithm is briefly introduced in Table 2. The state vector x includes the extrinsic parameters (translation vectors t I BI , t V BV and Euler angle ψ I B , ψ V B ), the IMU bias (b a and b g ), the camera initial parameter (t includes the accelerometer (f I x and f I y ) and gyro (ω I x and ω I y ) measurements as well as the extracted checkerboard corners (u x ,v x , u y , and v y ). Table 2. EKF updating process.

Simulation
A simulation test is designed to validate the performance of the extrinsic parameter calibration method. During the simulation, the turntable is assumed to demonstrate swinging motions around the x and y axes. The swinging rule is ω k = A k sin(2π f k + ϕ k ) + ω k0 , where k = x, y, ω k denotes the angular velocity, A k and f k denote the swinging amplitude and frequency, respectively, and ϕ k and ω k0 denote the initial phase and swinging center, respectively. The swinging parameters of the simulation are defined in Table 3. The true values for simulation data of the visual/inertial integrated system are defined in Table 4. [0. 10, 9.70, 0.10] Given the parameters in Tables 3 and 4, the true measurement of the gyroscope and accelerometer can be simulated by using the dynamics equation. The true measurement of the camera (pixel coordinates of the checkerboard corners) can be simulated by using the image model. When the errors in Table 5 are added into the ideal measurement data, real inertial sensor and camera outputs can be generated. The sampling rates of the IMU and camera are 100 Hz and 10 Hz, respectively. The alignment errors are shown in Figure 4, whereas the alignment error statistics are listed in Table 6. The calibration method can evaluate the extrinsic parameters correctly. The attitude error is <0.03 • for each Euler angle, and the position error is <0.10 mm for each position vector component. (pixel coordinates of the checkerboard corners) can be simulated by using the image model. When the errors in Table 5 are added into the ideal measurement data, real inertial sensor and camera outputs can be generated. The sampling rates of the IMU and camera are 100 Hz and 10 Hz, respectively. The alignment errors are shown in Figure 4, whereas the alignment error statistics are listed in Table 6. The calibration method can evaluate the extrinsic parameters correctly. The attitude error is <0.03° for each Euler angle, and the position error is <0.10 mm for each position vector component.

Experiment Setting
A calibration experiment is conducted to confirm the validity of the proposed method and to evaluate the accuracy of the system. Figure 5 shows the experiment architecture, whereas Table 7 presents the main devices. The system body frame coincides with the turntable frame for the sake of simplicity because the mechanical interface of the turntable frame is clearly defined.
The experiment is designed as follows. First, frames {I 0 }, {V 0 }, and {B 0 } are defined to coincide at the initialization time. The system body, camera, and MIMU coordinates are fixed with the turntable, camera, and MIMU, respectively. Second, the turntable is manually controlled to rotate around its axes, whereas the visual/inertial integrated system moves along with the turntable.

Experiment Setting
A calibration experiment is conducted to confirm the validity of the proposed method and to evaluate the accuracy of the system. Figure 5 shows the experiment architecture, whereas Table 7 presents the main devices. The system body frame coincides with the turntable frame for the sake of simplicity because the mechanical interface of the turntable frame is clearly defined.
The experiment is designed as follows. First, frames {I0}, {V0}, and {B0} are defined to coincide at the initialization time. The system body, camera, and MIMU coordinates are fixed with the turntable, camera, and MIMU, respectively. Second, the turntable is manually controlled to rotate around its axes, whereas the visual/inertial integrated system moves along with the turntable.    The intrinsic parameters obtained through Zhang's method [27] are shown in Table 8. The calibration achieved an accuracy of 0.08 pixel based on 18 images. 1 k 1 and k 2 are the radial distortion of the lens, while P 1 and P 2 are the tangential distortion.

Experiment Results and Discussion
The test results are presented in this section. The extrinsic calibration results before and after the checkerboard corner modification are nearly similar. The standard uncertainty of the modified method (0.15 mm) (Table 10) is lower than that of the unmodified method (0.18 mm) ( Table 9). The camera measurement residuals of the two methods are different as shown in Figure 6. The measurement residuals' 3σ bound of the method based on a linear constraint is approximately 0.193 pixels, whereas the measurement residuals' 3σ bound of the unmodified method is approximately 0.220 pixels. Thus, the motion blur correction described in Section 2.3 is effective. The intrinsic parameters obtained through Zhang's method [27] are shown in Table 8. The calibration achieved an accuracy of 0.08 pixel based on 18 images.

Experiment results and discussion
The test results are presented in this section. The extrinsic calibration results before and after the checkerboard corner modification are nearly similar. The standard uncertainty of the modified method (0.15 mm) (Table 10) is lower than that of the unmodified method (0.18 mm) ( Table 9). The camera measurement residuals of the two methods are different as shown in Figure 6. The measurement residuals' 3σ bound of the method based on a linear constraint is approximately 0.193 pixels, whereas the measurement residuals' 3σ bound of the unmodified method is approximately 0.220 pixels. Thus, the motion blur correction described in Section 2.3 is effective.   The difference of IMU calibration parameters before and after modification is not significant. The camera measurements can suppress the angle integral error due to IMU's drift, and increase the accuracy of IMU extrinsic parameters calibration results. Thus, more accuracy camera measurements will lead to more accuracy IMU extrinsic parameters calibration results. But the accuracy of IMU extrinsic parameters calibration results is closed before and after checkerboard corner modification, through compare the results in Tables 9 and 10. There may be three reasons explain it.

1.
For camera measurements, the rotation angle is computed based on all the checkerboard corners' pixel coordinates. The effects of motion blur are eliminate by involve all corners into computation process.

2.
The calibration process is not long, so the effect of accumulation error is not significant. 3.
There are system errors exits, such as the non-orthogonal of rotation axes, and the time delay of data acquisition, which also influence the error level.  Figure 7 summarize the experiment results. The origin of the MIMU frame is discussed in the sbg-IMU user manual, while that of the camera is the optical center of the lens. Therefore, the translation vector between the MIMU and camera can be roughly evaluated. The results that are evaluated based on the mechanical structure coincide with those that are evaluated by using EKF. The difference of IMU calibration parameters before and after modification is not significant. The camera measurements can suppress the angle integral error due to IMU's drift, and increase the accuracy of IMU extrinsic parameters calibration results. Thus, more accuracy camera measurements will lead to more accuracy IMU extrinsic parameters calibration results. But the accuracy of IMU extrinsic parameters calibration results is closed before and after checkerboard corner modification, through compare the results in Tables 9 and 10. There may be three reasons explain it.
1. For camera measurements, the rotation angle is computed based on all the checkerboard corners' pixel coordinates. The effects of motion blur are eliminate by involve all corners into computation process. 2. The calibration process is not long, so the effect of accumulation error is not significant. 3. There are system errors exits, such as the non-orthogonal of rotation axes, and the time delay of data acquisition, which also influence the error level. Table 10 and Figure 7 summarize the experiment results. The origin of the MIMU frame is discussed in the sbg-IMU user manual, while that of the camera is the optical center of the lens. Therefore, the translation vector between the MIMU and camera can be roughly evaluated. The results that are evaluated based on the mechanical structure coincide with those that are evaluated by using EKF.  The experiment results, the existing problems and possible reasons, the strategies for improving the results, and some directions for future work are presented below: 1. IMU and camera frames are aligned to the system body frame. The standard deviations of the three-axis position error are (0.13, 0.15, 0.11) mm and (0.12, 0.10, 0.13) mm for the MIMU and camera, respectively. Meanwhile, the standard deviations of the three-axis Euler angle error are (0.02°, 0.02°, 0.02°) and (0.03°, 0.04°, 0.02°) for the MIMU and camera, respectively. Compare the difference between the method with corner correction and without corner correction. We find the camera extrinsic parameters' accuracy of former is higher than the latter (Tables 9 and 10), and the camera measurement residuals of former is lower than the latter ( Figure 6). It indicates the corner correction described in Section 2.3 is effective. The reasons why the difference of IMU calibration parameters before and after modification is not significant have been briefly discussed. 2. There are three errors affecting the calibration accuracy. Firstly, the calibration errors of IMU and camera intrinsic parameters, which affect the measurements' accuracy. At present, a reasonable choice of camera calibration method ensures that imaging accuracy reaches the subpixel level, and IMU is factory calibrated. Secondly, the time delay between the IMU and camera data acquisition, which affects the calibration accuracy and stability of filter. We align the data by the time label (both IMU and camera data are marked on the time label, respectively), and don't evaluate the time delay exactly. Thirdly, we ignore the non-orthogonality of the turntable axes, and the turntable has been factory calibrated. Further research on a solution without orthogonal axes could be performed. 3. We can observe the convergence of each parameter in the EKF process. The experiment results show that the method is valid and is not restricted in the Kalman filter. Some optimal algorithms, such as the particle filter and Levenberg-Marquardt algorithm can also be used. The calibration parameters are obtained, and the complete visual/inertial integrated system is established. Future research may focus on the calibration in the navigation process, and the proposed method may be seen as a standard calibration step in factory production and user operation. The experiment results, the existing problems and possible reasons, the strategies for improving the results, and some directions for future work are presented below:
IMU and camera frames are aligned to the system body frame. The standard deviations of the three-axis position error are (0.13, 0.15, 0.11) mm and (0.12, 0.10, 0.13) mm for the MIMU and camera, respectively. Meanwhile, the standard deviations of the three-axis Euler angle error are (0.02 • , 0.02 • , 0.02 • ) and (0.03 • , 0.04 • , 0.02 • ) for the MIMU and camera, respectively. Compare the difference between the method with corner correction and without corner correction. We find the camera extrinsic parameters' accuracy of former is higher than the latter (Tables 9 and 10), and the camera measurement residuals of former is lower than the latter ( Figure 6). It indicates the corner correction described in Section 2.3 is effective. The reasons why the difference of IMU calibration parameters before and after modification is not significant have been briefly discussed.

2.
There are three errors affecting the calibration accuracy. Firstly, the calibration errors of IMU and camera intrinsic parameters, which affect the measurements' accuracy. At present, a reasonable choice of camera calibration method ensures that imaging accuracy reaches the sub-pixel level, and IMU is factory calibrated. Secondly, the time delay between the IMU and camera data acquisition, which affects the calibration accuracy and stability of filter. We align the data by the time label (both IMU and camera data are marked on the time label, respectively), and don't evaluate the time delay exactly. Thirdly, we ignore the non-orthogonality of the turntable axes, and the turntable has been factory calibrated. Further research on a solution without orthogonal axes could be performed.

3.
We can observe the convergence of each parameter in the EKF process. The experiment results show that the method is valid and is not restricted in the Kalman filter. Some optimal algorithms, such as the particle filter and Levenberg-Marquardt algorithm can also be used. The calibration parameters are obtained, and the complete visual/inertial integrated system is established. Future research may focus on the calibration in the navigation process, and the proposed method may be seen as a standard calibration step in factory production and user operation.

Conclusions
An extrinsic parameter calibration method for a visual/inertial integrated system is developed based on a swinging motion. A checkerboard corner detection algorithm is then utilized to detect checkerboard corners with a smearing effect. The extrinsic parameter calibration method is developed based on the imaging model and dynamic equation. This method is validated by performing a simulation and a real-world experiment, which results highlight the effectiveness of the proposed method. This method can also be seen as a standard calibration step and used for visual/inertial systems, especially for visual and inertial navigation integrated systems.