# A Multi-Projector Calibration Method for Virtual Reality Simulators with Analytically Defined Screens

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Hardware Components

#### 2.2. Multi-Projector Calibration Method with Analytically Defined Screens

#### 2.2.1. Compensated 3D Coordinates of CPs

**A**is the design matrix,

**X**is the vector of the unknowns,

**R**is the vector of the residuals and

**K**is the vector holding the independent factors. The design matrix

**A**is constructed from the equation shown in (2), which represents the mathematical form of an observed distance. In this equation, ${l}_{ij}$ is the approximated calculated value of the distance between i and j, $d{l}_{ij}$ is the differential value of ${l}_{ij}$, ${\theta}_{i{j}_{ca}}$ is the approximated calculated azimuth of the distance ${l}_{ij}$ and dz

_{j}, dz

_{i}, dx

_{j}and dxi are the unknowns, which are the corrections to the coordinates X and Z of points j and i (any pair of CPs). The equation of the azimuth is given in (3). The solution of the unknowns (dz

_{j}, dz

_{i}, dx

_{j}and dx

_{i}) is depicted in Equation (4), which is known as the normal equations. Finally, the X and Z compensated coordinates of CPs are calculated as shown in (5) and (6). Note that the Y coordinates are not compensated in this procedure, as the height h is directly measured on the screen surface, and thus is acquired with more accuracy.

#### 2.2.2. 2D/3D Correspondences of CPs

#### 2.2.3. Camera Interior Orientation

_{0}and y

_{0}). On the other hand, the tangential and radial distortion coefficients are computed following the method described in [32]. The equations used in OpenCV for correcting radial distortion are shown in Equations (7) and (8), where k

_{1}, k

_{2}and k

_{3}are the computed radial distortion coefficients. The tangential distortion is corrected via Equations (9) and (10), where p

_{1}and p

_{2}are the computed tangential distortion coefficients. The radius r is the distance from the distorted image point under consideration to the distortion center.

#### 2.2.4. Camera Exterior Orientation

#### 2.2.5. 2D/3D Correspondences of Chessboard

#### 2.2.6. Projector Calibration

_{i}, b

_{i}, c

_{i}are the 11 DLT parameters of a particular image. As one observed point provides 2 equations, a minimum of 6 points are needed to solve the 11 unknowns. It is known that the DLT parameters can be directly related to the six elements of the exterior orientation parameters of an image (X

_{0}, Y

_{0}, Z

_{0}, and orientation angles: camera direction α, nadir distance ν and swing κ) and to five elements of the interior orientation (principal point coordinates x

_{0}, y

_{0}, focal length c, relative y-scale λ and shear d) [33,34]. Therefore, solving these equations for a minimum of 6 observed points (points with 2D-3D correspondences) leads to the computation of the interior and exterior sensor orientation. If more points are available, the system can be solved with a LSF. Finally, it is worth mentioning that the DLT fails if all CPs lie in one plane. This situation cannot occur here, as CPs lie on a curved surface, the vertical cylinder.

## 3. Results

#### 3.1. Validation

#### 3.2. Image Warping

#### 3.3. Testing the Simulator with Users

## 4. Discussion

_{0}, while differences in angular values arrive to 10 degrees in some cases.

## 5. Conclusions

## Author Contributions

## Conflicts of Interest

## References

- Brown, M.; Majumder, A.; Yang, R. Camera-based calibration techniques for seamless multiprojector displays. IEEE Trans. Vis. Comput. Gr.
**2005**, 11, 193–206. [Google Scholar] [CrossRef] [PubMed] - Chen, H.; Sukthankar, R.; Wallace, G.; Li, K. Scalable alignment of large-format multi-projector displays using camera homography trees. In Proceedings of the IEEE Visualization, Boston, MA, USA, 27 October–1 November 2002; pp. 339–346. [Google Scholar]
- Raij, A.; Pollefeys, M. Auto-calibration of multi-projector display walls. In Proceedings of the 17th International Conference on Pattern Recognition, Cambridge, UK, 23–26 August 2004; pp. 14–17. [Google Scholar]
- Okatani, T.; Deguchi, K. Autocalibration of an ad hoc construction of multi-projector displays. In Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), New York, NY, USA, 17–22 June 2006; p. 8. [Google Scholar]
- Zhang, B.; Li, Y.F.; Wu, Y.H. Self-recalibration of a structured light system via plane-based homography. Pattern Recognit.
**2007**, 40, 1368–1377. [Google Scholar] [CrossRef] - Orghidan, R.; Salvi, J.; Gordan, M.; Florea, C.; Batlle, J. Structured light self-calibration with vanishing points. Mach. Vis. Appl.
**2014**, 25, 489–500. [Google Scholar] [CrossRef] - Huang, Z.; Xi, J.; Yu, Y.; Guo, Q. Accurate projector calibration based on a new point-to-point mapping relationship between the camera and projector images. Appl. Opt.
**2015**, 54, 347–356. [Google Scholar] [CrossRef] - Portalés, C.; Ribes-Gómez, E.; Pastor, B.; Gutiérrez, A. Calibration of a camera–projector monochromatic system. Photogramm. Rec.
**2015**, 30, 82–99. [Google Scholar] [CrossRef] - Ashdown, M.; Sato, Y. Steerable projector calibration. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 21–23 September 2005; p. 8. [Google Scholar]
- Kimura, M.; Mochimaru, M.; Kanade, T. Projector calibration using arbitrary planes and calibrated camera. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; pp. 1–2. [Google Scholar]
- Chien, H.J.; Chen, C.Y.; Chen, C.F. A target-adapted geometric calibration method for camera-projector system. In Proceedings of the 2010 25th International Conference of Image and Vision Computing New Zealand, Queenstown, New Zealand, 8–9 November 2010; pp. 1–8. [Google Scholar]
- Park, S.-Y.; Park, G.G. Active calibration of camera-projector systems based on planar homography. In Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey, 23–26 Augest 2010; pp. 320–323. [Google Scholar]
- Fernandez, S.; Salvi, J. Planar-based camera-projector calibration. In Proceedings of the 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA), Dubrovnik, Croatia, 4–6 September 2011; pp. 633–638. [Google Scholar]
- Moreno, D.; Taubin, G. Simple, accurate, and robust projector-camera calibration. In Proceedings of the 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, Zürich, Switzerland, 13–15 October 2012; pp. 464–471. [Google Scholar]
- Gockel, T.; Azad, P.; Dillmann, R. Calibration issues for projector-based 3d-scanning. In Proceedings of the Shape Modeling Applications, Genova, Italy, 7–9 June 2004; pp. 367–370. [Google Scholar]
- Liao, J.; Cai, L. A calibration method for uncoupling projector and camera of a structured light system. In Proceedings of the 2008 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Xian, China, 2–5 July 2008; pp. 770–774. [Google Scholar]
- Hong, W.; Gelb, D.; Trott, M. Automatic calibration of a projector-camera system with a see-through screen. In Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA, 30 September–3 October 2012; pp. 337–340. [Google Scholar]
- Knyaz, V.A. Automated calibration technique for photogrammetric system based on a multi-media projector and a ccd camera. In Proceedings of the ISPRS Commission V Symposium Image Engineering and Vision Metrology, Dresden, Germany, 25–27 September 2006. [Google Scholar]
- Raskar, R.; Brown, M.S.; Ruigang, Y.; Wei-Chao, C.; Welch, G.; Towles, H.; Scales, B.; Fuchs, H. Multi-projector displays using camera-based registration. In Proceedings of the Conference on Visualization ’99: Celebrating Ten Years, San Francisco, CA, USA, 24–29 October 1999; pp. 161–522. [Google Scholar]
- Tardif, J.P.; Roy, S.; Trudeau, M. Multi-projectors for arbitrary surfaces without explicit calibration nor reconstruction. In Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, Banff, AB, Canada, 6–10 October 2003; pp. 217–224. [Google Scholar]
- Harville, M.; Culbertson, B.; Sobel, I.; Gelb, D.; Fitzhugh, A.; Tanguay, D. Practical methods for geometric and photometric correction of tiled projector. In Proceedings of the 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), New York, NY, USA, 17–22 June 2006; p. 5. [Google Scholar]
- Sun, W.; Sobel, I.; Culbertson, B.; Gelb, D.; Robinson, I. Calibrating multi-projector cylindrically curved displays for “wallpaper” projection. In Proceedings of the 5th ACM/IEEE International Workshop on Projector camera systems, Marina del Rey, CA, USA, 10 August 2008; pp. 1–8. [Google Scholar]
- Sajadi, B.; Majumder, A. Auto-calibration of cylindrical multi-projector systems. In Proceedings of the 2010 IEEE Virtual Reality Conference (VR), Waltham, MA, USA, 20–24 March 2010; pp. 155–162. [Google Scholar]
- Sajadi, B.; Majumder, A. Markerless view-independent registration of multiple distorted projectors on extruded surfaces using an uncalibrated camera. IEEE Trans. Vis. Comput. Gr.
**2009**, 15, 1307–1316. [Google Scholar] [CrossRef] [PubMed] - Zhao, L.; Weng, D.; Li, D. The auto-geometric correction of multi-projector for cylindrical surface using bézier patches. J. Soc. Inf. Disp.
**2014**, 22, 473–481. [Google Scholar] [CrossRef] - Chen, B.-S.; Zhong, Q.; Li, H.-F.; Liu, X.; Xu, H.-S. Automatic geometrical calibration for multiprojector-type light field three-dimensional display. Optice
**2014**, 53, 073107. [Google Scholar] [CrossRef] - Nahon, M.A.; Reid, L.D. Simulator motion-drive algorithms—A designer's perspective. J. Guid. Control Dyn.
**1990**, 13, 356–362. [Google Scholar] [CrossRef] - Casas, S.; Portalés, C.; Riera, J.V.; Fernández, M. Heuristics for solving the parameter tuning problem in motion cueing algorithms. In Revista Iberoamericana de Automática e Informática industrial; CEA: New Delhi, India, 2017; pp. 193–204. [Google Scholar]
- Chueca Pazos, M.; Herráez Boquera, J.; Berné Valero, J.L. Métodos Topográficos; Editorial Paraninfo S.A.: Madrid, Spain, 1996; p. 746. [Google Scholar]
- Itseez. Opencv. Available online: http://opencv.org/ (accessed on 2 June 2017).
- Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell.
**2000**, 22, 1330–1334. [Google Scholar] [CrossRef] - Brown, D.C. Close-range camera calibration. Photogramm. Eng.
**1971**, 37, 855–866. [Google Scholar] - Dermanis, A. Free network solutions with the direct linear transformation method. ISPRS J. Photogramm. Remote Sens.
**1994**, 49, 2–12. [Google Scholar] [CrossRef] - Kraus, K. Photogrammetry. In Advanced Methods and Applications; Dümmler/Bonn: Bonn, Germany, 1997; p. 466. [Google Scholar]

**Figure 3.**Top view (

**left**) and side view (

**right**) of the spatial distribution of devices and control points (CPs), and representation of measured distances.

**Figure 5.**Spatial positioning of devices and 3D points, where the different coloured dots represent: black: camera projection center; cyan: projector projection center; red: control points at the cylindrical surface; green: checkerboard points at Z = 0; blue: checkerboard points at the cylindrical surface.

**Figure 6.**Residuals (×50) of the three projector calibration, where: (

**a**) Proj1; (

**b**) Proj2; (

**c**) Proj3.

**Figure 8.**Panoramic views of the resulting wall-paper like projections of a grid (

**top**) and a scene (

**bottom**). No photometric correction is applied in the overlapping areas to highlight the results of the method.

**Table 1.**Spatial coordinates of CPs from geometrical rules. Only the central CPs are depicted as an example.

Control Point | X | Y | Z |
---|---|---|---|

CP2 | −a/2 | h | −d |

CP10 | a/2 − m | h | −d − n |

CP3 | a/2 | h | −d |

CP6 | −a/2 | 0 | −d |

CP13 | a/2 − m | 0 | −d − n |

CP7 | a/2 | 0 | −d |

Control Point | CP10 | CP3 | CP1 | CP9 | CP11 | CP4 | O |
---|---|---|---|---|---|---|---|

CP2 | 0.835 | 1.920 | 2.045 | 1.018 | 2.745 | 3.040 | 1.570 |

CP10 | 1.190 | 2.585 | 1.760 | 2.250 | 2.780 | 1.570 | |

CP3 | 3.040 | 2.605 | 1.275 | 2.060 | 1.570 | ||

CP1 | 1.170 | 2.995 | 2.920 | 1.570 | |||

CP9 | 3.060 | 3.045 | 1.570 | ||||

CP11 | 0.930 | 1.570 | |||||

CP4 | 1.570 |

Control Point | CP10 | CP3 | CP1 | CP9 | CP11 | CP4 | O |
---|---|---|---|---|---|---|---|

CP2 | 0.851 | 1.905 | 2.032 | 0.992 | 2.761 | 3.090 | 1.570 |

CP10 | 1.157 | 2.608 | 1.756 | 2.260 | 2.795 | 1.567 | |

CP3 | 3.077 | 2.582 | 1.307 | 2.058 | 1.566 | ||

CP1 | 1.153 | 3.066 | 2.797 | 1.586 | |||

CP9 | 3.062 | 3.118 | 1.551 | ||||

CP11 | 0.862 | 1.552 | |||||

CP4 | 1.588 |

x_{0} | y_{0} | c | k1 | k2 | p1 | p2 | k3 |
---|---|---|---|---|---|---|---|

1384.811 | 826.200 | 2380.083 | −0.160384 | 0.124279 | −0.000252 | −0.000955 | −0.015237 |

**Table 5.**Camera exterior orientation at the three locations (units of translations in (m), angles in (deg)).

Camera | X_{0} | Y_{0} | Z_{0} | α | ν | κ |
---|---|---|---|---|---|---|

Cam1 | 0.554 | 1.192 | 0.476 | 80.3854 | 73.4311 | −82.3739 |

Cam2 | −0.671 | 1.017 | 1.406 | −69.7206 | 9.3806 | 69.8575 |

Cam3 | −0.634 | 1.176 | −0.405 | −84.0016 | 77.2742 | 83.9486 |

Projector | x_{0} | y_{0} | c | m | d |
---|---|---|---|---|---|

Proj1 | 873.064 | −56.964 | 1344.290 | −0.99987 | 0.00260 |

Proj2 | 738.275 | −53.301 | 1518.870 | −0.99997 | −0.00677 |

Proj3 | 731.679 | −46.186 | 1247.710 | −0.99998 | −0.00352 |

Projector | X_{0} | Y_{0} | Z_{0} | α | ν | κ |
---|---|---|---|---|---|---|

Proj1 | 0.389 | 1.431 | 0.215 | 88.5852 | 76.6541 | −85.5039 |

Proj2 | −0.283 | 1.457 | 0.752 | 67.4913 | 2.8702 | −67.6160 |

Proj3 | −0.438 | 1.397 | −0.156 | −89.1899 | 70.6764 | 84.9487 |

**Table 8.**Discrepancies between the ideal (Table 6) and the simulated projector interior orientation with approximate 3D coordinates (units in (pixels) for x

_{0}, y

_{0}and c).

Projector | dx_{0} | dy_{0} | dc | dm | dd |
---|---|---|---|---|---|

Proj1 | −45,650 | 9138 | −111,680 | 0.00011 | −0.00174 |

Proj2 | −150,398 | −4950 | 13,900 | −0.00001 | −0.00111 |

Proj3 | −218,024 | 13,286 | −111,970 | 0.00000 | −0.00463 |

**Table 9.**Discrepancies between the ideal (Table 7) and the simulated exterior orientation of projectors with approximate 3D coordinates (units of translations in (m), angles in (deg)).

Projector | dX_{0} | dY_{0} | dZ_{0} | dα | dν | dκ |
---|---|---|---|---|---|---|

Proj1 | −0.151 | −0.003 | −0.034 | 0.1736 | 41,517 | 22,518 |

Proj2 | −0.151 | 0.010 | 0.019 | 115,601 | 0.7653 | −114,308 |

Proj3 | 0.052 | −0.006 | −0.249 | −0.0710 | −53,606 | 93,404 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Portalés, C.; Casas, S.; Coma, I.; Fernández, M. A Multi-Projector Calibration Method for Virtual Reality Simulators with Analytically Defined Screens. *J. Imaging* **2017**, *3*, 19.
https://doi.org/10.3390/jimaging3020019

**AMA Style**

Portalés C, Casas S, Coma I, Fernández M. A Multi-Projector Calibration Method for Virtual Reality Simulators with Analytically Defined Screens. *Journal of Imaging*. 2017; 3(2):19.
https://doi.org/10.3390/jimaging3020019

**Chicago/Turabian Style**

Portalés, Cristina, Sergio Casas, Inmaculada Coma, and Marcos Fernández. 2017. "A Multi-Projector Calibration Method for Virtual Reality Simulators with Analytically Defined Screens" *Journal of Imaging* 3, no. 2: 19.
https://doi.org/10.3390/jimaging3020019