Calibration of a Catadioptric System and 3D Reconstruction Based on Surface Structured Light
Abstract
:1. Introduction
2. Materials and Methods
2.1. Related Basic Knowledge
2.1.1. Catadioptric Camera Model
2.1.2. Pinhole Imaging Model of the Traditional Camera
2.1.3. Projector Projection Model
2.2. System Calibration and 3D Reconstruction
2.2.1. Calibration of Ordinary Camera and Projector
2.2.2. Calibration of Catadioptric Camera
2.2.3. Position and Attitude Calculation of the Projector and Catadioptric Camera
2.3. Three-Dimensional Reconstruction of Catadioptric System Based on Surface Structured Light
2.3.1. Depth Estimation
2.3.2. Summary of Calibration and Reconstruction Steps
- The calibration toolbox [28] was used to calibrate the camera and projector to obtain the intrinsic parameter MP of the projector and the intrinsic parameter MC of the camera;
- The intrinsic parameters and mirror parameters of the catadioptric camera are obtained by the calibration toolbox [16] to calibrate the catadioptric camera;
- Taking the plane checkerboard as the reference plane of the world coordinate system, the extrinsic parameters between the camera and the world coordinate system are calculated as and ;
- The extrinsic parameters and between the projector and the world coordinate system are calculated with a fixed plane checkerboard assisted by an ordinary camera;
- Move the plane checkerboard, repeat steps 3 and 4, and calculate the extrinsic parameters and of the catadioptric camera and projector by using Equations (11) and (12);
- Take and as initial values, optimize and minimize the reprojection error, and obtain the final values of and ;
- Project the coded structured light, use Equation (17) to calculate the 3D-reconstructed point cloud of the object, and use MeshLab to display the point cloud results.
3. Results
3.1. Design and Calibration of Catadioptric System Based on Structured Light
3.2. Three-Dimensional Reconstruction of Catadioptric System Based on Surface Structured Light
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Paulson, B.; Lee, S. Miniaturized omnidirectional flexible side-view endoscope for rapid monitoring of thin tubular biostructures. Biomed. Opt. Express 2019, 5, 2279–2289. [Google Scholar] [CrossRef] [PubMed]
- Amorim, P.H.J.; Moraes, T.F. Reconstruction of panoramic dental images through bezier function optimization. Front. Bioeng. Biotechnol. 2020, 8, 794. [Google Scholar] [CrossRef] [PubMed]
- Panetta, K.; Rajendran, R. Tufts Dental Database: A Multimodal Panoramic X-ray Dataset for Benchmarking Diagnostic Systems. IEEE J. Biomed. Health Inform. 2022, 26, 650–1659. [Google Scholar] [CrossRef] [PubMed]
- Liu, S.Y.; Guo, P. Accurate and Robust Monocular SLAM with Omnidirectional Cameras. Sensors 2019, 19, 4494. [Google Scholar] [CrossRef]
- Cheng, D.W.; Wang, Q.W. Design method of a wide-angle AR display with a single-layer two-dimensional pupil expansion geometrical waveguide. Appl. Opt. 2022, 61, 5813–5822. [Google Scholar] [CrossRef]
- Cheng, D.W.; Hou, Q.C. Optical design and pupil swim analysis of a compact, large EPD and immersive VR head mounted display. Opt. Express 2022, 5, 6584–6602. [Google Scholar] [CrossRef] [PubMed]
- Chen, L.; Yuan, Q. Design of a compact dual-view endoscope based on a hybrid lens with annularly stitched aspheres. Opt. Commun. 2019, 453, 124346. [Google Scholar] [CrossRef]
- Li, Y.N.; Zhan, T. Broadband cholesteric liquid crystal lens for chromatic aberration correction in catadioptric virtual reality optics. Opt. Express 2021, 29, 6011–6020. [Google Scholar] [CrossRef] [PubMed]
- Zou, Y.Y.; Zhang, L. Synchronous measurement method of a multi-angle scattered light field. Appl. Opt. 2021, 61, 2275–2283. [Google Scholar] [CrossRef]
- Chen, Y.Z.; Zhou, F.Q. A Novel Mirrored Binocular Vision Sensor Based on Spherical Catadioptric Mirrors. IEEE Sens. J. 2021, 21, 18670–18681. [Google Scholar] [CrossRef]
- Barone, S.; Neri, P. Structured light stereo catadioptric scanner based on a spherical mirror. Opt. Lasers Eng. 2018, 107, 1–12. [Google Scholar] [CrossRef]
- Chen, X.; Zhou, F.Q. Mobile visual detecting system with a catadioptric vision sensor in pipeline. Optik 2019, 193, 162854. [Google Scholar] [CrossRef]
- Kuznetsov, A.O.; Gorevoy, A.V. Image rectification for prism-based stereoscopic optical systems. Comput. Vis. Image Underst. 2019, 182, 30–37. [Google Scholar] [CrossRef]
- Lin, H.Y.; Chung, Y.C. Self-localization of mobile robots using a single catadioptric camera with line feature extraction. Sensors 2021, 21, 4719. [Google Scholar] [CrossRef] [PubMed]
- Guo, X.D.; Wang, Z.B. Research on design, calibration and real-time image expansion technology of unmanned system variable-scale panoramic vision system. Sensors 2021, 14, 4708. [Google Scholar] [CrossRef] [PubMed]
- Active Vision Group. Available online: https://www.robots.ox.ac.uk/~cmei/Toolbox.html (accessed on 8 September 2021).
- Gao, Y.; Zhao, Y.Y. A Projector-Based Augmented Reality Navigation System for Computer-Assisted Surgery. Sensors 2021, 9, 2931. [Google Scholar] [CrossRef]
- Jin, Y.; Nan, G. High-accuracy projector calibration method for fringe projection profilometry considering perspective transformation. Opt. Express 2021, 29, 15053–15066. [Google Scholar]
- Jia, T.; Wang, B.N. Scene depth perception based on omnidirectional structured light. IEEE Trans. Image Process. 2016, 9, 4369–4378. [Google Scholar] [CrossRef]
- Cordova-Esparza, D.M.; Terven, J. Three-dimensional reconstruction of indoor and outdoor environments using a stereo catadioptric system. Appl. Sci. 2020, 10, 8851. [Google Scholar] [CrossRef]
- Minhtuan, H.; Xiao, C.Y. Complete grid pattern decoding method for a one-shot structured light system. Appl. Opt. 2020, 59, 2674–2685. [Google Scholar]
- Zhang, Q.; Xv, X.F. Phase-shift extraction and wave reconstruction in four-step phase-shifting interferometry. Opto-Electron. Eng. 2011, 38, 139–144. [Google Scholar]
- Li, J.; Guan, J. Error self-correction method for phase jump in multifrequency phase-shifting structured light. Appl. Opt. 2021, 60, 4. [Google Scholar]
- Mei, C.; Rives, P. Single View Point Omnidirectional Camera Calibration from Planar Grids. In Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy, 10–14 April 2007. [Google Scholar]
- Grossberg, M.D.; Nayar, S.K. A general imaging model and a method for finding its parameters. In Proceedings of the Eighth IEEE International Conference on Computer Vision, Vancouver, BC, Canada, 7–14 July 2001. [Google Scholar]
- Kimura, M.; Mochimaru, M. Projector Calibration using Arbitrary Planes and Calibrated Camera. In Proceedings of the 2007 IEEE Conference on Computer Vision & Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007. [Google Scholar]
- Falcao, G.; Hurtos, N. Plane-based calibration of a projector-camera system. VIBOT 2008, 1, 1–12. [Google Scholar]
- Procamcalib. Available online: https://code.google.com/archive/p/procamcalib/ (accessed on 11 October 2021).
- Zhao, X.X.; Ye, S.Z. Space reconstruction of audiovisual media based on artificial intelligence and virtual reality. J. Intell. Fuzzy Syst. 2021, 40, 7285–7296. [Google Scholar] [CrossRef]
- Li, F.; Shang, X.D. Single-shot depth sensing with pseudo two-dimensional sequence coded discrete binary pattern. IEEE Sens. J. 2021, 21, 11075–11083. [Google Scholar] [CrossRef]
- Zhang, C.; Xu, J. A robust surface coding method for optically challenging objects using structured light. IEEE Trans. Autom. Sci. Eng. 2014, 11, 775–788. [Google Scholar] [CrossRef]
- Xu, Y.; Aliaga, D.G. An adaptive correspondence algorithm for modeling scenes with strong interreflections. IEEE Trans. Vis. Comput. Graph. 2009, 15, 465–480. [Google Scholar] [CrossRef]
- Jia, T.; Yuan, X. Depth perception based on monochromatic shape encode-decode structured light method. Opt. Lasers Eng. 2020, 134, 106259. [Google Scholar] [CrossRef]
- Wang, Q.; Xiong, D.P. Optical image authentication scheme using dual polarization decoding configuration. Opt. Lasers Eng. 2019, 112, 151–161. [Google Scholar] [CrossRef]
- Almaraz-Cabral, C.C.; Gonzalez-Barbosa, J.J. Fringe projection profilometry for panoramic 3d reconstruction. Opt. Lasers Eng. 2016, 5, 106–112. [Google Scholar] [CrossRef]
Parameters | Catadioptric Camera | Projector |
---|---|---|
Intrinsic matrix | ||
Rotation matrix | ||
Translation matrix |
Measurement Object | Parameter | Length (mm) | Height (mm) |
---|---|---|---|
Cuboid | Standard parameter | 110 | 220 |
Reconstruction parameter | 110.75 | 216.15 | |
Relative error | 0.0068 | 0.0175 | |
Cone | Standard parameter | 140 | 220 |
Reconstruction parameter | 140.55 | 214.40 | |
Relative error | 0.00392 | 0.0255 |
Comparative Items | Jia [19] | Cesar-Cruz [20] | Cordova-Esparza [35] | Proposed Method |
---|---|---|---|---|
Camera System | One catadioptric camera and four projectors | One catadioptric camera and a catadioptric projector | One catadioptric camera and a catadioptric projector | One catadioptric camera and a projector |
Structured light | Hourglass spatial coding | Temporal phase unwrapping | 10 × 6 stereo-point spatial coding | Binary time-coding |
Camera projection model | Taylor polynomial model | Taylor polynomial model | Taylor polynomial model | Unified spherical model |
The experimental application | Scene depth perception | Three-dimensional reconstruction of the sphere | Measurement of the angle between two planes | Three-dimensional reconstruction of a cuboid |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lu, Z.; Lv, Y.; Ai, Z.; Suo, K.; Gong, X.; Wang, Y. Calibration of a Catadioptric System and 3D Reconstruction Based on Surface Structured Light. Sensors 2022, 22, 7385. https://doi.org/10.3390/s22197385
Lu Z, Lv Y, Ai Z, Suo K, Gong X, Wang Y. Calibration of a Catadioptric System and 3D Reconstruction Based on Surface Structured Light. Sensors. 2022; 22(19):7385. https://doi.org/10.3390/s22197385
Chicago/Turabian StyleLu, Zhenghai, Yaowen Lv, Zhiqing Ai, Ke Suo, Xuanrui Gong, and Yuxuan Wang. 2022. "Calibration of a Catadioptric System and 3D Reconstruction Based on Surface Structured Light" Sensors 22, no. 19: 7385. https://doi.org/10.3390/s22197385
APA StyleLu, Z., Lv, Y., Ai, Z., Suo, K., Gong, X., & Wang, Y. (2022). Calibration of a Catadioptric System and 3D Reconstruction Based on Surface Structured Light. Sensors, 22(19), 7385. https://doi.org/10.3390/s22197385