A New Calibration Method for Commercial RGB-D Sensors
Abstract
:1. Introduction
2. A Distortion Calibration Model for Depth Sensor
3. RGB-D Sensor Calibration
3.1. RGB-D Joint Calibration
3.2. Error and Distortion Model
4. Experimental Design and Data Collection
5. Calibration Results
6. Accuracy Assessment of the Calibration Models
7. Conclusions and Future Work
Acknowledgments
Author Contributions
Conflicts of Interest
References
- Occipital. Available online: http://structure.io/ (accessed on 22 November 2015).
- Xtion PRO LIVE. Available online: https://www.asus.com/3D-Sensor/Xtion_PRO_LIVE/ (accessed on 5 April 2016).
- Meet Kinect for Windows. Available online: https://dev.windows.com/en-us/kinect (accessed on 15 January 2016).
- Gokturk, S.B.; Yalcin, H.; Bamji, C. A time-of-flight depth sensor-system description, issues and solutions. In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, Washington, DC, USA, 27 June–2 July 2004. [Google Scholar]
- Yamazoe, H.; Habe, H.; Mitsugami, I.; Yagi, Y. Easy depth sensor calibration. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR), Tsukuba, Japan, 11–15 November 2012. [Google Scholar]
- Wang, K.; Zhang, G.; Bao, H. Robust 3D reconstruction with an RGB-D camera. IEEE Trans. Image Process. 2014, 23, 4893–4906. [Google Scholar] [CrossRef] [PubMed]
- Haggag, H.; Hossny, M.; Filippidisy, D.; Creighton, D.; Nahavandi, S.; Puriy, V. Measuring depth accuracy in RGBD cameras. In Proceedings of the 7th International Conference on Signal Processing and Communication Systems (ICSPCS), Carrara, Australia, 16–18 December 2013. [Google Scholar]
- Chow, J.C.K.; Lichti, D.D. Photogrammetric bundle adjustment with self-calibration of the primesense 3D camera technology: Microsoft Kinect. IEEE Access 2013, 22, 465–474. [Google Scholar] [CrossRef]
- Herrera, C.D.; Kannala, J.; Heikkila, J. Joint depth and color camera calibration with distortion correction. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34, 2058–2064. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.-T.; Shen, C.-A.; Yang, J.-S. Calibrated Kinect sensors for robot simultaneous localization and mapping. In Proceedings of the 2014 19th International Conference on Methods and Models in Automation and Robotics (MMAR), Miedzyzdroje, Poland, 2–5 September 2014. [Google Scholar]
- Shibo, L.; Qing, Z. A New approach to calibrate range image and color image from Kinect. In Proceedings of the 4th International Conference on Intelligent Human-Machine Systems and Cybernetics, Nanchang, China, 26–27 August 2012. [Google Scholar]
- Fryer, J.G. Camera calibration in non-topographic photogrammetry. In Non-Topographic Photogrammetry; Karara, H., Ed.; American Society for Photogrammetry and Remote Sensing: Bethesda, MD, USA, 1989; pp. 59–70. [Google Scholar]
- Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1330–1334. [Google Scholar] [CrossRef]
- Zhang, C.; Zhang, Z. Calibration between depth and color sensors for commodity depth cameras. In Proceedings of the 2011 IEEE International Conference on Multimedia and Expo (ICME), Barcelona, Spain, 11–15 July 2011. [Google Scholar]
- Pagliari, D.; Pinto, L. Calibration of kinect for Xbox one and comparison between the two generations of Microsoft sensors. Sensors 2015, 15, 27569–27589. [Google Scholar] [CrossRef] [PubMed]
- Lachat, E.; Macher, H.; Landes, T.; Grussenmeyer, P. Assessment and calibration of a RGB-D camera (kinect v2 sensor) towards a potential use for close-range 3D modeling. Remote Sens. 2015, 7, 13070–13097. [Google Scholar] [CrossRef]
- Liu, W.; Fan, Y.; Zhong, Z.; Lei, T. A new method for calibrating depth and color camera pair based on Kinect. In Proceedings of the 2012 International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, China, 16–18 July 2012. [Google Scholar]
- Gui, P.; Ye, Q.; Chen, H.; Zhang, T.; Yang, C. Accurately calibrate Kinect sensor using indoor control field. In Proceedings of the 2014 3rd International Workshop on Earth Observation and Remote Sensing Applications (EORSA), Changsha, China, 11–14 June 2014. [Google Scholar]
- Raposo, C.; Barreto, J.P.; Nunes, U. Fast and accurate calibration of a Kinect sensor. In Proceedings of the 2013 International Conference on 3D Vision-3DV, Seattle, WA, USA, 29 June–1 July 2013. [Google Scholar]
- Davoodianidaliki, M.; Saadatseresht, M. calibrate kinect to use in computer vision, simplified and precise. In Proceedings of the 8th Iranian Conference on Machine Vision and Image Processing (MVIP), Zanjan, Iran, 10–12 September 2013. [Google Scholar]
- Mallick, T.; Das, P.P.; Majumdar, A.K. Characterizations of noise in Kinect depth images: A review. IEEE Sens. J. 2014, 14, 1731–1740. [Google Scholar] [CrossRef]
- Khoshelham, K.; Elberink, S.O. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensor 2012, 12, 1437–1454. [Google Scholar] [CrossRef] [PubMed]
- Filippo, B.; Pretto, A.; Emanuele, M. Unsupervised intrinsic and extrinsic calibration of a camera-depth sensor couple. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014. [Google Scholar]
- Macknojia, R.; Chávez-Aragón, A.; Payeur, P.; Laganière, R. Calibration of a network of kinect sensors for robotic inspection over a large workspace. In Proceedings of the IEEE Workshop on Robot Vision (WORV), Clearwater Beach, FL, USA, 15–17 January 2013. [Google Scholar]
- Ju, Z.; Wang, Y.; Zeng, W.; Chen, S.; Liu, H. Depth and RGB image alignment for hand gesture segmentation using Kinect. In Proceedings of the 2013 International Conference on Machine Learning and Cybernetics, Tianjin, China, 14–17 July 2013. [Google Scholar]
- Kim, J.H.; Choi, J.S.; Koo, B.K. Calibration of multi-kinect and multi-camera setup for full 3D reconstruction. In Proceedings of the 44th International Symposium on Robotics (ISR), Seoul, Korea, 24–26 October 2013. [Google Scholar]
- Morvan, Y. Acquisition, Compression and Rendering of Depth and Texture for Multi-View Video. Available online: http://www.epixea.com/research/multi-view-coding-thesis.html (accessed on 1 June 2015).
- Jia, W.; Yi, W.-J.; Saniie, J.; Oruklu, E. 3D image reconstruction and human body tracking using stereo vision and Kinect technology. In Proceedings of the 2012 IEEE International Conference on Electro/Information Technology (EIT), Indianapolis, IN, USA, 6–8 May 2012; pp. 1–4. [Google Scholar]
- Khoshelham, K. Accuracy analysis of kinect depth data. In Proceedings of the ISPRS Calgary 2011 Workshop, Calgary, AB, Canada, 29–31 August 2011; pp. 133–138. [Google Scholar]
- Heikkila, J.; Silven, O. A Four-step camera calibration procedure with implicit image correction. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico, 17–19 June 1997. [Google Scholar]
- Fryer, J.G.; Brown, D.C. Lens distortion for close-range photogrammetry. Photogramm. Eng. Remote Sens. 1986, 52, 51–58. [Google Scholar]
- Tang, S.; Zhu, Q.; Chen, W.; Darwish, W.; Wu, B.; Hu, H.; Chen, M. Enhanced RGB-D mapping method for detailed 3D indoor and outdoor modeling. Sensors 2016, 16, 1589. [Google Scholar] [CrossRef] [PubMed]
Sensor | Phase 1 | Phase 2 | ||
---|---|---|---|---|
IR | RGB | Disparity | Depth | |
1 | 53 | 90 | ||
2 | 59 | 44 |
Parameter | Fx (Pixels) | Fy (Pixels) | Cx (Pixels) | Cy (Pixels) | K1 | K2 | K3 | P1 | P2 | |
---|---|---|---|---|---|---|---|---|---|---|
Sensor 1 | RGB Camera | 592.59 ± 2.23 | 590.48 ± 2.04 | 305.69 ± 1.84 | 235.92 ± 1.30 | 0.019 ± 0.012 | 1.144 ± 0.091 | −3.580 ± 0.291 | −0.004 ± 0.001 | −0.015 ± 0.002 |
IR Camera | 592.49 ± 2.17 | 591.22 ± 2.02 | 301.02 ± 1.76 | 239.9 ± 1.26 | −0.13 ± 0.013 | 1.219 ± 0.131 | −3.305 ± 0.476 | 0.000 ± 0.000 | −0.014 ± 0.001 | |
Sensor 2 | RGB Camera | 579.23 ± 2.22 | 580.29 ± 2.18 | 330.41 ± 1.43 | 242.63 ± 1.15 | 0.061 ± 0.014 | 0.744 ± 0.15 | −3.607 ± 0.512 | 0.005 ± 0.001 | 0.006 ± 0.001 |
IR Camera | 568.57 ± 2.15 | 570.35 ± 2.11 | 323.83 ± 1.37 | 247.41 ± 1.16 | −0.05 ± 0.018 | 0.343 ± 0.248 | −0.982 ± 0.993 | 0.003 ± 0.001 | 0.003 ± 0.001 |
Parameter | Sensor 1 | Sensor 2 |
---|---|---|
dx (mm) | −37.997 ± 0.154 | −32.603 ± 0.101 |
dy (mm) | −3.861 ± 0.117 | 0.431 ± 0.091 |
dz (mm) | −23.170 ± 0.656 | −22.479 ± 0.473 |
Rx (rad) | 0.0079 ± 0.0018 | 0.0110 ± 0.0021 |
Ry (rad) | −0.0024 ± 0.0025 | −0.0044 ± 0.0028 |
Rz (rad) | −0.0053 ± 0.0002 | 0.0013 ± 0.0003 |
Parameter | RGB Camera | IR Camera |
---|---|---|
Fx (pixels) | 566.80 | 566.80 |
Fy (pixels) | 566.80 | 566.80 |
Cx (pixels) | 320 | 320 |
Cy (pixels) | 240 | 240 |
Sensor | In-Factory Calibrated Value | Calibrated Value | ||
---|---|---|---|---|
a | b | a | b | |
1 | −3.38807 × 10−6 | 3.82665 × 10−3 | −3.42936 × 10−6 | 3.86688 × 10−3 |
2 | −3.38649 × 10−6 | 3.82538 × 10−3 | −3.34912 × 10−6 | 3.78253 × 10−3 |
RANSAC Threshold (m) | Recovered Angle (Degrees) | |
---|---|---|
Default Depth | Modeled Depth | |
0.001 | 79.8288 | 89.8004 |
0.002 | 99.8740 | 89.3294 |
0.005 | 91.5966 | 89.9098 |
0.010 | 92.2871 | 90.2850 |
0.020 | 90.4728 | 90.1596 |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Darwish, W.; Tang, S.; Li, W.; Chen, W. A New Calibration Method for Commercial RGB-D Sensors. Sensors 2017, 17, 1204. https://doi.org/10.3390/s17061204
Darwish W, Tang S, Li W, Chen W. A New Calibration Method for Commercial RGB-D Sensors. Sensors. 2017; 17(6):1204. https://doi.org/10.3390/s17061204
Chicago/Turabian StyleDarwish, Walid, Shenjun Tang, Wenbin Li, and Wu Chen. 2017. "A New Calibration Method for Commercial RGB-D Sensors" Sensors 17, no. 6: 1204. https://doi.org/10.3390/s17061204
APA StyleDarwish, W., Tang, S., Li, W., & Chen, W. (2017). A New Calibration Method for Commercial RGB-D Sensors. Sensors, 17(6), 1204. https://doi.org/10.3390/s17061204