# A Flexible Fringe Projection Vision System with Extended Mathematical Model for Accurate Three-Dimensional Measurement

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Principle on Absolute Phase to 3D Coordinates Transformation

## 3. Mathematical Model Extension with Least-Squares Parameter Estimation

#### 3.1. Error Analysis

- (1)
- The uncertainty in the computed model. The simplified geometrical model $f(\xb7)$, which does not consider the lens distortion and lens defocus, results in a systematic error. ${\sigma}_{f(\xb7)}$ denotes the standard deviation (std) of the simplified model.
- (2)
- The uncertainty in the system calibration parameters. ${\sigma}_{{f}_{x}^{c}}$, ${\sigma}_{{f}_{y}^{c}}$, ${\sigma}_{{u}_{0}^{c}}$, and ${\sigma}_{{v}_{0}^{c}}$ denote the std of the estimated camera parameters ${f}_{x}^{c}$, ${f}_{y}^{c}$, ${u}_{0}^{c}$, and ${v}_{0}^{c}$, respectively; ${\sigma}_{{f}^{p}}$ denotes the std of the projector focus length ${f}^{p}$; ${\sigma}_{\mathbf{R}}$ denotes the std of the rotation matrix between the camera and the projector $\mathbf{R}$; and ${\sigma}_{\mathbf{T}}$ denotes the std of the translation matrix between the camera and the projector $\mathbf{T}$.
- (3)
- The uncertainty in the phase map. The nonsinusoity of the fringe pattern will result in a phase map error. ${\sigma}_{\phi}$ denotes the std of the absolute phase map φ; ${\sigma}_{{\phi}_{0}}$ denotes the std of the absolute phase of the projection center ${\phi}_{0}$.

#### 3.2. Mathematical Model Extension

## 4. Realization and Experiments with Single Continuous Objects

#### 4.1. Experimental Setup

#### 4.2. System Calibration

#### 4.3. Derivation of the Extended Mathematical Model Parameters

#### 4.4. Experimental Results

#### 4.5. Performance Comparison with Kinect

## 5. Experiments with Multiple Discontinuous Objects

## 6. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Zhang, S. Recent progresses on real-time 3D shape measurement using digital fringe projection techniques. Opt. Lasers Eng.
**2010**, 48, 149–158. [Google Scholar] [CrossRef] - Wang, Z.; Nguyen, D.A.; Barnes, J.C. Some practical considerations in fringe projection profilometry. Opt. Lasers Eng.
**2010**, 48, 218–225. [Google Scholar] [CrossRef] - Villa, J.; Araiza, M.; Alaniz, D.; Ivanov, R.; Ortiz, M. Transformation of phase to (x,y,z)-coordinates for the calibration of a fringe projection profilometer. Opt. Lasers Eng.
**2012**, 50, 256–261. [Google Scholar] [CrossRef] - Liu, H.; Su, W.-H.; Reichard, K.; Yin, S. Calibration-based phase-shifting projected fringe profilometry for accurate absolute 3D surface profile measurement. Opt. Commun.
**2003**, 216, 65–80. [Google Scholar] [CrossRef] - Huang, L.; Chua, P.S.K.; Asundi, A. Least-squares calibration method for fringe projection profilometry considering camera lens distortion. Appl. Opt.
**2010**, 49, 1539–1548. [Google Scholar] [CrossRef] [PubMed] - Guo, H.; He, H.; Yu, Y.; Chen, M. Least-squares calibration method for fringe projection profilometry. Opt. Eng.
**2005**, 44. [Google Scholar] [CrossRef] - Du, H.; Wang, Z. Three-dimensional shape measurement with an arbitrarily arranged fringe projection profilometry system. Opt. Lett.
**2007**, 32, 2438–2440. [Google Scholar] [CrossRef] [PubMed] - Martinez, A.; Rayas, J.A.; Puga, H.J.; Genovese, K. Iterative estimation of the topography measurement by fringe-projection method with divergent illumination by considering the pitch variation along the x and z directions. Opt. Lasers Eng.
**2010**, 48, 877–881. [Google Scholar] [CrossRef] - Tian, A.; Jiang, Z.; Huang, Y. A flexible new three-dimensional measurement technique by projected fringe pattern. Opt. Laser Technol.
**2006**, 38, 585–589. [Google Scholar] [CrossRef] - Maurel, A.; Cobelli, P.; Pagneux, V.; Petitjeans, P. Experimental and theoretical inspection of the phase-to-height relation in Fourier transform profilometry. Appl. Opt.
**2009**, 48, 380–392. [Google Scholar] [CrossRef] [PubMed] - Wen, Y.; Li, S.; Cheng, H.; Su, X.; Zhang, Q. Universal calculation formula and calibration method in Fourier transform profilometry. Appl. Opt.
**2010**, 49, 6563–6569. [Google Scholar] [CrossRef] [PubMed] - Zhang, S.; Huang, P.S. Novel method for structured light system calibration. Opt. Eng.
**2006**, 45. [Google Scholar] [CrossRef] - Da, F.; Gai, S. Flexible three-dimensional measurement technique based on a digital light processing projector. Appl. Opt.
**2008**, 47, 377–385. [Google Scholar] [CrossRef] [PubMed] - Feng, S.; Chen, Q.; Zuo, C.; Sun, J.; Yu, S.L. High-speed real-time 3-D coordinates measurement based on fringe projection profilometry considering camera lens distortion. Opt. Commun.
**2014**, 329, 44–56. [Google Scholar] [CrossRef] - Pan, B.; Kemao, Q.; Huang, L.; Asundi, A. Phase error analysis and compensation for nonsinusoidal waveforms in phase-shifting digital fringe projection profilometry. Opt. Lett.
**2009**, 34, 416–418. [Google Scholar] [CrossRef] [PubMed] - Yao, J.; Xiong, C.; Zhou, Y.; Miao, H.; Chen, J. Phase error elimination considering gamma nonlinearity, system vibration, and noise for fringe projection profilometry. Opt. Eng.
**2014**, 53. [Google Scholar] [CrossRef] - Zhang, K.; Yao, J.; Chen, J.; Miao, H. Phase extraction algorithm considering high-order harmonics in fringe image processing. Appl. Opt.
**2015**, 54, 4989–4995. [Google Scholar] [CrossRef] [PubMed] - Zhou, P.; Liu, X.; He, Y.; Zhu, T. Phase error analysis and compensation considering ambient light for phase measuring profilometry. Opt. Lasers Eng.
**2014**, 55, 99–104. [Google Scholar] [CrossRef] - Sansoni, G.; Carocci, M.; Rodella, R. Three-dimensional vision based on a combination of gray-code and phase-shift light projection: Analysis and compensation of the systematic errors. Appl. Opt.
**1999**, 38, 6565–6573. [Google Scholar] [CrossRef] [PubMed] - Su, W. Color-encoded fringe projection for 3D shape measurements. Opt. Express
**2007**, 15, 13167–13181. [Google Scholar] [CrossRef] [PubMed] - Wang, Y.; Zhang, S. Novel phase-coding method for absolute phase retrieval. Opt. Lett.
**2012**, 37, 2067–2069. [Google Scholar] [CrossRef] [PubMed] - Xiao, S.; Tao, W.; Zhao, H. An improved phase to absolute depth transformation method and depth-of-field extension. Opt. Int. J. Light Electron Opt.
**2016**, 127, 511–516. [Google Scholar] [CrossRef] - Falcao, G.; Hurtos, N.; Massich, J. Plane-based calibration of a projector-camera system. VIBOT Master.
**2008**, 9, 1–12. [Google Scholar] - Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell.
**2000**, 22, 1330–1334. [Google Scholar] [CrossRef] - Strobl, K.; Hirzinger, G. More accurate pinhole camera calibration with imperfect planar target. In Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 6–13 November 2011; pp. 1068–1075.
- Huang, L.; Zhang, Q.; Asundi, A. Flexible camera calibration using not-measured imperfect target. Appl. Opt.
**2013**, 52, 6278–6286. [Google Scholar] [CrossRef] [PubMed]

**Figure 3.**Experimental setup of the fringe projection vision system: Fringe patterns are projected by the projector. The deformed fringe patterns are captured by the camera. The system calibration is implemented with the help of the calibration board.

**Figure 4.**Calibration fringe patterns: (

**a**) checkerboard for the camera calibration; (

**b**) projected checkerboard for the projector calibration; and (

**c**) projected sinusoidal fringe pattern for the depth calculation.

**Figure 5.**Camera calibration results. (

**a**) Reprojection error and (

**b**) relative positions of the calibration planes based on the camera coordinate system.

**Figure 7.**3D measurement by the fringe projection vision system. (

**a**) 3D point clouds in Matlab and (

**b**) curve of the cross section.

**Figure 9.**3D measurement by the fringe projection vision system. (

**a**) Cylinder and (

**b**) hand surfaces displayed with the Meshlab tool.

**Figure 10.**3D measurement with Kinect. (

**a**) Cylinder and (

**b**) hand surfaces measured with the Meshlab tool.

**Figure 12.**Plot of the sinusoidal fringe pattern with a phase shift of zero that is colored in blue, the phase codeword of the designed fringe pattern that is colored in red, and the wrapped phase that is colored in cyan.

**Figure 13.**Absolute phase map retrieval for spatially isolated objects. (

**a**) Captured designed fringe pattern for multiple discontinuous objects; (

**b**) graph for finding the phase zero-crossing points that include the chosen feature point; and (

**c**) frequency component obtained by an FFT.

**Figure 14.**3D measurement by the fringe projection vision system. (

**a**) Absolute phase map; (

**b**) measured 3D surface for discontinuous objects in Matlab; and (

**c**) measured 3D surface for discontinuous objects, shown in Meshlab tool.

Codeword | Period | Ideal Sinusoidal Fringe Pattern | Captured Sinusoidal Fringe Pattern |
---|---|---|---|

p1 | 2 | 80.5 | 82 ± 1 |

p2 | 3 | 53.2 | 54 ± 0.5 |

p3 | 4 | 40.5 | 41 ± 0.5 |

p4 | 5 | 32.6 | 33.5 ± 0.5 |

p5 | 6 | 27.3 | 28 ± 0.5 |

p6 | 8 | 20.5 | 21 ± 0.5 |

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Xiao, S.; Tao, W.; Zhao, H. A Flexible Fringe Projection Vision System with Extended Mathematical Model for Accurate Three-Dimensional Measurement. *Sensors* **2016**, *16*, 612.
https://doi.org/10.3390/s16050612

**AMA Style**

Xiao S, Tao W, Zhao H. A Flexible Fringe Projection Vision System with Extended Mathematical Model for Accurate Three-Dimensional Measurement. *Sensors*. 2016; 16(5):612.
https://doi.org/10.3390/s16050612

**Chicago/Turabian Style**

Xiao, Suzhi, Wei Tao, and Hui Zhao. 2016. "A Flexible Fringe Projection Vision System with Extended Mathematical Model for Accurate Three-Dimensional Measurement" *Sensors* 16, no. 5: 612.
https://doi.org/10.3390/s16050612