Dynamic Phase Measuring Profilometry Based on Tricolor Binary Fringe Encoding Combined Time-Division Multiplexing

Abstract

A dynamic phase measuring profilometry (PMP) based on tricolor binary fringe combined time-division multiplexing principle is proposed. Only one tricolor binary fringe combined by red (R), green (G), and blue (B) binary fringes with the same fringe width but without any color overlapping one another is needed and sent into the flash memory of a high-speed digital light projector (HDLP) in advance. A specialized time-division multiplexing timing sequence is designed to control the HDLP to project the tricolor binary fringe saved in the flash memory onto the measured dynamic object separately and sequentially at 234 fps, at the same time, the projected light source mode is set as monochrome mode which means that all the RGB LEDs remain lighting. Meanwhile, it also triggers a high frame rate monochrome camera synchronized with the HDLP to capture the corresponding deformed patterns in R, G and B channels. By filtering, the nearly unbroken phase-shifting sinusoidal deformed patterns for three-step PMP can be extracted from the captured deformed patterns. It is equivalent to the three-dimensional (3D) shape reconstruction of the measured dynamic object at 78 fps. Experimental results verify the feasibility and the validity of the proposed method. It is effective for measuring the dynamic object and can avoid the color cross-talk effectively.

1. Introduction

Optical three dimensional (3D) shape measurement based on fringe projection is widely used in many fields such as industry inspection [1], 3D thermal deformation measurement of electronic devices [2], 3D point cloud reconstruction [3], 3D printing [4], face recognition [5] and so on with its advantages of noncontacting measurement, full-field acquisition, high precision, and ease of information processing. The traditional static or online 3D shape measurement methods based on fringe projection may be hard to satisfy the demand of real-time or dynamic 3D shape measurement. With the rapid development of digital light projector (DLP) devices, digital imaging acquisition devices, and personal computers [6,7], many dynamic 3D shape measurement techniques have been proposed.

The existed methods for dynamic 3D shape measurement mainly contain Fourier transform profilometry (FTP) [8,9], single-one color fringe projection profilometry [10], and multiple fringes phase measuring profilometry (PMP) based on high-speed projection system [11,12]. The FTP proposed by Takeda et al. [13] can achieve dynamic 3D measurement [14] because it can reconstruct 3D shape from one deformed pattern, but its measuring accuracy is probably limited due to the filtering procedure. In order to improve the measuring accuracy, the single-one color fringe projection profilometry [15] becomes an effective method. When a color sinusoidal fringe whose red (R), green (G), and blue (B) components comprise three sinusoidal fringes with an equivalent shifting-phase of 2π/3 is projected onto the measured object [16], the corresponding color deformed pattern modulated by the profile of the measured object can be captured by a color charge coupled device (CCD) camera. The corresponding phase-shifting deformed patterns can be retrieved simply from the R, G, and B components of the captured color deformed pattern. So the 3D shape of the measured object can be reconstructed successfully with three-step PMP. However, the color cross-talk and the grayscale imbalance problems caused by the color overlapping among the R, G, and B channels one another may lead to extracting the phase-shifting deformed patterns incompletely [17]. Thus, some corresponding color decoupling compensation and grayscale imbalance correction methods are proposed [18,19,20,21]. Cao et al. proposed an improved RGB tricolor based fast phase measuring profilometry in which the chroma transfer function (CTF) was introduced to correct the color cross-talk and grayscale imbalance problems among R, G, and B channels [22]. This method can reduce the effect of these problems and improve the measuring accuracy, but it may be hard to avoid these problems completely and needs additional calibration experiments. Pan et al. proposed software-based and hardware-based methods to compensate for the color coupling and grayscale imbalance errors by designing a three-CCD camera system and three color filters detect system [20]. It worked well, the measuring system may be complicated due to the additional multiple CCD cameras and color filters. Zou et al. proposed a color fringe-projected technique based on bidimensional empirical mode decomposition to decouple to color cross-talk among color channels [19]. It did reduce the errors caused by the color cross-talk, but its computational process may be complicated and time consuming. Although the above methods can reduce the effect of color cross-talk and grayscale imbalance problems existed in color CCD camera, these methods may be hard to solve the above problems completely and more time consuming.

In order to avoid the color cross-talk problem completely, the multiple fringes PMP based on high-speed projection system [23] is proposed to realize dynamic 3D shape measurement. This method mainly utilized the high-speed digital light projector (HDLP) to project the N frames of phase-shifting sinusoidal fringes onto the measured object at N times of dynamic frame rate. The corresponding phase-shifting deformed patterns can be captured by a synchronous high frame rate monochrome camera. So the 3D shape of the measured object can be reconstructed with N-step PMP. Zhang et al. used the single-chip DLP technology for projecting three coded sinusoidal fringes rapidly and sequentially to realize dynamic 3D shape measurement by removing the color filters onto the color wheel of the projector [24]: the color cross-talk problem can be avoided. However, the operation may be inconvenient because the DLP needed to be reinstalled. With the development of the DLP technology, Zhu and Ma et al. solved the color cross-talk problem by utilizing a special DLP substituted for the ordinary commercial DLP and a high frame rate monochrome camera substituted for the color CCD camera to realize dynamic 3D shape measurement [25,26]. Although the color cross-talk problem can be avoided completely, the grayscale imbalance caused by the monochrome camera’s different sensitivity to R, G, and B light was needed additional correction. Furthermore, the sinusoidal fringe projection speed of the DLP is limited to less than 120 frames per second (fps) due to the grayscale values of the sinusoidal fringe is from 0 to 255 [27]. In order to improve the projection speed of the DLP, the dynamic 3D measurement based on defocusing binary fringe projection [26] is proposed. The binary fringe refresh rate of the DLP can be reach tens of kHz by using the recently developed digital light processing discovery technology [28] due to the binary feature of the binary fringe. The sinusoidal fringe pattern can be approximated by properly defocusing the projected binary fringe [29]. However, the binary fringe defocusing projection method needs an additional defocusing device and its measurement range may be limited by the lesser depth of field caused by defocusing, it may be difficult to calibrating the defocused projector [30,31].

In order to solve the above problems, a dynamic phase measuring profilometry based on tricolor binary fringe encoding combined time-division multiplexing (TDM) principle is proposed.

2. The Tricolor Binary Fringe Encoding Principle

In the traditional phase measuring profilometry (PMP) based on binary fringe projection [32], the traditional binary fringe is shown in Figure 1. The width for non-zero transmittance ($w_1$) of the binary fringe is always encoded the same as that for zero transmittance ($w_0$) in one period ($T_0$) as shown in Figure 1. The corresponding sinusoidal fringe pattern can be approximated by properly defocusing the binary fringe [29] or using the nonlinear error suppression of the large-step PMP based on binary fringe projection [30]. The duty cycle $P_{dc}$ can be expressed as

$$P_{dc}=\frac{w_1}{T_0}$$

(1)

Until now, the duty cycle is always 1/2 used in PMP based on binary fringe projection. In the proposed method, this traditional barrier is broken through. A new duty cycle binary fringe is introduced that means $P_{dc}$ may not be 1/2 but 1/3, meaning that the $w_1$ can be smaller than $w_0$ in one period. It is found that although the duty cycle is not 1/2 in the encoded binary fringe, its fundamental frequency components of Fourier spectrum contains the sinusoidal fringe pattern information. Just by filtering operation in spatial frequency domain, the nearly unbroken sinusoidal fringe pattern can be extracted effectively.

The introduced binary fringe can be modeled as the convolution of the rectangular window function and the comb function. Its gray value $g(x,y)$ can be expressed as

$$g(x,y)=A_{0}rect(\frac{y}{P_{dc}{T}_0})*comb(\frac{y}{T_0})$$

(2)

where $A_0$ is a constant represented as the non-zero grayscale of the introduced binary fringe, * denotes the convolution operation. Its Fourier spectrum $G(f_x,f_y)$ can be expressed as

$$G(f_x,f_y)=A_0{P}_{dc}{T}_0{\displaystyle \sum _{j=-\infty }^{j=\infty }\sin \mathrm{c}}(P_{dc}{T}_0{f}_y)\delta (f_y-j{f}_0)$$

(3)

where $f_0=1/T_0$ denotes the fundamental frequency of the Fourier spectrum. By introducing a proper rectangular window low-pass filter, the zero frequency component, the positive frequency component, and the negative frequency component $G^{\prime }(f_x,f_y)$ can be retained as

$$G^{\prime }(f_x,f_y)=A_0{P}_{dc}{T}_0\{\delta (f_y)+\sin \mathrm{c}(P_{dc})[\delta (f_y+f_0)+\delta (f_y-f_0)]\}$$

(4)

By inverse Fourier transform for $G^{\prime }(f_x,f_y)$, the nearly unbroken sinusoidal fringe pattern $g\prime (x,y)$ can be extracted as

$$g^{\prime }(x,y)=A_0{P}_{dc}{T}_0+2A_0{P}_{dc}{T}_0\sin \mathrm{c}(P_{dc})\cos (\frac{2\pi y}{T_0})$$

(5)

It can be simplified as the mathematic model of PMP [33] and expressed as:

$$g^{\prime }(x,y)=A(x,y)+B(x,y)\cos (\frac{2\pi y}{T_0})$$

(6)

where $A_0{P}_{dc}{T}_0$, simplified as $A(x,y)$, represents the background light intensity, and $2A_0{P}_{dc}{T}_0\sin \mathrm{c}(P_{dc})$, simplified as $B(x,y)$, reflects the contrast of the fringe pattern.

The corresponding sinusoidal fringe pattern extracting process is show in Figure 2. When the encoded binary fringe is projected onto the reference by the HDLP, the fringe pattern is captured by a high frame rate complementary metal oxide semiconductor (CMOS) monochrome camera, as shown in Figure 2a. By fast Fourier transform (FFT), its Fourier spectrum is shown in Figure 2b, it can be seen that although there many higher order harmonic frequencies existed in the frequency domain, the Fourier spectrum distribution is discrete. By introducing a proper rectangular window low-pass filter, the frequency components contained the sinusoidal fringe pattern information as shown in the dotted box in Figure 2b can be retained as shown in Figure 2c. By inverse Fourier transform (IFT), the nearly unbroken sinusoidal fringe pattern (see Figure 2d) can be extracted efficiently. In the same way, the nearly unbroken sinusoidal deformed pattern also can be extracted from the captured deformed pattern when the encoded binary fringe is projected onto the measured object.

Furthermore, the effects of the traditional binary fringe, and the introduced binary fringe on their Fourier spectrums, are analyzed. At the same time, the amplitudes of the Fourier spectrums in binary fringes are also analyzed, from Equation (4), the amplitude $A_{\max }(f_0,P_{dc})$ of the fundamental frequency spectrum in binary fringe can be expressed as

$$A_{\max }(f_0,P_{dc})=A_0{P}_{dc}{T}_0\sin \mathrm{c}(P_{dc})$$

(7)

The traditional 1/2 duty cycle binary fringe and the 1/3 duty cycle binary fringe as shown in Figure 3a,b, respectively, are taken as an example. In this example, their image sizes are all 300 pixels × 300 pixels. By FFT for them, their corresponding Fourier spectrums are shown in Figure 3c,d, respectively; it can be seen that the second order harmonic frequency spectrum vanishes in the traditional binary fringe but exists in the 1/3 duty cycle binary fringe. Although the second order harmonic frequency spectrum exists in the 1/3 duty cycle binary fringe, its proportion is small enough. It means that the shape reconstruction error caused by the second order harmonic frequency spectrum is small enough. According to Equation (7), it can also be seen that the amplitude of the fundamental frequency spectrum in traditional binary fringe is 4700 and that of the 1/3 duty cycle binary fringe is 4000 from Figure 3c,d. But if the zero frequency spectrums and the fundamental frequency spectrums in Figure 3c,d are filtered out respectively by introducing a proper rectangular window low-pass filter as shown in the rectangle box in Figure 3c,d with the above-mentioned process, it can be seen that these retained frequency spectrum components’ distributions contained the sinusoidal fringe information are similar. Then they aligned the same amplitude of the fundamental frequency spectrum by simply multiplying the latter spectrum with 4700/4000, it is found that the aligned frequency spectrums are very similar to those shown in Figure 3e. It reveals that the 1/3 duty cycle binary fringe has the same effect as the traditional binary fringe. By IFT for the aligned frequency spectrums in Figure 3e respectively, the corresponding nearly unbroken sinusoidal fringe patterns can be efficiently extracted from the traditional binary fringe and the 1/3 duty cycle binary fringe respectively. Figure 3f show the cutaway views of the extracted sinusoidal fringe patterns in one columns, it can be seen that the transmittance function of the sinusoidal fringe pattern extracted from the 1/3 duty cycle binary fringe is much close to that of the traditional binary fringe. Furthermore, it also reveals that the reconstructed phase information of the measured object by using the 1/3 duty cycle binary fringe can be close to that of traditional binary fringe.

In order to realize dynamic 3D shape measurement with the introduced binary fringe, a tricolor binary fringe is designed as shown in Figure 4a in which its R, G, and B components are encoded by three monochromatic binary fringes with the same duty cycle of 1/3 but shifted 1/3 period one by one. It can be seen that R, G and B components of the encoded tricolor binary fringe share the same fringe width of 1/3 periods and are independent without any color overlapping in the non-zero regions one by one as shown in Figure 4b–d, so the encoded tricolor binary fringe can avoid the color overlapping problem in the traditional composite color sinusoidal fringe. The mathematic model of the tricolor binary fringe $I_C(x,y)$ can be expressed as

$$I_C(x,y)=I_R(x,y)\overrightarrow{R}+I_G(x,y)\overrightarrow{G}+I_B(x,y)\overrightarrow{B}$$

(8)

where $I_R(x,y)$, $I_G(x,y)$, and $I_B(x,y)$ denote the grayscale distributions of the R, G, and B components in the tricolor binary fringe:

$$\{\begin{array}{l}{I}_R(x,y)=A_{0}rect(\frac{y}{P_{dc}{T}_0})*comb(\frac{y}{T_0})\\ I_G(x,y)=A_{0}rect(\frac{y}{P_{dc}{T}_0})*comb(\frac{y-T_0/3}{T_0})\\ I_B(x,y)=A_{0}rect(\frac{y}{P_{dc}{T}_0})*comb(\frac{y-2T_0/3}{T_0})\end{array}$$

(9)

3. The Dynamic PMP Principle Based on the Tricolor Binary Fringe

The schematic diagram of the dynamic PMP principle based on the proposed tricolor binary fringe is shown in Figure 5. Before measuring, the proposed tricolor binary fringe is encoded and saved into the flash memory of the HDLP in advance. Under the control of a special time-division multiplexing (TMD) timing sequence, the HDLP projects the encoded tricolor binary fringe saved in the flash memory with its R, G, and B channels ($I_R(x,y)$, $I_G(x,y)$, and $I_B(x,y)$) onto the reference plane separately and sequentially. At the same time, the projected light source mode was set as monochrome mode which means that all the RGB LEDs remain lighting mode, so that every frame binary fringe could be projected as a grayscale fringe, effectively avoiding the grayscale imbalance problem caused by the monochrome camera’s different sensitivity to the R, G, and B light. Meanwhile, a high frame rate monochrome CMOS camera synchronized with the HDLP was used to capture the corresponding three fringe patterns $I_{Rref}(x,y)$, $I_{Gref}(x,y)$, and $I_{Bref}(x,y)$ from R, G, and B channels, as shown in Figure 6a. The filtering operation is as discussed in Section 2, the corresponding extracted three nearly unbroken sinusoidal fringe patterns—$I_{FRref}(x,y)$, $I_{FGref}(x,y)$, and $I_{FBref}(x,y)$—can be obtained as shown in Figure 6a. Their cutaway views of the three extracted sinusoidal fringe patterns are shown in Figure 6b, it can be seen that the three extracted sinusoidal fringe patterns have an equivalent shifting-phase of 2π/3 one another. According to the calculation process of Equations (2)–(6), they can be expressed as

$$\{\begin{array}{l}{I}_{FRref}(x,y)=A(x,y)+B(x,y)\cos [\frac{2\pi y}{T}+{\phi }_0(x,y)]\\ I_{FGref}(x,y)=A(x,y)+B(x,y)\cos [\frac{2\pi y}{T}+{\phi }_0(x,y)+\frac{2\pi }{3}]\\ I_{FBref}(x,y)=A(x,y)+B(x,y)\cos [\frac{2\pi y}{T}+{\phi }_0(x,y)+\frac{4\pi }{3}]\end{array}$$

(10)

where $T$ denotes the period of the captured fringe pattern. ${\phi }_0(x,y)$ denotes the phase caused by the reference plane, it can be expressed as

$${\phi }_0(x,y)=\arctan [\frac{\sqrt{3}(I_{FRref}(x,y)-I_{FBref}(x,y))}{2I_{FGref}(x,y)-I_{FRref}(x,y)-I_{FBref}(x,y)}]$$

(11)

As ${\phi }_0(x,y)$ is wrapped in $(-\pi ,\pi ]$ due to the arctan function it should be unwrapped to be ${\Phi }_0(x,y)$ by a phase unwrapping algorithm [34] and saved in the computer in advance.

While measuring, under the same condition above-mentioned, the HDLP projects the encoded tricolor binary fringe in R, G, and B channels ($I_R(x,y)$, $I_G(x,y)$, and $I_B(x,y)$) onto the measured object separately and sequentially at three times of dynamic frame rate, the corresponding three deformed patterns ($I_{DR}(x,y)$, $I_{DG}(x,y)$, and $I_{DB}(x,y)$) from R, G, and B channels can be captured by the high frame rate CMOS monochrome camera synchronized with the HDLP as shown in Figure 5. In the same way as discussed in Section 2, the captured three deformed patterns—$I_{DR}(x,y)$, $I_{DG}(x,y)$, and $I_{DB}(x,y)$—which were processed the corresponding extracted nearly unbroken sinusoidal deformed patterns $I_{FDR}(x,y)$, $I_{FDG}(x,y)$, and $I_{FDB}(x,y)$ can be expressed as

$$\{\begin{array}{l}{I}_{FDR}(x,y)=A(x,y)+B(x,y)\cos [\frac{2\pi y}{T}+\phi (x,y)]\\ I_{FDG}(x,y)=A(x,y)+B(x,y)\cos [\frac{2\pi y}{T}+\phi (x,y)+\frac{2\pi }{3}]\\ I_{FDB}(x,y)=A(x,y)+B(x,y)\cos [\frac{2\pi y}{T}+\phi (x,y)+\frac{4\pi }{3}]\end{array}$$

(12)

where $\phi (x,y)$ is the phase modulated by the measured object located the reference plane; it can also be expressed as

$$\phi (x,y)=\arctan [\frac{\sqrt{3}(I_{FDR}(x,y)-I_{FDB}(x,y))}{2I_{FDG}(x,y)-I_{FDR}(x,y)-I_{FDB}(x,y)}]$$

(13)

It also should be unwrapped to $\Phi (x,y)$. The phase $\mathsf{\Psi }(x,y)$, which s modulated by the height of the measured object, is the difference $\Phi (x,y)$ and ${\Phi }_0(x,y)$. $\mathsf{\Psi }(x,y)$ can be expressed as

$$\mathsf{\Psi }(x,y)=\Phi (x,y)-{\Phi }_0(x,y)$$

(14)

So the 3D shape of the measured object is reconstructed by phase-to-height mapping relationship [35] as

$$\frac{1}{h(x,y)}=a(x,y)+b(x,y)\frac{1}{\mathsf{\Psi }(x,y)}+c(x,y)\frac{1}{{\mathsf{\Psi }}^2(x,y)}$$

(15)

where the system constants $a(x,y)$, $b(x,y)$, and $c(x,y)$ can be calibrated by several planes with known heights. Furthermore, due to the phase of the binary fringe will be slightly changed during the FFT process, it may introduce a phase error. However, this phase error is the system phase error, it can be eliminated by the three-step PMP algorithm used in the proposed method.

The corresponding reconstruction process of the proposed method is shown in Figure 7. The captured three deformed patterns, $I_{DR}(x,y)$, $I_{DG}(x,y)$, and $I_{DB}(x,y)$, from R, G, and B channels were processed as outlined in the above-mentioned process. By FFT, their Fourier spectrums (R spectrum, G spectrum, and B spectrum) are shown in Figure 7. After proper filtering, their zero frequency components and the fundamental frequency components as shown in the dotted rectangle regions are extracted respectively. By IFT, the corresponding three nearly unbroken sinusoidal deformed patterns with an equivalent shifting-phase of 2π/3 one another can be retrieved successfully. The wrapped phase modulated by the measured object can be calculated with the three deformed patterns combined three-step PMP and the 3D shape of the measured object can be reconstructed successfully as above-mentioned method. Due to the binary feature of the encoded binary fringe, the projection fringe refresh rate of the HDLP can be improved greatly, so the proposed method can be used to reconstruct the 3D shapes of the real-time changing or dynamic object.

The specialized TDM timing sequence is designed as shown in Figure 8. It can be generated by a microcontroller signal circuit which can actively control the HDLP and the high frame rate CMOS monochrome camera synchronously [36] in our research group. The frame rate of the HDLP for a 1-bit image can be 4225 fps while that of the high frame rate CMOS monochrome camera can be reach 337 fps. When the synchronized microcontroller signal is at the rising edge of the projector trigger signal, the binary fringe data $I_R(x,y)$ saved in the memory of the HDLP is output and completely projected onto the real-time changing or dynamic object by the HDLP at 1/500s with its monochrome light projection mode, the corresponding high frame rate CMOS monochrome camera synchronized with the HDLP starts to integrate at the rising edge of the camera trigger signal until it completes the process of capturing the current deformed pattern from R channel and the corresponding deformed pattern was $I_{DR}(x,y)$ is effectively saved into the personal computer (PC) by the universal serial bus in one period (1/234s). Then the HDLP refreshes its digital micromirror device (DMD) and starts projecting the next two binary fringes $I_G(x,y)$ and $I_B(x,y)$ in the next two periods, respectively, with the above projection process, the corresponding two deformed patterns ($I_{DG}(x,y)$ and $I_{DB}(x,y)$) from G and B channel can be captured and saved into the PC. Circularly, under the active control of the synchronized microcontroller signal, the three fringes ($I_R(x,y)$, $I_G(x,y)$, and $I_B(x,y)$) of the encoded tricolor binary fringe were projected onto the measured real-time changing or dynamic object at 234 fps and the corresponding three deformed patterns ($I_{DR}(x,y)$, $I_{DG}(x,y)$, and $I_{DB}(x,y)$) were captured synchronously. So the corresponding each group of the deformed patterns of the measured real-time changing or dynamic objects in different states can be captured at 78 fps. It is guaranteed that the sufficient 3D shape information of dynamic objects can be obtained at 78 fps with the proposed method. Furthermore, the color cross-talk problem can be avoided completely because the R, G, and B components of the encoded tricolor binary fringe are projected at different times.

4. Experiments and Analysis

The experimental system of the proposed method as shown in Figure 9 is set up by a HDLP (DLP light crafter 4500 with resolution of 912 pixel × 1140 pixel) and a high frame rate CMOS monochrome camera (HXC20 with frame rate of 234 fps for 800 pixel × 600 pixel image). The camera uses an 8 mm focal length lens (SE0813-2). Due to the DLP light crafter 4500 (DLP 4500) can control its R, G, and B LEDs on/off individually in each channel, which is independent of the image refresh timing sequence. Before projecting fringe, the R, G, and B LEDs of the DLP 4500 are set to lighting on mode in each channel and remain independent of the tricolor binary fringe projecting timing sequence. That is the projecting light source is set to be monochrome projection mode. So that the every frame binary fringe in R, G, and B channels of the tricolor binary fringe can be projected separately in sequence as a grayscale fringe combined TDM principle. Its projection frame rate can be 4225 fps for 1-bit binary fringe. In the experiment, the period of the proposed tricolor binary fringe is encoded 24 pixels, its R, G, and B components share the same duty cycle of 1/3 but shifted 1/3 periods (8 pixels). The obtained 3D shape information data is implemented on a computer with configuration of Intel(R) core (TM) i5-7500 CPU @3.40 GHz and 8 GB extendible physical memory.

4.1. Experiments for Measuring Static Object

In order to verify the effectiveness of the proposed method, many experiments were conducted. One of them is to measuring a crab model as shown in Figure 10a. Figure 10b shows the captured three deformed patterns from the R, G, and B channels of the projected tricolor binary fringe. The corresponding Fourier spectrum is shown in Figure 10c, it can be seen that the higher frequency components modulated by the measured object are hardly overlapped with the higher order harmonic components of the binary fringe. By filtering, the regions of interest (ROI) frequency components as shown in the dotted rectangle region as shown in Figure 10c can be retained. By IFT, the corresponding three nearly unbroken sinusoidal deformed patterns are extracted as shown in Figure 10d; Figure 10e,f shows the corresponding wrapped phase and the reconstructed 3D shape result of the measured crab model with the proposed method. It can be seen that the surface of the measured crab model can be well reconstructed. If the measured object is too complex to extract the effective sinusoidal deformed pattern from the captured deformed pattern due to the over spectrum overlapping between the higher frequency components modulated by the measured object and the higher order harmonic components of the binary fringe, the proposed method may be limited. Although the proposed method may be limited to measuring the overcomplex objects to some extent, it can reconstruct the 3D shapes of most objects with relative complex surfaces.

4.2. Accuracy Analysis

Due to the filtering operation used in the proposed method, in order to verify the measuring accuracy of the proposed method objectively, several compared experiments were conducted among FTP [13] based on a well-known FFT technique in which the filtering operation is used, Zhu’s method [25] (Zhu), and the proposed method (Prop). One of them is to measuring a “heart” model as shown in Figure 11a. Figure 11b shows one of the three captured deformed patterns with the proposed method and the corresponding extracted nearly unbroken sinusoidal deformed pattern is shown in Figure 11c. Figure 11d shows one of the three captured sinusoidal deformed patterns with Zhu’s method. Figure 11e shows the reconstructed 3D shape result with FTP from Figure 11c. Figure 11f shows the reconstructed 3D shape result with Zhu’s method from the three captured sinusoidal deformed patterns. Figure 11g shows the reconstructed 3D shape result with the proposed method from the extracted three deformed patterns as shown in Figure 11c. It can be seen that there some ripples on the surface of the reconstructed 3D shape with FTP, but the surface of the reconstructed 3D shape with Prop is smooth and close to that with Zhu. In order to further analyze the results in detail, the cutaway views at 140 columns from 1 to 255 rows of Figure 11e–g are shown in Figure 11h, the corresponding zoom in of “ear” part (the dotted circle region, A) and the smooth slope part (the dotted rectangle region, B) are shown in Figure 11i–j, respectively; it can be seen that the “ear” part with FTP is lost but well reconstructed with the proposed method and close to that of Zhu. It can be seen that if we directly used one of the three extracted nearly unbroken sinusoidal deformed patterns to reconstruct the 3D shape of the measured object with well-known FTP, it may lead to some measuring errors caused by filtering operation. However, in our proposed method, all the three extracted nearly unbroken sinusoidal deformed patterns are used to reconstruct the 3D shape of the measured object. The measuring errors can be improved by the constraint of multiple frames deformed patterns and the complementarity of the three extracted phase-shifting deformed patterns. It also can be seen that even though in the smooth region, the reconstructed surface of the proposed method is more close to that of Zhu than that of FTP. So the measuring accuracy of the proposed method is higher than that of FTP and close to that of Zhu.

In order to further analyze the measuring accuracy of the proposed method, lots of planes with known heights are measured.

Table 1. Measuring results for different planes with known heights (mm).

h0	3.00	5.00	9.00	14.00	25.00	35.00
havg	2.945	5.060	8.965	13.950	24.943	34.926
MAE	0.065	0.070	0.074	0.060	0.078	0.082
RMS	0.034	0.041	0.037	0.033	0.042	0.045

shows the measuring results of six planes with known heights of 3.00 mm, 5.00 mm, 9.00 mm, 14.00 mm, 25.00 mm, and 35.00 mm. $h_0$ denotes the measured plane with known height and $h_{avg}$ denotes the mean measuring height. The MAE denotes the mean absolute error and the RMS denotes the root of mean square error. It can be seen that the MAE is less than 0.082 mm and the RMS is less than 0.045 mm, so the proposed method has higher measuring accuracy.

At the same time, in order to show the details of the above measured planes used for calibration, the reconstructed result for measuring the plane with height of 5.00 mm is shown in Figure 12a, it can be seen that the reconstructed 5.00 mm plane’s surface is relatively much smooth with the proposed method. In order to show the accuracy of the reconstructed 5.00 mm plane in detail, the corresponding cutaway view at 150 column from 1 to 300 rows in Figure 12a is shown in Figure 12b, it can be seen that the height of the reconstructed 5.00 mm plane is between 4.925 mm and 5.077 mm which is consistent with the measuring result of 5.00 mm plane in Table 1: it effectively verified that the proposed method has higher measuring accuracy.

4.3. The Experiments for Measuring Dynamic Object

In order to verify the effectiveness and the practicality of the proposed method for measuring the 3D shapes of dynamic object, several experiments are conducted. One of them is to measuring the swinging “heart” model. The deformed patterns of the swinging “heart” model, in different states, are captured at 78 fps. The corresponding reconstructed 3D shapes of the swinging “heart” model are reconstructed with the proposed method. Figure 13a–c show three states (state 1, state 2, and state 3) of the captured deformed patterns from R channel. The corresponding reconstructed 3D shapes at above three states are shown in Figure 13d–f. It can be seen that the 3D shapes of the measured swinging “heart” model at different states can be also well reconstructed with the proposed method. Therefore, the proposed method is effective for measuring the dynamic objects. If the higher frame rate camera and HDLP are used, the deformed patterns contained the 3D shape information of the measured dynamic object will be obtained at higher speed, the proposed method has potential application in superfast, dynamic 3D shape measurement. Although, with the additional Fourier transform, low-pass filtering, and inverse Fourier transform used in the proposed method, which may be for more time consuming, the proposed method can realize dynamic 3D shape information acquisition. If the graphic processing unit (GPU) acceleration calculation technique [37] is used in the proposed method, it will make the proposed method realize the real-time dynamic 3D shape reconstruction which is into our further research.

5. Conclusions

A dynamic PMP based on the tricolor binary fringe combined time-division multiplexing (TDM) principle is proposed. In the proposed method, only one tricolor binary fringe, designed by three monochromatic binary fringes with the same fringe width but shifted 1/3 periods one another in R, G, and B channels, is needed; no defocusing projection is needed. The tricolor binary fringe is easy to be designed and encoded. Under the control of a special TMD timing sequence, the HDLP projects the tricolor binary fringe as three grayscale fringes onto the measured dynamic object separately and sequentially at 234 fps with the monochrome light projection mode, the high frame rate monochrome camera synchronized with the HDLP to capture the corresponding deformed patterns in R, G, and B channels. It guaranteed that sufficient 3D shape information of measured real-time changing or dynamic object in different states can be obtained at 78 fps. Just by FFT, filtering and IFT operations, the three nearly unbroken sinusoidal deformed patterns with an equivalent shifting-phase of 2π/3 extracted from the captured deformed patterns can be used to reconstruct the 3D shape of dynamic object with three-step PMP. The experimental results proved the effectiveness and the practicability of the proposed method for measuring real-time changing or dynamic object. The proposed method can effectively avoid the color cross-talk problem and the grayscale imbalance problem completely. It can also greatly improve the refresh frame rate of the HDLP. We must admit that the proposed method may be limited to measure objects with sharp edges.