Intensity-Averaged Double Three-Step Phase-Shifting Algorithm with Color-Encoded Fringe Projection

: Fringe projection proﬁlometry (FPP) has been broadly employed for three-dimensional shape measurements. However, the measurement accuracy suffers from gamma nonlinearity. This paper proposes an intensity-averaged double three-step phase-shifting (IDTP) algorithm making use of color-encoded fringe projection, which does not require complex calibration processes or extra fringe patterns. Speciﬁcally, two phase maps with π /2 phase shift are encoded into the red and blue channels of color fringe patterns. The average fringe patterns of the red and blue channels are approximately in sinusoidal waveform with little harmonics, thus can be directly used for accurate phase recovery. Additionally, an adaptive weight is also estimated for average operation to suppress the effect of color crosstalk. Both simulations and experiments demonstrate that the proposed IDTP algorithm can effectively eliminate nonlinear phase errors.


Introduction
Non-contact optical 3D surface imaging techniques have been extensively applied into various fields, such as mechanical engineering, industry inspection, and biomedical science [1][2][3]. As an important 3D surface imaging technique, FPP is one of the research hotspots and attracts much attention [4][5][6]. In general, a standard FPP system consists of a projection unit and an acquisition unit, which requires to project one or more designed fringe patterns onto the measured objects, capture the deformed fringe patterns of the measured objects, and extract the phase distribution for 3D reconstruction. Phase-shifting algorithms that require three or more fringe patterns are usually used for phase extraction because of high accuracy and high robustness [7][8][9][10]. However, the gamma nonlinearity will distort the intensities of the fringe patterns, then lead to nonlinear phase errors into the extracted phase distribution, and further result in 3D measurement errors [11].
Many methods have been developed for gamma correction during the past two decades [12][13][14][15][16]. Some methods require some complex calibration processes. For instance, Zhang et al. [17] established a look-up table between the wrapped phase and the nonlinear phase error. Hoang et al. [18] combined three-step and large-step phase-shifting algorithms to calibrate the gamma value. Liu et al. [19] constructed an intensity pre-compensation look-up table to modify the projected fringe patterns. Zhang et al. [20] firstly calibrated the amplitudes of harmonics, and then added specific harmonics into the projected fringe patterns. Yu et al. [21] used the probability distribution of the wrapped phase to estimate the gamma value. Subsequently, Liu et al. [22] also used the phase probability distribution to estimate the harmonic coefficients for phase error compensation. However, the requirement of the calibration processes limits the flexibility.
Some other methods require extra fringe patterns. For instance, Huang et al. [23] proposed a phase-averaged double three-step phase-shifting (PDTP) algorithm that used two groups of fringe patterns to calculate two phase maps with π/3 phase shift. Since two phase maps have nearly opposite errors, the phase error could be reduced from the average phase map. Based on the similar model, several double N-step phase-shifting algorithms have been developed to average two phase maps with π/N phase shift for phase error reduction [24][25][26]. Nevertheless, the measurement speed decreases because of the requirement of extra fringe patterns. Alternatively, Cai et al. [27,28] used Hilbert transform on the original fringe patterns to generate extra fringe patterns, and then their phase maps with opposite errors were averaged for gamma correction. However, the Hilbert transform-based method is sensitive to sharp edges or discontinuous parts.
To address the above problem, this paper proposes an effective IDTP algorithm based on color-encoded fringe projection without requiring complex calibration processes or extra fringe patterns. Two phase maps with π/2 phase shift are encoded into the red and blue channels of color fringe patterns. Differing from the PDTP algorithm that averages two phase maps with π/N phase shift for phase error compensation, the IDTP algorithm directly averages the intensities of two fringe patterns with π/2 phase shift and recovers the accurate phase map from the average fringe patterns. Additionally, to suppress the effect of color crosstalk, we also estimate an adaptive weight for the average operation. Both simulations and experiments have been carried out, and their results demonstrate the efficiency of the proposed IDTP algorithm.

Phase-Averaged Double Three-Step Phase-Shifting (PDTP) Algorithm
Three-step phase-shifting algorithm only requires three fringe patterns, and is sensitive to the gamma nonlinearity. For the three-step phase-shifting algorithm, the captured fringe patterns in sinusoidal waveform can be mathematically described as: where n = 1, 2, 3; δ 1 = 0, δ 2 = 2π/3, and δ 3 = 4π/3 are the phase shifts; A(x, y) is the average intensity; B(x, y) is the intensity modulation; and φ(x, y) is the phase relating to the 3D surface information. For convenience, the image coordinate (x, y) will be omitted in the following. Solving the above equations, we can compute the wrapped phase as: Note that the phase will be wrapped when we use the above equation that contains the arc tangent function. However, the captured fringe patterns will be distorted due to the gamma nonlinearity of the projector or the camera. The distorted fringe patterns in a non-sinusoidal waveform can be described as: where γ is the gamma value. Based on the Fourier series expansion, the distorted fringe patterns can be regarded as the ideal fringe patterns with harmonics as: where a m is the amplitude of the m-th harmonic. In general, as the harmonic order increases, the amplitude a m decreases [29]. Based on three-step phase-shifting algorithm, we can compute the distorted phase as: Since the amplitudes of high-order harmonics are usually very small, we can ignore m ≥ 6 harmonics, and the distorted phase can be further approximated as: φ ≈ tan −1 a 1 sin (φ) − a 2 sin (2φ) + a 4 sin (4φ) − a 5 sin (5φ) a 1 cos (φ) + a 2 cos (2φ) + a 4 cos (4φ) + a 5 cos (5φ) (6) Then, the nonlinear phase error can be approximated as: where c 1 and c 2 are two constants. It can be found that the frequency of the nonlinear phase error is three times that of the wrapped phase. Based on this characteristic, Huang et al. [23] proposed the PDTP algorithm for phase error compensation. The basic idea is that, if we shift the original phase by π/3 to obtain the shifted phase, the nonlinear phase error of the shifted phase can be approximated as: It is evident that the original phase and the shifted phase have approximate opposite errors ∆φ ≈ −∆φ . Therefore, the nonlinear phase errors can be compensated by simply averaging the original phase and the shifted phase. Without increasing the number of required fringe patterns, two phase maps with π/3 phase shift are encoded into the red and blue channels of three color fringe patterns to conduct the PDTP algorithm in this paper for comparison.

Intensity-Averaged Double Three-Step Phase-Shifting (IDTP) Algorithm
Instead of averaging two phase maps with π/3 phase shift for gamma correction, this paper proposes an effective IDTP algorithm that directly averages two groups of fringe patterns for accurate phase recovery. Figure 1 shows the principle of the proposed IDTP algorithm. To be specific, two phase maps with π/2 phase shift are encoded into the red and blue channels of color fringe patterns, which can be described as:   where the subscripts r and b denote red and blue channels, respectively. The green channel carries no information, and we set I n,g = 0. The red and blue channels of the distorted fringe patterns can be, respectively, expanded into Fourier series as: Figure 2a,b illustrates the red and blue channels of the distorted fringe patterns in a non-sinusoidal waveform. Therefore, the average fringe patterns with harmonics can be computed as: where a m = a m cos (mπ/4) can be also regarded as the amplitude of the m-th harmonic, and φ a = φ − π/4. Clearly, the average fringe patterns described by Equation (13) have a similar Fourier series form as the distorted fringe patterns described by Equation (4). As illustrated in Figure 2c, the average fringe patterns have an approximate sinusoidal waveform. If we continue to use three-step phase-shifting algorithm to compute the wrapped phase, the nonlinear phase error can be approximated as: can be also regarded as the amplitude of the m -th harmo and / 4 a φ φ = −π . Clearly, the average fringe patterns described by Equation (13) ha similar Fourier series form as the distorted fringe patterns described by Equation (4) illustrated in Figure 2c, the average fringe patterns have an approximate sinusoidal w form. If we continue to use three-step phase-shifting algorithm to compute the wrap phase, the nonlinear phase error can be approximated as:  Substituting m = 1, 2, 4, 5 into a m = a m cos (mπ/4), we can obtain: As for the average fringe pattern, the first order harmonic a 1 and the fifth order harmonic a 5 are smaller than that of the red channel of the distorted fringe patterns; the second order harmonic a 2 is zero; and the forth order harmonic a 4 is opposite to that of the Photonics 2022, 9, 173 5 of 11 red channel of the distorted fringe patterns. Substituting Equation (15) into Equation (14), we can rewrite the nonlinear phase error as: The nonlinear phase error was greatly reduced since the second harmonic disappears from the numerator. Figure 2d illustrates the undistorted phase, and the distorted phase of the red channel and the blue channel, and the corrected phase of their average fringe, respectively. Clearly, the corrected phase is very close to the undistorted phase. Figure 2e illustrates the corresponding nonlinear phase errors of the red channel, the blue channel, and their average fringe, respectively. It is evident that the ∆φ a of I n,a is much smaller than the ∆φ r of I n,r or the ∆φ b of I n,b , which theoretically validates the success of the proposed IDTP algorithm.

Influnce of the Color Crosstalk
However, most color projectors and color cameras are usually designed to have some spectra overlaps between color channels to avoid blind regions in the spectrum, which will change the intensity distribution of color fringe patterns [30][31][32]. This phenomenon is often referred as color crosstalk or color coupling. In general, the color fringe patterns with color crosstalk can be expressed as: where κ ij (i, j = r, g, b) are color crosstalk factors, and κ rb , κ br are usually far less than κ rr , κ bb . Because the green channel is I n,g = (I n,g ) γ = 0, the red and blue channels of the color fringe patterns with color crosstalk can be further expressed as: I n,r = κ rr I n,r + κ br I n,b To suppress the effect of color crosstalk, we introduce an adaptive weight α for I n,b . Therefore, the average fringe patterns of I n,r and αI n,b can be calculated as: If we assume that η = κ rr + ακ rb = κ br + ακ bb , the average fringe patterns with harmonics can be described as: It can be concluded that the nonlinear phase error ∆φ a of I n,a will be equal to the nonlinear phase error ∆φ a of I n,a because I n,a is proportional to I n,a , and ∆φ a can be also described as Equation (16). The key is how to correctly determine the adaptive weight α. In general, κ rr , κ bb κ rb , κ br ≈ 0, and we can estimate the adaptive weight α as: Especially, if no color crosstalk is considered, or κ rb = κ br = 0, we have α = κ bb /κ rr .

Simulations
Some simulations were carried out to compare the proposed IDTP algorithm and the conventional PDTP algorithm. The peaks function was used to simulate the measured object. The resolution of the captured fringe patterns was set to 450 × 450 pixels. The period of the fringe patterns was set to 150 pixels. The IDTP algorithm was implemented by encoding two phase maps with π/2 phase shift into the red and blue channels of color fringe patterns. For comparison, the PDTP algorithm was also implemented by encoding two phase maps with π/3 phase shift into the red and blue channels of the color fringe patterns. Note that the adaptive weight was assumed to be same at all pixels of the measured object, and thus its mean value was calculated to suppress the effect of color crosstalk.

Ignoring Color Crosstalk
Firstly, the color crosstalk was ignored, and we set κ rr = κ bb = 1, κ rb = κ br = 0, and γ = 2.5. Figure 3a shows the color fringe generated by the IDTP algorithm. Figure 3b-d show the phase errors of the red channel, the blue channel, and the average fringe, respectively. To quantify the accuracy, we further computed the root-mean-square (Rms) phase errors of Figure 3b-d, which were about 0.2406, 0.2397, and 0.0024 rad, respectively. The Rms phase error of Figure 3d is far less than that of Figure 3b,c. In contrast, Figure 3e shows the color fringe generated by the PDTP algorithm. Figure 3f-h shows the corresponding phase errors of the red channel, the blue channel, and the average phase, respectively. We also computed the Rms phase error of Figure 3f-h, which were about 0.2406, 0.2427, and 0.0390 rad, respectively. Similarly, the Rms phase error of Figure 3h is far less than that of Figure 3f,g, but higher than that of Figure 3d. These results validate that the proposed IDTP algorithm can eliminate the nonlinear phase error and achieve higher accuracy than the conventional PDTP algorithm.

Simulations
Some simulations were carried out to compare the proposed IDTP algorithm and the conventional PDTP algorithm. The peaks function was used to simulate the measured object. The resolution of the captured fringe patterns was set to 450 × 450 pixels. The period of the fringe patterns was set to 150 pixels. The IDTP algorithm was implemented by encoding two phase maps with π/2 phase shift into the red and blue channels of color fringe patterns. For comparison, the PDTP algorithm was also implemented by encoding two phase maps with π/3 phase shift into the red and blue channels of the color fringe patterns. Note that the adaptive weight was assumed to be same at all pixels of the measured object, and thus its mean value was calculated to suppress the effect of color crosstalk.

Ignoring Color Crosstalk
Firstly, the color crosstalk was ignored, and we set = =1 . Figure 3a shows the color fringe generated by the IDTP algorithm. Figure 3b-d show the phase errors of the red channel, the blue channel, and the average fringe, respectively. To quantify the accuracy, we further computed the root-mean-square (Rms) phase errors of Figure 3b-d, which were about 0.2406, 0.2397, and 0.0024 rad, respectively. The Rms phase error of Figure 3d is far less than that of Figure 3b,c. In contrast, Figure 3e shows the color fringe generated by the PDTP algorithm. Figure 3f-h shows the corresponding phase errors of the red channel, the blue channel, and the average phase, respectively. We also computed the Rms phase error of Figure 3f-h, which were about 0.2406, 0.2427, and 0.0390 rad, respectively. Similarly, the Rms phase error of Figure 3h is far less than that of Figure 3f,g, but higher than that of Figure 3d. These results validate that the proposed IDTP algorithm can eliminate the nonlinear phase error and achieve higher accuracy than the conventional PDTP algorithm.

Considering Color Crosstalk
Secondly, the color crosstalk was considered, and we set κ rr = 0.8, κ bb = 0.9, κ rb = κ br = 0.1, and γ = 2.5. Figure 4a shows the color fringe generated by the IDTP algorithm. Figure 4b-d show the phase errors of the red channel, the blue channel, and the average fringe, respectively, and the Rms phase errors are about 0.2087, 0.2114, and 0.0045 rad, respectively. Figure 4e shows the color fringe generated by the PDTP algorithm. 0.0505 rad, respectively. These results confirm the accuracy of the proposed IDTP algorithm compared with the conventional PDTP algorithm.

Considering Color Crosstalk
Secondly, the color crosstalk was considered, and we set . Figure 4a shows the color fringe generated by the IDTP algorithm. Figure 4b-d show the phase errors of the red channel, the blue channel, and the average fringe, respectively, and the Rms phase errors are about 0.2087, 0.2114, and 0.0045 rad, respectively. Figure 4e shows the color fringe generated by the PDTP algorithm. Figure 4f-h shows the corresponding phase errors of the red channel, the blue channel, and the average phase, respectively, and the Rms phase errors are about 0.2118, 0.2168, and 0.0505 rad, respectively. These results confirm the accuracy of the proposed IDTP algorithm compared with the conventional PDTP algorithm.

Different Gamma Values
Moreover, we also explored the accuracy of the IDTP algorithm and the PDTP algorithm under different gamma values, [1,4]  . Figure 5a illustrates the Rms phase errors of the IDTP algorithm. In contrast, Figure 5b illustrates the Rms phase errors of the PDTP algorithm. It is evident that the phase errors are almost proportional to the gamma value for both the IDTP algorithm and the PDTP algorithm. For the IDTP algorithm, the Rms phase error of the average fringe is very small, below 0.02 rad, even 4.0 γ = . In contrast, for the PDTP algorithm, the Rms phase error of the average phase is larger, and close to 0.1 rad when 4.0 γ = . These results confirm that the proposed IDTP algorithm can achieve higher accuracy than the conventional PDTP algorithm under different gamma values.

Different Gamma Values
Moreover, we also explored the accuracy of the IDTP algorithm and the PDTP algorithm under different gamma values, γ = [1,4]. The color crosstalk was set as κ rr = 0.8, κ bb = 0.9, κ rb = κ br = 0.1. Figure 5a illustrates the Rms phase errors of the IDTP algorithm. In contrast, Figure 5b illustrates the Rms phase errors of the PDTP algorithm. It is evident that the phase errors are almost proportional to the gamma value for both the IDTP algorithm and the PDTP algorithm. For the IDTP algorithm, the Rms phase error of the average fringe is very small, below 0.02 rad, even γ = 4.0. In contrast, for the PDTP algorithm, the Rms phase error of the average phase is larger, and close to 0.1 rad when γ = 4.0. These results confirm that the proposed IDTP algorithm can achieve higher accuracy than the conventional PDTP algorithm under different gamma values.

Experiments
To further compare the proposed IDTP algorithm and the conventional PDTP algorithm, some experiments were also carried out. A standard FPP system, including a DLP projector and a CMOS camera, was built up. The projector was Light-Crafter 4500 with a resolution of 912 × 1140 pixels. The camera was Basler a2A1920-160ucBAS with a resolution of 1920 × 1200 pixels. The lens mounted on the front of the camera was Computar M1214-MP2 with focal length of 12 mm. The period of the fringe patterns was set to 21 pixels. A three-step phase-shifting algorithm was used for the wrapped phase calculation, and three-frequency phase unwrapping algorithm was used for absolute phase recovery.
Firstly, a white sphere with smooth surface was measured. Color fringe patterns were generated by the IDTP algorithm and the PDTP algorithm, and then they were projected

Experiments
To further compare the proposed IDTP algorithm and the conventional PDTP algorithm, some experiments were also carried out. A standard FPP system, including a DLP projector and a CMOS camera, was built up. The projector was Light-Crafter 4500 with a resolution of 912 × 1140 pixels. The camera was Basler a2A1920-160ucBAS with a resolution of 1920 × 1200 pixels. The lens mounted on the front of the camera was Computar M1214-MP2 with focal length of 12 mm. The period of the fringe patterns was set to 21 pix-els. A three-step phase-shifting algorithm was used for the wrapped phase calculation, and three-frequency phase unwrapping algorithm was used for absolute phase recovery.
Firstly, a white sphere with smooth surface was measured. Color fringe patterns were generated by the IDTP algorithm and the PDTP algorithm, and then they were projected by the projector and captured by the camera. Figure 6a shows the distorted color fringe for the IDTP algorithm. Note that the adaptive weight was calculated by pixel to suppress the effect of color crosstalk. Figure 6b-d shows the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. Clearly, the absolute phases shown in Figure 6b,c contain severe ripple errors, which are mainly caused by gamma nonlinearity. In contrast, the absolute phase shown in Figure 6d contains invisible ripple errors. In contrast, Figure 6e shows the distorted color fringe for the PDTP algorithm. Figure 6f-h shows the absolute phases recovered from the red channel, the blue channel, and the average phase, respectively. Similarly, there are serious ripple errors in Figure 6f,g, and little ripple errors in Figure 6h. For the sake of clarity, Figure 7 illustrates the cross section of the recovered absolute phases of the sphere. The absolute phase recovered from the IDTP algorithm is smoother than that of the PDTP algorithm. It is evident that the proposed IDTP algorithm can effectively correct the gamma nonlinearity.

Experiments
To further compare the proposed IDTP algorithm and the conventional PDTP algorithm, some experiments were also carried out. A standard FPP system, including a DLP projector and a CMOS camera, was built up. The projector was Light-Crafter 4500 with a resolution of 912 × 1140 pixels. The camera was Basler a2A1920-160ucBAS with a resolution of 1920 × 1200 pixels. The lens mounted on the front of the camera was Computar M1214-MP2 with focal length of 12 mm. The period of the fringe patterns was set to 21 pixels. A three-step phase-shifting algorithm was used for the wrapped phase calculation, and three-frequency phase unwrapping algorithm was used for absolute phase recovery.
Firstly, a white sphere with smooth surface was measured. Color fringe patterns were generated by the IDTP algorithm and the PDTP algorithm, and then they were projected by the projector and captured by the camera. Figure 6a shows the distorted color fringe for the IDTP algorithm. Note that the adaptive weight was calculated by pixel to suppress the effect of color crosstalk. Figure 6b-d shows the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. Clearly, the absolute phases shown in Figure 6b,c contain severe ripple errors, which are mainly caused by gamma nonlinearity. In contrast, the absolute phase shown in Figure 6d contains invisible ripple errors. In contrast, Figure 6e shows the distorted color fringe for the PDTP algorithm. Figure 6f-h shows the absolute phases recovered from the red channel, the blue channel, and the average phase, respectively. Similarly, there are serious ripple errors in Figure 6f,g, and little ripple errors in Figure 6h. For the sake of clarity, Figure 7 illustrates the cross section of the recovered absolute phases of the sphere. The absolute phase recovered from the IDTP algorithm is smoother than that of the PDTP algorithm. It is evident that the proposed IDTP algorithm can effectively correct the gamma nonlinearity. Secondly, a white sculpture was also measured. For the IDTP algorithm, Figure 8a-d shows the corresponding distorted color fringe, and the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. For the PDTP algorithm, Figure 8e-h shows the corresponding distorted color fringe, and the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. Figure 9 illustrates the cross section of the recovered absolute phases of the sculpture. The experiment results confirm the performance of the proposed IDTP algorithm compared with the conventional PDTP algorithm. Figure 6. Comparison results of the sphere. (a) Distorted color fringe for the proposed IDTP algorithm, and the absolute phases recovered from (b) the red channel, (c) the blue channel, and (d) the average fringe, respectively. (e) Distorted color fringe for the conventional PDTP algorithm, and the absolute phases recovered from (f) the red channel, (g) the blue channel, and (h) the average phase, respectively. Secondly, a white sculpture was also measured. For the IDTP algorithm, Figure 8ad shows the corresponding distorted color fringe, and the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. For the PDTP algorithm, Figure 8e-h shows the corresponding distorted color fringe, and the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. Figure 9 illustrates the cross section of the recovered absolute phases of the sculpture. The experiment results confirm the performance of the proposed IDTP algorithm compared with the conventional PDTP algorithm.   Secondly, a white sculpture was also measured. For the IDTP algorithm, Figure 8ad shows the corresponding distorted color fringe, and the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. For the PDTP algorithm, Figure 8e-h shows the corresponding distorted color fringe, and the absolute phases recovered from the red channel, the blue channel, and the average fringe, respectively. Figure 9 illustrates the cross section of the recovered absolute phases of the sculpture. The experiment results confirm the performance of the proposed IDTP algorithm compared with the conventional PDTP algorithm.

Conclusions
An effective IDTP algorithm was proposed for gamma correction without requiring complex calibration processes or extra fringe patterns. Two phase maps with π/2 phase shift were encoded into the red and blue channels of color fringe patterns. The average fringe patterns with little harmonics were used to restore the accurate phase. The effect of

Conclusions
An effective IDTP algorithm was proposed for gamma correction without requiring complex calibration processes or extra fringe patterns. Two phase maps with π/2 phase shift were encoded into the red and blue channels of color fringe patterns. The average fringe patterns with little harmonics were used to restore the accurate phase. The effect of color crosstalk can be suppressed by using an adaptive weight for average operation. The simulations and experiments indicate that the IDTP algorithm can effectively correct the gamma nonlinearity. Additionally, the PDTP algorithm conducts phase-shifting algorithm twice to recover two phase maps, and then computes the average phase map. In contrast, the IDTP algorithm needs to compute the average intensity of two fringe patterns, and then conducts the phase-shifting algorithm once to recover a one phase map, thus the computational efficiency of the IDTP algorithm is higher than that of the PDTP algorithm. Finally, it is still challenging for the proposed method to measure colorful objects, which will be studied in our future research.