Efficient and Robust Phase Measuring Deflectometry for Vehicle Paint Surface Measurement

Abstract

Phase measuring deflectometry (PMD) is widely employed in automotive intelligent manufacturing for high-precision, large-area inspection of specular and free-form surfaces. However, the conventional three-frequency four-step phase-shifting method requires multiple projected patterns, which reduces measurement efficiency and exhibits poor robustness on highly curved surfaces. To overcome these limitations, this study proposed a novel encoding and decoding strategy that utilized only four phase-shifted fringes and one additional unequal phase-shifted fringe for phase retrieval and unwrapping. Moreover, an amplitude symmetry-based segmentation algorithm was introduced for background segmentation, and a phase self-correction approach was developed to mitigate jump errors in the fringe order map caused by surface imperfections. Experimental results demonstrated that the proposed method significantly improved the measurement efficiency compared with the conventional approach while improving measurement accuracy.

1. Introduction

In the automotive manufacturing industry, the surface quality of painted car bodies directly influences both the aesthetic appearance and long-term durability of vehicles, making it a critical aspect of pre-delivery quality control. Traditional inspection methods, however, rely heavily on manual visual inspection and assisted defect localization, which not only consumes substantial human resources but also poses safety risks when inspecting elevated areas, such as roofs and hoods, often requiring scaffolding or other auxiliary equipment. These subjective and inefficient approaches are increasingly inadequate to meet the growing demand for high-precision and automated surface inspection in modern automotive production. Consequently, the development of non-contact, high-precision optical measurement techniques, particularly phase measuring deflectometry (PMD), has emerged as a crucial pathway to enhance both the efficiency and reliability of vehicle paint surface inspection. PMD has multiple advantages in measuring mirror-like or specular reflective surfaces, including non-destructive testing, a wide dynamic range, simple system configuration, and full-field high-precision measurement capability [1,2,3,4], and therefore has been widely applied to three-dimensional measurement of automotive paint surfaces. However, two main challenges exist in current automotive paint surface inspection: low measurement efficiency, since full-field PMD typically requires phase information from two orthogonal directions to obtain the surface gradient and additional phase-shifted fringe patterns are often needed to perform multi-frequency heterodyne phase unwrapping to handle phase retrieval problems caused by surface discontinuities [5,6,7,8,9], which severely limits the measurement speed; and phase unwrapping errors caused by the properties of automotive paint materials, as the physical characteristics of paint surfaces reduce fringe modulation and the signal-to-noise ratio of the captured images, making phase unwrapping prone to errors. Therefore, improving measurement speed while maintaining measurement accuracy has become one of the key research focuses in automotive paint surface measurement.

To overcome the above challenges, extensive research has been conducted on phase unwrapping algorithms to develop advanced phase unwrapping techniques [10,11,12,13,14,15,16]. According to the computational strategies employed, phase unwrapping methods can generally be divided into three major categories: spatial phase unwrapping [17,18], temporal phase unwrapping, and deep-learning-based phase unwrapping. Spatial phase unwrapping provides a natural and intuitive way to recover continuous phase maps by exploiting spatial continuity, in which the unwrapped phase at each pixel is inferred from its local neighborhood, using methods such as the Goldstein method [5], the quality-guided method [19], the Flynn method [20], and the minimum Lp-norm method [21]. Although these methods require only a single wrapped phase map, they often fail in the presence of surface discontinuities, isolated regions, and complex illumination disturbances [22,23,24].

In contrast, temporal phase unwrapping is known for its high accuracy and robustness and can effectively handle discontinuous surfaces [7,23]. Two representative methods are multi-frequency heterodyne and Gray-code-assisted schemes. However, the need to project a large number of fringe patterns limits the measurement efficiency. In the TPU algorithm, each spatial pixel is processed independently, thereby preventing noise from propagating between pixels. Nevertheless, auxiliary decoding still requires additional projected patterns. Therefore, minimizing the number of projected patterns has become a major research objective. Wang et al. [25] embedded fringe sequence information into N-step phase-shifting patterns to avoid extra projections, but the designed frequency and phase accuracy are constrained by N. Similarly, hybrid patterns that combine sinusoidal and speckle components can reduce the total number of patterns, although speckle signals are highly sensitive to ambient light [26]. An et al. [27] embedded periodic information into phase shifts to reduce the number of required fringe patterns. Although effective for ideal diffuse surfaces, this method is highly prone to fringe-induced errors when measuring semi-transparent or specular materials, making it difficult to meet the requirements of automotive paint surface measurement.

In recent years, deep learning has developed rapidly and has been widely applied in many fields [28,29]. It has also been extensively used to address various problems in structured-light 3D reconstruction [30,31,32]. Recently, an improved method was proposed that combines an encoder–decoder network with a pixel-wise classification layer to directly classify the wrapped order of all pixels. However, since the classification accuracy cannot always be guaranteed, post-processing (e.g., clustering-based smoothing) is usually required to optimize the results, which are then added to the wrapped phase to reconstruct the true phase [33]. More critically, any misclassification leads to an integer-multiple error of $2\pi$ (at least $2\pi$), which is unacceptable in practical applications. Dardikman and Shaked proposed using a neural network to directly map the relationship between the wrapped phase and the unwrapped phase, but only preliminary results were demonstrated [34]. Overall, deep-learning-based phase unwrapping still suffers from the limitation of insufficient generalization performance.

2. Materials and Methods

2.1. Phase Shift Encoding with Embedded Fringe Order

The encoding pattern set comprises five images in total: four conventional phase-shifting patterns and an encoding pattern designed in the paper. The traditional four-step phase-shifting formula can be expressed by the following equation:

$$I_n(x,y)=a(x,y)+b(x,y)cos\left[\phi (x,y)+{\delta }_n\right],$$

(1)

where $n=1,2,3,4$, and phase shift ${\delta }_n=(n-1)·2\pi /4$. Here, $a(x,y)$ is the background intensity, $b(x,y)$ is the modulation, and $\phi (x,y)\in (0,2\pi ]$ is the encoded phase. They can be derived from the following equations:

$$\left\{\begin{array}{c}a(x,y)={\displaystyle \sum _{n=1}^4}{\displaystyle \frac{I_n(x,y)}{4}},\hfill \\ b(x,y)=\sqrt{\frac{{\left(I_1-I_3\right)}^2+{\left(I_2-I_4\right)}^2}{4}}.\hfill \\ \phi (x,y)=arctan\left({\displaystyle \frac{I_4-I_2}{I_1-I_3}}\right)\hfill \end{array}\right.$$

(2)

Unlike traditional temporal phase unwrapping schemes, this paper embeds the fringe order information into the phase-shift values. By encoding these phase-shift values, the unwrapping of the wrapped phase can be achieved with only a single additional pattern. The mapping of the encoded phase shift values onto the grayscale distribution of the projected pattern can be represented by the following equation:

$$I_u(x,y)=a(x,y)+b(x,y)cos\left[{\phi }_k(x,y)\right],$$

(3)

where ${\phi }_k(x,y)=\phi (x,y)+{\theta }_k$, where ${\theta }_k=(k-1)·2\pi /K$, K is the total number of periods, and $k=1,2,\dots ,K$. According to Equation (3), the fringe pattern obtained after embedding the periodic information into the phase shift is shown in Figure 1a,b, which present the amplitude map of the cosine values resulting from encoding six periods, corresponding to $K=6$.

Figure 1. Coding Pattern. (a) Fringe pattern encoded by phase shift; (b) encoding of phase shifts for six periods.

2.2. Effective Segmentation of Encoding Regions

After projecting the five encoded patterns onto the object surface, they are sequentially captured by a camera for decoding. The first step in the decoding process is to separate the encoded regions from the non-encoded regions. Our approach aims to maximize the contrast between these two areas to facilitate binary segmentation. According to Equation (4), by calculating the differences between phase shift diagram, taking the absolute values, and then summing these values, the overall amplitude can be amplified, as shown in Figure 2a,b. Since the grayscale values of the background region remain approximately consistent across different images, performing image subtraction can reduce the grayscale values of the background. This process enhances the contrast between the encoded region and the background:

$$I_{bin}=abs(I_1-I_3)+abs(I_2-I_4).$$

(4)

Figure 2. Schematic of the regional contrast enhancemenfiguret principle. (a) Amplitude variation in single-cycle four-step phase shifting; (b) principle of amplitude stretching in the encoded region.

Subsequently, a threshold $\alpha$ is applied to $I_{bin}$ to perform binarization, achieving the segmentation of the encoded regions.

$$mask(x,y)=\left\{\begin{array}{c}0,\mathrm{if}{I}_{bin}(x,y)<\alpha \\ 1,\mathrm{if}{I}_{bin}(x,y)\ge \alpha \end{array}\right.$$

(5)

Compared to the segmentation using modulation b, this segmentation method has two advantages. First, the computation time is reduced. The calculation of modulation involves squaring, summing, square rooting, and division, and outputs floating-point data, whereas the method proposed in this paper only requires subtraction, absolute value operation, addition, and outputs integer data, resulting in lower time consumption. Second, the histogram of $I_{bin}$ obtained by this method is smoother, as shown in Figure 3, which is more conducive to finding the minimum value as an automatic threshold for segmentation.

Figure 3. Histogram of the Image. (a) Histogram of modulation b; (b) histogram of $I_{bin}$.

2.3. Determination of the Fringe Order Map

The determination of the fringe order is based on the difference between $\phi$ and ${\phi }_k$, namely ${\theta }_k$. The parameters a, b, and $\phi$ can be retrieved from the four-step phase-shifting fringe patterns. Subsequently, the parameter ${\phi }_k$ can be determined based on them:

$${\phi }_k(x,y)=arccos\left[{\displaystyle \frac{I_u(x,y)-a(x,y)}{b(x,y)}}\right].$$

(6)

As illustrated in Figure 4a, the reverse cosine operation confines ${\phi }_k$ to a triangular pattern $0\to \pi \to 0$, whereas $\phi$ exhibits a sawtooth pattern $0\to 2\pi$. These two waveforms are characterized by a periodic parallel or symmetric relationship, with both the parallel segment and the symmetry axis defined by ${\theta }_k$ for the k-th period. Leveraging these properties, we can subsequently derive the subtracting phase ${\phi }_{sub}(x,y)$ and the adding phase ${\phi }_{sub}(x,y)$:

$${\phi }_{sub}(x,y)=\phi (x,y)-{\phi }_k(x,y),$$

(7)

$${\phi }_{add}(x,y)=2\pi -\phi (x,y)-{\phi }_k(x,y).$$

(8)

Figure 4. Decoding process. (a) Wrapped phase and coded phase; (b) phase addition and phase subtraction; (c) before and after orientation alignment; (d) wrapped phase and fringe order.

As shown in Figure 4b, the phase shift encoding quantity ${\theta }_k$, which is alternately embedded within patterns ${\phi }_{sub}$ and ${\phi }_{add}$, can be extracted by utilizing the horizontal gradients of ${\phi }_{sub}$ and ${\phi }_{add}$:

$${\theta }_k(x,y)=\left\{\begin{array}{cc}{\phi }_{sub}(x,y),\hfill & \mathrm{if}\nabla {\phi }_{sub}(x,y)<\nabla {\phi }_{add}(x,y)\hfill \\ {\phi }_{add}(x,y),\hfill & \mathrm{if}\nabla {\phi }_{sub}(x,y)\ge \nabla {\phi }_{add}(x,y)\hfill \end{array}\right.$$

(9)

To achieve phase unwrapping, we add a $2\pi$ compensation to all negative values in the retrieved phase ${\theta }_k$, as shown in Figure 4c:

$$\widetilde{{\theta }_k}(x,y)=\left\{\begin{array}{cc}{\theta }_k(x,y)+2\pi ,\hfill & \mathrm{if}{\theta }_k\le 0\hfill \\ {\theta }_k(x,y),\hfill & \mathrm{if}{\theta }_k>0\hfill \end{array}\right.$$

(10)

The fringe order k for each pixel can be determined by the following equation:

$$k(x,y)=round\left[K\times {\displaystyle \frac{\widetilde{{\theta }_k}(x,y)}{2\pi }}\right].$$

(11)

2.4. Self-Correction of Absolute Phase

Although the series encoding and decoding process is theoretically sound, it poses significant challenges in the practical measurement of vehicle paint surfaces, manifesting as considerable measurement errors. The root cause of this issue is the “orange peel” effect on the paint surface, combined with interference from ambient light and inter-reflections. As shown in Figure 5, the quality of the acquired fringe pattern is very poor. Using such low-quality images as input will undoubtedly increase the difficulty of decoding.

Figure 5. The influence of orange peel effect on fringes.

The accumulation of various errors during temporal phase unwrapping results in misalignment between the decoded fringe order map and the actual fringe periods, as shown in Figure 6a. Our method is specifically designed for scenarios involving relatively smooth surfaces, such as vehicle paint finishes. In such contexts, the phase in continuous regions is expected to remain consistent without abrupt changes, except at isolated local points. The encoding and decoding approach described in the paper often introduces repeated phase-jump artifacts in practical measurements, even though the resolved phase in most regions is accurate (with a signal-to-noise ratio of at least 0.5).

Figure 6. Principle diagram of absolute phase self correction. (a) Fringe order map before and after correction; (b) absolute phase before correction and reference phase.

Therefore, by integrating spatial information (to ensure smoothness of the absolute phase within continuous regions) and temporal information (to verify the correctness of the phase across most of the measured area), our algorithm achieves self-correction of the absolute phase.

First, the correction of the Absolute Phase is applied individually to each isolated region. As shown in Figure 6b, this particular row contains two isolated regions that must be corrected separately. The first step of the correction algorithm involves spatially unwrapping the Wrapped Phase along the encoding direction, producing the result indicated by the Absolute Phase in Figure 6b. The Reference Phase in Figure 6b illustrates the initial absolute phase unwrapped by the encoding and decoding method proposed in this paper. Subtracting the vector of Absolute Phase from the vector of Reference Phase yields the deviation vector Offset, as illustrated in Figure 6b. The deviation offset1, offset2 to be added to the absolute phase corresponds to the mode of the offset vector, as shown in the phase plot in Figure 6b. Adding offset1 to the Absolute Phase yields the corrected absolute phase. As shown in Figure 6b, the fringe order map of the corrected phase demonstrates perfect alignment with the fringe periods. The same method is applied to the isolated region on the right side of Figure 6a, where offset2 is calculated to achieve phase self-correction.

The correction algorithm is applied to all rows, resulting in the absolute phase shown in Figure 7a. However, since the initial correction only incorporates horizontal reference information, the calculation of global deviation becomes prone to errors when the isolated regions in the horizontal direction are small. To address this limitation, a second correction is performed by incorporating vertical reference information. This step follows a procedure similar to the first correction, but uses the output of the initial correction as the input for spatial unwrapping. The result after the second correction is shown in Figure 7b.

Figure 7. Absolute phase self-correction results. (a) First self correction result; (b) second self correction result.

3. Experiments

We conducted various experiments and tests on PaintPro at Hunan SpeedBot Robotics Ltd., Changsha, China. Figure 8 shows our system composition of PaintPro, which is built around a 23.8-inch LCD screen with a resolution of 1920 × 1080 (pixel pitch ≈ 0.2745 mm) and a refresh rate of 20 fps, paired with a Daheng ME2S-560-70U3M camera (resolution 2600 × 2160, pixel size 2.5 µm). When the display fills the camera’s field of view—as in our calibration setup—this configuration yields a spatial resolution on the display plane of approximately 0.20 mm horizontally and 0.14 mm vertically. In practical terms, this means the system operates in an oversampled regime where light from each display pixel is captured by about 2.7 camera pixels on average. This oversampling is instrumental for our phase measuring deflectometry approach, as it reduces discretization artifacts from the screen and provides the sub-pixel phase precision that underpins the robustness of the proposed unwrapping algorithm.

Figure 8. Photograph of the PMD system.

3.1. Determination of the Fringe Order

To validate the feasibility of the algorithm, tests were conducted under challenging imaging conditions on white car paint, as shown in Figure 9. The images respectively display the four-step phase-shifting patterns, the unequal phase-shifting patterns, and the binary mask segmented by the proposed method.

Figure 9. Captured fringe patterns and the computed initial values. (a) Measurement scenario; (b) captured four-step phase-shifting fringe patterns; (c) captured coded phase-shifting fringe pattern; (d) wrapped phase; (e) coded phase; (f) binary mask extracted by the proposed method.

As shown in Figure 10a, which displays the 1000th line of the encoded phase cosine value, the value range deviates from the ideal distribution between −1 and 1 due to system noise, and significant errors are present during the oscillation process. To address this, we preprocess the wrapped phase and encoded phase by resampling the image. However, it is crucial to note that we backed up the original wrapped phase. In other words, we only utilized the wrapped phase to perform phase unwrapping for obtaining the fringe order map. The wrapped phase ultimately used for the final unwrapping is the pristine, unaltered wrapped phase, which has not undergone any downsampling or image processing. This approach reduces noise while improving the efficiency of subsequent processing. Since the input to the arccosine function must be within the range of −1 to 1, manual truncation is required, manual truncation is required to ensure that the input values do not exceed the bounds represented by the red dashed line in Figure 10a. Figure 10b,c show the smoothness of the initial phase values and the resampled phase, respectively.

Figure 10. Preprocessing of the 1000th line. (a) Truncated cosine value range; (b) wrapped phase and coded phase; (c) smoothed signal after downsampling.

After obtaining the down-sampled wrapped phase and coded phase, the phase summation value extracted using Equation (8) is shown in Figure 11b, and the phase difference value extracted using Equation (7) is shown in Figure 11a,c, which show the phase addition and subtraction results for a randomly selected row. The resulting fringe order map, computed using Equations (9)–(11), is shown in Figure 12a. To obtain an initial fringe order map with a relatively high signal-to-noise ratio (SNR), median filtering was applied to the calculated fringe order map shown in Figure 12a, ensuring the subsequent self-correction of the absolute phase could proceed smoothly. The filtered result is presented in Figure 12b. Figure 12c shows the fringe order of a selected row before and after median filtering.

Figure 11. Addition and subtraction of wrapped phase and coded phase. (a) Phase addition; (b) phase subtraction; (c) line chart of phase difference and sum for a selected row.

Figure 12. Fringe Order Maps Before and After Filtering. (a) Fringe order map; (b) fringe order map after filtering; (c) fringe order of a selected row before and after median filtering.

An enhanced signal-to-noise ratio was achieved through median filtering, ensuring the successful execution of the subsequent phase correction procedure. Figure 13 depicts the results of our correction methodology, Figure 13a presents the initial absolute phase, Figure 13b presents the absolute phase following the initial correction applied along the coding direction, and Figure 13c presents the absolute phase after the secondary correction applied along the alternative direction. This demonstrates that the proposed method, by encoding and decoding the phase shift and incorporating spatial domain information for correction, successfully yields the desired absolute phase.

Figure 13. Absolute phase correction results. (a) Absolute phase before correction; (b) absolute phase after the first correction; (c) absolute phase after the second correction.

3.2. Comparative Analysis

The absolute phase calculated by the three-frequency four-step method has been extensively validated on our system and is considered highly reliable. To verify the effectiveness of the proposed method, the fringe order map calculated by our approach is compared with that of the three-frequency four-step method in Figure 14, while the absolute phase is compared with the three-frequency four-step method and An’s method [27] in Figure 15. We also compared the phase gradient maps of these methods, as shown in Figure 16. We performed surface fitting for each absolute phase in Figure 15 and calculated the corresponding root mean square error (RMSE). This error can reflect the extent to which phase noise and fringe order affect the phase retrieval results. It can be observed that the RMSE of the proposed method is the smallest among the three methods. The experimental results show that the proposed method not only significantly reduces the number of encoded patterns but also improves accuracy.

Figure 14. Comparison of fringe order results. (a) Fringe order map obtained by the multi-frequency heterodyne method; (b) fringe order map obtained by the proposed method.

Figure 15. Absolute phase comparison. (a) Absolute phase unwrapped by An’s method; (b) absolute phase unwrapped by the multi−frequency heterodyne method; (c) absolute phase unwrapped by the proposed method.

Figure 16. Phase gradient map. (a) Gradient map calculated by An’s method; (b) gradient map calculated by the multi-frequency heterodyne; (c) gradient map calculated by our method.

To ensure the correctness of our decoded fringe orders, surface gradients at each point on the object are indirectly derived from the absolute phase based on the principle of phase-measuring deflectometry. These gradient data are then processed through a Poisson integration algorithm to reconstruct the 3D height field, which effectively minimizes the cumulative error during the numerical integration process. The reconstruction results are shown in Figure 17. A multi-camera system with four synchronized units is employed in this setup. Through calibrated stereo vision parameters, direct feature point correspondence is established between each camera pair, enabling high-precision, single-frame 3D reconstruction of specular surfaces without requiring additional temporal sequences.

Figure 17. Point cloud comparison. (a) Result reconstructed by the three-frequency four-step phase-shifting method; (b) result reconstructed by the proposed method.

4. Discussion

In the field of 3D reconstruction and reverse engineering, measurement accuracy and speed are often the two most concerning indicators. When reconstructing specular objects, classical spatial phase unwrapping algorithms, such as the three-frequency four-step phase-shifting method, require a large number of encoded patterns and perform poorly in measuring high-curvature areas. The Gray code strategy can reduce the number of patterns to some extent when the number of fringe periods is small. However, due to inter-reflections and non-uniform illumination on specular surfaces, the binarization step of Gray code becomes extremely challenging, and modulation-based segmentation methods often fail.

This paper proposes embedding period information into the phase-shifting sequence. Reusing the background intensity and modulation data obtained from four-step phase-shifting, we determine the fringe period. Since the actual measurement process is complex and involves various error sources during reverse reconstruction, resampling is introduced to enhance measurement accuracy. In addition, spatial information is incorporated to enable self-correction of the absolute phase, ultimately realizing a 4+1 pattern unwrapping algorithm for industrial applications. The encoding method proposed in the paper can be extended from four-step phase shifting to other phase-shifting schemes with different numbers of steps and remains equally applicable.

A high-precision Phase Measuring Deflectometry (PMD) measurement system relies not only on excellent algorithms but also crucially on rational hardware configuration. For instance, parameters such as the display screen’s brightness, contrast, and encoded fringe period must be determined according to the specific measurement scenario. On the other hand, camera settings including resolution, exposure time, gain, and aperture also require appropriate configuration. Furthermore, synchronization between the display’s refresh rate and the camera’s frame rate must be carefully ensured.

In addition to hardware safeguards, systematic error analysis is equally essential. Assuming an additive, white, Gaussian noise model with zero mean and standard deviation ${\sigma }_n$ for the intensity [35] and under the condition of multi-period standard phase-shifting fringe patterns, the phase error standard deviation for the N-step PSP approach is defined as:

$${\sigma }_{\phi }=\sqrt{{\displaystyle \frac{2}{N}}}{\displaystyle \frac{{\sigma }_n}{BK}},$$

(12)

where ${\sigma }_{\phi }$ represents the standard deviation of the phase error. Noise suppression in Phase-Shifting Profilometry (PSP) is governed by three key parameters: the phase-shift steps (N), the intensity modulation (B), and the fringe frequency (K). Increasing N improves accuracy at the cost of longer acquisition time. Optimizing B enhances the signal-to-noise ratio. Compared to standard 3-step sinusoidal patterns, raising K is the most effective way to reduce phase error.

Furthermore, the proposed method still has certain limitations. Since the measured object in this study is vehicle paint surfaces, which are relatively smooth and cover a large area, the phase self-correction algorithm achieves very good results. For smaller scenarios or surfaces with high complexity, additional correction procedures tailored to practical conditions may be necessary on the basis of the present work.

5. Conclusions

This paper proposes a novel temporal phase unwrapping algorithm to achieve robust three-dimensional measurement of specular objects. The period information of each pixel is embedded into the phase-shifting amount. During decoding, parameters are utilized to extract both the wrapped phase and the encoded phase. To address noise and nonlinear interference in practical measurement scenarios, a resampling strategy is employed to minimize the loss of periodic encoding information while simultaneously denoising the signal. After retrieving the fringe order map, a self-correction mechanism is applied based on the physical properties of the object and the spatial information of the phase map. Experimental results demonstrate that in the reverse engineering of 3D reconstruction for painted specular surfaces, the proposed encoding and decoding method using only 4+1 patterns can simultaneously improve both measurement speed and accuracy.