Modal Reconstruction Based on Arbitrary High-Order Zernike Polynomials for Deflectometry

Abstract

Deflectometry is a non-destructive, full-field phase measuring method, which is usually used for inspecting optical specimens with special characteristics, such as highly reflective or specular surfaces, as well as free-form surfaces. One of the important steps in the Deflectometry method is to retrieve the surface from slope data of points on the sample map or surface reconstruction. This paper proposes a modal reconstruction method using an adjustable number of Zernike polynomials. In addition, the proposed method enables the analyses on practical surfaces that require an infinite number of Zernike terms to be represented. Experiments on simulated surfaces indicated that the algorithm is able to reveal the number of major-contributing Zernike terms, as well as reconstruct the surface with a micrometer-scale from slope data with a signal-to-noise ratio of 10.

1. Introduction

Phase Measuring Deflectometry (PMD) is a non-contact three-dimensional surface measuring method that has many potential applications in optical metrology. The main advantage this technique offers is the ability to full-field measure reflective as well as free-form surfaces with high dynamic range [1]. Together with its robustness and simple implementation, PMD is widely used for testing and quality control in precision manufacturing [2,3], free-form surface inspection [4,5,6], and 3D shape capturing in reverse engineering [7]. In PMD, active illumination is applied, the surface under test (SUT) is illuminated with coded light patterns, and the deformed patterns on the surface, which carry geometrical information, typically the slope data of the surface, are captured. By processing on the original and deformed fringe patterns, or the incident and reflected light to the surface, the slope data can be retrieved and used to regenerate the surface digitally [8].

The procedure that uses slope data in order to retrieve a 3D surface is called surface reconstruction. In general, it can be categorized into two main groups: zonal reconstruction and modal reconstruction [9]. In zonal methods [10,11,12], the reconstruction process is conducted locally, and slope data is integrated to determine the height relationship of points on the surface, and the geometrical information of a point is achieved by analyzing its neighbors. However, since the surface is reconstructed locally, it is more sensitive to noise, and the results are significantly reduced in quality when facing huge phase steps such as multi objects or surface disruption [9]. In contrast, modal reconstruction methods perform reconstruction globally, using slope data to perform fitting on an analytical model, mainly polynomials or sinusoidal curves. With a pre-defined model, we can easily have analytical slopes by taking the first derivatives of the initial model; then, measured surface slopes are used to approximate the model coefficients, which later can be used to reconstruct the surface by taking a weighted sum of modes in the former model [13]. Some of the common polynomials used to perform modal estimation namely Zernike [14,15], Legendre [16], and Chebyshev [13], each have their own characteristics and are selected based on the need of each reconstruction procedure.

This paper presents a method that allows performing wavefront reconstruction from slope data using an arbitrarily high number of Zernike terms. The gradient data are used to span out the least-square solution for the coefficients vector, which is later used to integrate back with their corresponding terms to represent the wavefront. By letting the total number of Zernike terms be a modifiable variable, it allows for making useful analyses when working with practical data, which will be discussed later in this paper. More detailed information on the Deflectometry method and the reconstruction procedure is proposed in Section 2. In Section 3 and Section 4, we carried out various analyses using simulated data in order to assess the performance of the algorithm under different circumstances.

2. Principle and Method

2.1. Deflectometry Principle

A basic setup of a PMD system consists of three main components, a charge-coupled device (CCD) camera, a liquid crystal display (LCD), and specimens, as shown in Figure 1 [17]. Here, each pixel on the LCD is considered a point source to the SUT, and the camera can be simplified as a pinhole model.

The driving principle of the method is the law of reflection, multiple vertical and horizontal sinusoidal phase-shifting fringe patterns are generated and shown on the LCD, which is placed opposite the SUT to perform the phase-shifting method [18]. These light patterns are projected to the SUT, and the deformed patterns, which carry geometrical information of the surface, are captured via the camera.

The fringe patterns displayed on the screen can be expressed as

$$I_n^s(x,y)=a^s+b^{s}cos(2\pi f^s{x}^s-2\pi \frac{n}{N}),$$

(1)

where $a^s$ and $b^s$ are the mean value and the amplitude, respectively, ($x^s$,$y^s$) is the coordinate of the pixel on the screen, $f^s$ is the frequency of the sinusoidal pattern (period/pixel), N is the total number of steps, and n is the shifting index which ranges from 0 to ($N-1$). Note that this and the next steps are applied for just one direction of the fringes, typically along the x-axis here, but implementations for the other direction can be carried out similarly. After generating and displaying them, the distorted fringes captured are

$$I_n(x,y)=A(x,y)+B(x,y)cos[\varphi (x,y)-2\pi \frac{n}{N}],$$

(2)

As can be seen from Equation (2), there are three unknowns to be determined: $\varphi (x,y)$, $A(x,y)$, and $B(x,y)$, so it required at least 3 shifting steps in order to make the calculation. However, a four-step phase-shifting algorithm is usually preferred because it is more convenient to implement in some cases since each step takes exactly ${90}^{°}$ or $\frac{\pi }{2}$ rad, also its better noise tolerance and higher accuracy [18]. Consequently, the four captured images ($I_1$, $I_2$, $I_3$, $I_4$) are also $\frac{\pi }{2}$-rad-delayed to one another, and the wrapped phase map can be solved as:

$$\varphi (x,y)=arctan\left(\frac{I_1-I_3}{I_0-I_2}\right),$$

(3)

The above solution is called wrapped phase map because the arctan() function is used. Therefore, the result is “wrapped” in the range of [$-\pi$, $\pi$]. This will go under the unwrapping process [19] in order to retrieve the absolute phase map. To utilize the unwrapping results, many strategies can be taken from the beginning step, at the fringe generation stage, such as using multi-frequency patterns [20,21,22], spatial-carrier frequency phase-shifting [23], or modifying phase steps [24,25,26], so it makes the unwrapping process simpler and enhances the absolute phase map accuracy. Once the absolute phase map is recovered, we can determine which set of points on the screen and the surface correspond to each other, combining with the geometry information of the system, which is obtained throughout the calibration process [27,28,29]. The slope at each spot on the surface in the x- and y- direction can be calculated.

By this stage, the slopes in both vertical and horizontal directions of every point on the surface are determined. As mentioned earlier, there are various types of polynomials that can be used to fit the slope data on. Generally, all these polynomials serve as a basis, and the surface is described as a linear combination of these bases [13]. Consequently, the slope, or the gradient, of the surface can also be interpreted as a linear combination of these basis gradients with the same coefficients. Since the polynomials are the analytical model, which is pre-defined, those gradients can be easily taken. Also, the surface gradients are what we already have from previous processes. We can mathematically approximate the coefficients of each element in the weighted sum. After that, those coefficients are used on the original bases in order to reconstruct the SUT.

2.2. Reconstruction Using Zernike Polynomials

To start with, Zernike polynomials are a sequence of continuous functions that form a complete orthogonal set over a unit circle. This means that each term is linearly independent of others, and the coefficients of terms that represents wavefront function are not affected by the number of terms [30]. Therefore, it enables the execution of wavefront mathematical procedures such as addition, subtraction, translation, rotation, and scaling so that it can serve as a handy basis set for wavefront reconstruction. Furthermore, Zernike polynomials are distinguished by their close correspondence to classical aberrations such as astigmatism, coma, and spherical aberration [31].

Considering the wavefront function as a linear combination of various Zernike terms, we have the mathematical expression of the wavefront as follows:

$$W(x,y)=\sum _{i=1}^M{a}_i{Z}_i(x,y)$$

(4)

where $Z_i$ represents the (i)th Zernike polynomial, $a_i$ is the corresponding coefficient, and M is the total number of Zernike terms used, which is an adjustable parameter. It also should be noted that the first Zernike polynomial is usually ignored since it just indicates the offset of the surface, thus not affecting the overall result. Taking derivative of both sides of Equation (4) along x- and y-direction, we have

$$\begin{array}{c}{S}_x(x_k,y_k)=\frac{\partial W(x_k,y_k)}{\partial x}={\displaystyle \sum _{i=1}^M}{a}_i\frac{\partial Z_i(x_k,y_k)}{\partial x}(k=1,2,\dots ,K),\\ S_y(x_k,y_k)=\frac{\partial W(x_k,y_k)}{\partial y}={\displaystyle \sum _{i=1}^M}{a}_i\frac{\partial Z_i(x_k,y_k)}{\partial y}(k=1,2,\dots ,K),\end{array}$$

(5)

where K is the total number of sampling points. Rewrite Equation (5) in matrix form, it becomes

$$S=Aa,$$

(6)

where

$$S=\left[\begin{array}{c}{S}_x\\ S_y\end{array}\right]={\left[\begin{array}{cccccccc}{S}_x(x_1,y_1)& S_x(x_2,y_2)& \dots & S_x(x_K,y_K)& S_y(x_1,y_1)& S_y(x_2,y_2)& \dots & S_y(x_K,y_K)\end{array}\right]}^T,$$

(7)

$$A=\left[\begin{array}{c}{A}_x\\ A_y\end{array}\right]=\left[\begin{array}{cccc}\frac{\partial Z_1(x_1,y_1)}{\partial x}& \frac{\partial Z_2(x_1,y_1)}{\partial x}& \dots & \frac{\partial Z_M(x_1,y_1)}{\partial x}\\ \dots & \dots & \dots & \dots \\ \frac{\partial Z_1(x_K,y_K)}{\partial x}& \frac{\partial Z_2(x_K,y_K)}{\partial x}& \dots & \frac{\partial Z_M(x_K,y_K)}{\partial x}\\ \frac{\partial Z_1(x_1,y_1)}{\partial y}& \frac{\partial Z_2(x_1,y_1)}{\partial y}& \dots & \frac{\partial Z_M(x_1,y_1)}{\partial y}\\ \dots & \dots & \dots & \dots \\ \frac{\partial Z_1(x_K,y_K)}{\partial y}& \frac{\partial Z_2(x_K,y_K)}{\partial y}& \dots & \frac{\partial Z_M(x_K,y_K)}{\partial y}\end{array}\right]$$

(8)

$$a={\left[\begin{array}{cccc}{a}_1& a_2& \dots & a_M\end{array}\right]}^T.$$

(9)

Here, S is a $2K\times 1$ vector containing the slope data of the surface, a is a $M\times 1$ vector containing the Zernike coefficients, and A is a $2K\times M$ matrix containing the derivatives of Zernike terms in x- and y-direction. The first 15 Zernike polynomials under Noll indexing and their derivatives are shown in

Table 1. First 15 Zernike polynomials and their gradients.

j	m	n	${\mathit{Z}}_{\mathit{j}}$	$\frac{\mathit{\partial }{\mathit{Z}}_{\mathit{j}}}{\mathit{\partial }\mathit{x}}$	$\frac{\mathit{\partial }{\mathit{Z}}_{\mathit{j}}}{\mathit{\partial }\mathit{y}}$
1	0	0	1	0	0
2	1	−1	$2\rho sin\theta$	0	2
3	1	1	$2\rho cos\theta$	2	0
4	2	−2	$\sqrt{6}{\rho }^{2}sin2\theta$	$2\sqrt{6}\rho sin\theta$	$2\sqrt{6}\rho cos\theta$
5	2	0	$\sqrt{3}(2{\rho }^2-1)$	$4\sqrt{3}\rho cos\theta$	$4\sqrt{3}\rho sin\theta$
6	2	2	$\sqrt{6}{\rho }^{2}cos2\theta$	$2\sqrt{6}\rho cos\theta$	$-2\sqrt{6}\rho sin\theta$
7	3	−3	$\sqrt{8}{\rho }^{3}sin3\theta$	$3\sqrt{8}{\rho }^{2}sin2\theta$	$3\sqrt{8}{\rho }^{2}cos2\theta$
8	3	−1	$\sqrt{8}(3{\rho }^3-2\rho )sin\theta$	$3\sqrt{8}{\rho }^{2}sin2\theta$	$\sqrt{8}[(6{\rho }^2-2)-3{\rho }^{2}cos2\theta ]$
9	3	1	$\sqrt{8}(3{\rho }^3-2\rho )cos\theta$	$\sqrt{8}[(6{\rho }^2-2)+3{\rho }^{2}cos2\theta ]$	$3\sqrt{8}{\rho }^{2}sin2\theta$
10	3	3	$\sqrt{8}{\rho }^{3}cos3\theta$	$3\sqrt{8}{\rho }^{2}cos2\theta$	$-3\sqrt{8}{\rho }^{2}sin2\theta$
11	4	−4	$\sqrt{10}{\rho }^{4}sin4\theta$	$4\sqrt{10}{\rho }^{3}sin3\theta$	$4\sqrt{10}{\rho }^{3}cos3\theta$
12	4	−2	$\sqrt{10}\left(4{\rho }^4-3{\rho }^2\right)sin2\theta$	$\sqrt{10}[8{\rho }^{3}sin2\theta cos\theta +(8{\rho }^3-6\rho )sin\theta ]$	$\sqrt{10}[8{\rho }^{3}sin2\theta sin\theta +(8{\rho }^3-6\rho )cos\theta ]$
13	4	0	$\sqrt{5}(6{\rho }^4-6{\rho }^2+1)$	$\sqrt{5}(24{\rho }^3-12\rho )cos\theta$	$\sqrt{5}(24{\rho }^3-12\rho )sin\theta$
14	4	2	$\sqrt{10}(4{\rho }^4-3{\rho }^2)cos2\theta$	$\sqrt{10}[8{\rho }^{3}cos2\theta cos\theta +(8{\rho }^3-6\rho )cos\theta ]$	$\sqrt{10}[8{\rho }^{3}cos2\theta sin\theta -(8{\rho }^3-6\rho )sin\theta ]$
15	4	4	$\sqrt{10}{\rho }^{4}cos4\theta$	$4\sqrt{10}{\rho }^{3}cos3\theta$	$-4\sqrt{10}{\rho }^{3}sin3\theta$

.

The least-square solution for Equation (6) is

$$a={\left(A^{T}A\right)}^{-1}{A}^{T}S,$$

(10)

where $A^{T}A$ is an $M\times M$ symmetric matrix, and $A^{T}S$ is a vector of length M with components that are weighted sums (or moments) over the slope data [9]. Having the solution of a, we can integrate it back into Equation (4) to reconstruct the wavefront function. The workflow can be summarized with the flowchart, as shown in Figure 2.

It is worth noting that, in practice, an infinite number of Zernike terms are required to demonstrate real wavefronts [32]. Therefore, it is impossible to reconstruct the exact surface, but the problem turns into finding the closest approximation for the wavefront. Intuitively, the more Zernike terms we can use to analyze the slope data, the better understanding of the wavefront we have. In general, there are two main problems when working with practical wavefronts: to find how many Zernike terms would represent the wavefront appropriately and to find the corresponding coefficients for each of the terms. While the coefficients can be easily retrieved using the procedure presented above, we cannot know in advance how many Zernike terms to use. However, since M, which is the number of Zernike terms used in Equation (4), is tunable, we can choose M as much as we want to make multiple approximations using the slope data until the result is converged, which means that the difference between reconstruction wavefronts of two consecutive estimations is smaller than a certain threshold. However, it is not essential to increase the Zernike terms one by one, but we can initially analyze the data with big incremental steps to find the range in which the result seems to be converged and then narrow down the range with smaller steps to find the optimal number of terms. With the ability to reconstruct wavefront with an arbitrarily high order of Zernike terms, it enhances the estimation results and spans out a better solution for the closest approximation problem.

3. Simulation

Although reliable physical constraints can be integrated for better surface reconstruction, the result of the reconstruction stage heavily relies on the slope data [1]. Therefore, without loss of generality, simulated data can be used to evaluate the performance of the reconstruction algorithm. Moreover, by using simulated data, multiple settings of the surface can be modified, such as the number of Zernike terms used to generate the surface or the level of noise. Simulated surfaces will be generated by taking a linear combination of multiple Zernike terms. Then, the slope data of surfaces can be calculated by taking the derivative of the surface. At this stage, noises can be added to the slope data in order to make it closer to that data of real cases and to test the noise-resistant ability of the algorithm. Here, a pre-defined surface is generated by taking a weighted sum of the first 40 Zernike terms, ranging from −40 to 40 mm in both the x- and y-directions, with the size of the sample points matrices of 512 by 512, which spans out a points’ interval of approximately 0.16 mm. Figure 3a is the illustration of the generated surface, and by taking the derivatives of the surface vertically and horizontally, we are able to have slope data of the surface in both directions, which are shown in Figure 3b,c. Moreover, the coefficients of each Zernike term up to the term of 40 that were used to generate the surface are illustrated in Figure 3d.

To verify the performance of the algorithm and make the comparison, another dataset with noise added to the ground truth slope data, SNR = 10 dB, will be used along with the original one. For reference convenience, we call the original slope data dataset 1, and the other is called dataset 2. With dataset 1, the reconstruction results are shown in Figure 4, as seen in Figure 4b. The reconstructed surface is almost identical to the original one since the error is at just about ${10}^{-15}$ mm. Moreover, Figure 4c shows the coefficients were also well estimated with an error scale only about ${10}^{-15}$. Regarding dataset 2 with noise applied, the results are shown in Figure 5, the reconstruction error achieved was about ${10}^{-3}$ mm, which is the micrometer scale, as shown in Figure 5b. With the estimated coefficients, the error shown in Figure 5c shows that the algorithm successfully retrieves the coefficients, with the error at about ${10}^{-4}$ throughout the range of terms.

On top of that, to ensure that the algorithm was able to approximate the proper number of Zernike terms that made up the surface, we used a variable number of Zernike terms in order to reconstruct the previous surface, which is the linear combination of 40 Zernike terms, and judge how accurate the reconstructed surfaces were with respect to the original one. As seen from Figure 6, the error of the reconstructed one experienced a significant drop when reaching 40 terms and flattened out when exceeding 40 terms, which is exactly the number of terms used to build the original surface, and that is also what we expected the algorithm to perform.

4. Discussion

As mentioned earlier, in practice, a wavefront is made from an infinite number of Zernike terms. Therefore, it is necessary to confirm that for an arbitrary surface, a number exists, or a range of the number of Zernike terms would contribute to the majority of the wavefront, or we should be able to describe it with an acceptable error. In order to verify that, we used a polynomial surface governed via the function:

$$z(x,y)=0.5x^2-0.3[3{(1-x)}^2{e}^{-x^2-{(y+1)}^2}-10(\frac{x}{5}-x^3-y^5)e^{-x^2-y^2}+\frac{1}{3}{e}^{-{(x+1)}^2-y^2}],$$

(11)

which is proposed by Ye, Jingfei, et al. in [33] to make comparisons of the trend in the results and to provide further explanations for the results, which was not expounded by Ye, using the proposed method. The demonstration of the surface is shown in Figure 7a. Since a polynomial function is used to generate the surface, it is not intentionally made of a specific number of Zernike terms as before. Therefore, the problem is to find the number of Zernike terms to make a good enough approximation of the original surface. In [33], Ye performed the reconstruction process in two circumstances, one using 22 Zernike terms and the other using 37 terms, and the results showed that both the RMS error and the PV error were significantly reduced when using more Zernike terms. However, it was lacking an explanation as to why those specific numbers of Zernike terms should be chosen, as well as clarifying whether using more terms will guarantee a reduction in reconstruction error or not. We performed a similar reconstruction process, and the results that are shown in

Table 2. Recontruction errors of the surface in Equation (11).

	PV Error (mm)	RMS Error (mm)
22 terms used	0.0659	0.0122
37 terms used	0.0085	0.0014

indicated a similar trend.

An illustration of the reconstructed surface using 37 terms is shown in Figure 7b; as can be observed, it is nearly identical to the original one. However, it is worth noting that the two reconstruction processes are not exactly the same. We performed on the circular aperture with sufficient data across the region, while Ye performed on just a nearly circular region, with some areas missing data. Therefore, the results are just to show the relevant trend in two independent-conduct simulations since the underlying process is similar but not to compare the performance of the reconstruction process. It is apparent that the more the Zernike terms are used, the lower the error will be, but how high the order of the Zernike terms to be used would be sufficient is the remaining question. Despite choosing some very high order, it will span out better results, as doing so will significantly raise the computational complexity. By conducting an analysis with a range of Zernike terms as proposed, we can explain the results well. The outcome of the analysis is shown in Figure 7c; as can be seen, the error line experienced a steep decrease reaching 32 terms, which explains the significant drop in the error between using 22 terms and 37 terms. Moreover, the chart shows why it is impractical to pick a randomly high order, where the error will only drop at some number of Zernike terms and flat out after that value before reaching another drop. For example, in this case, the result when using 45 terms is not much different than using 35 terms, but the computational complexity is. This also justifies the need to perform the assessment on the slope data to choose the suitable number of Zernike terms to perform the reconstruction. In Figure 7c, the errors were computed with respect to the original surface, but, in practice, where that original surface is what we try to reconstruct. Hence, it is unavailable. We can start the process at a certain term and use the reconstructed surface as the reference to compute the error of the following step, and the process is nearly the same.

On top of that, to record the performance of the algorithm under different circumstances, we fixed the number or Zernike terms at 40 to build the ground truth surface and let other dataset parameters vary one by one. In each case, we then performed surface reconstruction using exactly 40 terms in order to minimize the error from terms insufficient and compare the results. In the first experiment, we kept the sample matrix size at 512 by 512 and let the SNR vary from 1 to 80 in order to observe how the reconstruction error would change under different settings of noise in slope data. The results are shown in Figure 8a, and they make sense as the larger the SNR is, or the less noise in data, the better approximation the experiment made. For the second experiment, we kept the SNR at 10 and reconstructed with different sample sizes to measure the change in runtime with respect to the change in the size of sample matrices. From Figure 8b, we observed a linear dependence between the two attributes, when each direction of the sampling matrix doubled in size or total sampling points quadrupled, the runtime also increased by four times.

Moreover, not limited to the Deflectometry method only, this reconstruction procedure can also be applied to other measurement techniques that are required to perform the slope-to-surface conversion in general. Some of the techniques that can be listed are the Hartmann wavefront sensing [34,35], the lateral shearing interferometer [36,37,38], and the pupil plane wavefront sensing [39].

5. Conclusions

Wavefront reconstruction is a crucial stage in the Deflectometry method, which turns slope data of points into a 3D representation of the surface. Reconstruction methods can be divided into two main groups: zonal methods and modal methods. While zonal methods process the data locally, modal methods use analytical models, usually orthogonal polynomial sets, as a basis to perform reconstruction on the data as a whole. There are many polynomial sets that are valid for this purpose, but Zernike polynomials are usually preferred when working with optical instruments since Zernike terms can be interpreted into different types of aberrations.

To represent practical wavefronts, we require infinite Zernike terms. Therefore, the problem that arises is how to make the closest estimation, which requires knowing how many Zernike terms are needed and the coefficient of each term contributes to the weighted sum that spans the approximation of the wavefront.

In this paper, we proposed a method to reconstruct the wavefront with an arbitrary number of Zernike terms. Working on simulated data, the algorithm successfully estimated the coefficients for each of the Zernike terms used to generate the original wavefront, with the RMS error at about ${10}^{-4}$ to the ground truth coefficients, and reconstructed the wavefront with micrometer-scaled error. Moreover, the method could span out the exact number of terms used to generate the original wavefront, with the error enormously dropping reaching that number. The significance of this method is that by being able to tune the number of terms to perform reconstruction as high as needed, it allows for making multiple estimations to carry out analysis when working with practical data, where we do not know how many terms would be enough to make an acceptable approximation for the wavefront. Moreover, it can also be applied in other methods that require the conversion from gradient data to surface representation. Nevertheless, further studies on the reconstruction of non-circular regions or missing slope data can be investigated to enhance the proposed method.