Three-Dimensional Reconstruction Method for Machined Surface Topography Based on Gray Gradient Constraints

Featured Application

Applying on mechanical engineering.

Abstract

In the modern engineering field, recovering the machined surface topography is important for studying mechanical product function and surface characteristics by using the shape from shading (SFS)-based reconstruction method. However, due to the limitations of many constraints and oversmoothing, the existing SFS-based reconstruction methods are not suitable for machined surface topography. This paper presents a new three-dimensional (3D) reconstruction method of machined surface topography. By combining the basic principle of SFS and the analytic method, the analytic model of a surface gradient is established using the gray gradient as a constraint condition. By efficiently solving the effect of quantization errors and ambiguity of the gray scale on reconstruction accuracy using a wavelet denoising algorithm and image processing technology, the reconstruction algorithm is implemented for machined surface topography. Experimental results on synthetic images and machined surface topography images show that the proposed algorithm can accurately and efficiently recover the 3D shape of machined surface topography.

1. Introduction

Machined surface topography refers to the microgeometry shape left on the workpiece surface by the tools and chips during the process of machining [1], which not only has a significant impact on the function of the mechanical product, but also affects the surface characteristics of the mechanical parts, including wear, lubrication, friction, corrosion resistance, and reflection [2,3,4]. Accurate and effective surface topography information is of importance to the modern engineering field. At present, surface topographic information is mainly obtained by the contact measurement method and non-contact measurement method. The main equipment used in contact measurement method is the stylus profilometer. However, due to the limitations of measurement conditions and stylus, the measurement results of linear sampling points cannot represent the whole surface topography, and the workpiece surface cannot avoid being scratched [5]. Moreover, there are many types of non-contact measurement methods, including optical methods [6], ultrasonic methods [7], and three-dimensional (3D) reconstruction methods. Optical and ultrasonic methods have high requirements for their measurement environment, and they also have a limited measurement range. With the development of 3D reconstruction, 3D reconstruction measurement methods have been widely used for non-contact measurement. Wang et al. [8] proposed the 3D reconstruction method of wear particle surfaces based on a photometric stereo, and obtained the surface topographies of wear particles for further feature-based wear particle identification. Tang et al. [9] used the 3D reconstruction method based on the shape from a focus to determine the grinding wheel surface topography.

Three-dimensional reconstruction is the process of recovering 3D shapes of objects from single or multiple images [10,11,12], and the shape from shading (SFS) method is considered to be one of the fastest and most efficient 3D reconstruction methods [13,14,15]. Therefore, the SFS-based reconstruction method has attracted attention from numerous researchers. Maurer et al. [16] used a method that combined SFS and stereo to establish the functional energy with a detail-preserving anisotropic second-order smoothness term, and estimated the depth of the object surface. Lu et al. [17] proposed a Lambert–Phong hybrid model algorithm, and obtained the coordinate information of the highlighted region of the droplet surface by the mask regions with convolutional neural network (R-CNN). Furthermore, they used the Taylor expansion and Newton iteration method in the reflection model to obtain the heights of all the positions. Sun et al. [18] proposed a new roughness evaluation method based on the complex wavelet enhanced SFS transform, and recovered the 3D topography of a milling surface using the 3D models that were developed from the digital image. Zhu et al. [19] proposed a novel method that uses Fourier transform to fuse the depth from focus (DFF) and SFS, and achieved the reconstruction of the tool wear image in addition to ensuring the continuity of the depth map. Yang et al. [20] used the SFS algorithm to reconstruct the 3D shapes of the welding seam, and extracted the curvature information as the feature vector of the welds from the reconstructed topography. The SFS algorithm is essentially an ill-conditioned problem. Therefore, the SFS algorithm is generally combined with other methods in studies, which adds many constraints when reconstructing the surface topography. However, it easily affects the accuracy and smoothness of the reconstructed surface.

Our analytical model is effective, simple, and easy to use, and has been widely used in the machining field. By using the machining variables such as spindle speed, feed rate, depth of cut, and step over, Ozcelik and Bayramoglu [21] developed a statistical model for surface roughness estimation in a high-speed flat end milling process under wet cutting conditions. Özel and Karpat [22] proposed neural network modeling to predict the surface roughness and tool flank wear over the machining time for a variety of cutting conditions in finish hard turning, and developed regression models to capture process-specific parameters. Based on a constitutive relation of various materials and the mechanics of their cutting process, Ning and Liang [23,24,25] presented an analytical model for predicting temperature distribution in orthogonal machining. Moreover, Ning et al. [26,27,28] also established an analytical modeling of extended chip formation, which can predict the machining forces and calculate the average temperature at the primary shear zone.

For the aforementioned problems, this study proposes a new three-dimensional reconstruction method for effectively and accurately reconstructing the machined surface topography. By combining the basic principle of SFS and the analytic method, the analytic model of the surface gradient is established by using the gray gradient as a constraint condition. The 3D reconstruction algorithm of the machined surface topography is implemented by resolving the effect of quantization errors and ambiguity of gray scale on reconstruction accuracy. This paper will lay a foundation for the online detection of machined surface quality.

2. Reconstruction Principle

2.1. Basic Principle of SFS

The basic idea of SFS is that the illuminated patches of object surfaces correspond to the local regions in the gray image [29]. The gray scale of an image is mainly related to the following four factors: the surface orientation, the illumination direction, the direction of the camera relative to the object surface, and the reflection characteristics of the object surface, as shown in Figure 1.

The normal vectors of surface can be expressed as N (−p, −q, 1), and the function of the object surface is z = f(x, y). Hence, the components of the surface gradient in the x and y direction can be expressed as:

$$\{\begin{array}{l}p=\partial z/\partial x\\ q=\partial z/\partial y\end{array}$$

(1)

We assumed that the direction vector of the light source is L(−p₀, −q₀, 1), the intensity of the incident radiance is I₀, and the reflection coefficient of the object surface is ρ. θ_i represents the angle between the normal surface and light source direction. Based on the Lambert reflectance model [30], the reflected radiance L_r can be expressed as follows:

$$L_r=I_0\rho \cos {\theta }_i=I_0\rho \frac{p_{0}p+q_{0}q+1}{\sqrt{p^2+q^2+1}\sqrt{p_0^2+q_0^2+1}}$$

(2)

It is well-known that the gray scale of the image is proportional to the reflected radiance of the corresponding point on the object surface [31]. Hence, the gray scale of the image E(x, y) that corresponds to surface orientation (p, q) can be expressed as follows:

$$E\left(x,y\right)=I_0\rho \frac{1+p_{0}p+q_{0}q}{\sqrt{1+p^2+q^2}\sqrt{1+p_0^2+q_0^2}}$$

(3)

As shown in the previous equation, the surface orientation has two degrees of freedom, and the gray scale of the image can only provide one constraint. Therefore, the SFS algorithm is essentially an ill-problem, and a unique solution cannot be obtained. Although the constraints and restrictive assumptions can be added, they can only guarantee the smoothness of the solution. Based on the rough characteristics of machined surface topography, the existing SFS algorithm is unable to fulfill the reconstruction requirements.

2.2. Analytic Reconstruction Principle

As we mentioned above, the main problem is that conducting an analysis according to the gray scale of image requires the information of the topographical gradient. Furthermore, the gray scale mainly depends on the inclination angle of surface patches. Thus, we propose a reconstruction analytic method to obtain the surface gradient, as shown in Figure 2.

It was supposed that the vector of the light source was L(0, 0, 1), which takes the lens center as the coordinates of the origin. Furthermore, the z-axis of the coordinate system coincides with the optical axis of the imaging system, and the x and y axes are parallel to the imaging plane.

During the process of imaging, the surface topography is projected onto the image plane, and the surface topography data are converted to the gray scale data. The gray surface, which can be used to describe the gray scale distribution of an image, is converted into the object surface. A flatter object surface has a greater gray scale. A_i is a point on the image plane. A_g is the corresponding point on the gray surface with a normal vector of N_g(−p_g, −q_g, 1) and inclination angle of θ_g. A_w is the corresponding point on the object surface with a normal vector of N (−p, −q, 1) and inclination angle of θ_w. φ denotes the declination angle of points.

Based on the analytic geometry theory [32], the declination angle of A_g is equal to that of A_w, both of which are φ. Therefore, the gradient relationship between the object surface and gray surface can be expressed as follows:

$$\frac{p}{q}=\frac{p_g}{q_g}=tg\phi$$

(4)

Hence, the object surface gradient can be defined as:

$$\{\begin{array}{c}p=k{p}_g\\ q=k{q}_g\end{array}$$

(5)

where k is the gradient scale coefficient. We found that the solution of the SFS algorithm will evolve into a regularization problem by using the gray gradient as the constraint condition.

The gray gradient (p_g, q_g) is given by the gray scale of image E(x, y):

$$\{\begin{array}{c}{p}_g=E{(x,y)}_x\\ q_g=E{(x,y)}_y\end{array}$$

(6)

where E(x, y)_x and E(x, y)_y represent the gradient of gray scale E(x, y) in the x and y directions, respectively.

Therefore, the object surface gradient can be written as follows:

$$\{\begin{array}{c}p=kE{(x,y)}_x\\ q=kE{(x,y)}_y\end{array}$$

(7)

Furthermore, the vector of the light source L(0, 0, 1) indicates that p₀ and q₀ are both. By substituting Equation (7) into Equation (3), we obtain an updated formula:

$$E(x,y)=\frac{I_0\rho }{\sqrt{1+k^2[E{(x,y)}_x{}^2+E{(x,y)}_y{}^2]}}$$

(8)

Hence, the gradient scale coefficient is given as:

$$k(x,y)=\pm {\left(\frac{{(I_0\rho )}^2-E{(x,y)}^2}{E{(x,y)}^2[E{(x,y)}_x{}^2+E{(x,y)}_y{}^2]}\right)}^{1/2}$$

(9)

By substituting Equation (9) into Equation (7), the object surface gradient is expressed as follows:

$$\{\begin{array}{c}p=\pm {\left(\frac{{(I_0\rho )}^2-E{(x,y)}^2}{E{(x,y)}^2[E{(x,y)}_x{}^2+E{(x,y)}_y{}^2]}\right)}^{1/2}E{(x,y)}_x\\ q=\pm {\left(\frac{{(I_0\rho )}^2-E{(x,y)}^2}{E{(x,y)}^2[E{(x,y)}_x{}^2+E{(x,y)}_y{}^2]}\right)}^{1/2}E{(x,y)}_y\end{array}$$

(10)

Therefore, the surface gradient of each point can be obtained by Equation (10), and the relative height of surface topography can also be calculated by using the integral method.

3. Implementation of Reconstruction Method

Based on the above analytic reconstruction principle, ensuring the accuracy and stability of the gray information of the image is essential in the reconstruction of surface topography. However, machined surface topography has the fractal structure of desertification [33], and the gray scale of the corresponding image easily produces quantization errors and ambiguous information. Hence, it is necessary to solve these two problems and remove their effect on reconstruction accuracy.

3.1. The Quantization Error of Gray Scale

The original image is always obtained by sampling and quantifying the continuously changing gray values [34]. Figure 3 shows the image of a semi-cylindrical surface that satisfies a Lambert reflection model and its gray curve in the x-direction, respectively. It can be seen that the quantized gray values produce random errors with a uniform distribution, and the gray curve presents a discontinuous step-like change. Hence, if the derivative of gray is calculated by using the original image, it will inevitably result in abnormalities in the gray gradient, as shown in Figure 4.

The wavelet denoising algorithm is adopted to eliminate quantization errors [35], which can maintain the details of the gray information and smooth the discontinuity of gray values. By setting an appropriate wavelet decomposition level and quantization threshold for decomposition coefficients, the curves of gray scale and gray gradient were smoothed, as shown in Figure 3b and Figure 4, respectively. It can be seen that the problem of the gray random errors has effectively been solved.

3.2. The Ambiguous Gray Scale

According to Lambert’s reflection theory, the surfaces with an equivalent gradient have the same reflection characteristics. The surface with two peaks and valleys is shown in Figure 5a. The synthetic image of a light source located at (0, 0, 1) is shown in Figure 5b. It can be seen that the gray scales of the peaks are consistent with that of valleys and the peaks. However, the valleys in the surface cannot be distinguished in the synthetic image. Therefore, the image of a surface with multiple peaks and valleys is prone to producing ambiguous gray scale information, resulting in an inability to identify the convexity and concavity of the surface.

The relationship between the topography, gradient, and gray scale of the above-mentioned surface in the x-direction is shown in Figure 6. We can see that the ambiguous gray scale information will mutate the calculated surface gradient, and the mutational points are mainly concentrated in the valleys. Hence, for solving the ambiguous problem of the gray scale, it is important to identify the valley from the image and correct the corresponding calculated surface gradient.

Based on the principle of optical imaging, the points on the focal plane can form a clear image on the imaging sensor, while the image formed by the points outside the focal plane is not clear. Due to the focal plane usually being close to the peaks of surface topography, the images of the peaks are relatively clear. Hence, the clarity of image can be used to identify the peaks and valleys from the image. The image definition represents the clarity of image details and textures [36]. If the image contains richer details, there will be a sharper change in the gray scale that will ultimately produce a clearer image, as shown in Figure 7.

The entropy function evaluation algorithm is an effective evaluation method for image definitions, which calculates the information entropy of an image to evaluate the clarity. The entropy function H(E) can be defined as:

$$H(E)=-{\displaystyle \sum _{x=1}^m{\displaystyle \sum _{y=1}^{n}E(x,y)In[E(x,y)]}}$$

(11)

where m × n represents the image size. A larger H(E) will result in a clearer image.

Moreover, the machined surface topography has the statistical self-similarity and self-affinity, as the gray distribution of different rows in the image is similar [37,38]. Therefore, during the process of correcting the surface gradient, each peak value of a gray scale row is first extracted and the sub-images with the same size are collected by taking each peak value as the center. After this, the information entropy of each sub-image is calculated by the entropy function H(E), and the average entropy of all the sub-images is obtained. After this, the sub-image with an information entropy that is lower than the average entropy is defined as the surface valley. Finally, we recognized the column position of the above peak values of the valleys in the image. The gray scales between the minimums on both sides of the column are extracted, and the corresponding calculated surface gradient values are corrected.

3.3. Reconstruction Algorithm

Based on the above-mentioned points, a schematic of the reconstruction algorithm is presented, which is shown in Figure 8. The algorithm mainly consists of four procedures: denoising and smoothing of the original image, analytic calculation of the surface gradient, correction of the mutational surface gradient, and calculation of the topographic relative height.

(1) We read the data of the original image. The gray scale E(x, y) can be obtained without quantization errors by using the wavelet denoising algorithm.

(2) Based on the gray scale E(x, y), the gray gradient (E_x, E_y) can be obtained by the following equation:

$$\{\begin{array}{c}E{(x,y)}_x=(E(x+1,y)-E(x-1,y))/2\\ E{(x,y)}_y=(E(x,y+1)-E(x,y-1))/2\end{array}$$

(12)

Moreover, the surface gradient corresponding to the gray scale E(x, y) of each point in the image is calculated by Equation (10).

(3) The mutational points are found, and their corresponding calculated surface gradients are corrected by adopting the above-mentioned correction method.

(4) We assumed that z₀ is the height of the reference point (x₀, y₀) in the image, and subsequently set the height of the reference point. Thus, the relative height of any point on the object surface can be calculated as follows:

$$z(x,y)=z_0+{\displaystyle {\int }_{(x_0,y_0)}^{(x,y)}{p}_{c}dx+q_{c}dy}$$

(13)

4. Experiments

To verify the effectiveness and accuracy of the proposed reconstruction algorithm, we have conducted several experiments on both synthetic images and machined surface topography images. The image and data processing is performed on a PC with AMD Ryzen processors. The proposed reconstruction algorithm is implemented using MATLAB codes.

4.1. Experimental Results on Synthetic Images

The experiments employ the images of a semi-spherical surface and sinusoidal surface, which are the benchmark images in some studies [39,40]. The semi-spherical and sinusoidal surfaces are generated by the surface formula, which is shown in Figure 9a and Figure 10a, respectively. The diameter of the semi-spherical is five mm. The amplitudes of the sinusoidal surface are 2.5 mm and 0.5 mm, respectively, and the frequency difference is five times. The two synthetic images are formed by the Lambert model, where the vector of the light source is (0, 0, 1).

Figure 9b and Figure 10b illustrate the synthetic images of the two original surfaces, while the reconstructed surfaces are shown in Figure 9c and Figure 10c, respectively. Figure 9d and Figure 10d show the errors between the original surfaces and reconstructed surfaces. It can be seen that the reconstructed surfaces are similar to the original surfaces, with the maximum error values being less than 0.154 mm and 0.0178 mm, respectively. The reconstruction errors mainly occur near the boundary, while the errors of the other points are negligible. This is due to the discontinuity of the boundary surface resulting in the abnormal change in the gray scale. At the same time, it is apparent from the figure that although there is ambiguous gray-scale information in the synthetic image, the peaks and valleys of the surface can be accurately recovered using the proposed correction method.

To evaluate the performance of the proposed reconstruction algorithm, the comparison of the original surface and the reconstructed surface is studied using the mean absolute error (MAE) and root mean square error (RMSE). The results are shown in

Table 1. The comparison results. MAE: mean absolute error, RMSE: root mean square error.

Synthetic Image	MAE	RMSE
Semi-spherical	0.087	0.982
Sinusoidal	0.013	0.965

.

As shown in

Table 2. The comparison results of profiles.

	Turing Surface			Milling Surface			Boring Surface
	Ra (µm)	Rz (µm)	Rq (µm)	Ra (µm)	Rz (µm)	Rq (µm)	Ra (µm)	Rz (µm)	Rq (µm)
Measured data	3.331	13.325	3.642	1.746	6.984	1.972	1.691	6.765	1.886
Reconstructed data	3.525	13.621	3.924	1.820	7.121	2.091	1.743	6.893	1.998
Relative error (%)	5.5	2.17	7.18	4.06	1.92	5.69	2.98	1.85	5.6

, the MAE between the reconstruction surface and the original surface is small, and the maximum RMSE is only 0.982. It indicates that the reconstructed surfaces are consistent with the original surfaces. The verification results show that the proposed reconstruction algorithm performs well and has high accuracy.

4.2. Experimental Results on Machined Surface Topography Image

Machining surfaces from turning, milling, and boring operations were recovered using the presented algorithm for the investigation of effectiveness. By using a Leica DCM 3D white light confocal interference microscope, the images for reconstruction containing 512 × 512 pixels are collected, and the corresponding topographic data are measured. Moreover, the reconstruction results are compared with that of the existing SFS algorithm of Horn and Brooks. Figure 11, Figure 12 and Figure 13 show the measured results and reconstruction results of machined surface topography, respectively.

From the above-mentioned reconstruction results, it can be seen that some peaks of the reconstructed surface contain distorted topography. This is due to the effect of light or impurities created during the imaging process, resulting in abnormal gray scales in the image. This is shown in the red boxes in Figure 12a. Figure 11c, Figure 12c, and Figure 13c show the surface topography recovered by the Horn and Brooks algorithm. We can see that a considerable number of detailed features are lost, which means that we cannot clearly show the texture and undulation of the surface. Furthermore, the heights of the reconstruction surface are higher. It is mainly caused by the algorithm, including the excessive smooth constraints. Moreover, we can see that the whole undulation of topography recovered by the proposed algorithm is basically the same as that of the measured results. The reconstruction surface details are abundant, and the reconstruction effect is much better than that of the Horn and Brooks algorithm, as shown in Figure 11d, Figure 12d, and Figure 13d.

We extracted the profiles of the same position from the above-mentioned topography and the profile between the reconstructed topography and measured topography, which were compared. The results are shown in Figure 14.

It can be seen that the profiles reconstructed by the Horn and Brooks algorithm are different to the measured profile, and the deviations between them are larger. Hence, the reconstruction method based on the Horn and Brooks algorithm is not suitable for machined surface topography. On the contrary, the spacing and height of peaks and valleys between the profiles that were reconstructed by the proposed method and measured profiles are similar. Due to the interference of mutual reflection, secondary reflection, noise, or stray radiation, which are inevitable in the imaging process, the results in some details of the profiles are different. However, from the distribution of peaks and valleys, the profiles reconstructed by the proposed method are very close to the measured profile.

The arithmetic average roughness (Ra), maximum height of the profiles (Rz), and root mean square roughness (Rq) are used to compare the profile reconstructed by the proposed method and measured profile. The results are shown in Table 2.

As shown in Table 2, the relative error between the reconstructed profile and the measured profile is small: less than 10%. It indicates that the reconstructed data are very close to the measured data, and the reconstruction accuracy can meet the requirements of surface topography measurements. Therefore, the experimental results demonstrate that the proposed algorithm can accurately and effectively reconstruct the machined surface topography.

5. Conclusions

(1) Based on the basic principle of the SFS algorithm combined with the analytic method, a new three-dimensional (3D) reconstruction method is introduced to recover machined surface topography. By using the gray gradient as the constraint condition, the relationship between the surface gradient and gray gradient is described, and the analytic model of the surface gradient is developed. Moreover, the problem of SFS can evolve into a regularization problem using the proposed reconstruction principle.

(2) The quantization error and ambiguity of the gray scale has a significant influence on the reconstruction accuracy of machined surface topography. Hence, we apply the wavelet denoising algorithm and the image processing technology to effectively solve those problems, and develop the 3D reconstruction algorithm for machined surface topography.

(3) Experimental results on synthetic images and machined surface topography images reveal that the proposed algorithm is effective, and the profiles of the reconstructed topography are very close to the measured profiles with a relative error of less than 10%.

(4) The proposed three-dimensional reconstruction method solves the problem of accuracy and smoothness of the reconstructed surface. However, there must be a “more or less” limitation that the geometric features of specimen surface topography are relatively good. Hence, it is quite valuable to extend the measured surface.