Outlier Elimination in Rough Surface Profilometry with Focus Variation Microscopy

Abstract

Rough surfaces such as metal additive manufactured surfaces are quite challenging for measurement. Artifacts caused by irregular and difficult-to-measure geometries are inevitable. Removing all the artifacts would cause a portion of surface information to be missing. Different from previous works, the postprocessing in this paper includes an additional step to eliminate artifacts based on autocorrelation functions of particular subimages instead of simply removing them. This increases the accuracy with respect to surface roughness and provides a more comprehensive view on the topography. In addition, a dome shape LED array ring light is proposed to provide all-round lighting due to the high degree of irregularity of workpiece surfaces. The experimental results obtained from FVM are validated and compared with the given roughness values of a Rubert Microsurf 329 comparator test panel as well as measurement results of a metal additive workpiece by a confocal microscope.

1. Introduction

Metal additive manufacturing (AM) techniques play an important role nowadays in producing both prototypes and final components in many industrial fields [1]. The essential advantages of these techniques are the rapid manufacturing process compared to conventional methods for metal processing and the capability of complex designs such as lattice structures or honeycomb structures based on digital models produced from computer-aided design (CAD) software [1,2]. Profilometry of such surfaces and their texture parameters provides benefits for quality control and the investigation of AM processes. Compared to turned or ground surfaces, AM processes typically generate much rougher surfaces with high irregularity, which appears as sharp protrusions, deep recesses, critical undercuts, transitioning from open pores to close pores, etc. [3,4]. These irregularities vary due to different AM processes, such as power bed fusion (PBF), material extrusion, material jetting, sheet lamination, and directed energy deposition (DED) [5]. Most commonly used processes are based on PBF, which produces less rough surfaces and can be further divided into laser-PBF and electron beam-PBF [4]. Profilometry is quite a challenge due to such surface properties.

Appropriate measuring methods are tactile stylus profilometry [6,7], confocal scanning microscopy (CM) [8,9], X-ray computed tomography (XCT) [10,11,12], focus variation microscopy [3,13,14,15], SEM stereoscopy [16], and coherence scanning interferometry (CSI) [17]. Tactile stylus measurement, as a contact method, exhibits critical disadvantages compared to non-contact optical methods as there is a challenge with respect to softer materials at risk of damaging the workpiece surface. Although enabling easy employment [18], one of the challenges of applying non-contact optical methods is that the measurement may be influenced by steep flanks or recesses on the surface where light is barely reflected and finally captured by the camera. Previous works compared some non-contact optical instruments, such as CM, CSI, FVM, and XCT [18]. The profiles measured using CM contains much more spikes compared to the other instruments. There are many invalid pixels on the topography measured with CSI. FVM and XCT present an equivalent performance, however, FVM has lower installation effort and is thus more economical. The profile discrepancy between the surface height values obtained by two different instruments is typically in the order of 50%. It is hard to provide a relatively reliable conclusion about which instrument provides the highest topography fidelity since it depends on what type of surface is measured and how big is the roughness of the surface.

The most frequently used profile roughness parameters are the arithmetic mean value $R_a$ and maximum peak to valley deviation $R_z$ according to DIN EN ISO 4287 [8,19,20]. These parameters are extended to areal roughness parameters, which are expressed by $S_a$ (from $R_a$) and $S_z$ (from $R_z$) for instance [19]. Typical $S_a$ values of AM surfaces manufactured by PBF are in the range of 5 to 30 µm.

Generally, FVM is based on a focus detection of a singular pixel during a vertical scanning process (depth scan), leading to the height value related to the corresponding pixel based on the intensity contrast of a local area. This requires a certain degree of roughness of the workpiece surface and a small depth of field (DOF) of the system. Therefore, it is quite difficult to measure rather smooth surfaces which produce little contrast [3,13,14,15].

1.1. Background of Focus Variation Technology

FVM is an optical system that detects the focus position of a surface section for each pixel of a CCD/CMOS camera. The focus is obtained by searching for the maximum value $\mathsf{\sigma }$_max of the so-called contrast curve. In order to obtain the contrast curve of a single pixel on the CCD camera, a local window comprising 2n × 2n pixels centered with respect to this pixel is utilized. Intensities measured by pixels within this window are used to calculate the contrast [21,22]. In our algorithm it is expressed by the standard deviation of the intensities within the window:

$$\sigma (k,l,m)=\frac{1}{2n}\sqrt{\sum _{i=k-n}^{k+n}\sum _{j=l-n}^{l+n}{(I(i\Delta x,j\Delta y,m\Delta z)-\overline{I})}^2},$$

(1)

where $(k\Delta x,l\Delta y,m\Delta z)$ are the coordinates of the central pixel of the window, n is the number of pixels in one dimension inside of the window, x and y are the intervals between two pixels, z is the z-interval between two adjacent images of the focus scan, I is the intensity, and $\overline{I}$ is the average intensity inside the window.

In addition to obtaining a contrast curve by calculating standard deviation values, there are other methods to find out the focus position of a single pixel, e.g., the Laplace operation [22,23,24,25]. To increase the performance and summation robustness of two filter masks in two dimensions, Helmli and Scherer [22] suggest to square the second derivation. Another approach is presented by Tenengrad, who utilizes the summation of the square of the results after applying the two dimensional Sobel masks [26,27,28]. The variance of the intensities in the local window can be applied to detect the focus, and similarly another method which is based on the ratio between $I(k\Delta x,l\Delta y,m\Delta z)$ and $\overline{I}$ is introduced in [22]. During the investigation of different algorithms for calculating the contrast curve, it is found that using the standard deviation produces more accurate results for calculating $S_a$ and $S_z$, compared to Laplacian and Tenengrad methods.

1.2. Motivation Statement

In general, the algorithm applied to FVM includes several aspects: focus detection on each pixel, reconstruction, and postprocessing which includes invalid-points-searching, invalid-points-removing, and filling the missing points of the topography by neighboring points. In this paper, we add one more step to eliminate the artifacts generated by invalid points between the steps of invalid-points-searching and removing to present a more comprehensive topography, which is especially beneficial for the measurement results in critical cases when artifacts occur on an unneglectable portion of the surface. The elimination is based on autocorrelation functions (ACF) of particular subimages. Therefore, the detected artifacts appearing as spikes are not simply removed but corrected.

In previous work [29], the approaches of searching for invalid points (producing artifacts) are either introduced in a vague way or related to specific cases instead of general solutions. In this paper, a new general approach is introduced.

2. System Setup

In the experimental setup of the FVM used in this paper, a high-power green LED as coaxial illumination and a dome shape ring light employing red LEDs are used, as depicted in Figure 1a. The sketch of the setup is shown in Figure 1d. Due to the reasons for the mechanical arrangement, there are a mirror and a condenser between the collimator and beam splitter. The 3D printed ring light and its 3D model are shown in Figure 1b and Figure 1c, respectively. During the depth scan, a stack of images is captured as it is shown in Figure 1e. The contrast curve for a single pixel of the camera and the corresponding maximum contrast $\mathsf{\sigma }$_max in the curve are illustrated in Figure 1f. Two exposure times are applied to improve the measurement accuracy for a certain type of assessed surface with deep recesses. A measurement approach of two-exposure-times can be achieved by two measurements with different exposure times for instance, which is used in this paper or by using two exposure times in the same measurement process. The colors of the ring light will be consistent later with the coaxial illuminations. In this paper, the self-made ring light is dome shaped and has a working distance between $15$ mm and $20$ mm. Compared to cone shaped ring lights, the dome shape improves the efficiency of the illumination. It is designed according to the working distance of the microscope objective.

The $10\times$ objective lens has a numerical aperture (NA) of 0.45. For a wavelength of 623 nm, its lateral resolution and DOF are approx. 0.84 µm and 3.07 µm, respectively. The DOF $d_z$ depends on both, the wavelength of illumination light and the NA of the objective lens according to $d_z\sim {\displaystyle \frac{\lambda }{N{A}^2}}$. The camera in the used setup is a ALLIED VISION GF146B ASG Guppy CCD camera equipped with IEEE 1394a interface, which comprises 1392 × 1040 pixels, 8 bits resolution and frame rates up to 17.7 fps. The effective pixel numbers for the experiments are 1280 × 960 pixels with a field of view (FOV) of approx. 585 µm × 438 µm. The images captured by the camera are stored and later evaluated in the computer. After the evaluation, matrices containing the height information are processed by the 3D analysis software ’Mountains Map’ from Digital Surf (MMDS).

The step size Δz used in a measurement is 0.1 µm or 0.2 µm. In order to increase the measurement FOV, multiple laterally shifted measurements are performed and finally, the resulting surface topographies are stitched together with a lateral overlap of approx. 150 µm.

3. Methodology

3.1. Samples

The samples used throughout this paper are shown in Figure 2. Figure 2b shows a sample with a metal additive surface, named MA0 in the following. This topography is used to explain the algorithm in detail in Section 3.3. The microsurf sample shown in Figure 2a contains several plates of approximately 15 mm × 24 mm, produced by either gravel blasting or ball blasting. This is used to verify the algorithm performance, based on roughness $R_a$ and $R_z$ values given by the manufacturer. Figure 2c shows the metal additive surface named MA1.

3.2. Data Preparation

Validation is performed by measuring Rubert Microsurf 329 comparator test panel and MA1. For each sub-sample, eight topography data sets are stitched together laterally in one direction to obtain a total area of approximately 450 µm × 2.66 mm. On the test panel, three sub-plates representing gravel blasted surfaces and three sub-plates representing ball blasted surfaces are measured. Since only $R_a$ and $R_z$ values are given by the manufacturers, only these two parameters are indicated. Workpiece MA1 is measured by FVM and CM µsurf custom from Nanfocus AG. The measured ROI is roughly synchronized.

3.3. Artifact Analysis and Data Postprocessing

A window comprising 5 × 5 pixels is applied for calculating the standard deviation of the corresponding intensity values. This procedure is repeated, while the window is laterally shifted pixel by pixel. Artifacts such as the previously mentioned spikes can be reduced by increasing the number of pixels covered by the standard deviation window. However, in this case the evaluation time increases significantly and the lateral resolution decreases. Compromising among the lateral resolution, the calculation time and the accuracy, a suitable window size is 5 × 5 pixels.

A contrast curve generated from a single pixel is in principle similar to a Gaussian curve. However, if the contrast curve does not show a Gaussian the focus detection becomes more demanding. There are mainly three cases where $\mathsf{\sigma }$_max is not corresponding to the focus position: First, only noise signals are captured. Second, the signal to noise ratio is too small to detect the correct focus position. Third, a bigger standard deviation occurs at a position of a defocused image as shown in Figure 3. The contrast curve in this figure is a result after applying spline interpolation. The dashed line called `threshold for curve fitting’ is used to choose a part of the contrast curve above this threshold for a parabolic curve fitting. We chose a threshold value of 70% of the maximum value $\mathsf{\sigma }$_{max_0} of the corresponding contrast curve. The maximum value of the fitted curve is the value of $\mathsf{\sigma }$_max.

Figure 3 represents the contrast curve of the pixel marked by the yellow cross in Figure 4a,b. $\mathsf{\sigma }$_max corresponds to $z=$192 µm with respect to the image shown in Figure 4b, which is defocused but produces a higher standard deviation because the pixel is located at the boundary between the bright and dark regions. In this case, a special filter is needed which filters out the position $z=$192 µm so that the correct position $z=$ 72 µm will be found, which is related to the image shown in Figure 4a. Figure 4a–h explains how the filter works. Figure 4i–k illustrates how the invalid pixels are detected.

A method to determine the Root Mean Square roughness by analyzing local ACFs of speckle patterns in the Fresnel region is introduced in [30]. The ACF of an M × N image submatrix is calculated according to Equation (2),

$$\rho (m,n)=\sum _{k=0}^{M-1}\sum _{l=0}^{N-1}I(k,l)I(k+m,l+n),$$

(2)

where $m=0,1,\cdots ,M-1$ and $n=0,1,\cdots ,N-1$ [30,31]. ACF results related to the subarea around the yellow cross in (Figure 4a,b) are shown in Figure 4c,d, respectively. The window size used to calculate the ACF is chosen to be $100\times 100$ pixels. If the local area is in focus, this leads to a narrower maximum of the ACF shown in Figure 4c compared to the case shown in Figure 4d where the defocus results in a much broader maximum peak.

A series of such sub-images centered by the yellow cross pixel are extracted from the corresponding image stack. The distance of adjacent images is 5 µm corresponding to 25 times the step size 0.2 µm. ACF results for each subimage are calculated. A threshold (e.g., 70% of the maximum value) is applied to each ACF result, and the number $N_p$ of pixels with ACF values higher than this threshold is used as a parameter to preselect the focus region. After an interpolation between subsequently obtained $N_p$ values, a normalized curve (solid blue curve in Figure 4e) is obtained. The height position corresponding to the minimum value of this curve is chosen as the center position of the ACF filter, which is shown by the red dashed rectangular curve in Figure 4e. The filter window width is normally chosen to be 25% of the whole depth scan range. The original contrast curve is multiplied by the ACF filter function to form a new contrast curve for the focus re-detection. Not in all cases does the ACF filter lead to the correct depth range. For instance, at some defocus positions, camera noise dominates. This would lead to extremely low $N_p$ values as well, as shown in Figure 4f. Therefore, the positions where the camera noise dominates and which are far away from the height values of the workpiece need to be excluded.

A method to achieve this is to use a median filter comprising 100 × 100 pixels applied to the reconstructed surface shown in Figure 4i. The colors are artificially created and each color represents a certain height according to the color bar. Errors of wrongly calculated height values are represented by spikes. The result after applying the median filter shown in Figure 4j leads to a surface without spikes, which indicates the most probable height range of the real surface. Its maximum and minimum position are used to limit the center of the ACF filter. This results in the correct height range obtained by ACF filtering shown in Figure 4g. The difference between Figure 4i,j represents the high frequency components of the topography including the outliers as it is shown in Figure 4k. Because Figure 4j is an approximate topography, the outliers are approximations as well. Therefore, only the outliers beyond a certain threshold are seen as potentially problematic pixels. Outliers (spikes) are directed upwards and downwards. Therefore, two thresholds are applied. This results in Figure 4h, which show all the potentially problematic pixels in green (upwards) and black (downwards). The size of the green or black points is plotted by a bigger area for better visualization. The relative number of these problematic pixels is approximately 7.32% of the total number of pixels of the camera. Some of the pixels are gathered as clusters and some of them are singular pixels. Each cluster covers one or more subimages used for the ACF filtering, depending on the individual cluster size. Determination of the centers of these clusters is achieved by using the `k-means’ algorithm [32]. The red points in Figure 4h represent these centers. An area with the size of 100 × 100 pixels surrounding each center defines a subimage for the ACF filtering. The corrected reconstructed surface topography corresponding to the area according to Figure 4a is shown in Figure 4l.

Not all artifacts can be corrected. The artifacts are caused either by defocused images or the low signal-to-noise ratio (SNR) of certain pixels. The reason for low SNR is that no height information is recorded at all due to the low intensity of the corresponding pixels. In such a case, a higher exposure time is used to get more information on dark regions so that more light intensity related to those areas is obtained. This is called compensation. However, some other regions of the image might then be overexposed. Therefore, only the reconstructed areas related to the dark regions are replaced by the results from higher exposure time and thus compensated. However, due to the geometry, e.g., in the case of deep recesses, some regions are still dark even though the exposure time is increased. In addition, high exposure time brings a higher degree of defocus. As a result, invalid pixels will be left out and the corresponding height values will be reconstructed by using the information of the neighbouring points.

The topography dataset after the postprocessing is analyzed using MMDS, which evaluates the surface topography according to ISO 25178. First, topography datasets are stitched, if necessary. Next, the tilt of the determined surface is removed. Then, a median filter with a window size of $9\times 9$ pixels is applied to eliminate errors caused mainly by left out spikes. The calculation of areal roughness parameters $S_a$ and $S_z$ excludes the step of filling-missing-points.

4. Measurement Results

4.1. Comparison of Topographies before and after Using ACF Filter and Different Exposure Times

Figure 5a shows the surface topography corresponding to Figure 4i without correction by the ACF filter. The result after removing the invalid pixels is shown in Figure 5b. After the ACF filter is applied, the dark regions (Figure 5a) where spikes are located are mostly replaced by valid height values as shown in Figure 5c. The result after removing the remaining invalid pixels in Figure 5d presents fewer invalid pixels (3.12% instead of 7.32%). Therefore, more information of the surface is kept due to the ACF filtering. Figure 5e shows the results of the compensation and Figure 5f shows how it looks after removing the pixels which are still invalid in (e). There are even fewer invalid pixels appearing compared to Figure 5d (2.19% instead of 3.12%), which means that more information is kept.

4.2. Validation with Rubert Microsurf 329 Comparator Test Panel

The Rubert Microsurf 329 comparator test panel shown in Figure 2a is used for validation. Three sub-samples produced by gravel blasting show nominal roughness parameters $R_a$ of 10.5, 18, and 25 µm, marked as MSS10.5, MSS18, and MSS25 respectively. Similarly, MSK8, MSK13, and MSK18 represent ball blasted surfaces. Only $R_a$ and $R_z$ values are given by the manufacturer. The results are shown in

Table 1. Comparison of Rubert Microsurf 329 comparator test panel between $S_a$ and $S_z$ values measured by FVM and $R_a$ and $R_z$ µm values given by the manufacturer.

	Samples	Manufacturer Values (µm)	No Correction (µm/%)	Correction and Compensation (µm/%)
$R_a$/$S_a$	MSS-10.5	10.5	9.76/7	9.71/7.5
	MSS-18	18	19.02/5.6	18.9/5
	MSS-25	25	26.5/6	26.4/5.6
	MSK-8	8	9.53/19.1	9.43/17.87
	MSK-13	13	16.1/23.8	15.9/22.3
	MSK-18	18	19.64/9.1	19.54/8.6
$R_z$/$S_z$	MSS-10.5	63	119	100
	MSS-18	108	216	181
	MSS-25	150	249	245
	MSK-8	48	176	105
	MSK-13	78	125	114
	MSK-18	108	182.4	171

. `No correction’ indicates that the results are obtained without ACF filter and compensation, which differs from `correction and compensation’.

The results exclude the step of invalid-pixel-removing, in order to show the pure performance of correction and compensation. Therefore, measured $R_z$ values are much higher compared to the nominal values. In most cases, the manufacturers use tactile stylus instruments to measure $R_a$ and $R_z$ values according to ISO standards. Previous work pointed out that comparing the performance of stylus and optical methods, the degree of discrepancies depends on what kind of surface topography is being measured. In addition, it is more trustworthy to use statistical height parameters of a surface instead of those parameters representing maximum or minimum height values, which are sensitive to the spikes (artifacts) [33]. Therefore, the large discrepancies of $R_z$ between FVM and the nominal values of the manufacturer are to be expected. It is obvious that, except for MSS-10.5, the deviation of measured $S_a$ values is smaller compared to the given $R_a$ values when correction and compensation are applied. The same measurements of MSS-18 are repeated six times on different subareas, the obtained deviations are 16.7%, 17.2%, 5%, 13.3%, 1.1%, and 8.3%. This results in an average deviation of 10.26%, which means that a bigger accessed area can provide a more reliable assessment of the surface topography.

4.3. Measurement of MA Surface

According to [18], it turned out that the profile discrepancy of two different instruments among CM, CSI, FVM, and XCT is about 50%. In this section, a CM is used for the measurement comparison of MA1. $S_a$ values obtained by CM and FVM are 16.2 and 15.8 µm respectively, corresponding to a discrepancy of only 2.5%. $S_z$ values obtained by CM and FVM are 144 and 133 µm, respectively, the discrepancy of which is about 8%. The results include the step of removing invalid pixels. The similarity of CM and FVM measurement results is observed in Figure 6a,b although the color bars related to both results represent slightly different height ranges. Therefore, only the shapes should be used for comparison instead of colors. The surface measured by CM overlaps approximately two-third of the surface measured by FVM, which might be the main reason for the discrepancy.

The validation from the Rubert Microsurf 329 comparator test panel and MA1 prove that the algorithm introduced in this paper is applicable for both cases. It is expected that there are deviations among the measurement results and surface characterization for different surface topography measuring instruments. These deviations mainly occur in those regions of the surface, where steep slopes, recesses, and attached spherical geometry elements are located. Incompletely overlapping surface sections also contribute to deviations. Further effort will be needed in the future to obtain the same ROI for the comparison.

5. Conclusions

In this paper, an algorithm of postprocessing applied in FVM to reconstruct very rough surfaces of $S_a\approx$ 20 µm has been introduced and discussed in detail. The main idea of the algorithm is to apply two methods for focus detection simultaneously: the standard deviation of image intensities in a subimage as well as the ACF of subimages. To achieve satisfying results with the ACF filtering, a bigger ACF window of at least $100\times 100$ pixels is required. Therefore, the focus detected by this method represents only a coarse height range for the corresponding ACF window. The individual height value of each pixel inside the ACF window falls into this range. An advantage of the ACF method is that it enables the elimination of the influence of defocused images. The advantages and disadvantages of using the standard deviation as the second method are quite the opposite. The standard deviation window size is only $5\times 5$ pixels resulting in a better accuracy but it is influenced greatly by defocused images. Hence, the two approaches complement each other and thus their combination leads to a trustworthy algorithm.

A Microsurf test panel of known roughness values and an AM workpiece were measured to evaluate the performance of this algorithm, in comparison with nominal values given by the manufacturer and values obtained by a commercial CM. The deviation of measured surface roughness values $S_a$ of the Microsurf sample by FVM ranges from 1.1% to 20% compared to the nominal value. The discrepancy of $S_a$ values of the AM workpiece measured by CM and FVM is only 2.5%. The measured topographies of overlapping ROIs by CM and FV are quite consistent with a small portion of unavoidable discrepancies due to the different instruments.

The measurement results indicate that the introduced algorithm is applicable for AM surfaces. There are many other algorithms for focus detection that are worth studying. However, the effects of obtaining invalid height information when using a single contrast criterium are to be expected even if these algorithms are applied. In addition, a well-designed dome shaped LED array ring light in combination with coaxial illumination is important for the algorithm’s performance. Finally, due to the particular properties of AM surfaces with a high degree of irregularity, the use of two exposure times is also essential. Care should be taken when the estimated $S_a$ value of assessed surfaces is large or the surface is anisotropic, so that the FOV should be extended in one or two dimensions.

The system used to obtain the results of this paper is only a prototype. There are many ways to improve its performance. In further work, illuminations with different exposure times will be combined with different channels of a color CCD. The design of the ring light will also be optimized. More workpieces will be measured to test and further improve the algorithm.