Focus Estimation Methods for Use in Industrial SFF Imaging Systems

Abstract

A Shape-From-Focus (SFF) is a three-dimensional imaging method based on focus information. It is not yet widely used for in-line industrial inspection or measurement tasks. The main reasons are the time it takes to capture a 3D image of the inspected product and the presence of interference affecting image quality. This paper compares operators for estimating focus in source images in the scope of their use in constructing an industrial 3D scanner. Interferences were introduced by using additional illuminators and changing the acquisition parameters. The use of industrial-grade cameras, industrial-grade illuminators, and electrically controlled optics are discussed. A novel approach in the research is using an electrically tunable lens to move the position of the image plane during 3D image acquisition. The research was conducted on various surfaces found on typical industrial products. The research showed which focus estimation operators can be applied to SFF imaging within the range of interference considered. It was also confirmed that using the centre of gravity method for scene reconstruction allows for an increase in resolution compared to the maximum method.

1. Introduction

One of the rapidly growing fields of application of 3D imaging is the quality control (QC) of products on industrial manufacturing lines. QC tasks are carried out by using different vision systems and imaging methods. The selection of the imaging method depends on the range of inspection tasks to be carried out on the image, the required imaging resolution and the options for installation on the production line. Product inspection using a vision system is present in many industries for measurement tasks [1,2], inspection tasks [3,4], and production planning [5].

The main 3D imaging methods used in QC are laser triangulation [6], stereovision [7,8] and time of flight (ToF) [9]. The scope of the application of each method varies due to hardware configuration, required imaging resolutions, and the influence of interference on the ability to produce a 3D image [10].

Laser triangulation imaging requires the use of a camera and a laser illuminator and the selection of a system geometry [11]. Regardless of the selected system geometry, the 3D image contains no-data areas, occlusions, resulting from the impossibility of illuminating or imaging all points on the surface. Stereovision imaging requires two cameras or a camera with structured light illumination. Significantly better images are captured by using additional structured light illumination. However, this only partially prevents the formation of occlusion areas in the 3D image. The ToF method is used for imaging in workspace inspection tasks. It allows for the reading of spatial information by measuring the wave propagation time. The method allows large spaces to be imaged but with significantly worse measurement resolution.

This study uses the Shape-From-Focus (SFF) method to determine spatial information by analysing a stack of 2D images. Images are captured when the position of the image plane is modified. Images in the stack are captured at a different position in the image plane relative to the scanned object (Figure 1). By changing the position of the image plane, the focus of specific areas changes in the subsequent raw image. The analysis and selection of focused areas, together with information on the image plane position, allow the reconstruction of a 3D image.

The development of the SFF method expands the potential for quality control of products under industrial conditions. Research on the SFF method is expected to increase the range of measurements that automatic inspection and measurement systems can achieve.

Classic solutions were based on moving the imaging plane with the optics and camera on the z-axis. This method allowed the capture of image series of the XY-plane from different heights due to the camera’s movement in the z-axis (Figure 1a). The time required to register the image series depended on the number of images and the speed of the camera. The disadvantage of this solution is the presence of a motor installed on the z-axis of the vision system which propagates vibrations when the position of the image plane changes [12,13].

This paper presents an alternative solution for changing the position of the image plane in the z-axis without the need to move the camera. The use of a tunable liquid lens was proposed. Such a lens allows the optical power of the lens to be changed without camera movement and in a significantly shorter time (Figure 1b) [14,15,16].

An image stack was captured on a stand where the camera sensor was positioned parallel to the surface under examination. Using a tunable lens, the Z position of the image plane was changed, and successive images were registered. The tests were carried out on three flat surfaces made of different materials. The materials were selected based on an analysis of industrial QC tasks performed by vision systems. The test stand includes industrial-grade cameras, lenses, and illuminators. Industrial-grade materials were also selected for the stand to reproduce industrial conditions and possible interference accurately.

The influence of environmental conditions was tested on operators already described in the literature. The operators were implemented for focus estimation on the stack of images. Based on a literature review and preliminary research, 18 operators (digital filters) were classified. The use of operators was conducted by Pertuz [17] based on simulated scenes and images from a low-resolution camera. The stacks contained between 25 and 51 images. A study of operators in the field of focus evaluation was also carried out by Fu [18]. The authors simulated surface shapes in the form of a plane surface, a cone, and a sine wave. In addition, they recorded images of a hat, a doll, and a cup. Alonso [19] used focus estimation operators on several partial-focus images to generate an image in focus over the entire distance range.

A set of operators was selected and tested based on the literature analysis for potential use in industrial applications. The operators were selected based on the analysis of the available results and the described computational complexity. Computational complexity is essential for implementing a solution in a real-time system. Cited papers related to studies performed under laboratory conditions. Tang [20] described the use of the SFF method for imaging of a grinding wheel in a microscopic setup. Thelen [21] presented an algorithm for face reconstruction using the SFF method. The influence of the applied operator and the window size was also described. However, the cited articles did not reach the functional prototype stage; they were proofs of concept. The literature has also addressed issues related to the topic of autofocus [22,23,24,25].

Another approach not covered in this paper is blind image quality assessment. Such algorithms are image assessment techniques based on a pre-prepared model. Such algorithms allow image quality assessment without recalibrating the model for each new scene. However, this approach only provides a single rating for the entire image and cannot be used to select and assess specific regions of interest, which is required by the SFF method [26,27].

This article differs from the cited publications by imaging the surfaces of actual industrial products. Recorded images have great metrological potential due to the 5 MPx resolution of the sensor. A dedicated camera for industrial solutions was used. A test stand was constructed to allow reproducible algorithm testing using different industrial light sources. A controllable tunable liquid lens was used to change the position of the image plane. The paper identifies estimators that allow practical focus estimation in the presence of interference in SFF imaging. Discussed interferences were not digitally simulated. They were introduced through a modification of industrial illuminators or acquisition parameters.

2. Materials and Methods

A series of images were acquired on the test stand for three selected surfaces (Figure 2). All images were recorded using controllable liquid lenses integrated into the optical system. Image stacks were registered using different industrial light sources to change the surface illumination. The position of the vision system remained fixed. The position of the illuminator was adjusted for every geometry.

The three most widely used materials used in industry for process parts were selected for the study (Figure 3):

• Metallic surface;
• Plastic surface;
• Wooden surface.

Figure 3. Image of the surfaces used taken with a dome light illuminator: (a) metallic surface, (b) plastic surface, (c) wooden surface.

Figure 3. Image of the surfaces used taken with a dome light illuminator: (a) metallic surface, (b) plastic surface, (c) wooden surface.

2.1. Measurement Stand

The tests were carried out on images recorded on the dedicated stand. It provided repeatable mounting and illumination conditions. The used camera has the On-Semi Python5000 P1 sensor. The sensor parameters most relevant for SFF imaging are shown in

Table 1. List of camera sensor parameters.

Sensor Parameter	Value
Pixel Size	4.8 $\mathsf{\mu }$m
Resolution	2592 × 2048
Shutter type	Global
Max. FPS	51.8 FPS
Sensor size	1″

. The used sensor allows image acquisition with a 5.1 MPx resolution. The increased resolution, compared to solutions in the literature, enables imaging with higher measurement resolution and an increased field of view of the vision system. Increasing the camera resolution negatively affects the computation time required for focus assessment and processing into a point cloud. The selected sensor has a pixel size of 4.8 µm. The pixel size affects the depth of field of the optical system, which should be kept as small as possible in the intended application. The global shutter allows moving objects to be imaged without the image distortion typical of rolling shutter apertures. The sensor can acquire images at more than 50 frames per second (FPS). The bandwidth of the camera communication interface limits the image acquisition frequency to 22 FPS. The sensor with the chosen parameters has a diagonal dimension of 1″. This allows the use of c-mount lenses without the occurrence of the negative vignetting phenomenon.

A liquid lens was used to collect a stack of images recorded at different positions of the image plane. The operating principle of the lens is presented in Figure 4. Changing the image plane position is performed by changing the focusing power of the lens P. This is the equivalent of changing the lens focal length (f). The distance between the lens and the sensor (v) remains constant. In order to obtain an in-focus image of the element, it is necessary to position the imaging plane on the element. When imaging a plane distant from the vision system by u, it is necessary to adjust the focusing power of the optical system to a value of P according to the formula:

$$P=\frac{1}{f}=\frac{1}{u}+\frac{1}{v}$$

(1)

Figure 4a shows the layout after aligning the image plane with the green arrow ($u=u_1$). The green arrow will be captured on the sensor in focus. The red arrow, on the other hand, will be significantly blurred in the image. The tipping point of the arrow is being focused before the camera sensor. In the digital image, the red arrow tip will be blurred on the surface of a circle of confusion with diameter ${\epsilon }_1$.

A focused image of the red arrow is obtained by changing the optical power of the lens ($f=f_2$). This causes a change in the position of the image plane ($u=u_2$). In this case, the green arrow becomes blurred (Figure 4b). The green arrow tip is focused behind the sensor. As a result, it is projected onto the sensor surface as a circle of confusion with diameter ${\epsilon }_2$.

An EL-16-40-TC liquid lens was used to control the optical power of the optical system. The used lens allows the optical power to vary by ±3 D.

A fixed focal length lens was also used in the optical system. The parameters of the lens are listed in the

Table 2. List of fixed focal length lens parameters.

Lens Parameter	Value
Focal length	50 mm
Aperture	F#2.4
Minimum object distance	200 mm
Max. sensor size	1.1″

. The lens was selected to reduce the depth of field. A fujinon HF50XA-5M lens was selected with a focal length of 50 mm. The lens brightness was set manually to F#2.4. The fixed focal length lens has a minimum object distance of 200 mm, but when combined with the tunable liquid lens, this value was reduced to 50 mm. The system is designed for use with sensors up to 1.1″, so it is compatible with the used camera.

The calculations were performed on a computer with an Intel Core i7-10750H processor with 32 GB of RAM installed.

2.2. Interference Covered by the Studies

This study considered the effect of interference generated through environmental factors and the choice of image acquisition parameters. In terms of interference, the following were considered: illumination, surface texture, and contrast.

The impact of illumination on the quality of the resulting height map represents essential information for implementing SFF imaging in measurement systems. In this study, the impact of interference in the form of lighting was determined by changing the camera–illuminator geometry. The performance comparison involved different industrial illuminators: dome light (DL) (Effilux IDS4-00-150-2-W-24V), bar light (BR) (IFM O2D926), direct ring light (DRL) (VS Technology VL-LRD153180RGB), horizontal ring light (HRL) (VS Technology VL-HR110146W) and collimated light (CL) (CCS LFX100RD). Due to the varying amount of light reaching the camera sensor depending on the used geometry and the power of the illuminator, it was necessary to normalise the average intensity on the digital image. Normalisation was carried out by controlling the exposure time parameter at a preset average intensity of 128.

The texture impact on the quality of the resulting height map provides information about the effect of the object itself on the quality of the height map. This information is crucial when choosing an imaging technology for a specific measurement task. This publication considers three materials: a metal surface, a plastic surface, and a wooden surface (Figure 3).

Another selected interference whose impact will be considered is the contrast in the source images. The designer has some influence over factors determining contrast in a digital image, such as the choice of illuminator or image acquisition parameters. However, despite proper selection, it is possible to obtain images with little contrast, if only due to physical limitations or properties of the object being imaged. In the study, the change in contrast in the source images was achieved by changing the sensor exposure time. Decreasing the exposure time resulted in decreasing contrast in the digital image.

This study narrowed a range of imaged elements to flat surfaces. This approach aims to determine the effectiveness of the focus estimation operators for the specified conditions without concentrating on errors introduced by other scene reconstruction steps [28].

2.3. Considered Operators for Estimating Focus on a Digital Image

This comparative study consisted of the evaluation of operators for focus estimation on digital images. The operators were selected based on the literature in the field of focus evaluation in both the SFF and autofocus approaches. The selection of operators was driven by two indicators: efficiency and time required to perform the calculations. The selected methods are listed in a table together with a classification into four groups (

Table 3. Operators used to assess the sharpness of a digital image.

No.	Group	Operator	Symbol
1	Derivative	Robert’s	DRB
2	Derivative	Robert’s–Tenengrad	DRBT
3	Derivative	Sobel	DSL
4	Derivative	Sobel–Tenengrad	DSLT
5	Derivative	Prewitt	DPT
6	Derivative	Prewitt–Tenengrad	DPTT
7	Laplace	Laplace	LLP
8	Laplace	Laplace–Tenengrad	LLPT
9	Laplace	LOG	LLOG
10	Laplace	Modified Laplace	LML
11	Laplace	Modified Laplace–Tenengrad	LMLT
12	Statistical	Entropy	SETP
13	Statistical	Standard Deviation	SSTD
14	Statistical	Variance	SVAR
15	Statistical	Variance Normalized	SVRN
16	Intensity	Intensity Sum	ISUM
17	Intensity	Intensity thresholded count	ICNT
18	Intensity	Intensity Energy	IENG

). The first group is based on calculating the derivative at a point. The study included modifications of the classic filters by averaging in the neighbourhood (Tenengrad algorithm). The additional averaging allowed extending the kernel size to values larger than 3 × 3. The second group of filters are operators based on the computation of the Laplace transform together with a modification of neighbourhood averaging. The next group of filters are statistical filters. This group includes filters that consider the spread of pixel intensity values in the local window as the focus estimator. The last group of filters allows the estimation of focus directly on the intensity of a point or its neighbourhood without the need for greater computational complexity calculations. The symbols given in Table 3 will be used consistently throughout the article.

2.4. Method of Comparing the Focus of a Scene Point between Images

A three-dimensional matrix containing the focus estimations is obtained by applying the operators from the Table 3 to the acquired images. The estimation is calculated for each pixel of the captured images. In order to obtain a height map, it is necessary to compare the focus estimations in a z-axis (Figure 5).

The image defocusing resulting from image plane displacement is a normal process. Therefore, the curve corresponding to the focus estimator in consecutive images should have the character of a normal distribution. The other authors use this relationship to estimate depth by using Gaussian function interpolation [29]. The impact of interference in real systems makes it necessary to use more robust methods.

Two solutions were proposed:

• Use of a maximum function—the assumption is made that the in-focus image has the highest value of the focus estimator. As the image plane moves away from the imaged surface, the image becomes increasingly blurred, reducing the value of the calculated focus estimator. The method allows the surface plane to be determined with a resolution of up to one plane;
• Use of the centre of gravity (CoG) method—the application of this method makes it possible to increase the resolution of the determination of the surface plane compared to the maximum function [30].

The CoG algorithm involves calculating a weighted average where the weights for consecutive coordinates are the values of the calculated focus estimator (Equation (2)). An additional modification to the algorithm is to increase robustness to noise by disregarding points for which $W_i$ is less than half of the maximum value of the focus estimator in a given profile (Equation (3)). The CoG function returns non-integer values of the position of the focus plane. A non-integer value corresponds to the allocation of the focus plane between the image planes captured on the images.

$$CoG=\frac{\sum W_i\times i}{\sum W_i}$$

(2)

$$W_i=\left\{\begin{array}{cc}{W}_i\hfill & W_i\ge \frac{max\left(W\right)}{2}\hfill \\ 0\hfill & W_i<\frac{max\left(W\right)}{2}\hfill \end{array}\right.$$

(3)

where:

• $CoG$—the determined centre of gravity value corresponding to the position of the focus plane in the z-axis;
• i—coordinate corresponding to the image number (z-axis) in registered image stack;
• W–set of values of the focus estimator for a surface point in the image stack.

2.5. Method for Assessing the Quality of Height Map Representation

The results obtained in the form of a 2D matrix height map do not allow a direct comparison of focus estimation methods. It is necessary to propose metrics for comparison. Limiting the range of imaged scenes to flat surfaces makes it possible to define the accuracy of the scene representation as the spread of the obtained results around the expected value [31,32]. The expected value for the entire height map is constant due to the imaging of flat surface scenes. The root mean square error (RMSE) value was used as a determinant. The second proposed indicator of height map quality is the absolute fit error ($AFE$). It was defined as the absolute value from the difference between the height map mode and the expected value (Equation (4)). The expected value was taken heuristically separately for each set of images. This indicator shows the distance between the plane calculated according to a given operator from the actual value.

$$AFE=abs\left(mode\right(HM)-E(HM\left)\right)$$

(4)

where:

• $AFE$—absolute fit error;
• $mode\left(HM\right)$–value that appears most often in the height map values;
• $E\left(HM\right)$—expected value of height map for the surface.

3. Results

In this chapter, the achieved results will be compared. The results are arranged in subsections according to the considered parameter. Each surface was imaged six times to reduce the impact of random noise. Each measurement involved recording 200 images while varying the optical power of the tunable lens from −2.00 D to 0.00 D with a step of 0.01 D. The location of the imaging system remained unchanged. The imaged surface, acquisition parameters, and environmental parameters were altered.

3.1. Method of Comparing the Focus of a Scene Point between Images

Section 2.4 discussed the proposed methods for comparing scene point focus estimator along the collected images. A comparison of the proposed methods was chosen to be presented in the histogram analysis. The histograms were generated from the height maps. Figure 6 shows an example of the histograms obtained by imaging a plastic surface for two operators. The mode value for the maximum method in both cases reached 119. The CoG method returned a mode value of 118.7 for the DSLT operator and 118.5 for the SVAR operator. The notable difference in the histogram amplitude results from the 10× higher resolution obtained by the CoG method.

The validity of the sharp plane selection by the proposed methods depended heavily on the operator used. Operators particularly affected by noise returned a signal with a low SNR. This resulted in an incorrect determination of the focus plane. Due to the averaging, the CoG method sometimes returns an incorrect value from the middle of the range. Due to the use of 200 images, a focus plane was falsely obtained for a value of approximately 100 (Figure 7).

During the experiments, an attempt was also made to modify the pixel focus comparison method by additional filtering of the focus estimator values in the z-axis. A median filtering with a kernel size of 3 was applied. The effect is a smoothing of the profile (Figure 8). However, this effect was not observed to affect the focus plane selection results significantly. For this reason and due to the increase in calculation time, filtering was abandoned.

3.2. Measurement Results for Surfaces with Different Textures

The results obtained for the standard conditions provided a reference for further measurements. Standard conditions refer to imaging with DL illumination. A 7 × 7 window size was used in the calculations for the variable kernel size operators. The contrast was ensured by adjusting the sensor exposure time to the maximum value without pixel saturation.

Figure 9 shows the results obtained. Separate plots have been prepared for the two focus plane selection methods. The bar chart shows the AFE value. The colours of the bars correspond to the measurements made for the different surface types.

The results for the maximum method show three operators with very high error (above 100): DRB, DRBT, and ICNT. These results significantly deviate from the other operators. The remaining operators have AFEs below a value of 3.

The results for the CoG method differ from those presented for the maximum method. All values are in the range of up to 30. Only six operators have AFE results less than 3: DSL, DSLT, DPT, DPTT, LLPT, SSTD, and SVAR.

In both cases, the values obtained for the operator are comparable for each surface type. Slightly larger errors were obtained for the wooden surface.

The obtained RMES results are generally consistent with the AFE results (Figure 10). For the maximum method, some operators have values above 50. The best results—RMSE values below 6—are obtained for six operators: DSLT, DPTT, SETP, SSTD, SVAR, and SVRN.

All results for the CoG method are in the range of up to 30. Results below 6 are obtained for the DSLT, DPTT, SSTD, and SVAR operators.

A general relationship between the materials tested is visible. The lowest RMSE values are achieved for the metal surface and the highest for the wood surface.

Applying the maximum method for focus plane selection returns acceptable values for six operators: DSLT, DPTT, SETP, SSTD, SVAR, and SVRN. However, using the CoG method, comparable results are obtained for four operators: DSLT, DPTT, SSTD, and SVAR. Due to the achieving of unacceptable results for most of the considered focus estimation operators, the remaining results presented in this chapter are narrowed down to six operators: DSLT, DPTT, SETP, SSTD, SVAR, and SVRN. This increases the readability of the results by excluding operators that did not already provide the required robustness for the standard conditions.

3.3. Comparison of Results for Measurements Taken with Different Operator Kernel Sizes

Two of six operators selected in the previous section have a fixed kernel size: DSL and DPT. These are classical filters with a kernel size of 3 × 3. The modification proposed in this subsection does not affect their behaviour. The other operators allow the size of the kernel to be changed. For the “tennengard” operators, this changes the averaging region on the XY-plane. By changing the kernel size for the SSTD and SVAR operators, a different number of pixels is considered when calculating the corresponding statistical measure.

Measurements were carried out for a range of window sizes from 3 × 3 to 21 × 21. The window size for the calculations was carried out with a step of 2, due to the odd size requirement. The comparison was performed for a metallic surface captured with a DL illuminator.

For the maximum method, zero AFE error values were obtained for each operator over the entire range of mask sizes.

Figure 11 shows the AFE error values as a function of window size for operators with variable kernel sizes for the CoG method. For the CoG focus plane selection method, the AFE values are up to 1.2. The AFE values for the DSL and DPT filters do not depend on the window size and are also 0 for the height map considered. For such small AFE values, there is no observable trend between window size and error value.

The DSL and DPT operators reached RMSE values for the maximum method of 7.7 and 4.9, respectively, and values for the CoG method of 7.3 and 4.8, respectively. Figure 12 shows the RMSE error values for the other operators. For both methods, the error values are below 8 over the entire range examined. The RMSE value decreases as the window size increases. The DPTT and DSLT operators achieve the best results in both cases. The changes in RMSE values are largest for smaller window sizes. The larger the window size, the smaller the impact of the size change on the RMSE value.

When selecting the appropriate window size for a particular measurement application, consideration should also be given to the increase in required computing power, which increases as the window size increases.

3.4. Comparison of Results for Different Image Contrast

One of the interferences studied is the impact of contrast in the source image on the quality of the resulting height map. The contrast was not digitally changed by applying image transformations. The contrast was modified in an analogue approach corresponding to conditions encountered on industrial production lines. The contrast in the image was modified by adjusting the exposure time of the sensor of the vision system. Such a change under static conditions is equivalent to changing the illuminator power.

Measurements were taken for exposure time sequences ranging from 600 µs to 87 µs with a step of 50 µs (Figure 13). Twelve samples were thus obtained, showing the relationship of error values to contrast in the source images. The value 87 µs is obtained due to the minimum sensor exposure time. For a value of 600 µs, a pixel saturation effect is obtained. It was decided to extend the set to include images with saturation because this negative phenomenon could also be observed in industrial applications (Figure 13c). Saturation causes a clipping of the pixel intensity values in the digital image related to the analogue light signal. As a consequence, the contrast in the digital image is reduced.

Figure 14 shows the AFE values as a function of the sensor exposure time. The results for both methods of determining the focus plane are within a range of up to 2. The exception is the SSTD operator, which for low-contrast images—exposure times of 87 µs and 100 µs—obtained an error of more than 15. For the other operators, the error values are small enough that no trend of AFE depending on contrast was observed.

Noticeable differences can be observed in Figure 15. It shows the value of the RSME error as a contrast function for both methods of determining the focus plane. A noticeable trend is that the error decreases with increasing contrast. This is a common trend for each of the operators considered. Operators achieve the best results with a variable kernel size. For the maximum method for times above 100 µs, the RMSE error remains below 10. For exposure time above 200 µs, it reaches values below 4. For the CoG method, performance improvement for the DSLT, DPTT, and SVAR operators is noticeable. For exposure time above 150 µs, the error value falls below 5.

The exception to the decreasing trend is the value obtained for 600 µs. The error value increases. This effect is caused by the presence of saturated pixels, leading to a reduction in contrast in this image.

The impact of contrast was examined indirectly by changing the sensor exposure time. Changes in contrast had a marginal effect on the AFE value. The exception was the SSTD operator, which significantly deviated in AFE error value for exposure times below 150 µs. The low error values did not allow a trend in the effect of contrast on AFE to be observed.

The RMSE value better shows the effect of contrast on the height map representation. As the contrast is increased up to 550 µs, the value of the obtained RMSE decreases. The exception is the measurement for an exposure time of 600 µs, for which the error increases. This is caused by the saturation of some of the pixels. The saturation reduces the contrast and lowers the quality of the representation of the scene by the height map.

3.5. Comparison of Results for Measurements Obtained with Different Illuminators

The last type of interference investigated is related to the geometric configuration of the illuminator–camera system. Changes in the position and type of illuminator cause glare, local saturation of the image, or uneven illumination of the scene. This section will present the results obtained using illuminators with different geometries and various diffusion levels.

Figure 16 shows sample images of a captured metallic surface. Different effects are obtained depending on the used illuminator and how it is mounted with respect to the camera. A dedicated illuminator allows specific features of an object to be enhanced by changing the intensity of some regions of the image. The DL illuminator in Figure 16a produces a glare-free surface image. The HRL illuminator in Figure 16b enhances local surface defects in the form of scratches while maintaining uniform surface illumination. A BL illuminator placed at a low angle to the surface achieves similar scratch enhancement. However, the unevenness of the illumination of the scenes is noticeable. The centre of the image is significantly brighter than the edges of the image (Figure 16c).

The AFE for each illuminator was calculated. The AFE values obtained by applying the maximum method for each of the considered operators were 0. The results obtained for the CoG method are shown in Figure 17. Applying the CoG method, the AFE value ranged up to 1. The exception is the SSTD operator, which achieved an AFE value of 10 for images recorded by using the CL illuminator.

RMSE values varied depending on the illuminator and operator used (Figure 18). The worst results were obtained with the CL illuminator for determining the focus plane with the maximum function. RMSE values of around 40 were obtained for the DPT and DSL operators. The other operators achieved RMSE values of around 10. BL illuminator had similar values of around 10 for all operators. The other illuminators achieved better results. In their case, the operators with no fixed kernel size kept the RMSE value below 4.

The CoG method halved the RMSE for illuminators CL and BL. For the other operators, the error values decreased slightly. Operators with no fixed kernel size obtained RMSE value below 3.

4. Discussion

This article discusses the application of digital filters on images for use in 3D imaging systems by using the SFF method. This study compares the selected operators’ performance on images collected on a test stand. This study determines the impact of several factors crucial for using the SFF-based measurement system under industrial conditions. The aforementioned factors include the product texture, the contrast obtained during acquisition, the type of the illuminator, and its alignment with the camera. In addition, the research is extended to include algorithm parameters essential for implementing the SFF method in the measurement system: the operator window size and methods for comparing the focus of a scene point between stack images. The research scope was limited to imaging flat surfaces due to the intention to reduce the contribution of errors arising from the 3D image reproduction stage.

The results allow for the initial selection of the operators based on the standard conditions. The standard conditions have been defined as uniform illumination, strong contrast, and no supplementary interference. After rejecting some operators that already gave unsatisfactory results for standard conditions, additional tests have been carried out. The tests involve the application of interference, addressing scenarios encountered in the industry.

Investigations into the influence of mask size on representation quality reveal a diminishing error with increasing window size. However, this increase in window size also escalates the computational power required for the computations. Minor changes in the error value above a 7 × 7 mask size suggest that the computational cost associated with enlarging the window size beyond 7 × 7 outweighs the potential benefits.

Examination of the impact of contrast reveals a decreasing error value with increasing contrast in the image. Additionally, saturation’s negative effect on the scene’s representation quality has been observed. The DPTT, DSLT, and SVAR filters consistently yield the smallest error values.

Investigations into the influence of the illumination method on reproduction quality demonstrate the broad applicability of the SFF method. Increased error values are observed when scenes are illuminated with collimated light.

5. Conclusions

What sets this study apart is incorporating actual interference instead of digitally introduced noise. This type of interference is implemented through supplementary industrial illuminators, adjustments to acquisition parameters, and imaging of the surfaces of actual industrial products. An industrial-grade camera is employed, and a tunable liquid lens carries out the image plane displacement.

The results have identified the DSLT and DPTT operators as the primary candidates for further research and testing of the SFF method. These operators yield the best results in terms of error reduction and exhibit robustness to the disturbances considered in the study. Moreover, they demand lower computational power compared to other operators.

This paper pursued a different approach for focus plane selection methods, diverging from solutions found in the literature [17]. The three-point interpolation with a Gaussian function has been dismissed, and the proposed alternatives include the maximum and centre of gravity (CoG) methods.

This study confirms the applicability of both proposed focus plane selection methods. The method based on the maximum function proves sufficient for most cases considered. The absence of floating-point calculations results in less computing power and the possibility of using hardware acceleration methods [33,34,35]. The CoG method allows for the increased resolution in the z-axis. RMSE errors for the CoG method have been lower than for the maximum method.

The research presented in this paper can be expanded to include multi-channel images. We are currently working on incorporating data from the R, G, and B channels into the SFF method. This will allow for the generation of a 3D image that can be further enhanced with RGBD (red, green, blue, depth) information.

The next phase of the presented research could also involve an extension to 3D scene reprojection and spatial scene imaging.