A Nonlinear Diffusion Model with Smoothed Background Estimation to Enhance Degraded Images for Defect Detection

: It is important to detect the defect of products efﬁciently in modern industrial manufacturing. Image processing is one of common techniques to achieve defect detection successfully. To process images degraded by noise and lower contrast effects in some scenes, this paper presents a new energy functional with background ﬁtting, then deduces a novel model which approximates to estimate the smoothed background and performs the nonlinear diffusion on the residual image. Noise removal and background correction can be both successfully achieved while the defect feature is preserved. Finally, the proposed method and some other comparative methods are performed on several experiments with some classical degraded images. The numerical results and quantitative evaluation show the efﬁciency and advantages of the proposed method.


Introduction
Image processing has become more and more important in many fields, such as machine vision, geological prospecting, medical imaging, defect detection, and so on. In the past decades, many methods have been presented based on different views, and nonlinear diffusion is one of the most important techniques for the solid mathematical physics foundation [1][2][3]. Different from the uniform diffusion (holds the same velocity along any direction), nonlinear diffusion can be applied to describe lots of complex natural phenomenons. Nonlinear diffusion models are often driven by partial differential equations (PDEs) which can be derived by minimizing an energy functional and computing the Euler-Lagrange equation [4,5].
Perona and Malik (PM) proposed a nonlinear diffusion model with a fixed edge stop velocity function related to the local gradient mode [6]. It can be applied to achieve some denoising tasks with homogeneous features and noise; however, it is difficult to efficiently govern the diffusion process for some more complex cases. Rudin, Osher and Fatemi (ROF) proposed the classical total variation model to recover noisy images by minimizing a geometric energy functional [1]. The diffusion can be performed only along the edge to preserve the image features and remove the noise.
Though these traditional methods have been reported to work well in many cases, they are still found to show some unfavorable phenomena, such as the staircase effect, loss of texture features, and so on [7,8]. Especially in many industrial scenes, the lighting condition is limited, inhomogeneous and easy to be disturbed so that the observed images are often low-contrast, have an inhomogeneous background and are serious polluted. It is often not a good idea to directly apply the traditional models to these degraded images for defect detection since the mentioned unfavorable effects often become worse in such situations.
Defects detection is often implemented on the images captured by some vision devices, which are effected by some objective factors, such as noise level, brightness, background gray distribution, size and shape of the defects. Lots of efforts have been made to present novel models or algorithms based on different starting points and proper assumptions. Saitoh [9] presented a machine vision scheme for the inspection of brightness unevenness in LCD panel surfaces. An edge detection algorithm and a genetic algorithm were used to identify discontinuous points and extract the boundary of anomalous brightness region, respectively. Kim et al. [10] presented an adaptive multiple-level threshold method based on the statistical characteristics of the local area to segment spot-type defects from the background surface. Ng revised the Otsu method for selecting optimal threshold values for both unimodal and bimodal distributions and tested the performance on common defect detection applications [11]. Krummenacher proposed two machine learning methods to automatically detect the wheel defects, which automatically learn different types of wheel defects and predict during normal operation if a wheel has a defect or not [12].
In recent years, with different views or assumptions, several improved nonlinear diffusion models were presented for different tasks about image processing and defect detection. Tsai and Chao took a non-negative decreasing function with an annealing gradient threshold as the diffusion coefficient, and then proposed a novel anisotropic diffusion model (TCAD) to detect subtle defects embedded in inhomogeneous textures [13]. Chao and Tsai introduced a weighted diffusion function and a generalized diffusion coefficient function respectively to the classical PM model for defect detection in low-contrast surface images [14,15]. These two models both carry out a smoothing process for faultless areas and perform a sharpening process for defect areas. Based on the directional Laplacian, Wang et al. proposed a modified version (MPM) of the original Perona-Malik model [16]. A weighted Laplacian was incorporated in the PM model to perform the diffusion along the edge direction of the original image. Prasath and Vorotnikov proposed a weighted and balanced anisotropic diffusion scheme for image denoising and restoration [17]. Maiseli and Gao presented a robust edge detector based on an anisotropic diffusion-driven process [18]. In this model, noise suppression and edge enhancement are achieved respectively by anisotropic diffusion and the edge-sensitive detector.
Xu et al. [19] contributed a new filter by introducing a semi-adaptive threshold to an anisotropic diffusion process. In detail, a Gaussian filter is applied to recognize corrupted pixels in noise-free pixels and then a semi-adaptive threshold function is introduced to the diffusion coefficient for noise removal and edge preservation. Malarvela et al. found that the local probability value of image gray can be applied to adjust the indication degree of a low edge gradient in the feature space of a noisy image, similar to the action of an adaptive threshold parameter [20]. They proposed an improved anisotropic diffusion model which can adaptively adjust the implication of the image edge gradient in the diffusion process. More research studies about defect detection and image denoising related to other techniques refer to some recent papers [21][22][23][24][25].
Based on the above, it can be found most research focuses on designing novel diffusion patterns or using the observed information in new schemes. However, in complex scenes, inconsistent background and features should be properly treated for efficient defect detection. The gradient, Laplacian and some other geometric metrics are often used to describe the observed image features, and they are easily disturbed by an uneven background. In this paper, a novel hybrid energy based on L1 and L2 is introduced to a constrained optimization problem. Then the corresponding Euler-Lagrange equations are derived to form a nonlinear diffusion with a smoothed background estimation, which can remove the noise and preserve more details of the defect features.
This paper is organized as follows. In Section 2, the basic fundamentals of defect detection and nonlinear diffusion are introduced. Then the proposed model based on a novel hybrid functional is introduced in Section 3; the algorithm and numerical scheme are also presented. Section 4 shows the experiments and results.

Basis and Fundamentals
Image degradation is a common situation in transmitting image information and recording digital images, but in many practical applications, high-quality images are necessary, so effective methods to improve and recover degraded images are worthy of our study. In this section, we will briefly describe some important image denoising models based on partial differential equations and anisotropic diffusion.

Nonlinear Diffusion
Perona and Malik proposed a classical nonlinear diffusion model based on the linear diffusion (or heat equation) [6], and the diffusion process can be represented by a partial differential equation: ∂I ∂t (x, y, t) = div 1 1 + |∇I| 2 /k 2 · ∇I (1) where I(x, y, t) refers to the image at time t, div means the divergence operator, ∇I(x, y, t) means the spatial gradient of the image at time t, and k denotes a threshold linked to the features strength. This partial differential equation can be derived by minimizing a related energy functional: Here 1 1+|∇I| 2 /k 2 means a diffusion velocity, and it can be replaced by other function C(|∇I|) as necessary.
In the PM model, the diffusion velocity is determined by a fixed pattern at each iteration, which reduces the diffusion as the gradient module increases to protect the edges present in the image. Some degraded images can be successfully recovered by solving the above equation with a proper initial condition and boundary condition, even if the artifacts still occur in the result due to the instability of the diffusion process. The total variation model (TV) is another representative one of nonlinear diffusion models, and it was first proposed by Rudin, Osher and Fatami [1]. The bounded variation (BV) function is introduced to denote the ideal image and then the discontinuities edge can be well-preserved when removing the noise. This model has been widely applied to many other fields as a regularization technique [26]. It can be denoted as Here, f denotes the observed degraded image and λ > 0 is a Lagrangian multiplier which balances different terms. It can be obtained by minimizing a combined energy functional Differing from the diffusion controlling mechanism of the PM model, the TV model prevents smoothing across the edge everywhere, even if in an almost plain area. For preserving the discontinuous edge and smoothing the approximately flat area, it is often necessary to carefully design numerical schemes with a small time step.
Based on the TV model and the PM model, lots of research has been presented to improve the results at different views [3,[27][28][29]. The CTAD model [30] was presented to treat the restoration tasks on astronomical images. The local variance of the image gray is embedded in the diffusion velocity of the model for an adaptive process. Specifically, the advanced diffusion velocity is denoted as where σ 2 is the local variance of gray levels in a proper neighborhood.
A weighted Laplacian was incorporated the modified Perona-Malik model, where the directional Laplacian diffuses the image along the edge direction of the original image [16]. In this model, the diffusion is designed along a unit directional vector − → n = − f y , f x /|∇ f |, and the noise can be suppressed by the weighted Laplace operator. The related partial differential equation can be denoted as where C(|∇I|) = 1/ 1 + (|∇I|) 2 , α denotes a small positive constant and f is the observed degraded image. Malarvel et al. proposed a modified model that introduces both a probability value and gradient of local image gray to reduce the noise and smooth the noise-free part adaptively [20]. Thee diffusion velocity can be denoted as Here parameter k should be a positive fixed constant, and it is related to the strength to denoise and enhance the edge. The input variable p is related to the local gray distribution, and it can be calculated based on image I.
In this paper, we will first introduce several fields related to smoothed estimation and edge information, and then present a novel dynamic mechanism based on them to better determine the diffusion velocity for improvements of the denoising results.

Some Evaluation Metrics
To measure the qualities of image processing results, some metrics are often applied to calculate the evaluation values. The general evaluation metrics include but are not limited to mean square error (MSE), peak signal-to-noise ratio (PSNR), entropy (ET) and structural similarity (SSIM).
The dissimilarity between the referenced image J(x, y) and the result image I(x, y) can be measured by MSE, which means the loss of image quality. Assume that the image size is n x × n y , MSE can be calculated as The PSNR can be defined as ].
Here, L means the gray level. The lower the MSE, the lesser the error, and MSE is inversely proportional to PSNR.
The entropy is a measure widely applied in many fields such as data mining and image processing. A lesser entropy value implies lesser disorder. The common Shannon entropy (10) can be used to evaluate the disorder degree or variance of a result image: Here, H i = n i /(n x n y ) means the gray ratio on the ith bin of the normalized image histogram.
SSIM measures the image quality by computing the similarity between the result image and reference image. The SSIM value is limited in [0, 1] and a larger value indicates that the result image is more similar to the original one.
where µ I and µ u represent the average pixels of the two images, σ I and σ u represent the standard variance of I and u, cov(I, u) is the covariance, and constant k 1 = 0.01, k 2 = 0.03.

The Proposed Method
In many complex scenes, image will suffer from inconsistent noise and varying lighting condition, which leads to a nonuniform background. Traditional models can often successfully treat the consistent degradation and uniform background (no or little change in each parts) while being deficient for inconsistent cases. For example, the noise strength is very different in some parts of the image area, the edge consists of some strong parts and some weak parts, and the background gray of some parts are at different levels (inconsistent background). Here, we focus on the image degraded by Gaussian white noise and an inconsistent background.
In this paper, a background gray surface is introduced to the observation model for complex scenes, and then a novel functional based on the combination of L1 and L2 energy is presented to modify the traditional total variation energy. Then, novel nonlinear diffusion equations can be derived to remove the noise and separate the inconsistent background by minimizing the energy functional.

A Novel Energy Functional
The existence of an inconsistent background (which different in parts) is disadvantageous for treating the polluted and low-contrast image. More exactly, many geometric terms (such as gradient, Laplacian, etc.) are unavoidable to be polluted by not only noise, but also the inconsistent background. Accurate estimation of the background can help to remove the noise and enhance the image quality more efficiently. The observation can be generally denoted as f (x) = I(x) + v(x) + n(x), and a simple example is shown in the first row of Figure 1. Here, f is the observed image polluted by Gaussian white noise n with variance σ 2 and inconsistent background v, and I is the clear version of f . Assume that v is almost smooth everywhere, then a novel energy functional is presented as It can be found that the first term means the total variation of the ideal image I. The second term is the L 2 energy of the gradient field of background v and the minimization can be linked to solve a smoothing version of v. The third term is the total variation of I + v, and the minimization means an edge-preserved process. α and β are both weight factors to balance these terms.
Based on the observation model, the variance prior of the noise can be related to a fitting functional E 0 (I, v) = Ω (I + v − f ) 2 dxdy. In a dynamic minimization process, it can limit the solved u, v not far away from the observation f . Then an optimization problem to determine I, v can be denoted as Apply the Lagrange method and then the above problem is equivalent to an unconstrained optimization problem, that is Here, λ means a Lagrange multiplier to combine the geometric energy and the fitting error. Apply the following Euler-Lagrange equations to compute the gradient flow of (13), and introduce the artificial term ∂I ∂t , ∂v ∂t , the final evolution equations can be obtained as (14) In the evolution, the first equation will preserve the features, including defects to be detected due to the two total variation diffusion terms. The second equation is mainly to approximate the ideal background v while features are well-protected.
Set the homogeneous Neumann boundary condition on ∂Ω, that is where n means the normal vector on the boundary ∂Ω. Set the initial condition in Ω then Equations (14) can be solved by some numerical methods, such as finite difference.
Here, v 0 (x, y) means an initial estimation of the background. Figure 1 shows the traditional evolution and the proposed evolution on an artificial image with two suspect areas. The first row shows the original image, the noise, the inconsistent background and the degraded image. As shown in Figure 1d, the clear image is polluted by Gaussian noise (Signal to Noise Ratio, SNR = 20) and an inconsistent background. The top-left seems underexposed, and the bottom-right looks overexposed. With PM model performance, the noise removal and edge preservation can be well balanced under the influence of an inconsistent background (as shown in Figure 1e). Applying our model on the degraded image, noise and inconsistent background are both reduced while the edge is preserved (as shown in Figure 1f). The PSNR curve and SSIM curve are shown in Figure 1g,h.

Numerical Scheme
In this paper, the spatial domain Ω is discretized by a grid domain {(x i , y j ), x i = ih, y j = jh, i = 0, 1, 2, · · · , n x − 1, j = 0, 1, 2, · · · , n y − 1} and the time interval (0, T) is discretized as t n = nτ, n = 0, 1, 2, · · · , n t − 1. Then the numerical solution of I, v at (x i , y j , t n ) can be denoted as I n i,j , v n i,j . Apply the minmod scheme [1] m(a, b) = sgn(a)+sgn(b) 2 min(|a|, |b|) to approximate the TV diffusion terms, and apply central difference to approximate all other differential term, then an efficient numerical scheme for the proposed model (14) can be denoted as Here, TV m (I)i, j n is denoted as and the Lagrange multipliers can be determined by or After discretization of the boundary condition and initial condition by the central difference scheme, the numerical algorithm can be implemented efficiently.

Experiments and Results
In this section, the proposed method will be performed on some experiments for detailed evaluation. The test images are shown in Figure 2 and the first two are artificial ones. The subsequent four are randomly selected from classical image datasets, such as miscellaneous of USC-SIPI (http://sipi.usc.edu/database/, accessed on 1 September 2022). The last six are collected from real industrial scenes. Different Gaussian white noise and background surfaces will be acted on the first three test images for their degraded version, which will be processed by the proposed method and other comparative methods, such as PM [6], TV [1], CTAD [30], mPM (modified PM) [16], Malarvel [20], and so on. The last three will be processed respectively to verify the performance of the proposed method on actual data.

Experiment 1: Artificial Images Test
To evaluate the performance of the proposed method working on different degradation levels, in this experiment, Gaussian noise at different levels (SNR = 12, 14, 16, 18, 20) and a inhomogeneous background surface are added to two artificial images respectively as shown in Figure 3a. The same initial estimation of the restoration image without an inhomogeneous background is configured to solve the five models.
The results achieved by different methods are shown in Figures 3b-f and 4b-f. The one to five columns refer the results of PM, TV, CTAD, Malarvel, and our method. It is found that the noise can be reduced to some extent in different levels by each method. The treatment effects of the four traditional methods are affected by an inhomogeneous background and lead to lower evaluations. Benefit from the background fitting mechanism, the proposed model can achieve better results.
The results are evaluated by two metrics (PSNR and SSIM) for full comparison. It can be found that the proposed method achieves the best performance in each degradation level. About the artificial image 1, our PSNR is on average 6.88% higher than the other four methods, while the SSIM is at least 2.05% higher than the traditional results. More details refer to Figure 3g,h and Table 1. A similar situation can be found on the test of artificial image 2, as shown in Figure 4 and Table 2. The iteration numbers are shown in Figure 5. Fewer iterations are required by our method.

Experiment 2: Natural Images Test
In this experiment, a more inhomogeneous artificial background is added to the two natural images with five levels of noise (SNR = 12, 14, 16, 18, 20) as shown in Figures 6a, 7a, 8a and 9a.
The four natural images results achieved by different methods are shown in Figures 6b-f, 7b-f, 8b-f and 9b-f. Each column refers the results of a method; one to five refer to the results of PM, TV, CTAD, Malarvel, and our method. It is found that the noise can be reduced to some extent in different levels by each method. However, the deficiency in treating the inhomogeneous background prevented the four traditional methods from achieving satisfying results. The proposed model was found to be more successful in obtaining high quality results.
PSNR and SSIM of the results are listed in Tables 3-6. It can be found that the proposed method achieves the best performance in each degradation level. Our method achieves significant improvements of PSNR (at least 9.97% in average on the Fruits image and 7.44% on the House image) for the well treatments on an inhomogeneous background. A similar situation can be found about SSIM; in fact, our results are at least 1.15% larger on the Fruits image and 1.14% larger on the House image than other methods. The advancement of the performance can be also found in tests on the Dogs image and Plane image.
The iteration numbers are shown in Figure 10. Fewer iterations are required by our method.

Experiment 3: Industrial Images Test
In this experiment, the five methods are applied to process six industrial images with some defects as shown in Figure 2g-l.
The six defect images results achieved by different methods are shown in Figure 11a-f. There is no reference image that can be applied to calculate PSNR or SSIM, so the standard variance (STD) and entropy will be applied to evaluate the results.
It is difficult to locate the defect in the original version of industrial image 1 directly by vision. After the five methods applied on indust1, the defect becomes clearer than the original, and it can be found that the proposed result is better than others (as shown in Figure 11a). The inhomogeneous background is removed from the main part of our result, while it can still be found in other results.
The suspect defect in the second image is very weak in vision, and it is also difficult to be found. The traditional methods strengthen the background difference between top part and bottom part. In this situation, it is disadvantageous to find a weak defect deeply hidden in a part of the background for the over-smoothing on each part. Our method excludes the inhomogeneous background, and the weak defect is more significant than other results (as shown in Figure 11b).
Though the main defects in the last four images are easy to be found and all the methods treat it well, some small features (maybe defects in other scenes) are smoothed in the results of PM, TV and CTAD. It seems that the Malarvel's model better protects the features than our model because it preserves abundant details (suspect area). In fact, most of them are unnecessary details or noise; it may result in a more expensive check cost.
STD and entropy of the results are listed in Table 7. It can be found that the proposed method achieves the minimal STD or entropy on each image, that is, the results show less confusion and more regularity in the view of information theory.

Conclusions
Many research studies have been presented to improve the image quality for defect detection, and very few of them take the inconsistent background into account. From our results, it is difficult for traditional models to treat some defect images well for the effects of inconsistent backgrounds. This paper presents novel nonlinear diffusion equations to achieve smoothed background estimation and denoising. It is helpful to improve the image quality for defect detection tasks, especially in some low-contrast cases. The numerical results and quantitative evaluation show the efficiency and advantages of proposed method. More efforts can be made to develop advanced models and algorithms for more practical solutions.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.