Eliminating Non-Uniform Smearing and Suppressing the Gibbs Effect on Reconstructed Images

In this work, the problem is considered for eliminating a non-uniform rectilinear smearing of an image by mathematical and computer-based way, for example, a picture of several cars taken with a fixed camera and moving with different speeds. The problem is described by a set of 1-dimensional Fredholm integral equations (IEs) of the first kind of convolution type with a 1-dimensional point spread function (PSF) at uniform smearing and by a set of new 1-dimensional IEs of a general type (not convolution type) with a 2-dimensional PSF at non-uniform smearing. The problem is also described by one 2-dimensional IE of convolution type with a 2-dimensional PSF at uniform smearing and by a new 2-dimensional IE of a general type with a 4-dimensional PSF at non-uniform smearing. The problem for solving Fredholm IE of the first kind is ill-posed (unstable). Therefore, IEs of convolution type are solved by the Fourier transform (FT) method and Tikhonov's regularization (TR), and IEs of general type are solved by the quadrature/cubature and TR methods. Moreover, the magnitude of the image smear Δ is determined by the original “spectral method”, which increases the accuracy of image restoration. It is shown that the use of a set of 1-dimensional IEs is preferable to one 2-dimensional IE in the case of non-uniform smearing. In the inverse problem (image restoration), the Gibbs effect (the effect of false waves) in the image may occur. It can be edge or inner. The edge effect is well suppressed by the proposed technique “diffusing the edges". In the case of an inner effect, it is eliminated with difficulty, but the image smearing itself plays the role of diffusing and suppresses the inner Gibbs effect to a large extent. It is shown (in the presence of impulse noise in an image) that the well-known Tukey median filter can distort the image itself, and the Gonzalez adaptive filter also distorts the image (but to a lesser extent). We propose a modified adaptive filter. A software package was developed in MatLab and illustrative calculations are performed.


Introduction
Consider one of the actual problems of distorted images processing -the elimination of image smearing by mathematical and computer-based way ( [1][2][3][4][5][6][7], et al.). Smearing may be due to the shift of the image recording device (photo camera, video camera, telescope, microscope), the mismatch of the movement of the observation object (the earth's surface, details on the conveyor) and the tracking Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 9 March 2020 doi:10.20944/preprints202003.0149.v1 device or the movement of the object itself (patient in tomograph, animal, car, plane) for exposure time [4,7].
The problem of smear eliminating by a mathematical way consists of two tasks: a direct problem (computer smear modeling) and an inverse problem (smear eliminating by equations' solving) [7] (p. 106-109). However, the inverse problem is very sensitive to the accuracy of the knowledge of smear magnitude Δ (as shown, for example, in [8]), especially if the smear is variable, i.e. Δ = Δ(x), where x is the coordinate along the smear [5,6]. In this paper, we develop the "spectral method" proposed in [7][8][9] and use it to determine the smear Δ and Δ(x).
In this paper, we focus on the rarely considered image smearing, which is rectilinear, but non-uniform when the camera or object moves during the exposure, as well as on various types of integral equations used in the inverse problem. We will give special attention to the Gibbs effect (the effect of false waves), which often appears on images [4,5,7].
The main goal of this work is a comparative consideration of two variants of rectilinear image smearing (uniform and non-uniform) and suppression of the Gibbs effect, which reduces the restored image quality.
Example: a smeared image of movable objects (athletes-runners, animals, cars, planes) moving with different speeds, taken with a fixed camera. Note that a variant was considered in [1,2] when a camera (or object) moved during the exposure rectilinearly with a certain speed v(t), where t is time.
This variant is quite complicated, since it requires solving the nonlinear equation δ(t(x)) = x with respect to t, where [6]. In this paper and in [5,6], the simpler variant of smear is considered: Δ = Δ(x), where x is the coordinate along the smear.
However, there is no a way in [5,6] to determine the dependence Δ(x) from a smeared image. In this paper, we propose such a way ("divided spectra") based on the "spectral method".
The article is organized as follows. Section 2 describes the uniform rectilinear image smearing mathematically as the direct problem (integral calculation) and inverse problem (image restoration via solving a set of 1-dimensional IEs or one 2-dimensional IE by the FT and TR method). Section 3 describes the non-uniform rectilinear image smearing via solving by the method of quadrature/cubature and TR a set of 1-dimensional IEs or one 2-dimensional IE of general type (not convolution type). In section 4, an illustrative numerical example is given: 3 cars moving with the same smear Δ. The smear is determined by the spectral method. The results of image restoration by the FT and TR methods are presented. The question of restoration error is stated. In section 5, the results for example solving are presented in the case of various smeares of automobiles. A way of "divided spectra" for determining the dependence Δ(x) is proposed. Section 6 describes the impulse noise filtering and its influence on the image itself. A modified median filter for impulse noise suppressing is proposed. In all sections, we use the technique "diffusing the image edges" that we proposed for suppressing the Gibbs effect.

of 17
Let us recall the well-known case of uniform smearing an image [4,7,9]. Consider the direct and inverse problems.

Direct problem
The direct problem of uniform rectilinear smearing an image is to calculate the integral [6]   + where Δ = const is the smear magnitude; the axes x and  are directed along the smear, and the axis y is perpendicular to the smear (plays the role of a parameter); wy() is the given non-smeared image, and gy(x) is the calculated (modeled) smeared image in each y-line. In order to calculate g according to (1), we developed in MatLab the main program Autos.m and m-functions smearing.m [7] for an arbitrary smear angle θ and smear.m [5] for θ = 0. In addition, the MatLab system has m-functions fspecial.m and imfilter.m for modeling g [10].
We notice that the description of the smearing problem by an integral (and, as a consequence, by an integral equation, see below) is the most adequate mathematical description of the image smearing process.

Inverse problem
The inverse (more important and complex) problem can be solved by two approaches.
In the first approach, in order to eliminate the smearing, we solve a set of 1-dimensional Fredholm integral equations (IEs) of the first kind of convolutional type (for each value of y) [6,7,9]: where The IE (2) is obtained from the relation (1); axes x and ξ are directed along the smear and axis y perpendicular to the smear; h is the kernel of IE mathematically, and the point spread function (PSF) physically and technically [3,4,7,11,13]. The function h, as a rule, is difference, or spatially invariant, which means that the smearing is uniform and the smear magnitude Δ is the same at all points of the image (Δ = const).
The following algorithm describes the solution of inverse problem (the first approach).
moreover, the x and ξ axes are directed horizontally, and y and η are directed vertically downward.
In this approach, the direct problem is calculated using the m-functions fspecial.m and imfilter.m [10]. And the solution of the 2-dimensional IE (6) (the inverse problem) by the TR method and the 2-dimensional FT is equal to where )) , . The MatLab contains the m-function deconvreg.m [10] for solving IE (6) by the TR and FT methods according to (7), (8).
The following algorithm describes the solution of inverse problem (the second approach).
according to (7) Output: w(x, y) We give the well-known formulas (1)- (8) in order to compare the various approaches below.

Mathematical description of non-uniform rectilinear smearing
Taking into account the formulas (1)-(8), we discribe non-uniform rectilinear smearing of image along the smear direction. Let us consider two approaches.
3.1. The first (time) approach [1,2,6] In this approach, the speed of moving object (or camera) is assumed to be known as a function where  is the exposure time. This approach was considered in detail in [6], and we consider the second, simpler approach. Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 9 March 2020 doi:10.20944/preprints202003.0149.v1 3.2. The second (spatial) approach [5,6] Suppose that we have determined a dependence Δ = Δ(x) of the smear Δ on the x coordinate directed along the smear. This dependence may be determined from the smeared image in some way, for example, by the method of "blind" deconvolution [11] or by the spectral method [9] (see section 4).

The direct problem
In this case, the PSF h is not difference, or spatially invariant, and the direct problem is written in the form (cf. (1)): To calculate gy(x) according to (9), we developed the m-function smear_n.m [5].

The inverse problem
The inverse problem in the case of the second approach is written in the form of a set of 1-dimensional Fredholm integral equations of the first kind of general type (not convolutional type) for each value of y [5,6]: The FT method cannot be applied for solving IE (10), since the IE (10) is not a convolutional type equation, but the quadrature method can be applied, which reduces the IE (10) to a system of linear algebraic equations (SLAE) for each y [7] (p. 126): where A is the matrix, associated with h (the same for all y-rows), y w is the desired vector, y g is the right-hand side of the SLAE. A stable solution of SLAE (12) is provided by the Tikhonov regularization method (TRM) [7]: where  > 0 is the regularization parameter (about the  choice method see below Figure 5), I is the identity matrix, T A is the transposed matrix, and  y w is the regularized solution in each y-row, equal to In order to realize the formulas (10)-(14) on a computer, we developed the m-function desmearq_n.m. Note that the quadrature method with Tikhonov's regularization (TR) according to Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 9 March 2020 doi:10.20944/preprints202003.0149.v1 (13), (14) can be used also for solving IE of convolution type (2) with the PSF (3), i.e. for uniform smearing. For that, we developed the m-function desmearq.m.
The inverse problem in the framework of the second approach can be written also in the form of a 2-dimensional Fredholm IE of the first kind of general type [18] (cf. (6)): Integral equation (15) can be solved by the quadrature method (more precisely, cubature method) (cf. [18, p. 167]).
According to this method, each of the integrals in (15)  The solution of a 2-dimensional IE by the cubature method took place, for example, in [18] (p. 168), however, with a small matrix 11×11×7×7, i.e. i.e. a matrix of gigantic size 160000×160000, which is implemented with difficulty even on powerful computers. We see that the cubature method for solving IE (15) is a cumbersome method and its application to restore a non-uniform smeared image is complex.
It is possible to use the iteration methods of Jacobi, Landweber, Friedman, Bakushinsky, et al. [2,15,18] for solving 2-dimensional IE (15). These methods usually do not use giant arrays, however, they require a successful initial approximation for w, the number of iterations (playing the role of a regularization parameter), etc.
As a result, we conclude that the most effective technique in the case of non-uniform smearing is the technique (10)- (14), based on line-by-line image processing by solving 1-dimensional IE (10) and SLAE (12)

Illustrative example
The following numerical example is solved. Figure 1 shows grayscale (gray) initial image J of Image J has intensities from 0 to 255, which causes inconvenience in filtering impulse noise, which has values 0 and 255 (see details in section 6).
To avoid this inconvenience, we replaced the intensities 0 and 255 of the image J with the values 1 and 254. We obtained the image I with intensities from 1 to 254.
We use the image I, 143×1307 pixels for further processing,. Note that the driver's head is visible in the left car, which (along with the wheels) will be one of the reference points in image processing.

Uniform smearing of image using Boundary Conditions
Further, we assume that the cars move at the same speed and therefore give the same smear in the image: Δ = const = 20 pixels or the cars are stationary, but the camera moves uniformly during the exposure. Image I is smeared horizontally according to (1)  In order to restore the image by solving inverse problem, it is necessary to know the angle θ and the smear Δ. It is possible to estimate these parameters visually, but only with an error, or it is impossible to estimate them at all. And, as we showed in [8,19], even an error of θ in 1-2 degree and an error of Δ in 1-2 pixel lead to a significant error (~ 10%) in image restoration even by Tikhonov regularization or Wiener parametric filtering method.
In [7][8][9]19], we developed the so-called "spectral method", which allows to determine the distortion type (smearing, defocusing, etc.) and estimate the distortion parameters based on the ( value is chosen by the minimum of curve 1 in Figure 5); relative error is 0359 . 0 rel =  (according to (17)).

Uniform smearing of image with truncation
The Boundary Conditions (BCs) [10,22] used in m-function imfilter.m are an artificial technique intended for determining the intensities of an undistorted image outside its edges. We proposed [7,23] instead of BCs the technique of image truncation, which does not require extraboundary data.

Uniform smearing of image with diffusing the image edges
In [7,23], we proposed a way of diffusing the smeared image edges to suppress the edge Gibbs

Error estimation of image restoration
In order to estimate numerically the restoration quality and choose the regularization parameter , we propose a formula for calculating relative error in the form of the standard deviation of the calculated image  w from the exact image w [9]: where M is the number of rows and N is the number of columns in the image. Such an expression for the image error can be used only in the case of model image processing when w is known (e.g., the image in Figure 1b). Figure 5 presents the curves ) ( rel   for different variants of solutions. One can choose the value of  from the minima of these curves. . 1 -FT and TR method (see Fig. 2c); 2 -FT and TR method with truncation (Fig. 3b); 3 -FT and TR method with diffused edges (Fig. 4b); 4 -quadrature and ТR method with diffusing and division (Fig. 9); 5 -FT and TR method with noise, d = 0.005, 0.01, 0.02 (Fig. 10e).
One can also use the following expression for restoration error: where PSNR is the so-called peak signal-to-noise ratio, used widely in engineering, acoustics, etc.
However, formula (17) is more convenient and obvious for image processing. Indeed, if, for example, the error according to (17) equals 035 . 0 rel =  , then this means that the restoration error is 3.5%, and this value does not depend on the system of units w. Convenient and clear! And if we use the formula (18) and get an error, for example, 8 dB, then it is difficult to judge whether this is a large or small error in image restoration.

Direct and inverse problems of non-uniform smearing of image
The next step is non-uniform smearing of an image. We believe that the cars move with different speeds and therefore they have different smears on the image, for example, if (i<=360) Δ=15; elseif (i<=820) Δ=20; else Δ=25; where i = 1 ... 1307 is the number of discrete reading along x. Figure 6 shows an image smeared non-uniformly according to (9) and (19) with diffusing the edges for suppressing the Gibbs edge effect using m-function smear_n.   Fig. 6.
However, Figure 7 shows that the spectrum is the sum of three spectra and this creates difficulties with the definition of We propose a way that we will call the "way of divided spectra".

The way of divided spectra
Let us divide the original piecewise-uniform smeared image in Figure 6  Next, we calculate the FT (spectrum) in modulus |G| for each part of the image (Figure 8def, cf.

Figure 2b
). From the spectra, we determine the smears' magnitudes by the formula (16) (10)- (14), namely, by the quadrature method with Tikhonov's regularization. Figure 9 presents the result of image restoration using the developed m-function desmearq_n.m. The image was restored almost accurately and without the Gibbs effect.

Noise accounting
In the above results, we did not take into account the influence of noise, therefore, we obtained clear restored images in Figures 2,4,9. Let us consider the influence of noise on the results of image processing for an example of impulse noise with intensities 0 and 255 [10,24,25].
In the Tukey median filter [4,7,24,25]  We propose a filter that we call a modified median filter (m-function modmed.m). According to this filter, it is supposed that the original noise-free image has no points with intensity 0 or 255, i.e.  Figure 1).
The following algorithm describes the operations of the modified median filter. Note that the intensities in the image in Figure 10a were artificially put equal from 1 to 254, as the modified median filter requires.
We also note that the impulse noise level (the share of noising) was put equal to d = 0.005, 0.01 and 0.02 (see Figures 10 and 5) in order to determine the dependence of the modified median filter error on the noise level. We see from a comparison of the images in Figures 10a and 10c that this filter well eliminates noise, but slightly distorts the image itself ( Figure 10c) and this leads to a noticeable distortion of the spectrum (Figure 10d) and the restored image (Figure 10e).
This means that the Tukey median filter has an error in restoring the center point of a sliding window. If we pass through medfilt2.m even a noise-free image, for example, the smeared image S1 in Figure 10a, we get the image Sf1 = medfilt2(S1, [3 3], 'symmetric'); which differents from S1 on 2.4%, i.e. the Tukey median filter distorts an image even in the noise absence. And since both the Gonzalez filter and our modified filter refer to the Tukey filter, they also distort an image (cf. Figures   10a and 10c). The relative error of the Tukey filter is 0244 . 0 rel =  , the Gonzalez filter is 0135 . 0 rel =  , and the modified filter is 0017 . 0 rel =  at noise d = 0.005.

Conclusion
In this paper, we describe a technique of restoring the smeared images in the case when the smearing is rectilinear horizontal, but non-uniform, in particular, piecewise uniform (example: the cars on a road). It is proposed to solve a set of 1-dimensional integral equations (IEs) (the first approach) or one 2-dimensional IE (the second approach). In both approaches, the equations are not convolution type IEs, therefore, they are solved by the quadrature/cubature method with Tikhonov's regularization. It is shown that the first approach is more preferable than the second approach.
Numerical examples confirm this. We developed programs in the MatLab system and performed illustrative calculations.
In order to determine the image smear of a variable quantity Δ = Δ(x), we proposed a new modification of the "spectral method" (the "way of divided spectra"), which increases the accuracy of image restoration.
It is also shown that to improve the quality of image restoration, the image edges' diffusing should be used for suppressing the Gibbs edge effect (a false wave effect). The Gibbs effect may also be inner. In this case, it is suppressed by image smearing, which reduces the jump in intensities.
We proposed a new filter for eliminating impulse noise in the image -a modified median filter that suppresses noise but almost does not distort the image.
The technique can be used in practice for restoring grouped images of several objects (people, planes, cars) moving with different speeds and therefore receiving different smears Δ in the image during exposure by a fixed camera.