Image Enhancement Based on Rough Set and Fractional Order Differentiator

: In the paper, an image enhancement algorithm based on a rough set and fractional order differentiator is proposed. By combining the rough set theory with a Gaussian mixture model, a new image segmentation algorithm with higher immunity is obtained. This image segmentation algorithm can obtain more image layers with concentrating information and preserve more image details than traditional algorithms. After preprocessing, the segmentation layers will be enhanced by a new adaptive fractional order differential mask in the Fourier domain. Experimental results and numerical analysis have veriﬁed the effectiveness of the proposed algorithm.


Introduction
The development of graphics technology and applications generates a huge number of images carrying valuable information, such as medical examination results, traffic condition diagrams and remote sensing images [1]. The wide range of graphics causes the problem that the vagueness of images will lead to information leakage and even misguidedness [2]. Under these circumstances, image enhancement methods are of great significance to highlight necessary information.
Image segmentation is a critical step before the enhancement process because of its ability to gather similar regions. Numerous image segmentation algorithms have been introduced in recent years based on different properties, such as rough set theory [3][4][5], genetic algorithms [6][7][8] and fuzzy set theory [9][10][11]. Furthermore, the image segmentation algorithms can be divided into several types, such as region-based algorithms, edge detection algorithms and clustering algorithms. These algorithms keep innovating on the original basis to seek a better segmentation effect. However, while these algorithms have been demonstrated to be an effective tool to handle uncertain or incomplete information, they still have the problems of missing detailed information, instability and introducing unrelated information in segmentation results. In this article, a new image segmentation algorithm based on roughness is proposed, which not only considers the difference in image regions but also uses similarity with neighbor pixels to cluster difference image patterns, combining the advantages of region-based and clustering algorithms. The preprocessed results show that this algorithm has more segmentation outputs and preserves more details and the patterns in each layer share higher similarity.
The quality of an image is enhanced through successive applications such as histogram manipulation [12], gamma correction [13] and contrast stretching [14]. While these methods have improved the image quality to some extent, they have their restrictions. For example, histogram-based methods are suitable for images that only have one peak in their histograms. Furthermore, these methods may not highlight necessary information as they are applied on the whole image without considering the gradient of the gray level. The digital fractional order differentiator is an effective tool to deal with these problems and has good stability in image enhancement. Recently, the interest in image enhancement techniques based on a fractional differential operator mask has grown in the field of image processing. Liu et al. [15] introduced the fractional differential wavelet algorithm which can greatly improve the high-frequency components of the signal and the very low frequency of the nonlinear retained signal. Kaur et al. [16] proposed a technique based on the Riesz fractional derivative in the fractional Fourier transform domain that outperformed the existing techniques by providing improvement in various performance parameters. Singh et al. [17] employed a piecewise gamma corrected fractional differential order masking framework for image quality enhancement. Experiments showed that the proposed approach had advantages compared with the conventional and state-of-the-art nonreal time algorithms. While these algorithms showed improvement, they still had problems of instability and cannot manage different kinds of information. Aiming at solving these problems, this paper proposes a new algorithm based on rough set theory and a fractional order differential operator.
The core contributions of the paper to achieve higher enhancement quality are identified as follows:

1
A new image segmentation algorithm based on rough set theory and a Gaussian mixture model is proposed, which combines the advantage of addressing uncertain boundaries and centralizing similar information from rough set theory and the property of clustering from a Gaussian mixture model. 2 We use 2D Fourier transform to change the gray level to image gradient and use filters to preserve different types of information. By applying a four-direction fractional differential mask on the Fourier domain, the proposed algorithm not only prevents the blur and shadow that could happen but also achieves the best enhancement for different types of frequency. 3 Entropy is employed as a cost function to calculate the optimal fractional order to preserve more image information and avoid degradation, which makes the enhanced images more suitable for computer vision processing.
The rest of the paper is organized as follows: Section 2 introduces the background knowledge of the proposed algorithm. Section 3 shows the detailed process of image segmentation and enhancement. Section 4 lists the experimental results to verify the performance and effectiveness. Section 5 concludes the whole paper.

Preliminary
In the image segmentation process, rough set theory is used to search for similar pixel points while a Gaussian mixture model is employed to cluster gray levels. After segmentation, Fourier transform is used to turn gray levels into frequency, then a fractional order differential operator is employed to enhance 2D signals in the image enhancement process.

Rough Set Theory
The rough set theory, first introduced by Pawlak in 1982 [18], has been proved to be an efficient analysis algorithm for studying uncertain information and incomplete data. It is effective in mining potential relationships between items without the need for additional relevant information.
For a given nonempty set U with limited elements x i , R is an equivalent relation on U and [x i ] R indicates the set of all the elements that are indistinguishable from x i under R. When X cannot be precisely described by the attribute of R, X is an undefined set or rough set of R. The R-lower and R-upper approximations of set X are defined as the following, respectively One example of the relationship between R-upper and R-lower approximations of set X and X is shown in Figure 1. Then, the R-upper and R-lower approximations have the following properties: The boundary region in rough set theory is B R (X) = R(X) − R(X). If the R-upper and the R-lower approximations are identical, i.e., B R (X) = ∅, then the set X is called an exact set.
The roughness of the set is caused by the boundary region. The larger the boundary region is, the less accurate set X is. In order to express this relationship, the measure of roughness is defined as The roughness measure can reflect the uncertainty of set X. For example, in Figure 1, the roughness measure of set X is 18 26 .

Gaussian Mixture Model
A Gaussian model is a common variable distribution model which is widely used in the field of mathematical statistics. The probability density function of a 1D Gaussian distribution is defined as follows: where θ = (µ, σ 2 ), µ and σ 2 represent expectation and variance, respectively. A Gaussian mixture model (GMM) [19] is regarded as a model composed of several Gaussian models, which are hidden variables of the GMM. The GMM can be used to smoothly approximate any given curve.
The probability density distribution of the GMM is where x is the observed data, K is the number of Gaussian models, α k is the component proportion of the kth model and θ k = (µ k , σ 2 k ). An example of a GMM of three Gaussian models with component α k = 1 3 is shown in Figure 2. The hidden variables α k and θ k of the GMM are needed to approximate a given curve. In this paper, an expectation maximum algorithm is employed to calculate the hidden variables. After building the GMM, intensity g is clustered by assigning it to the class with the largest posterior probability.

Fourier Transform
Fourier transform has been considered as a classical signal processing algorithm and one of the most critical analytical tools in many fields. Fourier transform can turn a square integrable function into a combination of circular functions. This algorithm can be used to transform the signal between the space and frequency domain.
In image processing, as the distribution function of an image is not continuous, 2D Fourier transform needs to be discretized. For an M × N image, the discrete 2D Fourier transform and its inverse transform are defined as: where F(u, v) is the distribution of the image gradient. The bright or dark parts on the F(u, v) represent the value of the gradient. If the gradient is large, then the brightness on F(u, v) is high, otherwise it is low. The energy distribution of the image can be obtained by observing F(u, v). After shifting F(u, v) to the origin, a conclusion can be drawn that the frequency is symmetrically distributed. The 2D Fourier transform of image "3.2.25" from the USC-SIPI image database "Aerials" and its shifted frequency image are shown in Figure 3.

Fractional Differential Operator
Fractional differentiation is the broad sense of integer-order differentiation and includes the nth derivative as a particular case [20,21]. There are three classical definitions of a fractional differentiator, the Grünwald-Letnikov definition, the Riemann-Liouville definition and the Caputo definition [22]. This paper uses the Grünwald-Letnikov fractional derivative definition as it has the advantages of being able to convert to convolution in numerical implementation and is suitable for image signal processing. For 1D signal According to Equation (11), if the period is divided by h = 1, Then, the fractional differential expression is obtained [23]: Under the definition of Grünwald-Letnikov, the fractional differential of y = 1 2 x 2 for different orders α is shown in Figure 4. For an M × N image f , as the gray level is finite, the largest variance is bounded. Furthermore, considering the discreteness of the image, the minimum interval in f (x, y) is the distance between adjacent pixels, which means h = 1. Then, in the x and y direction, n respectively. For image f , the fractional differential is defined as follows in the x and y direction: Then, a four-direction fractional order differentiator mask can be built as shown in  This paper employs the four-direction fractional order differentiator mask to scan the 2D transformed image gradient. Then, Fourier inverse transform is used to obtain the enhanced image.

Image Segmentation Algorithm
The intention of the segmentation algorithm is to produce image layers containing similar pixels, which can avoid burrs later in the image enhancement process. The rough set approach is employed to find pixels similar to the neighborhood while a Gaussian mixture model is used to cluster gray levels into several categories.

Histogram and Histon
Let f be the original M × N image, then the histogram is calculated by where δ(i) = 1, i = 0 0, i = 0 is the unit impulse function and L denotes the largest gray level.
Considering the P × Q neighborhood around pixel f (m, n), the total distance between all the pixels in the neighborhood and f (m, n) is calculated by By setting a threshold T 0 , the matrix X of size M × N is built by Then, the pixels in the neighborhood are assumed to be similar to pixel f (m, n) if d(m, n) < T 0 and their similarity decreases exponentially when d(m, n) ≥ T 0 .
Taking the discrepancy between distributions in different regions into consideration, the threshold T 0 is supposed to be larger for small regions to expand the R-upper approximation of the histogram, where k is the threshold of proportion of pixel intensity g. In this article, we set k = 0.003, T 1 = 40 and T 2 = 15. Then, the histon [24] is defined as The histogram and histon of the origin image are considered as the R-lower approximation and R-upper approximation in the rough set theory, respectively. The histogram value represents the number of pixels in the set of the gth gray level while the histon value of the gth gray level represents the number of all the pixels considered similar to g.
We substitute the histogram and histon into Equations (3)-(5). We can see the value of histograms will always be smaller than the histon. Furthermore, the histogram and histon value of gray level g 1 and g 2 will be the sum of their corresponding value. Then, it is verified that the histogram and histon satisfy all the properties in the rough set theory, which means they can be seen as the R-lower approximation and R-upper approximation.

Cluster Based on Gaussian Mixture Model
According to the rough set theory, the roughness measure is defined as However, the definition of histogram and histon determines their discreteness, which means the roughness measure function is discontinuous and it is difficult to separate its peaks and valleys. Under this circumstance, instead of using the δ(i) function, this paper uses the Gaussian kernel function to smooth the histogram and histon. Then, a Gaussian mixture model is applied to fit the roughness measure function.
The probability density distribution of the roughness measure function is where α k represents the component proportion of each Gaussian model, θ k = (µ k , σ 2 k ) and K are chosen by cross validation. For every given gray level g, it is assigned to the component Gaussian model yielding the highest posterior probability.
The segmentation process is described as follows: Step 1: For an M × N image f , for every channel, calculate its histogram h i (g) by using Equation (15).
Step 2: Use histogram h i (g) to consider the gth gray level as a small or large region and assign the corresponding threshold by Equation (18).
Step 3: Calculate the M × N weight matrix X by using Equation (17).
Step 4: Calculate the histon H i (g) by using Equation (19).
Step 5: Use the Gaussian kernel function to smooth h i (g) and H i (g) and calculate the rough measure function ρ i (g) with Equation (20).
Step 6: Use a Gaussian mixture model to fit the rough measure model and cluster the gray level.
Step 7: Use the cluster result to segment image f and obtain a series of segmentation results S.
The segmentation layers of image "3.2.25" are shown in Figure 6. The segmentation method proposed in the paper combined the rough set measure with the Gaussian mixture model. Compared with traditional segmentation methods which only divide an image into the object and background, this method has more segmentation layers which will preserve more details and pixels in every layer sharing higher similarity.

Adaptive Fractional Order Differential Mask for Image Enhancement in Fourier Domain
After preprocessing segmentation layers, this paper first uses 2D Fourier transform to turn gray levels into an image gradient, then applies filters to extract and enhance boundary and base information, respectively. Finally, a four-direction adaptive fractional differential mask based on entropy is employed to enhance images.

Preprocess
As the range of gray levels in segmentation layers might be wide, we compress the segmentation results S by using a logarithmic function. Then, to avoid burrs in the image enhancement process, blank regions are filled artificially in each segmentation layer with average nonzero pixel intensity by where F(x, y) is the image to be enhanced, s(x, y) is the compressed segmentation results,

Adaptive Fractional Differential-Based Image Enhancement Process in Fourier Domain
Then, the frequency images are obtained by 2D Fourier transform. By setting threshold d 0 , a high-pass filter (H p) and low-pass filter (Lp) are obtained to extract the boundary details and basic information of the image. Let The Fourier frequency images of the filtered images by H p and Lp and their inverse transform results are shown in Figure 8. Considering that the choice of fractional order will affect the image enhancement performance, this paper uses an unsupervised optimization algorithm to find the optimal fractional order. The concept of information entropy [25] is introduced to measure the distribution of gray value in an image to calculate the optimal parameter. The higher the value of entropy is, the more information the image transports. The formula of information entropy is where P(g) is the frequency of the gth gray level and L denotes the largest gray level of image f . After obtaining the optimal fractional order parameter, a fractional mask is used to enhance the image. The image enhancement process is described as follows: Step 1: Use a segmentation method above to obtain a series of image layers.
Step 2: Calculate the H p and Lp filters to extract the boundary and fundamental information.
Step 3: Calculate the optimal fractional order parameter α for filtered images F Step 5: Combine the enhanced results.
The block diagram of proposed algorithm is shown in Figure 9.

Experimental Results
In order to verify the image enhancement performance, we apply the algorithm on the USC-SIPI image database "Aerials", which was distributed in 1977 and consists of 37 color and one monochrome high-altitude aerial images, and list the enhanced results of the proposed method for six images (https://sipi.usc.edu/database/database.php?volume= aerials, accessed on 25 February 2022). The image enhancement performance of this method is compared with that of the 2-D DFOSGD method [26], which also builds an adaptive fractional order differentiator enhancement model. The comparison results are listed in Figures 10-15.      It is noted that compared with the original images, the proposed algorithm shows good enhancement results. Apart from this, the proposed algorithm extracts more texture details and makes the unobtrusive pattern more observable than the algorithm in [26].

Numerical Analysis
To further prove the efficiency of the proposed algorithm, a series of numerical analyses has been carried out with classical image enhancement algorithms such as 2D DFOSGD [26], LIME [27], Dong [28], MSRCP [29] and MSRCR [29].

Information Entropy Analysis
Shannon's information theory introduces the concept of information entropy and uses it to measure the distribution of gray values of an image. As this paper uses the model based on an entropy adaptive model to choose fractional order α and, in the other algorithms, we use the same process to choose optimal α to carry out the comparison experiments.
The comparison of the information entropy of enhanced images is shown in Table 1. The experimental results show that the entropy of the images enhanced by the proposed scheme is larger than that of the others, which means they can provide more information for later computer vision processing.
As the PSNR is the measure for gray images, in the experiment we average the gray level of the three channels of original and enhanced images as inputs to calculate MSE and PSNR. The comparison of PSNR of enhanced images is shown in Table 2. By analyzing the comparable results from Table 2, we find that the results of the proposed algorithm are better than most existing algorithms. Some results are smaller because the proposed algorithm emphasizes more interesting regions. While the information is preserved, the difference in gray level increases, which will increase the value of MSE.

Structural Similarity
To have a better understanding of the degradation of the image, this paper uses structural similarity (SSIM) [30] as another index. The SSIM is an indicator of the similarity between two images, which is defined as where µ x and µ y are the average of x and y, respectively, σ 2 x and σ 2 y are the variance of x and y, respectively, σ xy is the covariance of x and y and c 1 = (0.01L) 2 , c 2 = (0.03L) 2 are constant numbers to stabilize the equation. The range of SSIM is [0, 1]. When the two images are exactly the same, the value of SSIM is equal to 1. The comparison results are listed in Table 3. The comparison shows that the results of the proposed algorithm are better than the other algorithms except for the image "3.2.25". To have a better understanding, we made a comparison between the results of two algorithms, as shown in Figure 16. From Figure 16, it is observed that SSIM values of the other algorithms are higher, which is because their enhancement performances are not obvious. While the proposed algorithm preserves and enhances the image structure, it sensibly modifies the gray levels and leads to the lowest SSIM.

Measure of Enhancement by Entropy
The measure of enhancement by entropy (EMEE), proposed by Agaian et al. [31], is based on the modification of Weber's and Fisher's Laws. EMEE uses Weber's ratio in the calculation of entropy and gives the measure of contrast in terms of entropy [32]. EMEE is more sensitive to change in contrast to other metrics, with a larger EMEE value indicating higher contrast in images. EMEE is defined as where image f is divided in to L × L blocks, 0 < α < 1 is a constant, f max and f min are the maximum and minimum values of the pixels in each block. The compared results are shown in Table 4. The comparison shows that the proposed algorithm has the second highest EMEE value among all the six algorithms. Moreover, while 2D DFOSGD has the highest EMEE value, it might lose information of an image as the high contrast will reduce the detail pattern. The blind/referenceless image spatial quality evaluator (BRISQUE) [33] is a model based on natural scene statistics and can measure image naturalness based on measured deviations from a natural image model. The first step of BRISQUE is extracting natural scene statistics and transforming image gray level to luminance. Then, we calculate fracture vectors and use a learning algorithm to evaluate the image quality score, with a smaller BRISQUE score indicating better perceptual quality. The comparison results are shown in Table 5. The results show that the proposed algorithm has the lowest BRISQUE for most of the test images, while for image "3.2.25" MSRCP and LIME have better performance. From Figure 16, it is observed that the change in both algorithms is slight, which leads to the lower BRISQUE value.

Conclusions
In this paper, a new image enhancement algorithm based on a rough set and fractional order differentiator is proposed. In the proposed segmentation process, the rough set theory and Gaussian mixture model are combined to cluster gray levels into several image layers. In addition, compressing gray levels and filling in blank regions are performed to further centralize the information. Moreover, in the image enhancement process, 2D Fourier transform is employed to turn gray levels into a gradient, then an adaptive fractional order differential operator based on entropy is proposed to enhance the information of images. Experiments have verified the effectiveness of this algorithm. In the future, we intend to extend this enhancement process by studying more efficient segmentation methods and enhancement algorithms that can achieve higher structure similarities.