1. Introduction
Noise removal is one of the most important applications of image processing; it is used for image enhancement and also for a wide range of intermediate tasks in more complex applications, for example, as a stage in computer vision algorithms. However, removing noise from images is difficult because several types of noise are strongly correlated to the signal making difficult to discriminate between noise and the signal; also, there are gaps in our understanding of the human visual system (HVS) that prevent scientist from developing low-distortion techniques for signal filtering, and as a result, most algorithms reduce noise in images at the expense of damaging edges, creating visual artifacts, or removing details of the image.
Current approaches to noise removal include techniques such as linear and nonlinear filters. An example of a linear filters are the smoothing filter explained in [
1,
2,
3,
4]; a side effect of such filters is that they smooth edges too, which is a drawback since our visual perception needs edge information to recognize objects. Another widely used linear filter is the average filter; this is a sliding-window filter that computes each pixel of the filtered image as the spatial average of pixels in the neighborhood around the center pixel of the filter’s window; like the smoothing filter, average filters tend to blur edges. In addition to these disadvantages of linear filters, their digital implementations are usually bulky and slow. Refs. [
5,
6] give an overall panorama of noise removal using linear filters in computer vision.
On the other hand, the class of non-linear filters include the median filter which estimates each pixel of the filtered image as the median of the pixels in a square sliding-window; authors of [
7] discussed some modifications of median filters and their properties; for example, the max-median filter proposed by Arce in [
8,
9]; this filter removes noise whereas preserves geometrical features and is the base for the weighted median filter which generalizes the median filter; in a weighted median filter, positive integer weights are assigned to each position in the filter window and then the estimated pixel of the filtered image is computed as the median of the weighted neighborhood in a sliding-window manner. In general, the median filter is a nonlinear filter that preserves edges whereas removes impulsive noise.
Neuvo et al. introduced a class of median-type filters, which is called the FIR-median hybrid (FMH) filter with predictive FIR substructures as discussed in [
7]. The input signal,
, is filtered using
M linear phase FIR filters, and the output of the FMH filter is the median of the outputs of the filter bank. Shmaliy et al. [
10] developed the Unbiased Finite Impulse Response (UFIR) filter: a filter that reduces noise and preserves edges. Shmaliy developed this filter intending to reduce the Time Interval Error (TIE) of Global Position Systems (GPS) signals. The main characteristics of this filter are that it requires two constraints: first, the horizon value
N, which provides control over an initial estimation; and second, the filter holds the unbiased property: the input–output relationship of the signals exhibits the same energy, refs. [
10,
11,
12,
13].
Despite the efforts of various authors, such as the hybrid systems developed by Neuvo and the UFIR filters proposed by Shmaliy, which outperform the classical median filter, the weighted median filter, and other similar filters, their performance still requires further improvement, as noted by the aforementioned authors.
This motivates our work; we aim to develop a filtering structure with higher performance in preserving edges while reducing Gaussian noise compared to previous approaches. This paper presents the following contributions:
Developed a new class of non-linear filter to enhance images based on the FMH (
Section 4).
Developed three different FMH structures based on three symmetric window configurations (
Section 4).
Evaluate the performance of our proposed filtering structures and compare them to other approaches using the Root-Mean-Square Error and Signal-to-Noise Ratio (
Section 6).
We envision our proposed structures as a basis for applications aimed at reducing Gaussian noise, such as image preprocessing tasks, texture-recognition-based applications, and other computer vision tasks.
The remainder of this paper is organized as follows: First, in
Section 2, we discuss previous works closely related to the filter we proposed; then, we outline the theory we utilized of both the polynomial image model and the UFIR filter in
Section 3. Next, in
Section 4, we present FMH structures we propose for image processing. In
Section 5, describe the materials we utilized and the methodology we followed to conduct our experiments, and in
Section 6, we report the results of the computational evaluations in different scenarios. Finally, the general conclusions in
Section 7 and references.
2. Overview of Related Work on Nonlinear Hybrid Structures
Standard FIR filters are useful for noise reduction; however, they output a distorted image in some cases. An alternative to overcome this drawback is to use nonlinear filters which introduce less distortion as discussed in [
8,
9]. Nonlinear filters are able to remove more noise while introducing less distortion in the filtered image. It is important to reduce distortion in the case of edges because they are one of the most important features of an image; Edges are high-frequency content, so when one looks to remove high-frequency noise from an image, filter designers often apply a low-pass filter, and as a result, the filter smooths the edges. To get around this problem, they can use a median filter as discussed earlier in this paper. The classical median filter slides a window over the sequence
, so for each window centered at
k, it contains a set of samples (
) as explained in [
7] in the form:
where
M is the range in values over positive integers. The median smoother computes the output from the observation
in this manner:
Given the limitations the classical median filter exhibits, many authors have worked to develop a variety of structures to implement both linear and nonlinear FIR filters, and regarding median filters, we can note, for example, the filter developed by Neuvo et al. in [
7], where he proposed an FIR Median Hybrid Structure (FMH), which is a hybrid structure that implements digital filters using nonlinear operators. The model of this structure is defined as:
where
and
, are the backward and forward estimates of measurement
respectively. In this structure, the observed signal
is filtered using a forward filter and backwards filter; and then, the estimated signal
is computed using a nonlinear operator (
); the options for this NLOP includes the mean, median, maximum, minimum among other operators that designers find useful. The Backward and Forward filters are defined:
and
Also, Neuvo defined the general form of the FMH structure as illustrated in
Figure 1. Therefore, the estimates computed by the FMH structure computed by (
4) and (
5), this model can be adapted for image processing as follows:
and
Here, the linear operator is applied to both the output of each filter and the input image to compute the output image,
, as follows:
Algorithm 1 outlines the procedure for forward and backward filtering using the FMH structure used the batch filters as shown in [
14,
15].
| Algorithm 1: FIR-Median Hybrid Filtering Algorithm. |
![Symmetry 18 00871 i001 Symmetry 18 00871 i001]() |
3. Background for the Proposed Filter
In this section, we explain the model for signals considered in this study and the basic concepts of the UFIR filter, which is the core of the proposed filters presented in
Section 4.
A two-dimensional image is often defined as a matrix
of size
. To make two-dimensional filtering simpler, one can rewrite this matrix as a row-ordered vector or a column-ordered vector. So, we represent two-dimensional images using a lexicographic form as follows.
To represent a two-dimensional image using (
9) and (
10), one replaces each vector with the discrete time-invariant deterministic signal
; To filter such signal, a filter is applied to (
9) first and then to (
10), or vice versa. Signal
can be modeled on a horizon of
N points in the state-space domain as discussed in [
14,
15].
Utilizing the finite Taylor series of order
expand signal
, we get:
where
is defined as sampling time and
,
, is the
-state of the signal at index
, so the signal is characterized with
K states. An image can be accurately represented using both (
9) and (
10); therefore, each of the vectors may also be considered a deterministic one-dimensional signal.
Assuming that the original signal is discrete and that
is a measurement sampled in the presence of zero-mean additive white Gaussian noise (AWGN) (
), the noisy signal is defined as follows:
and
where
is the
state vector,
is the measurement representing the image, the
measurement matrix is defined as
, and the
triangular matrix
is specified as follows:
Shmaliy et al. showed in [
10] that the first state,
, can be estimated in an unbiased manner from
by computing the convolution
, provided that the impulse response
of degree
l of the FIR filter is a monic polynomial (with
).
From this foundation, the following low-degree (
) polynomials were derived in [
11] to serve as impulse responses for the UFIR filter:
The
l-degree unbiased FIR filter requires only the optimal horizon
to minimize the MSE. To implement the UFIR filter for noise reduction in images, we use the algorithm whose pseudocode is presented in Algorithm 2.
| Algorithm 2: Unbiased FIR (UFIR) Filtering Algorithm |
![Symmetry 18 00871 i002 Symmetry 18 00871 i002]() |
5. Materials and Methods
We conducted a series of experiments to evaluate the performance of the proposed hybrid UFIR filters. First, we examined the effect of varying the parameter N for fine-tuning; for this purpose, we used a grayscale dataset. Next, we tested the filter using synthetic binary images, as edge degradation is more difficult to mask in this scenario. In the third experiment, we applied the filter to grayscale images to evaluate its performance under these conditions. The final experiment was designed to identify the highest-performing filter structure among those proposed in the previous section. In addition, we conducted a Mean Opinion Score (MOS) test to assess performance from a qualitative perspective. The materials, evaluation metrics, and methodology used in these experiments are described next.
5.1. The Materials
To assess the performance of the filter according to the type of input image, we used two datasets: one consisting of synthetic binary images and another consisting of grayscale images. The characteristics of the datasets were as follows. The first dataset consisted of twenty binary synthetic images of size 512 × 512, whereas the second dataset consisted of 2000 grayscale images of size 512 × 512.
5.2. The Metrics for Assessment of the Filters Performance
To establish a comparison between the schemes considered in this study and our proposed filter, we employed edge-oriented metrics to measure the ability of the compared filters to preserve edges, in addition to the well-known Root Mean Squared Error (RMSE) and Peak Signal-to-Noise Ratio (PSNR). We selected the Structural Similarity Index Measure (SSIM), the Normalized Edge Response (NSER), and the Gradient Conduction Mean Square Error (GcMSE).
The SSIM is closely related to the Human Visual System (HVS), as it uses information about image luminance, contrast, and structure. The primary application of this metric is to evaluate structure preservation and noise removal [
16].
Another metric we considered is the NSER. This metric was designed with low-level early vision features in mind, according to [
17], and it shows a high correlation between the visual perception of image quality and the resulting non-shift edge map.
Finally, we also considered the GcMSE, which is an edge-aware version of the Mean Square Error (MSE) and provides higher performance compared to MSE and SSIM [
18].
We also conducted a Mean Opinion Score (MOS) evaluation, as it is a recognized and reliable metric for assessing filter performance from a subjective perspective.
Building on this basis, we compared the performance of closely related schemes proposed by other authors with that of our approach and reported the results in
Section 6 using the aforementioned metrics.
5.3. Filter Parameter Tuning Methodology
The order of the UFIR filter used in the conducted tests was linear; thus,
, and the step size
p was set as follows:
which, according to [
13] is the optimal value for this parameter.
5.4. Methodology to Assess the Performance of the Filter
First, we filtered several grayscale images to identify the optimal value of the parameter N. In the next experiment, we evaluated the filter’s performance on binary images with wide flat areas and scattered edges. Finally, we examined grayscale images filtered using the proposed scheme. For these tests, each image was corrupted with zero-mean Gaussian noise () at variances , , , and . For each variance level, the noisy image was filtered and RMSE, PSNR, SSIM, NSER, and GcMSE were computed. This process was then repeated for the grayscale image dataset.
To assess the performance from a qualitative perspective, we conducted a Mean Opinion Score test. We asked 132 individuals to observe a set of images that were filtered using the filters considered in this study, and they were asked to choose the image that preserved the details the best after being filtered.
The final experiment consisted of identifying the best-performing filter structure according to the selected metrics. To this end, Gaussian noise was added to each image in the grayscale dataset, and the resulting images were filtered. Next, the evaluation metrics were computed, and a Friedman test was conducted to identify the best-performing filter. This statistical test allows us to determine whether statistically significant differences exist among the filters considered in this study and, in this manner, to identify the best-performing approach.
6. Computational Simulations
As a first step, we determined the optimal value of the filter parameter
N to properly assess the filter’s performance in subsequent experiments. To this end, we set
and evaluated the filter for
. We then computed the evaluation metrics to observe how performance varied with
N; based on this analysis, we identified the value of
N that yields a properly tuned filter.
Figure 6a–e show the evaluation of the filters for each metric. The results show that the UFMH-XC filter exhibits the best performance for SSIM and achieves a tie with UFMH-C in terms of RMSE and PSNR. Likewise, UFMH-C provides better reconstruction quality for GcMSE and ties with UFMH-XC in RMSE and PSNR. Finally, the NSER metric indicates improved edge preservation by the UFMH-X filter. From the plots in
Figure 6a–e, we observe that
is the value at which the metrics stabilize. Accordingly, we set
for all subsequent experiments reported in this paper.
In the second experiment, the input image for the filtering structure was a synthetic binary image with wide flat areas and few edges; an example is shown in
Figure 7a. As can be seen, the image has sharp edges and flat areas, making it difficult to mask edge distortion, which is the rationale behind this test. The resulting image after adding Gaussian noise (
) is shown in
Figure 7b.
The noisy image (
) was then filtered using the FMH [
7], and the filtered image is shown in
Figure 7c. The output image exhibits slight variations in flat areas; however, these variations are more noticeable near the edges. Forward and backward filtering using a hybrid structure significantly improved the results for the same noisy image, as expected from [
14,
15].
Next, the noisy image was filtered using the FMH-based structure proposed in [
8] as a reference framework. In our implementation, this structure is adapted by replacing the FMH core with a UFMH (UFIR-based) filter. The filtering was oriented at 45 degrees from the central pixel. This approach exhibited improved performance compared to the conventional FMH. Its inherent characteristic of processing pixels along 45-degree directions explains this improvement. In contrast, the FMH is limited to processing pixels in row-wise and column-wise directions.
The resulting output image is shown in
Figure 7d. This figure shows that the noise in flat areas was reduced; however, distortion similarly increases near the edges, as observed in the case of the FMH. The main difference between the UFMH-X and FMH is that there are two edges for each processing direction in the former, whereas this is not the case for the latter.
The third structure evaluated in this test was the FMH proposed in [
9], which we again adopt as a reference framework. The structure proposed therein employs a cross-shaped window. In our implementation, this framework is adapted by replacing the FMH core with a UFMH filter, resulting in the UFMH-C structure. To illustrate the results achieved with this approach, a noisy image was filtered using the UFMH-C structure, and the resulting image is shown in
Figure 7e.
Finally, the UFMH-XC structure was tested. We verified that this scheme achieves higher performance compared to the FMH, UFMH-X, and UFMH-C filters. This is because the UFMH-XC was designed to exploit their advantages while mitigating most of the disadvantages of the filtering schemes discussed in this paper, as shown in
Figure 7f. In this figure, the noise in flat areas is significantly reduced, and the distortion near the edges is lower compared to the other approaches.
For the third experiment, we repeated the test using the grayscale image dataset discussed earlier. Again, Gaussian noise was added, and the noisy images were filtered using the FMH, UFMH-X, UFMH-C, and UFMH-XC structure, as in the previous tests. The results were consistent with those obtained for the synthetic binary images, as shown in
Figure 8a–f.
To assess performance from a qualitative perspective, we conducted a Mean Opinion Score (MOS) test. A total of 132 participants were asked to evaluate a set of six images filtered using the methods considered in this study and to select the image that best preserved details after filtering. All images had a resolution of pixels.
We added Gaussian noise with zero mean and variance . The images were filtered using the UFMH-C, UFMH-S, and UFMH-XC structures, as well as the median filter, UFMH-BF, FMH, and FAH, with the same parameters as before.
Each participant received a link to a Google Form containing the questionnaire, in which each question presented filtered images and asked participants to select the one they perceived as having the best quality. Participants repeated this procedure for five additional images. Since the questionnaire was distributed via a link, participants were able to complete the survey using their available devices (e.g., laptop, desktop, tablet, or cellphone).
The results clearly show that the UFMH-XC structure outperforms the other approaches, as illustrated in
Figure 9. The associated confidence intervals further support the distinctiveness of these results, reinforcing the reliability of the MOS-based evaluation.
To further analyze
Figure 9, we present
Figure 10, which breaks down the choice percentages for individual images and shows that they are almost uniformly distributed and contain no outliers, suggesting that participants were consistently convinced that the UFMH-XC structure was the best choice.
Next, we conducted an experiment using the Friedman test to identify the best-performing filter structure. First, we defined the null hypothesis (
) as follows: there is no significant difference in the performance of the filters considered in this study, along with the corresponding alternative hypothesis (
). Second, we computed the average rankings of the evaluated metrics, assigning rank 1 to the best-performing method for each metric, i.e., higher values were ranked as 1 for quality-related metrics (e.g., SSIM, PSNR), whereas lower values were ranked as 1 for error-related metrics. Finally, the average rankings were used to identify the best-performing filter. The results of the test are shown in
Figure 11 for a noise variance of
, and for other noise variances in the figures in the Annex.
Figure 11,
Figure A1,
Figure A2 and
Figure A3 show bar plots ordered according to the average rank for each filter. The Friedman test revealed statistically significant differences (
) among the evaluated filters for all considered metrics. According to the average rankings, the FAH structure achieved the best performance in terms of PSNR, RMSE, SSIM, and GcMSE, and exhibited the most consistent performance across the evaluated structures. For NSER, the UFMH-X structure achieved the best performance for noise variances
, while for a noise variance of
, the median filter ranked first. Overall, these results indicate that the FAH structure provides the most robust performance across the majority of quantitative metrics.
From the qualitative and quantitative experiments conducted to assess the filter’s performance, we observed that the results of both evaluations are consistent in indicating that the proposed filter exhibits higher performance compared to other approaches.
7. Conclusions
In this paper, we introduced a new unbiased filter with an FMH structure for reducing Gaussian noise in digital images, based on a framework that incorporates a UFIR filter as its core. We also explored the effect of using symmetric window-shape configurations on the filter’s performance. We conducted a series of experiments, first to fine-tune the filter and then to assess its performance using the RMSE, PSNR, NSER, SSIM, and GcMSE metrics, as well as a Mean Opinion Score (MOS) test.
We found that, for the target application, the parameter N selected through experimentation was , as higher orders yield only marginal improvements in performance.
According to the results in
Section 6, for noise variances of
, the UFMH-C structure achieves the best performance in terms of NSER. In contrast, the FAH structure exhibits the most consistent performance among the evaluated methods. In particular, the FAH structure effectively preserves edges while reducing noise in flat regions and achieves the best performance in terms of PSNR, RMSE, SSIM, and GcMSE. These results indicate that, according to the selected quantitative metrics, the FAH structure provides the best overall performance under these conditions.
However, for a noise variance of , the median filter ranks first, while the FAH structure consistently ranks second. In contrast, the X-shaped UFIR filter achieves second place in terms of NSER.
Although the MOS test indicates that the UFMH-XC structure provides the best edge-preserving performance, the quantitative metrics suggest a different outcome. This apparent discrepancy highlights a well-known limitation, namely the gap between current objective metrics and the Human Visual System (HVS). Some metrics primarily assess noise reduction, even in the presence of structural distortion, while others focus on edge preservation but do not fully capture both aspects simultaneously. In this context, the MOS-based assessment provides a more comprehensive evaluation of perceptual quality. These results suggest that objective metrics may not be fully suitable for assessing the performance of this filter; consequently, the performance of the UFMH-XC structure should be complemented with qualitative measures.
Future work includes assessing the filter’s performance under other types of noise, comparing the proposed filter with state-of-the-art edge-preserving methods, and evaluating its performance on applications such as ultrasound and night-vision image enhancement.