Intelligent Frequency Domain Image Filtering Based on a Multilayer Neural Network with Multi-Valued Neurons

Abstract

Neural networks have shown significant promise in the field of image processing, particularly for tasks such as denoising and restoration, due to their capacity to model complex nonlinear relationships between inputs and outputs. In this study, we explored the application of a complex-valued neural network—a multilayer neural network with multi-valued neurons (MLMVN)—for filtering two types of noise in digital images: additive Gaussian noise and multiplicative speckle noise. The proposed approach involves processing images as a set of overlapping patches in the frequency domain using MLMVN. Training was performed using a batch learning algorithm, which proved to be more efficient for big learning sets: it results in fewer learning epochs and a better generalization capability. Experimental results demonstrated that MLMVN achieves noise filtering quality comparable to well-established methods, such as the BM3D, Lee, and Frost filters. These findings suggest that MLMVN offers a viable framework for image denoising, particularly in scenarios where frequency domain processing is advantageous. Also, complex-valued logistic and hyperbolic tangent activation functions were used for multi-valued neurons for the first time and have shown their efficiency.

1. Introduction

The primary challenge in effective image and signal processing is the presence of noise. Noise refers to random deviations of a signal from its true value, caused by various external or internal factors within an image. As a result, a significant portion of image processing research and techniques focuses on noise reduction and removal.

In this article, we consider intelligent filtering of two common types of noise: additive Gaussian noise and speckle noise. Additive Gaussian noise with the Gaussian probability distribution function is usually caused by the following factors [1,2]:

• Discrete nature of light (photons), resulting in the fluctuation in the number of photons detected by an image sensor in a given period of time;
• Imperfections in electronic circuits, including amplifiers and analog-to-digital converters;
• Distribution of microscopic grains of metallic silver or dye particles in film emulsions while working with photographic film.

Speckle noise is a multiplicative noise, which may also contain an additive component. Speckle noise can be observed in images that are acquired using a coherent source (for example, laser, radar, and ultrasound). The cause of speckles is the microscopic variations in the surface roughness within one pixel, so when coherent radiation strikes the surface and becomes reflected, a received signal is subjected to random variations in phase and amplitude [3].

Noise filtering in images is a long-standing problem. However, it remains highly relevant today. Since the advent of digital imaging, numerous noise filtering methods have been developed. These range from a variety of spatial and frequency domain linear and nonlinear filters to filtering methods based on the use of neural networks and fuzzy logic.

One of the most efficient filters to deal with additive and speckle noise in images is the BM3D filter [4]. It is an efficient noise reduction method that operates in the frequency domain, utilizing the 3D Fourier transform on blocks constructed from statistically similar image patches. The BM3D filter has demonstrated high efficiency in reducing both Gaussian and speckle noise.

A variety of image denoising methods, including anisotropic diffusion, total variation minimization, weighted nuclear norm minimization (WNNM), trainable nonlinear reaction diffusion (TNRD), and shrinkage fields, have been explored in [5,6]. Techniques based on anisotropic diffusion, detailed in [7,8,9,10,11,12], employ a partial differential equation to iteratively smooth pixel intensities in low-gradient regions while preserving edges in high-gradient areas, effectively reducing noise while maintaining critical image features. Similarly, TNRD, introduced in [13,14], was demonstrated to be effective for filtering additive Gaussian noise in [5]. Total variation minimization, described in [15], reduces noise by minimizing the integral of the signal gradient, producing a denoised image that retains essential features. Other interesting noise filtering techniques are based on the use of shrinkage fields and nuclear norm minimization. The former one utilizes a structure called cascade of shrinkage fields (CSF) [16], offering computationally efficient denoising for high-resolution images. In turn, WNNM [17] constructs a low-rank matrix from similar image patches to estimate denoised patches, demonstrating high effectiveness in filtering additive Gaussian noise.

It is also worth mentioning such classical filters as the Lee filter [18], the Frost filter [19], and a family of order statistics-based filters [20,21,22,23]. The Lee and Frost filters are usually used to filter multiplicative speckle noise. These filters use the local statistics of an image fragment to calculate the updated intensity value of a target pixel. For example, the Lee filter uses the local mean value and weights based on the local and global variance of the image to calculate the pixel intensity. The Frost filter, in turn, uses an exponentially weighted value, with weights calculated using the local variance and mean.

Order statistics filters are based on the analysis of a variational series and the application of linear or nonlinear averaging over the intensities selected according to some special ranking criteria [24]. This family of filters includes basic filters such as the median filter and minimum and maximum filters [20,21,22]. But the main representatives of this family are based on ranking intensities in a local window around a pixel to be processed based on their closeness to the intensity of a pixel of interest in terms of values or their ranks [23,24,25]. These latter filters provide a decent ability to suppress noise and better preserve image details.

The widespread popularity of neural networks has led to their extensive adoption in various denoising methods, as well as other image processing tasks. A notable example is provided in [26], where machine learning techniques, and specifically neural networks, were employed for digital image restoration, noise removal, and object detection.

A comprehensive overview of deep learning-based image denoising methods using feedforward and U-shaped neural network architectures was presented in [27]. The study analyzed the advantages and limitations of these architectures, as well as various learning strategies employed for efficient image denoising.

Convolutional neural networks (CNNs) have captured the attention of researchers worldwide, including those working in the field of image filtering. Multiple CNN architectures were analyzed for filtering different types of noise in digital images in [28,29,30]. Among these, the Hybrid CNN [31] and the Pre-trained RLN [32] were specifically designed to address speckle noise and were reported to achieve performance comparable to, and in some cases surpassing, classical filters such as the Lee and Frost filters.

The denoising convolutional neural network (DnCNN) was proposed in [33]. DnCNN employs a deep architecture with multiple convolutional layers of various types and does not include pooling layers. The network was trained to reduce Gaussian noise in grayscale and color images, achieving results that were comparable to, and in some cases better than, those of BM3D, WNNM, TNRD, and CSF.

CNNs have also been adapted for specialized imaging domains. In [34], a deep CNN inspired by DnCNN was effectively utilized to filter a combination of additive Gaussian and Poisson noise in X-ray images obtained during the Multi-Shock (MShock) experiments conducted at the National Ignition Facility (NIF). According to the research, this approach achieved superior filtering results in terms of PSNR compared to filters such as the mean and Butterworth.

In addition to CNNs, neural networks with different architectures have also been successfully applied to denoising tasks. In [35], a multilayer perceptron (MLP) with four hidden layers, each containing 2047 neurons, was proposed for filtering additive Gaussian noise. The network was trained on a dataset designed from image fragments in the spatial domain, comprising 362 million training samples. The MLP with this architecture slightly outperformed the BM3D algorithm or showed comparable results in Gaussian noise filtering for specific test images.

Various types of neural networks, including those based on despeckling autoencoders, were evaluated for reducing speckle noise in ultrasound images in [36]. The study demonstrated that denoising solutions utilizing neural networks, such as Di-Conv-AE-Net, DGAN-Net [37,38], and D-U-NET, based on U-Net [39], outperformed widely used filtering algorithms like BM3D.

In [40], the stacked sparse denoising autoencoder (SSDA) architecture was proposed for denoising and inpainting grayscale images. This approach is based on sparse coding methods, which reconstruct images from a sparse linear combination of an overcomplete dictionary. It was demonstrated that SSDA could achieve performance comparable to the Bayes least squares with Gaussian scale mixture (BLS-GSM) method [41] and K-SVD [42].

A neural network framework named QIS-SPFT (QIS Serial–Parallel Fusion Transformer), which integrates CNN and transformer components, was employed to suppress Poisson (photon shot) noise during the QIS (quanta image sensor) imaging process was proposed in [43]. QIS-SPFT demonstrated superior performance in terms of PSNR compared to other filtering techniques, such as MLE, TD-BM3D, and QIS-Net.

In this paper, we focus on reducing the effects of speckle noise and additive Gaussian noise on digital images in the frequency domain using a complex-valued neural network.

Our idea is to perform image filtering by processing overlapping image fragments (patches) similarly how it was performed in [44,45]. However, while in that work a neural network is used to process patches taken from the spatial domain, we will focus on processing patches represented in the frequency domain using a multilayer neural network with multi-valued neurons (MLMVN). The idea behind this processing can be considered somewhat similar to the idea behind the BM3D filter. This filter performs processing by grouping similar image fragments into 3D blocks and processing them as a single unit. The idea behind this is that if patches are similar, then the noise affecting them should also be similar and can be filtered in a similar way. In our case, we would like to employ MLMVN as an intelligent processor that performs frequency domain convolution of image patches with adaptive convolutional kernels generated from the learning process.

The idea of using MLMVN as a framework for this task is based on the nature of image representation in the frequency domain, which relies on complex numbers. In turn, MLMVN is a type of neural network that shares the same architecture as the classical MLP but at the same time is based on multi-valued neurons (MVNs). MVNs, which are comprehensively described in [46], are artificial neurons whose weights, inputs, and outputs are complex numbers. This fact makes them a perfect processing unit for a vast number of problems where complex numbers are essential. This includes image processing in the frequency domain. MLMVN, along with its derivative-free backpropagation learning algorithm, was introduced in [47]. A number of studies have been conducted that build upon the concepts of MVNs and MLMVN [48,49,50,51,52]. Since their introduction, MVNs and MLMVN have demonstrated successful practical applications in various areas [53,54,55,56,57,58,59].

The structure of this paper is as follows. The Section 2 describes the process of noise filtering based on the use of a set of overlapping patches in the frequency domain. The respective algorithm is described, and the process of finding convolutional filter kernels in the frequency domain using MLMVN is presented. Then the training set design is described.

In Section 3, experimental results on filtering additive Gaussian noise and speckle noise using the MLMVN are presented. A detailed description of the learning process is provided, followed by a comparison of the filtering results obtained using our approach with those of some well-known popular filtering algorithms. Additionally, an analysis of the convolutional kernels generated from the learning process is conducted.

2. Frequency Domain Filtering of Overlapping Patches

2.1. Frequency Domain Filtering of Overlapping Patches

For some simplicity, but without loss of generality, we focus in this work on filtering grayscale images. Filtering of color images can be reduced to channel-wise processing of grayscale images, and therefore this simplification really does not affect anything.

Let $f(·)$ be an image. We may consider it as a superposition of a desired component $g(·)$ and a noise component $q(·)$. As we focus here on additive and multiplicative noises, the respective representations of this superposition are additive $f(·)=g(·)+q(·)$ and multiplicative $f(·)=g(·)\times q(·)$. The process of image filtering consists of finding an approximation of an ideal image $\widehat{f}(·)$ such that the difference between a filtered image and the ideal (noise-free) image is minimized according to a given metric. Usually, the root mean square error (RMSE) is used as this metric.

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{MN}}\sum _{i=1}^M\sum _{j=1}^N{\left(A(i,j)-B(i,j)\right)}^2}$$

Minimization of RMSE means maximization of the peak signal-to-noise ratio (PSNR).

$$\mathrm{PSNR}=20{log}_{10}\left({\displaystyle \frac{L}{\mathrm{RMSE}}}\right),$$

where L is the maximum possible pixel intensity value for a grayscale 8-bit image $L=255$. Specifically, PSNR is usually used to evaluate the quality of filtering.

Many classical image filtering methods [18,19,20,21,22,23] usually utilize linear and nonlinear filtering in the spatial domain. However, filtering in the frequency domain can also be highly effective. For example, one of the most effective filters for Gaussian and speckle noise reduction—BM3D—processes images in the frequency domain. It is a common understanding that a visible noise component in the case of additive and multiplicative noise primarily affects higher frequencies. This can be observed in Figure 1.

Hence, filtering of this kind of noise should typically be utilized via low-pass filtering preserving low frequencies and carefully and selectively suppressing high-frequency components affected by noise. This means that we may consider the process of filtering in the frequency domain as the detection of high-frequency components affected by noise followed by their suppression. A crucial part of this process is in fact the accurate identification of the components affected by noise and their correction. It is important to keep in mind that high-frequency components also contain information about small details and object boundaries. Therefore, while we are interested in noise filtering, we need to do our best to preserve useful information. By simply suppressing all high-frequency components, we may lose important information in a filtered image.

The design of filters is a sophisticated task. Some filters work effectively for a specific type of noise but prove to be ineffective for other types. This demands either modifying an existing filter or developing a new one. Such a situation drives the search for tools that enable the efficient design of new filters by implementing complex functional dependencies between the ideal and noisy images, thereby facilitating high-quality noise removal.

We employ here a neural network as such a tool. The idea behind the use of a neural network in image filtering is based on the ability of the network to learn from its environment. As a neural network may learn a lot of things from data, there are many reasons to believe that it may learn how to detect and suppress noise while at the same time preserving image details. Neural networks have successfully been applied to image filtering for more than a decade. Successful use of MLP for noise filtering was shown, for example, in [44,45]. So far, all applications of neural networks in image filtering have been in the spatial domain. That is, in all these applications a neural network developed a certain spatial domain filter. Here we suggest using a neural network for frequency domain filtering. We would like to use a neural network as a tool that is able to determine which frequencies are affected by noise and which are not or less affected. Simultaneously a neural network should be able to synthesize (design) convolutional kernels for noise filtering in the frequency domain resulting from the learning process. It is natural to use the MLMVN [47] for solving this problem, as a complex-valued network that is suitable to work with the complex-valued data in the frequency domain.

Nowadays images typically may have a large size. Thus, on the one hand, processing a large digital image as a whole using a neural network can be a highly resource-consuming task. On the other hand, to design a robust filter through the learning process, we would need to use many patches from many various images rather than some large image(s) as a whole. Thus, our idea is to focus on filtering relatively small overlapping patches. Each patch in this case should be filtered as a whole, while a resulting image should be created by averaging over all the overlapping pixels.

Hence, we focus on training MLMVN to design frequency domain convolutional kernels based on taking the Fourier transform of a noisy patch from an artificially corrupted image as an input and the Fourier transform of a respective clean patch from a noise-free image as a desired output. This approach also simplifies the adaptation of a neural network to specific data, as fragments contain significantly fewer details compared to the entire image. While processing local regions, the global context of an image is preserved through the overlapping areas of the fragments. Since fragments are much smaller than an entire image, their processing requires considerably fewer computational resources. Additionally, each fragment can be treated as an independent unit, enabling the parallel processing of multiple fragments.

Thus, our idea is basically to reconstruct the Fourier transform of a noise-free patch from the Fourier transform of its noisy version using a neural network whose input is the Fourier transform of a noisy patch and desired output is the Fourier transform of a corresponding noise-free patch. To create a respective representative learning set, many clean images should be corrupted by noise, noisy patches should be randomly selected from each image, their Fourier transforms should be used as inputs,  corresponding clean patches should be selected, starting with the same coordinates from the respective clean images, and their Fourier transforms should be used as desired outputs.

We suggest employing MLMVN with a single hidden layer and an output layer with the same number of output neurons as the number of Fourier coefficients in a respective patch’s Fourier transform. With this topology of MLMVN, its two layers of neurons perform the following tasks. Every neuron in a single hidden layer develops a frequency domain convolutional kernel through the learning process. Therefore, each neuron in a single hidden layer performs a frequency domain convolution, multiplying component-wise its weights by the respective Fourier transform coefficients of a patch to be processed.

$${\mathcal{F}}^{-1}(\mathcal{F}\left(w\right)\odot \mathcal{F}\left(p\right))=w★p$$

Each neuron in the output layer estimates a respective Fourier coefficient of a noise-free patch based on the outputs of all hidden layer neurons.

The selection of patches for constructing the training set and the learning process is described below in Section 2.2. After a learning set is created, the learning process should start. After a neural network develops its weights through the learning process, it should be used to filter images. To extract patches from an image, we use a window of size $m\times n$. By moving this window across the image using steps $s_x$ and $s_y$ (where $s_x$ corresponds to the step along the x-axis and $s_y$ to the step along the y-axis), we extract the intensities contained within the window into a patch.

As a result of this operation, a set of overlapping patches is generated if $s_x<m$ and $s_y<n$. This procedure is illustrated in Figure 2.

For the reader’s convenience, we provide an algorithm (Algorithm 1) for extracting a set of overlapping patches from an image to be processed. It should also be mentioned that the provided algorithm describes the process of building a set of patches for the actual image filtering process.

Algorithm 1: Extracting a set of overlapping patches from the input image

1: Input:   2: I: An image of size $M\times N$   3: m: The patch width   4: n: The patch height   5: $s_x$: The step size along x-axis   6: $s_y$: The step size along y-axis   7: Output:   8: P: A set of overlapping patches   9: Procedure: 10: $P\leftarrow \varnothing$ 11: for $i\leftarrow 1$ to $M-m$ by $s_x$ do 12:     for $j\leftarrow 1$ to $N-n$ by $s_y$ do 13:         Extract a patch p: $p\leftarrow I[i:i+m,j:j+n]$ 14:         $P\leftarrow P\cup \left\{p\right\}$ 15:     end for 16: end for 17: return P

Since we perform processing in the frequency domain, after a set of overlapping patches P is built, we need to create $P_{\mathcal{F}}$—a set of Fourier transforms of these patches—in the following way:

$$P_{\mathcal{F}}=\left\{{\displaystyle \frac{1}{nm}}\mathcal{F}\left(p\right)|p\in P\right\}.$$

The vectorized Fourier transform $p_{\mathcal{F}}\in P_{\mathcal{F}}$ should be used as an input sample for MLMVN. As was mentioned above, we employ a shallow MLMVN with a $k-q-k$ topology for processing frequency domain data. This means that a neural network consists of an input layer with k inputs, a single hidden layer with q neurons, and an output layer with k neurons. The number of inputs k, which is also equal to the number of neurons in the output layer, depends on the size of the image fragments being processed ($k=mn$). The number of hidden neurons q should be determined based on experimental testing.

It is important to make a remark about activation functions, which we use in our network. Two activation functions have been considered so far for MVNs—discrete and continuous ones. But both these functions project a weighted sum of MVNs onto the unit circle [46]. In MLMVN, which is used here, all hidden neurons employ a standard continuous MVN activation function $f\left(z\right)={\displaystyle \frac{z}{\left|z\right|}}$. At the same time, neither discrete nor continuous activation functions producing an output located on the unit circle can be used for output layer neurons. As our goal is to use MLMVN to estimate the Fourier transform of a clean patch, output neurons should produce an output not necessarily located on the unit circle. This requires the use of a different type of activation function capable of producing an output with an arbitrary (not necessarily unitary) magnitude. We used three such activation functions in our experiments: a linear activation function (when a weighted sum becomes an output), a complex-valued sigmoid activation function, and complex-valued hyperbolic tangent activation function. The respective results and analysis of the network behavior with all three of these activation functions are presented in Section 3.

As a result of processing elements of $P_{\mathcal{F}}$ with a neural network, we obtain a set of vectorized filtered overlapping patches ${\tilde{P}}_{\mathcal{F}}$ in the frequency domain. Elements of ${\tilde{P}}_{\mathcal{F}}$ can be used to obtain a set of filtered overlapping patches in the spatial domain in the following way:

$$\tilde{P}=\left\{crop\left(nm{\mathcal{F}}^{-1}\left({\tilde{p}}_{\mathcal{F}}\right)\right)|{\tilde{p}}_{\mathcal{F}}\in {\tilde{P}}_{\mathcal{F}}\right\}.$$

The goal of cropping the resulting image patch in the spatial domain is to remove possible unwanted artifacts and distortions that occur while processing the boundary regions of an image. The set $\tilde{P}$ is used for the “synthesis” of a filtered image.

It is important to note that after processing a patch with MLMVN, we restore a zero-frequency coefficient of the Fourier transform by setting it equal to a zero-frequency coefficient of the respective input (unprocessed) patch. This step is valid because our modeled noise has a zero mean. Thus, we can preserve a respective mean over a patch that is being processed. This is important to avoid a shift in patch intensities, which would occur if the mean value were modified.

For reconstructing an image from $\tilde{P}$, we use a window of size $t\times r$. By “moving” this window starting from the top-left corner with vertical step t in the range $[1,M-t]$ and with horizontal step r in the range $[1,N-r]$, and collecting all patches that lie inside the window at each step, we are able to reconstruct an image fragment of size $t\times r$. This should be carried out by combining the intensities of these overlapping patches. Intensities in areas where patches overlap are restored by applying an aggregation function. In this work, we employed median and mean aggregation functions. Below, an algorithm (Algorithm 2) for the “synthesis” of a filtered image from processed overlapping patches in the frequency domain is provided.

In general, the image noise filtering process proposed in this paper can be illustrated in Figure 3.

The algorithm for the complete process of filtering a noisy image is outlined here (Algorithm 3).

Algorithm 2: “Synthesis” of a filtered image from processed overlapping patches

1: Input:   2: $\tilde{P}$: A set of filtered overlapping patches   3: m: The width of a patch   4: n: The height of a patch   5: t: The width of a window   6: r: The height of a window   7: $agr$: An aggregation function   8: Output:   9: $I_{\mathrm{filtered}}$: A filtered image 10: Procedure: 11: for $i\leftarrow 1$ to $M-t$ by t do 12:     for $j\leftarrow 1$ to $N-r$ by r do 13:         $R\leftarrow \varnothing$ 14:         Collect the patches from $\tilde{P}$: 15:            $R\leftarrow \left\{p_{(x,y)}\mid p_{(x,y)}\in \tilde{P},\neg (x+m<i\vee x>i+t\vee y+n<j\vee y>j+r)\right\}$ 16:         Set the image intensities in the window: 17:            $I_{\mathrm{filtered}}[i:i+t,j:j+r]\leftarrow agr\left(\left\{p_{(x,y)}\mid p_{(x,y)}\in R\right\}\right)$ 18:     end for 19: end for 20: return $I_{\mathrm{filtered}}$

2.2. Organization of the Learning Process

The learning process we utilized in this work for training MLMVN is based on batch learning with validation. The main advantage of learning with validation is that it makes it possible to verify whether a neural network has developed a generalization capability. As learning with validation is based on the minimization of the validation error, it stops when a network is capable of generalizing with a desired accuracy. This also helps to avoid overfitting, which may often affect learning based on the minimization of the learning error (that is, the error on the learning set). Overfitting may occur from repeated attempts to “memorize” a learning set by achieving a low learning error. But the actual goal of learning is to develop a generalization capability (that is, to deal with the data that were not used to adjust the weights), not to “memorize” a learning set. This leads to improvement in robustness. In the learning process with validation, a dataset is divided into three subsets: a learning (training) subset, a validation subset, and a test subset. The training subset is used to adjust the weights, while the validation subset is used to verify whether a desired generalization capability has been achieved. Then the test subset is used to verify the results of the learning process.

Algorithm 3: The process of filtering noise in a digital image using MLMVN

1: Input:   2: $I_{\mathrm{noisy}}$: A noisy image   3: m: The width of a patch   4: n: The height of a patch   5: Output:   6: $I_{\mathrm{filtered}}$: A filtered image   7: Procedure:   8: Normalize the image: ${\widehat{I}}_{\mathrm{noisy}}\leftarrow {\displaystyle \frac{1}{255}}{I}_{\mathrm{noisy}}$   9: Split the image ${\widehat{I}}_{\mathrm{noisy}}$ into patches forming the set P: $P\leftarrow \mathrm{split}\left({\widehat{I}}_{\mathrm{noisy}}\right)$ 10: $\tilde{P}\leftarrow \varnothing$ 11: for $p\in P$ do 12:     Convert from the spatial to the frequency domain: $p_{\mathcal{F}}\leftarrow \mathcal{F}\left(p\right)$ 13:     Normalize $p_{\mathcal{F}}$: ${\widehat{p}}_{\mathcal{F}}\leftarrow {\displaystyle \frac{1}{nm}}{p}_{\mathcal{F}}$ 14:     Process ${\widehat{p}}_{\mathcal{F}}$ with the MLMVN: ${\widehat{p}}_{\mathcal{F}}^{\mathrm{out}}\leftarrow \mathrm{MLMVN}\left({\widehat{p}}_{\mathcal{F}}\right)$ 15:     Restore the zero-frequency coefficient: ${\widehat{p}}_{\mathcal{F}}^{\mathrm{out}}[0,0]\leftarrow {\widehat{p}}_{\mathcal{F}}[0,0]$ 16:     Denormalize ${\widehat{p}}_{\mathcal{F}}^{\mathrm{out}}$: $p_{\mathcal{F}}^{\mathrm{out}}\leftarrow \left(nm\right){\widehat{p}}_{\mathcal{F}}^{\mathrm{out}}$ 17:     Convert $p_{\mathcal{F}}^{\mathrm{out}}$ to the spatial domain: $p^{\mathrm{out}}\leftarrow {\mathcal{F}}^{-1}\left(p_{\mathcal{F}}^{\mathrm{out}}\right)$ 18:     $\tilde{P}\leftarrow \tilde{P}\cup \left\{p^{\mathrm{out}}\right\}$ 19: end for 20: Synthesize the filtered image $I_{\mathrm{filtered}}$ using set $\tilde{P}$: $I_{\mathrm{filtered}}\leftarrow \mathrm{synthesize}\left(\tilde{P}\right)$ 21: return $I_{\mathrm{filtered}}$

Model training was performed separately for each type of noise. To design training, validation, and test sets, a set of grayscale images of different sizes was used. It was split into three subsets, $S_t$ (training), $S_v$ (validation), and $S_f$ (test), such that $S_t\cap S_v\cap S_f=\varnothing$. The training dataset consists of pairs of vectorized Fourier transforms of noisy and corresponding clean image patches. The training dataset was built using set $S_t$, which comprised 300 grayscale images. It is important to note that as a part of preprocessing, we normalize the images by dividing their intensities by 255, so they have a range $[0,1]$ after normalization. This is important because a neural network learns faster and generalizes better when it deals with normalized data. For each image $I\in S_t$, an image $I_{\mathrm{noisy}}$ was created by applying noise (additive Gaussian noise or multiplicative speckle noise). Additive Gaussian noise was modeled with the following levels, $0.1\sigma$, $0.2\sigma$, and $0.3\sigma$, while multiplicative speckle noise was modeled with levels $0.2\sigma$, $0.3\sigma$, and $0.4\sigma$, where $\sigma$ is the standard deviation of the ideal (clean) image. After applying the corresponding noise, h random patches of size $m\times n$ were picked from the clean image I. In our experiments, we selected 200 patches per image. The same number of patches, of the same size and at the same spatial coordinates, were picked from each corresponding noisy image $I_{\mathrm{noisy}}$. As a result, two sets of image fragments were created:

$$P^{\mathrm{ideal}}=\left\{C_{(x_j,y_j)}^{\left(i\right)}|i=\overline{1,l},j=\overline{1,h}\right\}$$

and

$$P^{\mathrm{noisy}}=\left\{D_{(x_j,y_j)}^{\left(i\right)}|i=\overline{1,l},j=\overline{1,h}\right\}$$

where $(x_j,y_j)$ are spatial coordinates of top-left corner of patch and l is the number of images in $S_t$. Each element of the $P^{\mathrm{ideal}}$ and $P^{\mathrm{noisy}}$ sets was converted from the spatial domain to the frequency domain by computing a two-dimensional Fourier transform and normalized using the factor ${\displaystyle \frac{1}{nm}}$. After vectorizing each image fragment in the frequency domain, the following sets were built:

$$P_{\mathcal{F}}^{\mathrm{ideal}}=\left\{\mathrm{vec}{\left({\displaystyle \frac{1}{nm}}\mathcal{F}\left(C\right)\right)}^T|C\in P^{\mathrm{ideal}}\right\}$$

and

$$P_{\mathcal{F}}^{\mathrm{noisy}}=\left\{\mathrm{vec}{\left({\displaystyle \frac{1}{nm}}\mathcal{F}\left(D\right)\right)}^T|D\in P^{\mathrm{noisy}}\right\}$$

Thus, the training set for MLMVN was formed from elements of $P_{\mathcal{F}}^{\mathrm{ideal}}$ and $P_{\mathcal{F}}^{\mathrm{noisy}}$ in the following way:

$$L=\left\{\left(T_i^{\mathrm{in}},T_i^{\mathrm{out}}\right)|i=\overline{1,lh},T_i^{\mathrm{in}}\in P_{\mathcal{F}}^{\mathrm{noisy}},T_i^{\mathrm{out}}\in P_{\mathcal{F}}^{\mathrm{ideal}}\right\}$$

The validation dataset was built using g full-size images from $S_v$. Additive Gaussian or multiplicative speckle noise was applied to each image from $S_v$, thereby creating a set of noisy images $S_v^{\mathrm{noisy}}$ similarly to how noise was added to create a training set. Using pairs of ideal and noisy images, the validation dataset was constructed in the following way:

$$V=\left\{\left(V_i^{\mathrm{in}},V_i^{\mathrm{out}}\right)|i=\overline{1,g},V_i^{\mathrm{in}}\in S_v^{\mathrm{noisy}},V_i^{\mathrm{out}}\in S_v\right\}$$

In our work, MLMVN training was performed using the batch learning algorithm proposed in [60,61]. This algorithm is based on a derivative-free approach and enables the correction of neuron weights across multiple learning samples simultaneously (that is, across an entire batch).

As noted above, correction of the network weights was performed using samples from the training dataset L. During our experiments, we used a training dataset with 60,000 learning samples created from 300 grayscale images (as 200 patches were randomly selected from each of the 300 images). To employ the batch learning algorithm, learning samples were grouped into batches containing b samples each. We employed a batch size of 20,000 learning samples. The justification for this choice is provided in Section 3.2. Our learning process is based on the maximization of the mean value of the validation PSNR evaluated over the images from the validation set. During our experiments, we employed a validation set, which consists of five pairs of noisy and clean grayscale images.

After each batch learning step was completed, a validation step was performed. During the validation, image pairs from the validation dataset V were processed using the algorithm described in the previous section of this article (see Algorithm 3). The PSNR of the filtered image relative to the clean image was computed for each pair of images from V. The average PSNR value across all elements from V was compared with the threshold PSNR value, and the learning process either was stopped if the current average validation PSNR reached or exceeded a pre-determined threshold value or continued if it was below it. The algorithm for the learning process is outlined here (Algorithm 4)

The final performance evaluation of the proposed filtering approach was performed using the images from the $S_f$ set. It is important to note again that these images were not used in the design of either the training set or the validation set.

Algorithm 4: The MLMVN learning process

1: Input:   2: L: A training set   3: V: A validation set   4: ${\mathrm{PSNR}}_v$: The desired PSNR on the validation set   5: ${\mathrm{PSNR}}_b$: The besired PSNR on a batch   6: n: Tha size of a batch   7: iterations_limit: The maximum number of iterations per batch   8: Output:   9: A trained MLMVN 10: Procedure: 11: Build a set of batches B: $B\leftarrow \{b_1,\dots ,b_k\mid b_i\subseteq L,b_i\cap b_j=\varnothing \mathrm{when}i\ne j,|b_i|=n\}$ 12: ${\mathrm{PSNR}}_v^{cur}\leftarrow 0$ 13: repeat 14:     for $b\in B$ do 15:         ${\mathrm{PSNR}}_b^{cur}\leftarrow 0$ 16:         $\mathrm{iteration}\leftarrow 0$ 17:         while ${\mathrm{PSNR}}_b^{cur}<{\mathrm{PSNR}}_b$ and $\mathrm{iteration}<\mathrm{iteration\_limit}$ do 18:            Update the MLMVN weights 19:            Process the samples from b with the MLMVN: $\tilde{b}\leftarrow \{\mathrm{MLMVN}\left(s_i\right)\mid s_i\in b\}$ 20:            ${\mathrm{PSNR}}_b^{cur}\leftarrow \mathrm{mean}\left(\{\mathrm{PSNR}({\tilde{s}}_i,s_i)\mid {\tilde{s}}_i\in \tilde{b},s_i\in b\}\right)$ 21:            $\mathrm{iteration}\leftarrow \mathrm{iteration}+1$ 22:         end while 23:         Process the images from V with the MLMVN: $\tilde{V}\leftarrow \{\mathrm{MLMVN}\left(v_i\right)\mid v_i\in V\}$ 24:         ${\mathrm{PSNR}}_v^{cur}\leftarrow \mathrm{mean}\left(\{\mathrm{PSNR}({\tilde{v}}_i,v_i)\mid {\tilde{v}}_i\in \tilde{V},v_i\in V\}\right)$ 25:         if ${\mathrm{PSNR}}_v^{cur}>={\mathrm{PSNR}}_v$ then 26:            break 27:         end if 28:     end for 29: until ${\mathrm{PSNR}}_v^{cur}>={\mathrm{PSNR}}_v$ 30: return A trained MLMVN

3. Results

3.1. Convolutional Kernels Resulting from Learning and Their Analysis

As mentioned above, we employ an MLMVN topology of $k-q-k$ for filtering additive Gaussian noise and multiplicative noise in digital images. Here k is the number of input and output neurons and q is the number of neurons in a single hidden layer.

Let us consider a neuron s from a single hidden layer of our MLMVN. Let $x=(x_0,x_1,x_2,\dots ,x_k)$ be the input to the neuron ($x_0$ is a pseudo input corresponding to the bias and it always equals 1) and $w=(w_0,w_1,\dots ,w_k)$ be the neuron’s weights. The output of the neuron can be expressed as

$$O=f\left(\sum _{i=0}^k{x}_i{w}_i\right),$$

or equivalently as

$$O=f\left(x_0{w}_0+\sum _{i=1}^k{x}_i{w}_i\right)=f\left(x_0{w}_0+R\right).$$

In this expression,

$$R=\sum _{i=1}^k{x}_i{w}_i$$

Since the vector x results from the vectorization of an image fragment of dimensions $m\times n$ in the frequency domain, it can also be represented as a matrix $X_{\mathcal{F}}$, which is in its original shape. Similarly, the weights of the neuron (excluding the bias $w_0$) can be represented as a matrix $W_{\mathcal{F}}$. Consequently, the last formula can be rewritten as

$$R=\sum \left(X_{\mathcal{F}}\odot W_{\mathcal{F}}\right),$$

where the operator ⊙ denotes elementwise multiplication. According to the Convolution Theorem, this expression can be further expanded as

$$R=\sum \left(X_{\mathcal{F}}\odot W_{\mathcal{F}}\right)=\sum \mathcal{F}\left(X_s★W_s\right),$$

where $X_s={\mathcal{F}}^{-1}\left(X_{\mathcal{F}}\right)$ and $W_s={\mathcal{F}}^{-1}\left(W_{\mathcal{F}}\right)$, where ★ denotes the convolution operation. Based on this interpretation, the neurons in the hidden layer of MLMVN can be considered as convolutional processors (with their weights acting as convolutional kernels) and simultaneously as analyzers of the resulting convolutions. On the other hand, the neurons in the output layer act as ultimate processing units that integrate the outputs of neurons from the preceding layer, producing the result of filtering in the frequency domain.

Let us discover in detail how a neural network processes an image fragment. Each neuron in a single hidden layer actually performs filtering by utilization of the frequency domain convolution with a kernel K. Let $I_s$ be an image fragment in the spatial domain to be filtered, and $I_{\mathcal{F}}$ is its representation in the frequency domain, that is, the Fourier transform of $I_s$. Let $O_{\mathcal{F}}$ be the output of MLMVN for input $I_{\mathcal{F}}$. The output $O_{\mathcal{F}}$ is nothing but an approximation of the frequency domain representation of the processed image fragment $I_s$. Its spatial domain representation should be obtained by applying the inverse Fourier transform as follows:

$$O_s={\mathcal{F}}^{-1}\left(O_{\mathcal{F}}\right).$$

If we define $K_{\mathcal{F}}=O_{\mathcal{F}}\oslash I_{\mathcal{F}}$ and $K_s={\mathcal{F}}^{-1}\left(K_{\mathcal{F}}\right)$, we can express the relationship as follows:

$$O_{\mathcal{F}}=I_{\mathcal{F}}\odot K_{\mathcal{F}}=\mathcal{F}\left(I_s\right)\odot \mathcal{F}\left(K_s\right).$$

According to the Convolution Theorem,

$$\mathcal{F}(I_s★K_s)=\mathcal{F}\left(I_s\right)\odot \mathcal{F}\left(K_s\right).$$

Applying the inverse Fourier transform to both sides of the last equation, we obtain

$$I_s★K_s=O_s,$$

where $K_s$ is the convolutional kernel in the spatial domain. To discover what kind of filters resulted from the learning process, we need to discover the convolutional kernels generated from this process. Let us consider the power spectra $\left|K_{\mathcal{F}}\right|$ of some randomly selected convolutional kernels in the frequency domain resulting from the learning process as the weights of the respective neurons. These power spectra are just absolute values of the respective weights, which should be reshaped into matrices. They are shown in Figure 4.

Analysis of the power spectra shown in Figure 4 makes it possible to understand the frequency characteristics of filters resulting from the learning process. Figure 4a,b,d show that a neural network develops mostly various low-pass filters to reduce additive Gaussian noise. Basically Figure 4c also shows a low-pass filter, just a bit more sophisticated. It is clear from the high–low-frequency area “hills” in these examples that we are dealing with low-pass filters. However, it is important to mention that small “hills” appear in the medium- and high-frequency areas, which means that while these filters care about preserving low frequencies, they deal with high frequencies carefully, selectively (and, evidently, adaptively) preserving some of them. Analyzing Figure 4e–h, we may conclude that speckle noise filtering is a more sophisticated task, and a neural network adapts to this task in a different way. While the filters shown in Figure 4e,h are low-pass filters with some high-frequency areas carefully preserved, the filters shown in Figure 4f,g are much more sophisticated. They preserve five various-frequency areas each (this is clearly visible from the five “hills”), suppressing other frequencies at the same time. This means that a neural network adapts to specific noise by suppressing selective frequencies and preserving other frequencies (not only low frequencies, but preserving some specific frequencies across the bound), which might be important for preserving image details.

3.2. Batch Size and the Learning Results

The training dataset, as described earlier, consists of pairs of noisy and clean image fragments (patches) in the frequency domain. The learning process continued until a certain PSNR level was reached for the validation set, regardless of the learning error (that is, the error on the learning set) at that moment.

Another important preprocessing step is to perform mirroring (reflection padding) on a processed image. This step helps to avoid boundary effects while processing an image by ensuring that boundary intensities are processed in the same way as the rest of the intensities.

As was mentioned above, to organize the learning process, we used a batch learning algorithm [60,61]. In our work, we conduct experiments with various batch sizes. After performing many experiments with patches of various sizes, we found that a patch size of $8\times 8$ is optimal as the one leading to better results. Thus, a patch size of $8\times 8$ determines that our neural network should have 64 inputs and 64 outputs. To discover how a batch size influences the quality of filtering, the following experiment was conducted. MLMVN with a topology of 64-256-64 was trained to filter additive Gaussian noise with a standard deviation of 0.2$\sigma$ using different batch sizes. The network utilized a standard activation function for MVNs with continuous output in the hidden layer neurons, while an identity activation function was used in the output layer neurons. The learning epoch threshold was set to 3, while the iteration limit per batch was set to 100. The results of processing images from the test set with this MLMVN trained using various batch sizes are provided in

Table 1. The PSNR of the noisy images (additive Gaussian noise with SD = $0.2\sigma$) from the test dataset processed by MLMVN trained using different batch sizes. The best results are highlighted in bold.

Number of samples in a batch	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
1000	29.8493	29.2612	27.1164	29.9298	28.7228
2000	29.6757	29.3202	26.9995	30.0297	28.8767
3000	29.6286	29.2861	27.0174	30.0147	28.8323
4000	30.1186	29.5799	27.3663	30.207	29.1725
5000	30.2527	29.621	27.4764	30.1601	29.2054
6000	30.2331	29.6	27.4192	30.244	29.1891
10,000	30.4166	29.6002	27.5986	30.0765	29.128
20,000	29.8485	29.3861	27.3098	30.1236	28.9371
Number of samples in a batch	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
1000	28.3432	33.3609	31.4869	33.7677	33.9199
2000	28.4496	33.6422	31.63	34.0288	34.2254
3000	28.4347	33.5338	31.5108	33.7992	34.2069
4000	28.6853	33.8204	31.8179	34.4496	34.448
5000	28.717	33.8101	31.8114	34.3776	34.5142
6000	28.7273	33.8849	31.876	34.5031	34.5963
10,000	28.7651	33.5517	31.7046	34.1539	34.4115
20,000	28.5961	33.1434	31.3268	33.4583	33.9033

. For convenience, we also provide the ideal images from the test dataset that were used for the evaluation of the MLMVN filtering capabilities in Figure 5. According to our observations, larger batch sizes allow for achieving better results. This can be explained by the well-known fact that LLS-based methods for solving overdetermined systems of linear algebraic equations, which are the key elements of batch learning, show more accurate results for systems that are more overdetermined to some level, that is, for the systems where the number of equations is significantly larger than the number of unknowns. The number of equations in this case is determined by the number of samples in a respective batch.

During our experiments, we considered various activation functions for the neurons in the hidden and output layers (

Table 2. Activation functions used in output layer MVNs in this work.

Name	Expression
Identity activation function	$f\left(z\right)=z$
Classical activation function for MVN with a continuous output	$f\left(z\right)={\displaystyle \frac{z}{\left|z\right|}}$
Complex-valued sigmoid activation function	$f\left(z\right)=\left\{\begin{array}{cc}\hfill z,\hfill & \hfill \left|z\right|\le 1\hfill \\ \hfill {\displaystyle \frac{1}{1+e^{-\left|z\right|}}}{e}^{iarg\left(z\right)},\hfill & \hfill \left|z\right|>1\hfill \end{array}\right.$
Complex-valued hyperbolic tangent activation function	$f\left(z\right)=tanh\left(\left|z\right|\right)e^{iarg\left(z\right)}$

). It is well known that the main role of an activation function is to limit the neuron’s output range. In our case, an expected output of MLMVN is a filtered image patch in the frequency domain, whose magnitude should be in the range $[0,1]$, as we work with normalized images and their normalized Fourier transforms. Therefore, the important role of an activation function in our case is to keep the magnitude of the resulting Fourier transform coefficients within the range $[0,1]$ (Figure 6). The only exception is the linear (“identity”) activation function, which simply passes the computed weighted sums to the output without any modification.

When performing filtered image “synthesis” from a collection of filtered overlapping patches (Algorithm 2), an aggregation function should be used to process the overlapping pixels in the patches. In this work, we employed median and mean, both taken over the overlapping values as aggregation functions. According to the results of our experiments, the median aggregation function performs slightly better.

As part of our experiments, we trained several MLMVN models to filter additive Gaussian noise and speckle noise. As was already mentioned, we employed the MLMVN topology 64-1024-64 because it has shown the best results. This topology processes 8 × 8 image patches in the frequency domain. MLMVN-G-1, MLMVN-G-2, and MLMVN-G-3 were trained for denoising images with additive Gaussian noise, while MLMVN-S-1, MLMVN-S-2, and MLMVN-S-3 were trained for denoising images with multiplicative speckle noise. In all MLMVNs neurons in the hidden layer employed a standard continuous MVN activation function. The output layers of MLMVN-G-1 and MLMVN-S-1 employed the identity activation function, MLMVN-G-2 and MLMVN-S-2 employed the complex-valued sigmoid activation function, and MLMVN-G-3 and MLMVN-S-3 employed the complex hyperbolic tangent activation function. Figure 7 shows convergence curves for all these MLMVN models. These curves represents the mean PSNR over the validation set vs. the number of epochs/batches.

3.3. Simulation Results for Filtering Additive Gaussian Noise and Speckle Noise

Below, we provide the results of filtering noisy grayscale images corrupted by additive Gaussian noise (

Table 3. Results of filtering grayscale images affected by additive Gaussian noise with a standard deviation of 0.1$\sigma$, where $\sigma$ is the standard deviation of the ideal image. The best results are shown in bold, and the best results using our approach are highlighted in color.

	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
Noisy image PSNR	31.4534	32.4111	29.6285	34.1881	31.8786
BM3D 0.5$\sigma$	33.3987	33.6144	31.2485	34.9508	32.9469
BM3D 1$\sigma$	37.4319	35.6073	34.2258	36.6911	34.6627
BM3D 1.5$\sigma$	36.119	34.2096	33.1037	35.5024	33.7332
MLMVN-G-1	35.4248	32.3049	31.6007	32.0308	31.7697
MLMVN-G-2	35.4047	32.2347	31.5742	32.0051	31.7367
MLMVN-G-3	35.4594	32.3276	31.6167	32.1362	31.8193
	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
Noisy image PSNR	31.4643	34.9899	33.4381	35.0817	35.1862
BM3D 0.5$\sigma$	32.3518	36.7648	35.1181	37.1523	36.7276
BM3D 1$\sigma$	33.8656	40.1366	38.2556	41.7187	39.8033
BM3D 1.5$\sigma$	32.403	38.3289	36.94	41.1617	38.9479
MLMVN-G-1	31.555	35.5246	34.306	37.7523	37.2053
MLMVN-G-2	31.5289	35.5263	34.2972	37.7286	37.2716
MLMVN-G-3	31.6001	35.5819	34.3893	37.8504	37.2804

,

Table 4. Results of filtering grayscale images affected by additive Gaussian noise with a standard deviation of 0.2$\sigma$, where $\sigma$ is the standard deviation of the ideal image. The best results are shown in bold, and the best results using our approach are highlighted in color.

	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
Noisy image PSNR	25.5579	26.385	23.9992	28.1943	25.9918
BM3D 0.5$\sigma$	28.1544	28.133	26.35	29.491	27.6905
BM3D 1$\sigma$	33.7363	31.2135	30.1307	32.4521	30.6029
BM3D 1.5$\sigma$	32.5886	29.5837	28.4187	30.8032	28.7687
MLMVN-G-1	31.701	30.1547	28.5963	30.6758	29.6646
MLMVN-G-2	31.6896	30.1126	28.5971	30.6637	29.6592
MLMVN-G-3	31.6822	30.1428	28.5871	30.74	29.678
	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
Noisy image PSNR	25.5342	28.9904	27.4271	29.0807	29.1782
BM3D 0.5$\sigma$	27.0836	31.2659	29.6573	31.6465	31.345
BM3D 1$\sigma$	29.8181	35.8772	34.288	38.2263	36.4391
BM3D 1.5$\sigma$	28.0811	33.8929	32.6458	37.3526	35.4323
MLMVN-G-1	29.3247	34.2611	32.4535	35.729	35.685
MLMVN-G-2	29.3218	34.2212	32.4293	35.6606	35.669
MLMVN-G-3	29.3326	34.2682	32.4706	35.7497	35.6836

and

Table 5. Results of filtering grayscale images affected by additive Gaussian noise with a standard deviation of 0.3$\sigma$, where $\sigma$ is the standard deviation of the ideal image. The best results are shown in bold, and the best results using our approach are highlighted in color.

	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
Noisy image PSNR	22.1619	22.9098	20.8601	24.6972	22.5851
BM3D 0.5$\sigma$	25.1578	25.0337	23.8199	26.3476	24.7072
BM3D 1$\sigma$	31.5803	28.8525	27.6952	30.2132	28.2913
BM3D 1.5$\sigma$	30.599	27.2943	26.0956	28.6788	26.4801
MLMVN-G-1	27.8965	27.4263	25.4873	28.734	26.979
MLMVN-G-2	27.9158	27.4332	25.5081	28.7452	27.0003
MLMVN-G-3	27.865	27.4018	25.4714	28.7631	26.9621
	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
Noisy image PSNR	22.1456	25.493	23.9169	25.5879	25.6751
BM3D 0.5$\sigma$	24.1643	28.0635	26.4294	28.3924	28.1542
BM3D 1$\sigma$	27.7561	33.763	32.1677	35.978	34.4722
BM3D 1.5$\sigma$	26.2848	32.1272	30.6002	34.8368	33.4473
MLMVN-G-1	26.6104	31.725	29.6582	32.2901	32.6759
MLMVN-G-2	26.6285	31.7205	29.6736	32.2841	32.6955
MLMVN-G-3	26.5878	31.7009	29.6442	32.2805	32.6411

) and multiplicative speckle noise (

Table 6. Results of filtering grayscale images affected by multiplicative speckle noise with a standard deviation of 0.2$\sigma$, where $\sigma$ is the standard deviation of the ideal image. The best results are shown in bold, and the best results using our approach are highlighted in color.

	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
Noisy image PSNR	32.6753	32.1159	30.3498	35.3354	31.2367
BM3D 0.5$\sigma$	36.7301	34.6453	33.3135	37.0061	33.7141
BM3D 1$\sigma$	34.6252	32.043	31.3216	33.6298	31.8137
BM3D 1.5$\sigma$	33.1557	29.8412	28.7203	31.0331	28.8251
Frost	29.8109	25.8298	25.2361	24.3977	26.0143
Lee	31.4611	28.2385	26.7904	27.4653	27.7698
MLMVN-S-1	36.2585	33.1139	34.7767	35.0894	33.4302
MLMVN-S-2	35.7398	32.8295	34.4242	34.8853	33.0994
MLMVN-S-3	35.9468	32.9511	34.6287	34.9894	33.2681
	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
Noisy image PSNR	31.2046	34.6086	33.7206	35.7485	31.8837
BM3D 0.5$\sigma$	33.3862	40.3269	37.9508	40.786	35.6958
BM3D 1$\sigma$	30.4868	36.2671	35.3317	40.2067	37.5485
BM3D 1.5$\sigma$	28.2379	34.0124	32.9935	38.1526	35.7779
Frost	25.3471	30.4103	27.7662	30.1288	31.1936
Lee	27.0814	32.7945	30.0945	34.2327	34.3365
MLMVN-S-1	34.0183	38.7255	37.6577	39.5762	37.1491
MLMVN-S-2	33.7173	38.4703	37.2599	38.6982	37.1038
MLMVN-S-3	33.8792	38.7025	37.4987	39.1209	37.1174

,

Table 7. Results of filtering grayscale images affected by multiplicative speckle noise with a standard deviation of 0.3$\sigma$, where $\sigma$ is the standard deviation of the ideal image. The best results are shown in bold, and the best results using our approach are highlighted in color.

	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
Noisy image PSNR	29.1548	28.6296	27.3704	31.8288	27.7893
BM3D 0.5$\sigma$	34.2037	31.7623	31.0264	33.9564	31.1511
BM3D 1$\sigma$	32.576	29.4154	28.4644	30.9181	28.8088
BM3D 1.5$\sigma$	31.3591	27.5322	26.5578	29.005	26.6459
Frost	29.3017	25.5846	24.9998	24.3581	25.8356
Lee	30.7351	27.8267	26.4379	27.3616	27.5166
MLMVN-S-1	33.4248	31.2865	31.8917	33.6882	31.4565
MLMVN-S-2	32.9115	30.9381	31.7142	33.3546	31.1229
MLMVN-S-3	33.1212	31.0855	31.798	33.5095	31.2845
	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
Noisy image PSNR	27.7252	31.0785	30.229	32.2943	28.3538
BM3D 0.5$\sigma$	30.4968	37.1901	35.2554	38.5173	32.8427
BM3D 1$\sigma$	28.1887	33.9383	32.9065	37.7703	35.4343
BM3D 1.5$\sigma$	26.5971	32.4034	30.9719	35.5195	33.7933
Frost	25.2128	30.1896	27.6313	29.9569	30.7026
Lee	26.8865	32.4212	29.8526	33.7633	33.3033
MLMVN-S-1	31.8259	36.6313	35.3114	37.4804	35.4004
MLMVN-S-2	31.5377	35.9756	34.7158	36.3381	35.0165
MLMVN-S-3	31.677	36.3271	35.0075	36.851	35.1973

and

Table 8. Results of filtering grayscale images affected by multiplicative speckle noise with a standard deviation of 0.4$\sigma$, where $\sigma$ is the standard deviation of the ideal image. The best results are shown in bold, and the best results using our approach are highlighted in color.

	Train station	Music hall	Big Ben	Cambridge bldg	Gaudi bldg
Noisy image PSNR	26.7005	26.1868	25.1993	29.3454	25.3672
BM3D 0.5$\sigma$	32.4979	29.8616	29.2817	31.9489	29.4405
BM3D 1$\sigma$	31.1296	27.7225	26.7524	29.4297	27.1516
BM3D 1.5$\sigma$	30.0805	26.1482	25.2561	27.8733	25.5813
Frost	28.6894	25.267	24.6709	24.3052	25.6198
Lee	29.8741	27.2948	25.962	27.2305	27.2076
MLMVN-S-1	30.886	29.402	29.3543	32.1551	29.4495
MLMVN-S-2	30.5167	29.1159	29.3764	31.7958	29.2278
MLMVN-S-3	30.6627	29.2276	29.3627	31.9539	29.3293
	Barcelona plaza	Sailboat	Manhattan	Airplane	Fighter-jet
Noisy image PSNR	25.2747	28.584	27.7327	29.8389	25.858
BM3D 0.5$\sigma$	28.6345	35.0658	33.3732	36.879	30.7953
BM3D 1$\sigma$	26.9191	32.719	31.3977	35.9913	33.9965
BM3D 1.5$\sigma$	25.6379	31.5217	29.8046	33.8131	32.5234
Frost	25.0076	29.9046	27.4366	29.7582	30.1304
Lee	26.5529	31.9175	29.4938	33.1938	32.1024
MLMVN-S-1	29.5072	34.3743	32.963	35.2263	33.2604
MLMVN-S-2	29.3592	33.6725	32.421	34.1796	32.8563
MLMVN-S-3	29.4303	34.0011	32.6647	34.6213	33.0585

) at various levels using MLMVN and popular filtering methods for comparison. For the BM3D filter, we considered multiple values of the noise standard deviation parameter because, in real-world tasks, this information is usually unknown. For both Lee and Frost filters, we used a filter window size of 3 × 3. The damping coefficient for the Frost filter was set to 2. We utilized MLMVN-G-1, MLMVN-G-2, and MLMVN-G-3, and MLMVN-S-1, MLMVN-S-2, and MLMVN-S-3 for filtering additive Gaussian noise and speckle noise, respectively. The evaluation of the MLMVN filtering capabilities was performed using images from the test dataset (Figure 5).

Figure 8 and Figure 9 present the results of filtering additive Gaussian noise and speckle noise, respectively, using the MLMVN and the BM3D filter. Images (a–d) illustrate the cases where the MLMVN achieved a higher PSNR than the BM3D filter. Images (e–h) show the cases where the PSNR values for both methods are approximately equal. Finally, images (i–l) depict the cases where the BM3D filter outperformed the MLMVN in terms of PSNR.

4. Discussion

Existing studies on the use of neural networks in image filtering have proven the effectiveness of this approach. Thus, MLPs, CNNs, AEs, and other types of neural networks have been utilized for image denoising and restoration. These results can be explained by the ability of neural networks to model complex nonlinear functional relationships between inputs and outputs.

In this paper, we considered the application of MLMVN as a framework for filtering additive Gaussian noise and multiplicative speckle noise in the frequency domain. The results obtained in our study demonstrate that MLMVN is effective for solving such tasks. MLMVN proved its efficiency in the development of frequency domain convolutional kernels generated from the learning process and processing overlapping fragments (patches) of images. This approach enables the use of a neural network with fewer neurons and parameters than, for example, in [35,44,45] and facilitates parallelism in programmatic implementation.

Our work also demonstrates the effectiveness of the batch learning algorithm in the training process. Noise filtering results obtained using MLMVN were presented and compared with other methods. The performance of MLMVN was found to be comparable to that of the BM3D, Lee, and Frost filters. Moreover, for some small detailed images our approach performed better because it better preserves small details. For additive Gaussian noise our approach works better for more heavily corrupted images. At the same time, our approach shows better results for speckle noise filtering than traditionally used filters. These findings indicate that MLMVN has potential for solving a wide range of image processing problems, especially those naturally suited to frequency domain analysis. We have also shown that multi-valued neurons may successfully employ the complex-valued logistic and the complex-valued hyperbolic tangent activation functions. This expands the MVNs’ functionality and makes it possible to utilize them for input/output mappings whose outputs are not necessarily located on the unit circle.

The results presented in this paper create a good background for further work on intelligent image processing in the frequency domain using MLMVN. For example, Poisson noise filtering and filtering of speckle noise with both multiplicative and additive components would be attractive subjects for further work. It would also be attractive and useful to discover possibilities of restoration of blurred images using a similar approach, that is, utilizing deconvolution in the frequency domain using MLMVN.