Post-Filtering of Noisy Images Compressed by HEIF

Abstract

Modern imaging systems produce a great volume of image data. In many practical situations, it is necessary to compress them for faster transferring or more efficient storage. Then, a compression has to be applied. If images are noisy, lossless compression is almost useless, and lossy compression is characterized by a specific noise filtering effect that depends on the image, noise, and coder properties. Here, we considered a modern HEIF coder applied to grayscale (component) images of different complexity corrupted by additive white Gaussian noise. It has recently been shown that an optimal operation point (OOP) might exist in this case. Note that the OOP is a value of quality factor where the compressed image quality (according to a used quality metric) is the closest to the corresponding noise-free image. The lossy compression of noisy images leads to both noise reduction and distortions introduced into the information component, thus, a compromise should be found between the compressed image quality and compression ratio attained. The OOP is one possible compromise, if it exists, for a given noisy image. However, it has also recently been demonstrated that the compressed image quality can be significantly improved if post-filtering is applied under the condition that the quality factor is slightly larger than the one corresponding to the OOP. Therefore, we considered the efficiency of post-filtering where a block-matching 3-dimensional (BM3D) filter was applied. It was shown that the positive effect of such post-filtering could reach a few dB in terms of the PSNR and PSNR-HVS-M metrics. The largest benefits took place for simple structure images and a high intensity of noise. It was also demonstrated that the filter parameters have to be adapted to the properties of residual noise that become more non-Gaussian if the compression ratio increases. Practical recommendations on the use of compression parameters and post-filtering are given.

1. Introduction and Related Work

An obvious tendency in modern computer vision (CV), remote sensing (RS), and medical diagnostics (MD) is the increased number of acquired images and their size [1,2,3,4]. For some applications, the acquired images should be processed and passed to potential customers or decision undertaking systems quite quickly. For other practical situations, the acquired data have to be effectively saved for their future use such as in medical and multitemporal imaging [3,5]. In these cases, efficient compression may be needed [5,6,7].

Existing compression techniques can be divided into two large groups: lossless [6] and lossy [7]. Both are widely used depending on the application, priority of requirements, and imposed restrictions. Lossless compression methods introduce no distortions into compressed images (data) but the produced compression ratio (CR) is usually not large (especially for images contaminated by noise) and cannot be varied (for given data, it can only be slightly improved by using a more efficient coder). If the attained CR is not appropriate, one has to apply a lossy compression technique [7,8,9]. The advantages of the compression techniques in this group consist of a considerably larger attained CR and a possibility of its variation, however, this is at the expense of distortions being introduced. A typical situation is that a larger produced CR leads to larger distortions that can become inappropriate or even annoying [1,8]. Then, one has to provide a reasonable trade-off between the attained CR and quality of the lossy compressed images, which happens in optical RS [9] and medical [10] imaging. This takes place in SSIM-based optimization [11], approaches to compression based on the just noticeable difference (JND) [12], and a better portable graphics (BPG) coder [13]. Note that such a trade-off should often be provided quickly and with the appropriate accuracy [13,14].

A positive feature in this sense is that the dependences of a metric characterizing lossy compressed image quality (e.g., mean square error (MSE) and CR) on a parameter that controls compression (PCC), such as the quality factor (QF) for compression techniques JPEG and HEIF [15], are usually monotonous (e.g., the MSE increases if the QF decreases for the mentioned coders). This allows for a potential reaching of the desired metric value with better or worse accuracy depending on the coder used, PCC employed, the metric exploited in image quality characterization, the procedure used for PCC variation, etc. [16]. In some practical situations, it is possible to ensure the high accuracy of providing a desired metric value if the PCC values are not discrete [16], but at the expense of several iterations when multiple compressions/decompressions of a considered image are needed. In other situations, the provided accuracy is lower (but can still be appropriate) with less computational expenses [15,17]. One reason can be the discrete nature of PCC [17]; another reason could be the residual errors of a two-stage procedure [13,15] or metric value prediction [12]. It is worth noting here that distortions can be characterized by conventional metrics such as the MSE or peak signal-to-noise ratio (PSNR), visual quality metrics (e.g., SSIM [11]), and their visibility threshold (see the just noticeable distortion (JND) concept in [12]). It is also possible to characterize introduced distortions by their impact on the classification or recognition of compressed images [11,18].

The described dependences and properties take place in lossy image compression if an image subject to compression is noise-free or, at the very least, the noise can hardly be noticed. However, for many classes of images, this assumption is not valid, for example, synthetic aperture radar (SAR) images that are always noisy due to speckle [19]. Conventional color images acquired in bad conditions can also be noisy [20]. A certain percentage of noisy (junk) component images are always present in hyperspectral [21] images. Noise also takes place in medical, night light, astronomic, and other images [22,23]. Some drones use cheap and low quality digital cameras, and as a result, the acquired images can also be noisy [24].

If a noisy image is subject to lossy compression, specific effects, discovered for SAR [25,26] and other images [27,28] about three decades ago, take place. The specific effect of noise reduction due to lossy compression was shown for both JPEG and wavelet-based techniques in [29,30] and [27,28], respectively. The concept of an optimal operation point (OOP) that can be observed was introduced in [28]. The OOP is the value of a used PCC when a minimal “distance” between the compressed and noise-free images is observed according to a considered quality metric. First, a possible OOP existence was demonstrated for conventional metrics such as PSNR; later, the possibility of OOP occurrence was shown [31] for many visual quality metrics including PSNR-HVS-M [32] and MS-SSIM [33]. For most metrics, the OOP relates to the global maximum, but there are a few metrics (MSE, MDSI [34]) for which the OOP corresponds to the global minimum.

As above-mentioned, the initial results concerning the OOP were obtained for JPEG and wavelet-based coders including JPEG2000 [35]. In state-of-the-art papers [36,37], a possible OOP existence was shown for coders based on discrete cosine transform (DCT) including a better portable graphics (BPG) coder [38,39]. Finally, the existence of an OOP has recently been demonstrated [17] for coders such as HEIF and AVIF [40], which are part of the video coder HEVC. In other words, it is possible to state that the existence of an OOP for some practical situations is typical for the lossy compression of noisy images including not only the case of pure additive noise, but also the case of signal-dependent noise [41]. Note that for many coders and noise types, fully automatic procedures for OOP determination have been proposed [37,41] based on the assumption that the noise type and characteristics are either known in advance or can be quite accurately estimated by the corresponding blind techniques for signal-dependent [42,43] and pure additive [44] noise types.

The OOP is not only a theoretical phenomenon, but also has considerable practical importance. An attractive advantage of the OOP is that, if an OOP exists for a given noisy image and a used coder, there is a certain sense of the image compressing in the OOP or its close neighborhood. Then, the benefit is twofold: good quality of the compressed image is provided, and the produced CR is quite large [17,36]. Meanwhile, two questions arise. First, there are images, coders, and noise intensities for which the OOP is absent (e.g., complex structure images corrupted by not very intensive noise). Then, there is how to set the coder PCC for such images to provide reasonably high CR and compressed image quality. Second, although the filtering effect in the OOP can be quite large, denoising due to lossy compression is not perfect. A good filter is able to provide an output PSNR that is a few dB larger than the PSNR in the OOP.

Keeping this in mind, an approach presuming a more “careful” (compared with the OOP) lossy compression combined with post-filtering applied to a decompressed image could be useful. Such an approach, to the best of our knowledge, was first proposed for DCT-based coders in [45,46]. Recently, a similar approach was studied for the BPG coder, where the DCT-based post-filtering was first analyzed [47], followed by the consideration of more complex non-local post-processing [48]. The post-processing of decompressed images is widely known [49,50], but the main goal for the corresponding methods is usually deblocking. The novelty of the approach in [47,48] consists of the fact that post-filtering after decompression was applied to remove residual noise for noisy images compressed by a modern BPG coder. In particular, the following has been shown:

(1)

A positive effect of post-filtering can be provided for images of different complexity under the condition that the CR is smaller than the CR for the OOP (parameter Q, which serves as the PCC for the BPG coder, is smaller than Q_OOP);

(2)

The optimal parameters of the DCT-based filter [51] change depending on Q and Q_OOP and noise intensity; general dependences can be found in [47];

(3)

The same has been observed [48] for the BM3D filter [52], where this filter, in general, performs slightly better than the DCT-based filter [47];

(4)

It is possible to predict the post-filter performance [48] and set its parameters in a way close to optimal; if Q increases, CR increases, but the general efficiency of post-filtering applied to decompressed images decreases; in this sense, Q ≈ Q_OOP-4 can be recommended for practical use.

Note here that the BPG and HEIF encoders have recently been proposed and can both be part of HEVC [7,53]. Moreover, the characteristics of BPG and HEIF in the case of the lossy compression of noisy images are very similar [17]. Meanwhile, the ways to find the OOP for the BPG and HEIF coders are different [17]. One way to find the OOP for HEIF was proposed in [17]. However, an opportunity to apply post-filtering for noisy images compressed by HEIF, to the best of our knowledge, has not been investigated yet. This was the main goal of this paper, and it deals with the novelty of the obtained results.

As a starting point, we considered the case of single-channel images contaminated by additive white Gaussian noise (AWGN) with a zero mean for the following reasons. AWGN is a standard noise model used in image processing applications such as the blind estimation of noise variance [44], filtering [54,55], and lossy compression [36,37]. If the noise observed in the original image is signal-dependent, it is possible to convert it to additive by using a proper variance stabilizing transform (VST) [17,56]. If such a VST is applied, lossy compression is employed after direct VST; the corresponding inverse transform should be used after decompression. Furthermore, although many modern images are multichannel nowadays [41,57], we focused on single-channel image lossy compression for two reasons. First, component-wise compression is still an easy way for volume reduction. Second, such a preliminary analysis of component-wise compression will, hopefully, bring an understanding of what should be conducted for multichannel data [41,57].

Summarizing the aforesaid, the novelty of this paper consists of the following:

(1)

To the best of our knowledge, the procedure of the post-filtering of noisy images lossy compressed by HEIF has never been considered, and the post-filtering of noisy images compressed by HEIF has several peculiarities given in the next items;

(2)

The filter choice is based on the presented statistical analysis of residual noise and introduced distortions where, to the best of our knowledge, such analysis has never been carried out;

(3)

The recommendations on setting the QF for the proposed compression procedure were based on the analysis results presented in Section 4.1 and Section 4.2. These recommendations differ from the recommendations for the BPG coder since the HEIF and BPG coders are controlled by different parameters (QF and Q) with different properties (CR increases if the QF becomes smaller for HEIF, and if Q increases for BPG, the QF varies in the limits from 1 to 100 whilst Q varies from 1 to 51, and so on);

(4)

Although the general tendencies of setting the post-filter parameter β appeared similar for the BPG and HEIF coders (for the larger CR optimal, β should be smaller), the obtained dependencies differed from the dependencies of different metrics on Q and β for the BPG coder presented in our earlier paper [47,48].

The rest of this paper is structured as follows. The image/noise model and the fundamental rate/distortion dependencies are considered in Section 2. A brief analysis of residual noise and distortions after the lossy compression of noisy images by HEIF are given in Section 3. The performance of post-filtering carried out by the BM3D filter is analyzed in Section 4, with a further discussion of several practical aspects including compressed image classification and object recognition. Finally, the conclusions are given in Section 5.

2. Image/Noise Model and Basic Rate/Distortion Dependences for HEIF

According to the assumed AWGN model, a noisy image can be represented as

$$I_{ij}^n=I_{ij}^{true}+n_{ij},$$

(1)

where $I_{ij}^{true}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ is the true (noise-free image), $n_{ij}$ denotes the AWGN in the $ij$-th pixel, and $I_{Im}$ and $J_{Im}$ define the image size. It is assumed that AWGN has a zero mean and its variance ${\sigma }^2$ is a priori known or has been accurately pre-estimated.

The performance characteristics of lossy compression strongly depend on the image content (complexity). Taking this into account, we mainly used three test images, namely the test image Frisco (Figure 1a), which has a simple structure (characterized by a large percentage of pixels that belong to homogeneous image regions), the typical image Fr03 (Figure 1b), and the image Diego (Figure 1c), which contains many textures and small details with a very small percentage of pixels belonging to homogeneous image regions. All three images are represented as 8-bit data arrays and have a size of 512 × 512 pixels.

Recall here that the HEIF coder, similarly to JPEG, is controlled by a QF varying from 1 (low quality of compressed images with large CR) to 100 (insignificant compression with very small losses). One specific feature of HEIF (here we used the version available at https://github.com/bigcat88/pillow_heif) is that the compression characteristics for pairs of neighbor values of QF (e.g., 3 and 4, 5 and 6) are practically identical [17]. The CR for HEIF (similarly to many other compression techniques) not only depends on the QS, but also on the image complexity and noise properties. To demonstrate this, consider the data in

Table 1. Compression ratio for the Frisco and Diego images for different QF and ${\sigma }^2$.

QF	Frisco				Diego
	${\mathit{\sigma }}^2=25$	${\mathit{\sigma }}^2$ = 100	${\mathit{\sigma }}^2=196$	QOOP	${\mathit{\sigma }}^2=25$	${\mathit{\sigma }}^2=100$	${\mathit{\sigma }}^2=196$	QOOP
9	158.29	175.34	175.11	OOPs were observed for QF equal to 24, 22, and 19 for σ = 5 σ = 10, and σ = 14, respectively.	77.972	78.982	80.289	OOP was observed for QF equal to 29 for only σ = 14.
19	80.362	82.176	78.392		30.552	29.427	27.187
29	43.137	17.629	8.963		11.260	9.068	7.001
39	11.050	5.219	4.248		5.072	4.199	3.759
49	4.619	3.225	2.839		3.426	3.014	2.793
59	2.970	2.362	2.162		2.339	2.133	2.032
69	2.174	1.816	1.684		1.851	1.729	1.645
79	1.733	1.494	1.400		1.530	1.437	1.375
89	1.459	1.259	1.180		1.281	1.208	1.161
99	1.415	1.187	1.112		1.199	1.133	1.090

.

As expected, the CR increased when the QF was reduced. As also expected, the CR for the simple structure image was larger than the CR for the complex structure image for the same QF and ${\sigma }^2$. Quite small CR values (smaller than 3) were observed for both images for QF ≥ 59, where it was possible to note near-lossy compression. Meanwhile, there were two interesting observations. First, the CR decreased if the ${\sigma }^2$ increased for a large QF (e.g., QF = 79); for a small QF, the CR for a larger ${\sigma }^2$ was larger or practically the same (see data for QF = 9). Second, there was an interval of QF values (19–39) where the CR increased rapidly when the QF decreased, and this interval contained the QF_OOP for the considered values of ${\sigma }^2$. Recall that the noise-filtering effect due to lossy compression is associated with the appearance of considerably more zero-valued DCT coefficients after quantization, and then the CR becomes significantly larger.

Figure 2 presents the dependences of the MSE calculated between the compressed and true images on the CR for two test images and two values of ${\sigma }^2$, 25 and 196, for five coders. As can be observed, OOPs were observed for all five coders (including the advanced DCT-based coder ADCT [58]) for both noise variances for the test image Frisco as well as for three coders (including HEIF) for the test image Diego, contaminated by AWGN with ${\sigma }^2=196.7$.

The plots presented in Figure 2 also reflect the main tendencies typical for the lossy compression of noisy images. First, the presence of the OOP is more probable for simpler structure images—compare the results in Figure 2a,b, where OOPs were absent for all coders for the test image Diego and present for all five coders for the test image Frisco. Second, the OOP presence is more likely for larger noise intensity—compare the results for the test image Diego for two values of the noise variance. Third, OOPs could be observed for approximately the same PCC values for different images contaminated by AWGN with the same variance—this follows from data presented in Table 1 (Notes column). Fourth, OOPs are “more obvious” for better coders–see data in Figure 2d where the MSE_tc in the OOPs (minimal values of MSE_tc) was smaller for modern coders than for JPEG (tc relates to metrics calculated between the compressed and true images). Fifth, if an OOP exists, then the CR in it will be quite large—from about 10 for complex structure images (Figure 2c) to 20–60 for simple structure images depending on the noise variance. One more positive feature of HEIF is that it performed better than AVIF (see data in Figure 2c,d) and approximately at the same level as the BPG coder. Also recall that HEIF was slightly faster than BPG.

The values of MSE_tc were non-zero for all of the QF, test images, and noise variance values for HEIF (as well as other coders) for the plots in Figure 2. This means that there was some residual noise in the compressed image as well as some distortions of information content. A question is whether these residual noise and distortions can be additionally diminished by decompressed image post-filtering. Available results for the BPG coder have shown that this is possible [47,48], so why then should it not be possible for HEIF?

Note that the choice of a proper filter and its parameters depends on the properties of the noise to be removed. Therefore, similarly to [59], it was worth carrying out a statistical analysis for the mixture of residual noise and distortions for HEIF.

3. Distortion Statistical Analysis

In lossy image compression based on orthogonal transforms, distortions are due to the operation of transform coefficient quantization. This holds for both wavelet- and DCT-based coders. It has been found by several researchers for DCT-based [59], BPG [60], and JPEG [61] coders that:

(1)

The MSE of the introduced distortions increases (in most cases) monotonically if the PCC is changed to increase the CR [59,60];

(2)

Introduced distortions are “non-stationary”, in the sense that they are more intensive for the so-called active image areas (textures, neighborhoods of sharp edges, and high-contrast small-sized objects) [59];

(3)

Due to this, the integral MSE is usually larger for complex structure images with a larger percentage of pixels that belong to locally active areas [60,61]; meanwhile, it is less likely to have a non-Gaussian (heavy-tailed) distribution of introduced errors for complex-structure images than for simple-structure ones;

(4)

There is a tendency to have heavier tails of introduced errors for a larger CR [59];

(5)

In the lossy compression of noisy images with possible post-filtering, it is worth considering not only the statistics of the introduced errors, but also the statistics of the residual noise and distortions [60];

(6)

From the post-filtering viewpoint, it is desirable to not only analyze the statistics of the residual noise and distortions, but also their spatial correlation, since the removal of spatially correlated noise is a more complex task than AWGN suppression, and the efficiency of spatially-correlated noise removal is usually less than AWGN suppression for the same noise variance [62].

The statistics and spatial correlation characteristics of distortions may also depend on the noise statistics and spatial correlation if one is dealing with spatially-correlated noise, but this question was out of the scope of our study. Furthermore, we did not consider the case of signal-dependent noise, assuming that either the noise was close to pure additive or that an image contaminated by signal-dependent noise had been converted before compression by a proper variance stabilizing transform to be contaminated by additive noise [41,56]. Thus, our goal here was to study the statistical characteristics of residual noise/distortions for images corrupted by AWGN and compressed by HEIF.

We considered the noise/distortion parameter arrays ${∆}_{ij}=I_{ij}^{true}-I_{ij}^c$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ for two test images for three values of QF and two values of the noise variance, namely, 25 and 100. QF values were chosen to (1) correspond to the OOP for the image Frisco, or to quality degradation for the image Diego (the smallest considered QF); (2) relate to the case when the MSE_tc was still almost the same as σ² (the largest considered QF); and (3) correspond to intermediate situations (intermediate values of QF).

To study the statistics of ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ for each case, we calculated the mean, variance, skewness, and kurtosis (note that joint analysis of these parameters allows for the rough analysis of data Gaussianity). In addition, D’Agostino’s statistical K-squared test was applied [63], which aims to gauge the compatibility of given data with the null hypothesis that the data are a realization of independent, identically distributed Gaussian random variables. This test exploits two parameters. The parameter Stat is determined as s² + k², where s is the z-score returned by the skewtest (test whether the skew is different from the normal distribution), and k is the z-score returned by the kurtosistest (test whether a dataset has normal kurtosis). The parameter P is a 2-sided Chi-squared probability for the hypothesis test. According to the recommendations in [63], we assumed that alpha = 0.05 was a statistically significant level.

The obtained data are presented in

Table 2. Statistics of the introduced distortions.

Image	σ	QF	CR	Mean	Variance	Kurtosis	Skewness	Stat	P	Gaussian or not
Frisco	5	34	31.58	−0.007	11.08	3.36	−0.0029	20524	0	No
Frisco	5	47	5.23	0.005	22.59	0.078	0.0028	62.7	2.38 × 10−14	No
Frisco	5	60	2.97	−0.005	25.14	0.010	0.0046	1.994	0.3689	Yes
Frisco	10	24	61.14	−0.070	24.58	4.95	0.0287	28482	0	No
Frisco	10	37	5.96	−0.011	85.40	0.129	0.0025	161.7	7.61 × 10−36	No
Frisco	10	50	3.22	−0.001	98.96	0.018	0.0045	4.54	0.1031	Yes
Diego	5	48	3.42	−0.043	33.69	0.032	0.0024	11.03	0.0040	No
Diego	5	56	2.61	−0.006	28.64	0.015	0.0063	4.07	0.1302	Yes
Diego	5	64	2.11	−0.010	26.56	0.008	0.0065	2.56	0.2785	Yes
Diego	10	40	4.20	−0.041	107.83	0.069	0.0054	50.74	9.55 × 10−12	No
Diego	10	58	2.25	−0.007	101.91	0.010	0.0041	1.873	0.3919	Yes
Diego	10	56	2.37	−0.012	102.47	0.007	0.0026	0.779	0.6772	Yes

. In addition to the aforementioned statistical parameters, we present the CR values useful for further analysis. The conclusions that can be drawn are as follows:

(1)

There were cases when ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ had a Gaussian distribution; this happened for the complex structure image Diego when the QF was large enough and the CR was quite small, and also took place for the test image Frisco when the CR was quite small;

(2)

On the contrary, ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ became non-Gaussian when the QF decreased and, respectively, the CR increased; the conclusions in items 1 and 2 were confirmed by both the analysis of kurtosis and D’Agostino’s K-squared test;

(3)

Mean and skewness of ${∆}_{ij}$, $i=1,.., I_{Im}, j=1,..,J_{Im}$ were both close to zero;

(4)

For the same σ², the variance of ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ was within wide limits; this ranged from 11 to 34 for σ² = 25 and from 24 to 108 for σ² = 100, where the minimal values corresponded to the OOP observed for the test image Frisco, and the maximal values related to the small QF for the test image Diego, for which the OOP was absent. This means that the variance can be either slightly larger than σ² or several times smaller than σ², which could be problematic in terms of setting the filter parameters correctly.

Let us also visualize ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ using the preliminary transformation ${{\delta }_{ij}=5∆}_{ij}+128.$ Our goal here is to check whether we have something like white noise or the structure of ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$, which is more complex. Figure 3 shows ${\delta }_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ for QF = 50, σ² = 100, for two test images. It was interesting to see that fluctuations in homogeneous regions of the test image Frisco were considerably less intensive compared with the fluctuations in locally active areas of this image as well as for practically the entire image of Diego. This was due to the noise-filtering effect, which appeared more in homogeneous image regions. Meanwhile, it is also possible to state that some “structures” were present in ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$ due to the presence of some structures (objects, textures, edges) in the information component of the images subject to lossy compression.

As a result, we cannot expect that positive features of known denoising techniques can be realized for the considered application to the full extent since they are designed to suppress ${∆}_{ij}$, $i=1,..,I_{Im}, j=1,..,J_{Im}$, which is obviously not exactly additive white Gaussian noise. However, it is possible to hope that some positive effects due to post-filtering can be achieved.

4. Post-Filtering Opportunities and Results

4.1. Post-Filtering Opportunities

Thus, we have to remove noise that has a probability density function (pdf) close to Gaussian (with possibly heavier tails), is almost white, and has an approximately known variance (MSE of residual noise and distortions that can be either slightly smaller or larger than the variance of the noise σ² in the original image). Although there has been a huge number of image denoising techniques designed thus far (see survey papers [64,65,66] and references therein), our intention here was not to find the best filter for our application. Instead, our aim was to approximately estimate the potential of post-filtering and identify the main tendencies. Because of this, we decided to consider the famous block matching 3-dimensional (BM3D) filter [52,67], which is based on finding similar patches and using Haar and discrete cosine transforms for 3D denoising of the formed sets of similar blocks.

This filter has the following positive features that are useful for the considered application. First, the BM3D filter has a preset parameter β used for the threshold T setting as βσ. By changing β, it is possible to slightly vary the filter properties—a larger β leads to better noise suppression at the expense of worse edge/detail/texture preservation. In our case, this property can be useful to understand how the optimal or quasi-optimal threshold can be adjusted depending on the original σ², and possibly some other information on the image at hand to be compressed. Second, a filter is quite insensitive if the noise pdf is not exactly Gaussian or not exactly white. This property is also important for the considered application. Third, the BM3D filter is one of the best filters in the sense of AWGN suppression, and its performance is comparable to that of neural network-based filters [68] (neural network-based filters mostly demonstrate better performance for unrealistically large values of σ).

4.2. Post-Filtering Performance Analysis

Filter performance is usually characterized by the output peak signal-to-noise ratio or its increase compared with the PSNR of the original noisy images [67]. Meanwhile, it has become popular to also analyze the visual quality metrics or their improvement due to filtering [69]. Below, we consider the metrics PSNR-HVS-M [32] and FSIM [70], which are among the best for characterizing denoising efficiency and the visual quality of images in general [68,69]. Recall here that we considered grayscale images, and hence not all visual quality metrics (mainly intended on characterizing the color image quality) could be applied. PSNR-HVS-M is expressed in dB and larger values correspond to a better visual quality. FSIM varies between 0 and 1, and larger values relate to better quality.

Post-filtering can be useful only under certain conditions (i.e., if it is efficient enough and if the provided CR is appropriately large). Because of this, we did not consider a QF > 51, for which the provided CR was small (see data in Table 1). Meanwhile, we also did not analyze a QF < 23, since it was expected that the filtering efficiency would be low [49]. In other words, we studied the behavior of post-filtering efficiency for the neighborhood of the optimal operation point (if it existed, see data in Table 2) and for a QF that produced a smaller CR than in the OOP. Namely, the interval of QF analysis was from 23 to 51.

Some preliminary results are presented in

Table 3. Post-filtering performance characteristics, σ² = 50.

BM3D
QF	Frisco			Fr03
	βopt	PSNR, dB	PSNR-HVS-M, dB	βopt	PSNR, dB	PSNR-HVS-M, dB
23	1.8	35.01	32.44	1.3	27.56	28.59
25	2	35.63	33.31	1.3	28.20	29.59
27	2.2	36.21	34.05	1.3	28.83	30.57
29	2.2	36.74	34.75	1.3	29.44	31.55
31	2.2	37.26	35.44	1.4	30.06	32.38
33	2.2	37.78	36.06	1.7	30.67	33.36
35	2.2	38.16	36.55	2	31.20	34.13
37	2.2	38.52	36.97	2.1	31.72	34.83
39	2.2	38.78	37.40	2.1	32.13	35.44
41	2.3	39.03	37.76	2.3	32.48	35.82
43	2.3	39.13	37.99	2.3	32.76	36.27
45	2.4	39.25	38.15	2.3	32.98	36.58
47	2.4	39.40	38.36	2.3	33.19	36.86
49	2.4	39.41	38.50	2.4	33.33	37.15
51	2.4	39.44	38.59	2.4	33.45	37.30

for σ² = 50. The optimal values for β_opt, which provide the maximal output PSNR, are given, and the produced PSNR-HVS-M values are also presented. Recall here that the input PSNR was equal to 31.12 dB and the output PSNR for the BM3D filter applied to an uncompressed noisy image was equal to 39.45 dB for the image Frisco and 33.72 dB for the image Fr03. As seen, filtering was more efficient for the simpler structure image Frisco. For the complex structure image Fr03, the PSNR improvement was only 2.6 dB.

Analysis of the data in Table 3 allowed us to draw the following conclusions:

(1)

If the QF decreases (i.e., the CR increases and the noise-filtering effect of lossy compression becomes larger), the PSNR and PSNR-HVS for post-filtering decrease and the optimal β is also reduced; this means that the positive effect of post-filtering becomes smaller if the QF increases, and in fact, post-filtering for a small QF becomes useless;

(2)

There are QF values for which post-filtering is expedient; for example, for QF = 41, the values of PSNR_tc for the compressed images were equal to 34.74 dB for Frisco and 30.38 dB for Fr03, respectively, while the PSNR values after post-filtering were equal to 39.03 dB for Frisco and 32.48 dB for Fr03, respectively;

(3)

Certainly, post-filtering is less efficient for complex structure images, and this property is in good agreement with the general denoising properties [51,55,69];

(4)

The PSNR-HVS-M metric demonstrated the same properties and confirmed the conclusions based on the PSNR analysis;

(5)

A reduction in β_opt takes place if the QF decreases; this means that, in fact, the MSE_tc might be reduced compared with σ², and one needs to set a smaller threshold to make the filter perform in a quasi-optimal manner.

It is possible to represent the obtained data as surfaces depending on QF and β. Two examples are presented in Figure 4.

As can be seen, the surfaces are smooth, and the maxima in the neighborhood of the optimal β are not “obvious” (i.e., for β, which slightly differed from β_opt, the performance was practically the same as for β_opt). This means that errors in setting β do not considerably influence the post-filtering efficiency. Meanwhile, the reduction in the metric output values with QF decreasing was not rapid for QF = 40, but the reduction speed increased for a smaller QF.

Let us now see what happens for other values of noise variance. The data obtained for σ² = 100 are given in

Table 4. Post-filtering performance characteristics, σ² = 100.

BM3D
QF	Frisco			Fr03
	βopt	PSNR, dB	PSNR-HVS-M, dB	βopt	PSNR, dB	PSNR-HVS-M, dB
23	2	34.66	31.86	1.7	27.45	28.12
25	2	35.24	32.60	1.7	28.01	29.04
27	2	35.73	33.23	1.7	28.61	29.88
29	2.4	36.18	33.80	1.7	29.16	30.64
31	2.4	36.53	34.20	2	29.64	31.30
33	2.4	36.85	34.61	2.1	30.05	31.93
35	2.4	37.11	35.00	2.3	30.42	32.40
37	2.4	37.26	35.27	2.3	30.70	32.81
39	2.4	37.36	35.48	2.3	30.92	33.13
41	2.4	37.45	35.64	2.3	31.10	33.38
43	2.4	37.63	35.78	2.3	31.25	33.59
45	2.4	37.64	35.94	2.3	31.37	33.79
47	2.4	37.75	36.07	2.3	31.45	33.96
49	2.4	37.78	36.12	2.3	31.54	34.12
51	2.4	37.71	36.18	2.3	31.60	34.21

.

Recall that the input PSNR in this case was equal to 28.12 dB and the output PSNR for the BM3D filter applied to the uncompressed noisy image was 37.66 dB for the image Frisco and 31.71 dB for the image Fr03. Thus, filtering was again more efficient for the simpler structure image Frisco. For the image Fr03, the PSNR improvement was equal to 3.6 dB.

An analysis of the data in Table 4 allowed us to draw conclusions similar to the previous case of σ² = 50:

(1)

If the QF is reduced (which corresponds to an increase in CR), the PSNR and PSNR-HVS for the post-filtered images decrease, and the optimal β also decreases; in other words, the positive effect of post-filtering becomes smaller, and starting from a certain QS, post-filtering becomes useless;

(2)

At the same time, there exist QF values for which the use of post-filtering is reasonable; for example, for QF = 39, the values of PSNR_tc for the compressed images were equal to 28.66 dB for Frisco and 28.11 dB for Fr03, respectively. In turn, the PSNR values after post-filtering were equal to 37.36 dB for Frisco and 30.92 dB for Fr03, respectively;

(3)

Post-filtering is less efficient for complex structure images; for any QF, the output PSNRs were significantly smaller for test image Fr03 than for test image Frisco;

(4)

The PSNR-HVS-M metric showed the same dependencies;

(5)

A reduction in β_opt is observed if the QF decreases; thus, one needs to set a smaller threshold to provide the filter operation in a quasi-optimal manner.

The functions of the metrics on QF and β are represented in Figure 5.

As seen, the surfaces are again quite smooth; however, the maxima in the neighborhood of optimal β have become more “obvious” compared with the case of σ² = 50 (Figure 4); nevertheless, for β, while it slightly differed from β_opt, its performance was almost the same as for β_opt. Thus, the post-filtering efficiency remained high for β within the limits of 2.0 to 2.5. Again, the reduction in metric output values with a decrease in QF was not rapid for QF = 36, while the reduction speed increased for a smaller QF.

Consider now the case of σ² = 200. The obtained data are shown in

Table 5. Post-filtering performance characteristics, σ² = 200.

BM3D
QF	Frisco			Fr03
	βopt	PSNR, dB	PSNR-HVS-M, dB	βopt	PSNR, dB	PSNR-HVS-M, dB
23	2.3	34.15	30.99	1.8	27.18	27.40
25	2.3	34.54	31.48	1.9	27.69	28.12
27	2.3	34.85	31.86	1.9	28.15	28.72
29	2.3	35.09	32.17	2.1	28.54	29.23
31	2.3	35.42	32.64	2.2	28.81	29.66
33	2.3	35.60	32.96	2.2	29.05	29.95
35	2.3	35.63	33.07	2.2	29.28	30.34
37	2.3	35.78	33.32	2.2	29.41	30.53
39	2.4	35.93	33.45	2.3	29.56	30.76
41	2.4	35.93	33.49	2.3	29.66	30.91
43	2.4	35.98	33.62	2.3	29.69	31.02
45	2.4	36.02	33.75	2.3	29.77	31.10
47	2.5	36.10	33.77	2.3	29.81	31.18
49	2.5	36.15	33.82	2.3	29.85	31.26
51	2.5	36.06	33.87	2.3	29.88	31.31

.

The analysis of the data in Table 5 resulted in the following conclusions:

(1)

If the QF is reduced (i.e., the CR increases), both the PSNR and PSNR-HVS for the post-filtered images become worse, and the optimal β also reduces; this shows that the positive effect of post-filtering reduces, and starting from a QS around 23, post-filtering becomes useless;

(2)

There are QF values for which the post-filtering is beneficial; for example, for QF = 37, the values of PSNR_tc for the compressed images were equal to 25.65 dB for Frisco and 25.39 dB for Fr03, respectively. In turn, the PSNR values after post-filtering were equal to 35.78 dB for Frisco and 29.41 dB for Fr03, respectively;

(3)

Again, the post-filtering was less efficient for the complex structure image Fr03 than for the test image Frisco;

(4)

The PSNR-HVS-M metric had similar dependencies;

(5)

A reduction in β_opt is observed if the QF decreases; this means that one has to set a smaller threshold to provide the filter operating in a quasi-optimal manner.

(6)

The functions of the metrics on QF and β are represented in Figure 6.

Their analysis showed that the surfaces had maxima in the neighborhood of optimal β; small deviations of β from β_opt did not essentially reduce the performance. The post-filtering efficiency was high for a β of about 2.2. The reduction in the metric output values with QF decreasing was not rapid for QF ≥ 34 but the reduction speed increased for smaller QF.

The expedience of decompressed image post-filtering can be also studied in another way. Figure 7 presents the dependencies PSNR_{p pf} = PSNR_pf − PSNR_tc (Q = 1) and ∆PSNR_{p tc} = PSNR_tc − PSNR_tc (Q = 1) on the CR obtained by varying the QF and CR, respectively, for two test images and two noise variance values. Here, the PSNR_pf was the PSNR calculated between the decompressed image post-filtered with the optimal β and the true image, and PSNR_tc, as we recall, is the PSNR calculated between the compressed and true images.

Analysis of ∆PSNR_{p tc} (Figure 7a–c) showed that in all three cases, OOPs existed (there was a maxima of ∆PSNR_tc (CR)). Meanwhile, the maximal ∆PSNR_{p tc} for the test image Fr03 (Figure 7a) was very small (about 0.5 dB). The maximal values of ∆PSNR_{p tc} were larger for σ² = 200 (σ² is denoted as Var in the Figure title, see Figure 7c), and exceeded 6 dB for the test image Frisco (Figure 7b).

As seen, the PSNR_{p pf}, in the considered limits of CR, was always larger than PSNR_{p tc}. However, the difference between them decreased if the CR increased. In the neighborhood of the OOP (CR ≈ 10 in Figure 7a, CR ≈ 60 in Figure 7b, CR ≈ 14 in Figure 7c) and for a larger CR, the benefit due to post-filtering became negligible (about 0.5 dB and less) (i.e., post-filtering became practically useless).

Consider now the dependences PSNR-HVS-M_{p pf} = PSNR-HVS-M_pf − PSNR-HVS-M_tc (Q = 1) and PSNR-HVS-M_{p tc} = PSNR-HVS-M_tc − PSNR-HVS-M_tc (Q = 1) on the CR presented in Figure 7d–f. As seen, there were two cases when all PSNR-HVS-M_{p tc} were negative (Figure 7d,f) (i.e., there was no OOP for the test image Fr03). Meanwhile, there was an OOP according to the metric PSNR-HVS-M for the test image Frisco (Figure 7e), although the PSNR-HVS-M_{p tc} in this OOP was not large (slightly exceeded 1 dB).

As in the previous case of analysis, the PSNR-HVS-M_{p pf} was always larger than PSNR-HVS-M_{p tc} across the entire range of analyzed CR. However, the difference between PSNR-HVS-M_{p pf} and PSNR-HVS-M_{p tc} quickly reduced if the CR was increased (i.e., post-filtering became useless). In our opinion, it became practically useless for CR > 10 in Figure 7d, CR > 59 in Figure 7e, CR > 14 in Figure 7f) when the difference between the PSNR-HVS-M_{p pf} and PSNR-HVS-M_{p tc} was less than 1 dB (recall that a difference of less than 1 dB in the PSNR and PSNR-HVS-M of two images could hardly be noticed).

Nevertheless, in our opinion, the use of post-filtering could be reasonable for a smaller CR (e.g., for CR ≈ 8 (Figure 7a,d), CR ≈ 30 (Figure 7b,e), and CR ≈ 11 (Figure 7c,f), i.e., for a CR about 1.5 smaller than that one that corresponds to the OOP (if it exists)).

Based on the results presented above in this subsection and in Table 1 (as well as the results obtained for other test images and several noise variance values), it is possible to recommend setting the QF at 8–10 larger than the QF for the OOP. Keeping in mind the recommendations in [17], we offer the following recommendations on setting the QF, depending on the noise variance (supposed to be known in advance or accurately pre-estimated) for post-filtering to be reasonably applied. The recommended QF values are given in

Table 6. The image compression parameters depending on the AWGN variance.

σ2	20–30	31–44	45–64	65–90	91–130	131–180	185
QF for HEIF	43	41	39	37	36	34	32
PSNRpf	32–42	31–41	30–40	29–39	28–38	27–37	26–36
PSNR-HVS-Mpf	36–39	34.5–37	33–36	31–34	29.5–32	28–30.5	27–29.5
CRpf	3–6	3.5–7	4–8	4.5–9	4.5–10	4.5–9.5	4.5–9
QFOOP	≈33	≈31	≈29	≈27	≈26	≈25	≈24
PSNROOP	29–38.5	28–37.5	27–36.5	26.5–36	26–35	25.5–34	25–33
PSNR-HVS-MOOP	33.5–37	32–35.5	30.5–34	29–33	27.5–32	26.5–30.5	26–30
CROOP	5.5–24	6–27	7–30	8–31	8–27	7–23	6–17

. As seen, the recommended QF decreases if the AWGN variance increases. In addition, we provide the ranges of PSNR_pf, PSNR-HVS-M_pf, and CR_pf. The parameter QF_OOP as well as the ranges of PSNR_OOP, PSNR-HVS-M_OOP, and CR_OOP are also presented.

As seen, the values of PSNR_pf were larger than the corresponding PSNR_OOP by about 1–3 dB, and the PSNR-HVS-M_pf values were sufficiently larger than the corresponding PSNR-HVS-M_OOP for a noise variance smaller than 130. Meanwhile, the CR for image compression with post-filtering was 1.5–4 times smaller than for QF_OOP, which is the price for better quality.

To ensure that the presented tendencies and recommendations were valid, we analyzed six additional images. Three of them were the optical test images Baboon (that has complex structure), Goldhill (middle-complexity structure), and Peppers (low complexity image). These images can be found at https://ponomarenko.info/agu.htm (accessed on 1 March 2025). Three other images were medical (Figure 8). These images have quite a simple structure since they contain rather large homogeneous image regions.

The results obtained for the six test images above-mentioned are shown in

Table 7. Results for the additional test images.

Image	Noise Variance	Original PSNR, dB	PSNRf (BM3D with β = 2.6 Without Compression, dB	PSNROOP, dB	PSNRpf, dB	Optimal β for Post-Filtering
Pepper	25	34.1	37.67	34.39	36.24	2.2
	100	28.1	35.0	32.12	34.07	2.3
	196	25.2	33.93	30.99	33.07	2.5
Baboon	25	34.1	35.25	32.82	33.95	2.2
	100	28.1	30.57	28.08	28.99	2.3
	196	25.2	28.58	25.3	27.29	2.1
Goldhill	25	34.1	37.18	34.04	35.51	2.3
	100	28.1	33.67	30.48	32.67	2.2
	196	25.2	32.26	29.2	31.4	2.2
Med5	25	34.1	37.86	35.14	37.03	2.1
	100	28.1	35.54	33.34	35.0	2.4
	196	25.2	34.61	32.34	34.03	2.9
Med7	25	34.1	41.59	38.71	40.33	2.1
	100	28.1	37.0	34.43	36.03	2.5
	196	25.2	34.67	32.37	33.82	2.5
Med8	25	34.1	42.78	40.27	42.1	2.4
	100	28.1	40.16	37.67	39.56	2.8
	196	25.2	38.77	36.28	38.18	3.0

. We present the PSNR for the original noisy image, PSNR_f for the filter output applied to the non-compressed image, PSNR_OOP for the OOP (according to data in Table 1), PSNR_pf for the post-filtered images if the Q is set according to the recommendation in Table 6 and β is optimal, and the optimal β for each particular case.

The first conclusion that can be drawn from the comparison of data in Table 6 and Table 7 is that the PSNR_OOP and PSNR_pf given in Table 7 were within the corresponding limits presented in Table 6. It can be clearly seen that the PSNR_OOP was always by about 3 dB smaller than the PSNR_f (i.e., the noise filtering due to lossy compression was considerably less efficient than the traditional filtering applied to the uncompressed image). A comparison of the PSNR_OOP and PSNR_pf for the same image and the same noise variance in Table 7 demonstrated that PSNR_pf was 0.9–2.1 dB larger than the PSNR_OOP, where the largest benefit for the proposed approach was observed for simple structure images contaminated by intensive noise. The second conclusion is that the optimal β was in the range of 2.1 to 3.0, where the recommendation to set β ≈ 2.3 can be given for practice. Finally, the obtained data showed that: (1) the proposed approach is quite general and applicable to images of different origin; and (2) the results depend more on the image complexity than on the image origin domain.

Let us present some compression results. Figure 9a shows the noisy image Frisco, σ² = 100. The image compressed by HEIF with QF = 25 (in the OOP) is presented in Figure 9b, while the same image compressed by HEIF with QF = 36 and post-filtered is shown in Figure 9c. As seen, noise in the homogeneous image regions was better suppressed while the edges and details were sharper (compare images in Figure 9c,b). The trade-off for this improvement was a lower level of CR. Figure 9d presents the noisy image Diego, σ² = 100. Noise was only visible in small homogeneous regions. The compressed image (QF = 25) is shown in Figure 9e, and the image compressed by HEIF with QF = 36 and post-filtered is given in Figure 9f. Noise in the homogeneous parts was suppressed for both output images, but the edges and details were less smeared for the post-filtered image (Figure 9e).

5. Discussion

In the sections above, we considered the simplest possible model of AWGN that contaminates a single component (grayscale) image. A question that arose is what are the differences that can be met in practice? Note that one can deal with an image corrupted by signal-dependent noise. Then, two ways are possible. One can follow the quite standard approach described in our paper [17] (i.e., apply a proper VST to convert the original image corrupted by signal-dependent noise to the image contaminated by pure additive noise). After this, the image should be compressed with QF, as recommended in Section 4.2, and stored or passed via the communication line. At the decompression and post-processing stage, just standard decompression by HEIF is carried out; after this, post-filtering is applied, and then inverse VST is performed. Within this procedure, the following is supposed: (1) the parameters of signal-dependent noise are a priori known or pre-estimated with high accuracy; (2) a proper variance-stabilizing transform exists and allows for the variance of pure additive noise to be determined; and (3) a priori information for carrying out inverse VST is available at the decompression stage (e.g., this information is passed as auxiliary data together with the compressed image in header). Another method is to study the application of HEIF to images corrupted by signal-dependent noise without VST, with proper recommendations. However, this method presumes additional investigations that need to be carried out.

Second, what does one do if the image to be compressed is multichannel? Currently, we are not ready to answer this question precisely. However, several analogies and preliminary solutions can be found. Note that a variety of possible situations is possible. It can be that a multichannel image contains one or two component images that are corrupted by the noise while other components are practically noise-free. Then, a noisy component image can be compressed separately, as described above, while other component images can be compressed (without post-filtering) separately or jointly, according to the approaches typical for the compression of noise-free multichannel images [7]. However, to the best of our knowledge, there have been no attempts to apply HEIF for the compression of multispectral or hyperspectral data.

If all or a large group of component images are noisy, then the approach described in [42] seems reasonable, as it presumes the use of proper VST for all component images and their joint compression (note that joint compression usually produces an approximately twice larger CR than component-wise compression for the same quality of compressed data). However, such an approach using HEIF as the basis needs additional studies.

An important item is to understand (predict) whether an OOP exists or not for HEIF. Currently, we do not know of such studies. However, since it has been shown for the BPG coder that such a prediction is possible [38], and the BPG and HEIF coders have similar performance characteristics [17], then it is reasonable to expect that the prediction of the existence of an OOP for HEIF is also possible.

In other words, there are many options in which to study the applicability of HEIF for the compression of remote sensing images, both single- and multichannel.

Finally, a future direction could focus on whether or not the compression of noisy images with post-filtering can influence the image classification [11], feature selection [71], image segmentation and edge detection [72], and object recognition [18]. For example, the National Research Foundation of Ukraine (https://nrfu.org.ua/en/, accessed on 1 March 2025) is interested in such studies within Project No. 2023. 04/0039 “Geospatial monitoring system for the war impact on the agriculture of Ukraine based on satellite data” (2024–2025).

In this sense, our recent studies [18] have shown the following. For quite a large range of CR variation, lossy compression and noise do not lead to a significant reduction in object recognition accuracy if modern neural networks (especially those in the YOLO (You Only Look Once) family [73]) are applied. Our preliminary results show that the use of post-filtering for the classification of compressed noisy images with visible noise allows for an increase in the total accuracy of correct classification by about 2% compared with the classification of noisy images compressed in the OOP. However, a neural network that carries out classification needs to be trained for post-filtered compressed noisy images.

6. Conclusions and Future Work

Our paper concerns the lossy compression of single-channel noisy images with the use of post-filtering after decompression to improve the image quality for the HEIF coder. This approach was compared with image compression in the optimal operation point if it existed, or for similar values of quality factor. It was shown that a positive effect of post-filtering exists for images of different complexity as well as different intensities of the noise. However, this positive effect varies: it can be large if a compressed image has a simple structure and the noise intensity is quite large and if the CR for a given QF is considerably smaller than the QF for the OOP. An opposite situation takes place if a given QF is quite close to the QF in the OOP. Because of this, we recommend reaching the compromise solutions presented in Table 6 depending on the noise variance for an 8-bit representation of data. This allows compression to be carried out with further post-filtering after decompression in fully automatic mode (this is important for practice). The output PSNR and PSNR-HVS-M were in this case considerably better than the PSNR and PSNR-HVS-M in the OOP, but the attained CR was 1.5–2 times smaller than for the QF that corresponded to the OOP.

There are many unsolved problems for HEIF. For example, it is desirable to carry out studies for cases of signal-dependent noise, multichannel images, and the classification of RS, CV, and medical data. The problem is that the variety of noise properties in multichannel images of different origin is so immense that we do not expect general solutions. At the same time, prominent solutions for particular situations can be achieved by the 3D compression of all components jointly or components properly combined in groups [41].