BPG-Based Automatic Lossy Compression of Noisy Images with the Prediction of an Optimal Operation Existence and Its Parameters

Abstract

With a resolution improvement, the size of modern remote sensing images increases. This makes it desirable to compress them, mostly by using lossy compression techniques. Often the images to be compressed (or some component images of multichannel remote sensing data) are noisy. The lossy compression of such images has several peculiarities dealing with specific noise filtering effects and evaluation of the compression technique’s performance. In particular, an optimal operation point (OOP) may exist where quality of a compressed image is closer to the corresponding noise-free (true) image than the uncompressed (original, noisy) image quality, according to certain criterion (metrics). In such a case, it is reasonable to automatically compress an image under interest in the OOP neighborhood, but without having the true image at disposal in practice, it is impossible to accurately determine if the OOP does exist. Here we show that, by a simple and fast preliminary analysis and pre-training, it is possible to predict the OOPs existence and the metric values in it with appropriate accuracy. The study is carried out for a better portable graphics (BPG) coder for additive white Gaussian noise, focusing mainly on one-component (grayscale) images. The results allow for concluding that prediction is possible for an improvement (reduction) in the quality metrics of PSNR and PSNR-HVS-M. In turn, this allows for decision-making about the existence or absence of an OOP. If an OOP is absent, a more “careful” compression is recommended. Having such rules, it then becomes possible to carry out the compression automatically. Additionally, possible modifications for the cases of signal-dependent noise and the joint compression of three-component images are considered and the possible existence of an OOP for these cases is demonstrated.

1. Introduction

Remote sensing (RS) systems and other imaging tools currently provide valuable data for agriculture, forestry, hydrology, ecological monitoring, non-destructive testing in industry, etc. [1,2,3,4]. Using imaging data produced by modern systems, it is possible to estimate the parameters of sensed territories of large areas, to control their change in time, to detect objects of interest and to solve many other important tasks. Meanwhile, a better spatial resolution and more frequent observation result in a fast increase in the data volume where the images have to be transferred, processed, stored and interpreted [5,6].

At the stages of data transferring and storage, image compression is often applied [7,8,9]. A lossless compression [7,8,10] preserves all the information contained in RS and imaging data, but the compression ratio (CR) attained for the used methods can be inappropriate in practice. To obtain a larger CR, near-lossless and lossy compression is mostly applied [7,9,10,11,12]; however, distortions are then inevitably introduced and the important task of providing an appropriate trade-off between the introduced distortions and the reached CR arises [13,14]. The priority of requirements depends upon the application at hand and it can be necessary or desired to provide a given CR (or a CR not less than a given threshold). Meanwhile, it can also be necessary (or desired) to provide a desired quality (or a quality not worse than desired), but there can also be other requirements or restrictions that might concern the necessity for relying on standards or to reach a trade-off quickly, to save resources and to carry out a compression fully automatically, etc. In this paper, we focus on providing high quality with the desire to have quite a high CR and to perform the compression quite quickly and automatically. If the quality degradation due to a lossy compression is limited, it is possible to expect that the RS data classification or object (e.g., crack) detection are performed well enough, and that the RS data being visualized are of a proper quality for analysis and so on [15,16,17].

In many cases, it is supposed that the images to be compressed are noise-free or, at least, that the noise is invisible. This is often valid in practice but, meanwhile, there are quite a number of practical situations when noise is visible and its presence cannot be neglected; for example, noise (speckle) is always seen in radar images [18,19], some component images of hyperspectral and multispectral RS data are noisy [20], the signal-to-noise ratio can be low in night-light images [21] or in images acquired in bad illumination conditions and so on. The influence of noise first attracted the attention of researchers more than 20 years ago [22,23]. It has been demonstrated that the lossy compression of noisy images has two main peculiarities. First, a specific noise filtering effect is observed. Second, due to this, a so-called optimal operation point (OOP) can be observed, i.e., such a parameter of a coder that the “distance” between the compressed and true (noise-free) images is minimal. By distance, we mean some similarity measure, and this can be a mean square error (MSE, then the OOP would correspond to its minimum) or the peak signal-to-noise ratio (PSNR, then the OOP would correspond to its maximum) [22,23,24]. The OOP can also be observed for visual quality metrics [24,25] such as PSNR-HVS-M [26] and MS-SSIM [27]. Note that the use of visual quality metrics has become popular in many modern applications including in stereoscopic, panoramic and 360 degree imaging [28,29,30].

Compression in the OOP (or its neighborhood if the OOP is determined with some error), if it exists, has two advantages. First, the provided CR is usually quite high. Second, the quality of the compressed image appears to be better than the quality of the uncompressed (original, noisy, or compressed in a lossless manner) image. This can be, e.g., favorable for image classification [31]. Additionally, note that the OOP might exist not only for additive noise but also for other types of noise including signal-dependent and multiplicative for methods that employ a variance stabilizing transform (VST) or that perform compression without VST. An OOP might also exist for different coders, based both on a discrete cosine transform (DCT) or wavelets (for example, JPEG2000) [32]. Recently, the possible existence of an OOP [33,34] has been demonstrated for the coder, BPG (better portable graphics} [35,36,37]. This compression technique has several advantages. It outperforms JPEG considerably in the sense of a better quality for a given size of compressed images and the compression is fast enough with clear rules of providing a desired PSNR. Due to this, the encoder has become popular in different portable devices as well as for online encoding applications and these advantages were the reasons why BPG has attracted our attention [36,37]. The current paper incorporates the results of [36] and concentrates on the prediction of the OOP’s existence and estimating the performance parameters for it. This is explained by the following. Since the true image is absent, the OOP cannot be determined exactly and its existence or absence can only be predicted (at least, this has been shown for coders other than BPG coders) [24,29]. Assuming that it is also possible for BPG, it can then be supposed that it will be possible to give recommendations for how to set the compression parameters (namely, the quality parameter Q for BPG) to provide an appropriate quality. This is the main goal of this paper.

The paper contributions consist in the following. First, based on the results of many test images corrupted by AWGN having different values of noise variance, a Q for a possible OOP is established depending on the noise standard deviation; this dependence is shown to be logarithmic opposite to the linear dependences established in [24] for other coders. Second, a modification of the procedure [24] is proposed and its thorough analysis is carried out. In particular, it is proposed to use rational functions in curve fitting into scatter-plots. This analysis shows that there are, at least, two statistical parameters that can be quickly and easily calculated in the DCT domain that can be used for predicting an improvement or reduction in two metrics. Third, the accuracy of prediction is analyzed and factors such as noise realization and the number of blocks that influence the accuracy are considered. Based on this analysis, practical recommendations are given that allows compression automation. Fourth, initial data concerning the lossy compression of single-channel images corrupted by Poisson noise and three-channel images corrupted by AWGN are presented, demonstrating the possible existence of OOPs for these cases. Finally, we show that the BPG coder is more efficient compared to other coders earlier considered in [24].

The paper is structured as follows. Section 2 describes the image/noise model, considered metrics and basic dependences. Section 3 analyzes the dependences more in detail. The methodology of prediction and its accuracy analysis are given in Section 4. Decision-making and practical recommendations are discussed in Section 5. The cases of signal-dependent noise and three-channel image compression are also briefly studied in this Section. Finally, the conclusions are given.

2. Image/Noise Model and Compression Efficiency Criteria

It is a well-known fact that compression characteristics depend on sufficient image properties. Depending on the image complexity, the CR can vary by several (or even tens) of times for the same PSNR or, equivalently, the quality of compressed images can be rather different for the same CR. Because of this, to ensure universality of the conclusions and recommendations, the corresponding analysis and method synthesis should be performed for a set of images with a very wide variation of properties. A set of test images needs to contain simple, medium, and complex structure images where the complexity can be described or characterized by a percentage of pixels that belong to an image’s homogeneous regions or entropy for a noise-free case and so on. For a better understanding, we give six images as the examples of images of different complexity (Figure 1). More than a half of the pixels in Figure 1a belong to quasi-homogeneous regions, a certain part of the pixels in Figure 1c–f relate to homogeneous regions, whilst there are practically no homogeneous regions in the image in Figure 1b.

There are different ways to characterize image complexity. We have noticed that the performance of the lossy compression of noisy images correlates with the image entropy for noise-free cases. Entropy E is the following: 5.82 for Frisco, 7.33 for Diego, 7.46 for Fr01, 7.40 for Fr02, 7.38 for Fr03, and 7.29 for Fr04. It will be shown later that the plots for the test images having similar E values for the noise-free versions usually have a similar behaviour. Note that the marginal cases are of the most interest; however, “average” cases (of middle complexity) are also worth considering. Because of this, there is a tendency in the image processing community for the creation of image databases and for their use in the design and verification of image processing methods [38,39]. As a starting point for analyzing the lossy compression of noisy images using BPG, we concentrate on the case of grayscale images.

As a noise model, we consider the AWGN, which is known to be the simplest model. Just such a model is commonly used as a starting point in research, for example, in the prediction of the potential efficiency of image denoising [40]; thus, one can present an observed noisy grayscale image as:

$$I_{ij}^n=I_{ij}^{true}+n_{ij},$$

(1)

where $I_{ij}^{true}$, $i=1,...,I_{Im}, j=1,...,J_{Im}$ is the true or noise-free image, $n_{ij}$ denotes AWGN in the $ij$-th pixel, and $I_{Im}$ and $J_{Im}$ define the considered image size. It is possible to assume the noise mean equals to zero and the AWGN variance is equal to ${\sigma }^2$. Moreover, we assume that ${\sigma }^2$ is either a priori known or pre-estimated with a high accuracy [41].

Quality of the original noisy image can be characterized in different ways. One standard way is to calculate the peak signal-to-noise ratio as:

$$PSN{R}^n=10lo{g}_{10}\left(\frac{{255}^2}{{\sigma }^2}\right),$$

(2)

under the assumption that an image is represented as an 8-bit two-dimensional (2D) data array. Another way is to employ some visual quality metrics, here, we use two of them. The first one is PSNR-HVS-M [26] (a peak signal-to-noise ratio taking into account the human vision system (HVS) and masking (M)), and the multi-scale structural similarity metric (MS-SSIM) [27]. Both are among the best in characterizing the visual quality of images with distortions typical for remote sensing [42], in particular those due to noise and lossy compression. They can be calculated quickly, are applicable to a single-channel (grayscale images), and are based on different principles. The latter is important since no elementary full-reference metric is perfect and, thus, while carrying out an analysis and making conclusions, it is desired to rely on the results obtained for several visual quality metrics.

PSNR-HVS-M is defined as:

$$PSNR-HVS-M^n=10lo{g}_{10}\left(\frac{{255}^2}{MSE-HVS-M^n}\right),$$

(3)

where $PSNR-HVS-M^n$ is determined in 8 × 8 blocks in the DCT domain, taking into account the masking effect and lower sensitivity of a human eye to distortions in high spatial frequencies rather than distortions in low spatial frequencies. Note that PSNR and PSNR-HVS-M [26] are both expressed in dB with larger values relating to better quality. Usually, for AWGN and similar distortions, PSNR-HVS-M is slightly larger than PSNR due to masking effect.

For these two metrics, it is important that the distortion visibility thresholds have been determined [38]. Distortions are usually invisible if PSNR exceeds 36 dB and PSNR-HVS-M is larger than 41 dB. This happens if the noise variance is larger than 15…20. Because of this, in this paper we concentrate on considering $PSN{R}^n$ smaller than 36 dB which is typical for the aforementioned applications.

The metric MS-SSIM [27] extracts and employs structural information from the scene. It is in the limits from zero to unity where larger values correspond to a better visual quality. The distortion invisibility threshold is approximately equal to 0.99.

If a lossy compression method is applied, one obtains a compressed image $I_{ij}^c$, $i=1,...,I_{Im}, j=1,...,J_{Im}$ that differs from $I_{ij}^n$, $i=1,...,I_{Im}, j=1,...,J_{Im}$ and depends on the compression controlling parameter (CCP). For different coders, different parameters play the role of the CCP. This can be the quality factor, scaling factor, quantization step, and bits per pixel [12,24,31,32]. For the BPG coder, the quality parameter Q allows changing the image quality and compression ratio. Performing a multiple compression of $I^n$using different Q (which are integers in the limits from 1 to 51 for the BPG encoder), the rate-distortion curve $Met{r}^{nc}\left(Q\right)$ can be obtained for any image where$Met{r}^{nc}$ is a metric under interest calculated between noisy (original) and compressed images.

Such metrics behave in a reasonable manner, i.e., their values become worse (smaller for all three considered metrics) if the Q increases. This is clearly seen in all four plots presented in Figure 2. In Figure 2a,b, the dependences of $PSN{R}^{nc}\left(Q\right)$ for six test images for two values of the noise variance are presented. For very small Q < 7, the original and compressed images practically do not differ and the compression is near-lossless. Then, one has a practically linear part of all curves; thus, it is possible to say that the part starts at Q = 7 and ends at such Q that $PSN{R}^{nc}\left(Q\right)\approx PSN{R}^n$. For larger Q, the dependences for different images diverge where larger $PSN{R}^{nc}$ values are observed for simpler structure images. For the middle part, it is possible to approximate the curves as:

$$PSN{R}^{nc}\left(Q\right)\approx 63-Q, dB.$$

(4)

The dependences $PSNR-HVS-M^{nc}\left(Q\right)$ and $MS-SSI{M}^{nc}\left(Q\right)$ are monotonous. They are “going jointly” for all test images for Q < 25. A joint analysis of all four dependences shows that for Q < 29 the introduced distortions are invisible. This means that it is possible to carry out a visually lossless compression and then, after image transferring (storage) and decompression, to perform noise post-filtering if necessary.

In the case of simulations, i.e., if one has a true image, adds noise to it, and then compresses this image, it is also possible to calculate the metric $Met{r}^{tc}$ between the compressed and true images, and to get the dependence$Met{r}^{tc}\left(Q\right)$. The properties of such dependences are the most interesting. Two examples are given in Figure 3. For Q < 29, the image quality is “stable” and it is practically the same as for the uncompressed image. Then, with further Q increasing, three options are possible: (1) the image quality starts to improve and, after attaining the OOP (associated with the dependence maximum), the quality starts to quickly decrease; (2) the quality continues to be almost the same and even local maxima are possible, and then a fast reduction takes the place; (3) the quality starts to decrease. The first situation takes place for the metric $MS-SSI{M}^{tc}$ for all five images except the test image, Diego (Figure 3b), and for the metric $PSNR-HVS-M^{tc}$for the test image, Frisco (Figure 3a). The second situation is observed with the metric $PSNR-HVS-M^{tc}$ for some middle complexity test images (Figure 3a). The third situation takes place for the complex structure image, Diego, according to both visual quality metrics (Figure 3a,b). Obviously, the compression in the OOP can be recommended for the first situation, lossy compression in the neighborhood of the local maxima is reasonable in the second situation, and it is unclear what to do in the third situation.

Thus, we come to several questions. The first and the most complicated deals with the fact that in practice one does not have the true image and, thus, the dependence $Met{r}^{tc}\left(Q\right)$ cannot be attained; therefore, the Q in the OOP cannot be determined and it is impossible to understand if the OOP exists or not. Another, less important, problem is what to do if an OOP does not exist.

The arisen question will become clearer after an additional analysis performed below. The plots in Figure 4 visualizes what CR can be provided by a BPG encoder applied to noisy images. Note that the plots in Figure 4 are given for six single-channel RS images of different complexity. The conclusions are the following. First, the CR (for the same Q) depends on the image complexity. For example, for Q = 28 and a noise variance equal to 64, the attained CR for the simplest and most complex test images, Frisco and Diego, differ sufficiently (they are about 4.3 and 3.7, respectively, i.e., one deals with a near-lossless compression). For Q = 35, the situation is the following: the CR for Frisco is about 35.3 whilst the CR for Diego is about 6.8, i.e., the CR differs from a near-lossless case and it is considerably larger for the simple structure image. Second, from comparing the data in Figure 4a,b it is possible to state that the CR for noisier images is smaller. For example, for ${\sigma }^2=196$ and Q = 35, the CR values for the images Frisco and Diego are equal to 5.5 and 6.4, respectively. A sharp increase in the CR starts when Q occurs larger than Q_OOP, especially for simple structure images.

Third, there is no sufficient difference in the general tendencies for images of different origin. To partly prove this, we have obtained the dependences of the considered visual quality metrics on Q for two well-known test images, Lenna and Baboon, as well as an artificial image, RSA (the images are presented in Figure 5). These dependences are given in Figure 3c,d. As one can see, the results for the highly textural images, Diego and Baboon, are very close. Similarly, the results for the simple structure images, RSA, Frisco and Lena are similar as well.

3. Properties of Optimal Operation Point

We have already mentioned that we focused on applying the BPG encoder [35]. It has several obvious advantages stimulating to consider its application for RS and other types of images. First, BPG provides a higher compression ratio compared to JPEG and many other encoders for the same quality. Second, BPG is supported by most web browsers. Third, it supports the same formats as JPEG (grayscale, YCbCr 4:2:0, 4:2:2, and 4:4:4), and the most popular RGB, YCgCo and CMYK color spaces are supported. The available versions are able to work with data from 8 to 14 bits per channel. In this paper, we present the results obtained using the grayscale and color (4:2:2) BPG version 0.9.8 given at https://bellard.org/bpg/, accessed on 25 July 2022.

Our main intention in this section is to understand what the Q_OOP is and how it can be determined for a given image, metric, and noise variance. The preliminary observations that follow from the plots in Figure 3 are the following. For all images for which OOPs are observed, they take place for approximately the same Q. The OOPs for the metrics, PSNR-HVS-M and MS-SSIM, are observed for practically the same Q. This is a good property that allows to assume that an OOP does not depend on the image at hand and metric used, but, probably, depends on the noise variance.

This hypothesis is based on the results obtained earlier for other coders [24,43,44,45]. In [43], it has been demonstrated that for an OOP (according to $PSN{R}^{tc}$) the following condition is satisfied:

$$PSN{R}^{nc}\left(Q\right)\approx PSN{R}^n.$$

(5)

This means that for ${\sigma }^2=64$ $PSN{R}^{nc}\left(Q\right)\approx 30 \mathrm{dB}$, for ${\sigma }^2=100$ $PSN{R}^{nc}\left(Q\right)\approx 28 \mathrm{dB}$ and for ${\sigma }^2=196$ $PSN{R}^{nc}\left(Q\right)\approx 25 \mathrm{dB}$. Taking into account the expression (4), this should occur for Q = 33, 35, and 38, respectively. The plots presented in Figure 6 for noise variances equal to 64 and 100 show that this really is the case. Additional studies carried out for other variance values (25, 144, 196, and 289) and other test images have shown that expression (5) is really valid for Q = Q_OOP. Substituting (4) into (5) and carrying out simple transformations, it is also possible to obtain (for 8-bit images):

$$Q_{OOP}\approx 14.9+20lo{g}_{10}\left(\sigma \right).$$

(6)

Then, knowing the noise variance a priori or estimating it with appropriate accuracy, it is possible to predict for what value of Q the OOP is possible. Meanwhile, the values of $PSN{R}^{tc}$ for Q calculated according to (6) can differ significantly. For example, for data in Figure 6a, $PSN{R}^{tc}$ is from 28.5 dB to 36.5 dB where the first case (image Diego) corresponds to an OOP absence whilst the second case (image Frisco) relates to an “obvious” OOP.

The following has been demonstrated in [45] for DCT-based coders. First, OOPs according to visual quality metrics are observed less often than OOPs according to PSNR. Second, an OOP according to visual quality metrics is observed for slightly (by about 5…10%) smaller quantization steps than for an optimal PSNR.

The same is observed for the BPG encoder. Figure 7 shows dependences $PSNR-HVS-M^{tc}\left(Q\right)$ and $MS-SSI{M}^{tc}\left(Q\right)$ for a noise variance equal to 100. They can be compared to the plots in Figure 6b. As one can see, the OOP (if it is observed) takes place for the same Q. Less OOPs are observed according to the visual quality metric $PSNR-HVS-M^{tc}$ (Figure 7a), than according to the standard metric $PSN{R}^{tc}$ (Figure 6b). Similar effects have been observed for other test images and noise variances.

Thus, we can suppose that OOPs according to all considered metrics take place for the same Q determined according to (6); however, it might be so that, for a given image and noise variance, an OOP exists according to $PSN{R}^{tc}$ but does not exist according to $PSNR-HVS-M^{tc}$. Even if the image is compressed in an OOP and the noise is partly removed, residual distortions can be clearly visible ($PSNR-HVS-M^{tc}$ is smaller than 41 dB and $MS-SSI{M}^{tc}$ is smaller than 0.99).

The carried-out analysis also shows that it is worth analyzing many test images and many values of noise variance to obtain statistical data that allow for making certain conclusions.

4. Prediction of OOP Existence and the Parameters in It

4.1. The Main Idea and Preliminary Results

Our idea of OOP prediction is based on several assumptions. First, we assume that we can predict (estimate) the difference $\Delta Metr=Met{r}^{tc}\left(Q_{OOP}\right)-Met{r}^n.$ If this difference is positive, the OOP exists; if negative but quite close to zero, then we deal with situation 2 described in Section 2 (see the examples for four test images in Figure 6a); if negative and its absolute value is large enough, then, being compressed with Q_OOP (6), such an image can be sufficiently degraded. Second, we suppose that such a prediction can be accurate enough to undertake reliable decisions (to avoid making wrong decisions). Third, it is assumed that such a prediction can be performed easily and quickly, i.e., considerably faster than the compression itself.

In fact, we already have experience in solving the aforementioned task—the procedure for predicting the OOP’s existence and metric values in the OOP have been proposed in [24]. More in detail, two statistical parameters, $P_{2\sigma }$ and $P_{2.7\sigma }$, that can be easily computed in a set of 8 × 8 pixel blocks in the DCT domain were considered. For both of them, the expressions that allow for calculating $\Delta PSNR$ and$\Delta PSNR-HVS-M$ using $P_{2\sigma }$ or $P_{2.7\sigma }$ as the argument were obtained. In this paper, we employ the ideas of [24] for the BPG coder, keeping in mind that it is based on DCT and, in this sense, statistics in the DCT domain can be highly correlated with the BPG performance characteristics.

The parameter $P_{2\sigma }$ is calculated as:

$$\begin{gathered} P_{2\sigma }={{\displaystyle \sum }}_{m=1}^M\frac{P_{2\sigma }\left(m\right)}{M}, P_{2\sigma }\left(m\right)=\left({{\displaystyle \sum }}_{k=0}^7{{\displaystyle \sum }}_{l=0}^7\delta \left(k,l,m\right)\right\}, \\ \delta \left(k,l,m\right)=1, if \left|D\left(k,l,m\right)\right|<2\sigma \\ 0 \mathrm{otherwise}, \end{gathered}$$

(7)

where M is the number of the considered blocks,$D\left(k,l,m\right)$ is the kl-th DCT coefficient in the m-th block, m = 1, …, M. In other words, $P_{2\sigma }$ is an estimate of the probability that absolute values of DCT coefficients in blocks are smaller than 2σ supposedly a priori known.

Similarly:

$$\begin{gathered} P_{2.7\sigma }={{\displaystyle \sum }}_{m=1}^M\frac{P_{2.7\sigma }\left(m\right)}{M}, P_{2.7\sigma }\left(m\right)=\left({{\displaystyle \sum }}_{k=0}^7{{\displaystyle \sum }}_{l=0}^7\delta \left(k,l,m\right)\right)-1, \\ \delta \left(k,l,m\right)=1, if \left|D\left(k,l,m\right)\right|>2.7\sigma \\ 0 \mathrm{otherwise}, \end{gathered}$$

(8)

Both statistical parameters have come from the theory of DCT-based denoising [42]. It has been shown there that such parameters are able to jointly characterize the image complexity and noise intensity. For example, $P_{2\sigma }$ is small for complex structure images and/or low intensity noise.

The next task is to obtain dependences that connect $\Delta PSNR$ and $\Delta PSNR-HVS-M$ with the input parameters, $P_{2\sigma }$ or $P_{2.7\sigma }$. This stage is carried out off-line and can be treated as specific training. The main task is, such as in machine learning, to take into account all possible situations, i.e., a wide variety of image properties and noise intensities.

At this stage, scatter-plots of ∆Metr on the input parameters have been formed. Each scatter-plot has been obtained as follows: noise with a given noise variance has been added to a considered test image that has then been compressed using Q_OOP (6). Then, the input and output parameters have been determined. In general, eleven test images of different complexity have been exploited and the noise variance was varied in the limits from 0.25 to 400. The obtained scatter-plots are presented in Figure 8, Figure 9, Figure 10 and Figure 11.

Preliminary analysis shows the following:

• The points of the scatter-plots $\Delta PSNR$ vs. $P_{2\sigma }$ (Figure 8) and $\Delta PSNR$ vs. $P_{2.7\sigma }$ (Figure 10) are placed in a very compact manner, clearly showing that there are monotonous increasing and decreasing dependences in the output parameter on the input parameter, respectively; both the input parameters are in the limits from zero to unity, for small $P_{2\sigma }$ and large $P_{2.7\sigma }$, $\Delta PSNR$ are negative and close to −2 dB (as an example, see the data for the test image Diego in Figure 5a), and in this case the OOP is absent and it is not worth compressing the images using Q_OOP (6);
• The points of the scatter-plots $\Delta PSNR-HVS-M$ vs. $P_{2\sigma }$ (Figure 9) and $\Delta PSNR-HVS-M$ vs. $P_{2.7\sigma }$ (Figure 11) are placed in a less compact manner; meanwhile, there are obvious tendencies of $\Delta PSNR-HVS-M$ increasing with $P_{2\sigma }$ increasing and $P_{2.7\sigma }$ decreasing; a visual analysis allows supposing that these dependencies are monotonous; the scatter-plot points are placed more compactly for large $P_{2\sigma }$ and small $P_{2.7\sigma }$ that correspond to simpler structure images and/or more intensive noise;
• It is possible to expect that a good curve fitting is possible in both cases that allows establishing approximate analytic dependences between the output and input parameters (examples of the fitted curves are given in all Figure 8, Figure 9, Figure 10 and Figure 11);
• There are points where these curves cross the zero level: $P_{2\sigma } \approx 0.73$ and $P_{2.7\sigma } \approx 0.16$ for $\Delta PSNR$; $P_{2\sigma }\approx 0.84$ and $P_{2.7\sigma } \approx 0.05$ for $\Delta PSNR-HVS-M.$

4.2. Curve Fitting Details

Having, in general, shown the possibility of a good curve fitting, we come to the next questions—what the best (appropriate) fitting is and how to characterize its accuracy. There are well developed theories of LMSE and robust fitting [46,47]. A visual analysis of the scatter-plots in Figure 7, Figure 8, Figure 9 and Figure 10 shows that there are no obvious outliers in the data; thus, there is no obvious necessity to apply a robust fitting. LMSE regression is implemented in many software tools including MATLAB and others; therefore, we can use one of them (the MATLAB Curve Fitting Tool in our case). To characterize the fitting accuracy, it is common [46] to use the root mean square (RMSE) that should be as small as possible, as well as a goodness-of-the-fit R² and/or adjusted goodness-of-the-fit AdjR² that have to be as large (close to the unity) as possible.

Standard curve fitting tools usually provide several options of fitting functions including polynomials, exponential or weighted sum of exponents, rational functions, Fourier series, etc. Finding the best fitting function is more an engineering and heuristic rather than scientific task. Moreover, some solutions can be very close to each other according to quantitative criteria that does not allow for giving a unique practical recommendation.

The curves fitted for the scatter-plots in Figure 8, Figure 9, Figure 10 and Figure 11 have been obtained using rational functions. For the data in Figure 8, one has an RMSE about 0.37 and R² and AdjR² about 0.98. For the scatter-plot in Figure 9, the RMSE is about 1.72 and the R² and AdjR² are about 0.85. For the data in Figure 10, one has an RMSE about 0.43 and R² and AdjR² about 0.975. For the scatter-plot in Figure 11, the RMSE is about 1.66 and the R² and AdjR² are about 0.86. Supposing that the fitting is good enough, it is possible to conclude that the fitting for $\Delta PSNR$ is considerably more accurate than for $\Delta PSNR-HVS-M$ (this is not because of bad fitting but because of larger sparsity of data for $\Delta PSNR-HVS-M$). It is better to use $P_{2\sigma }$ in predicting the $\Delta PSNR$ and $P_{2.7\sigma }$ in predicting the $\Delta PSNR-HVS-M$. The parameters of the fitted curves are presented in

Table 1. Parameters of the fitted curves.

Dependence	Expression	Parameters
$\Delta PSNR$$\mathrm{on} P_{2\sigma }$	f(x) = (p1x+ p2)/(x3+ q1x2+ q2x + q3)	p1 = 1.533 × 104, p2 = −1.112 × 104, q1 = 75.71, q2 = −6291, q3 = 6139
$\Delta PSNR$$\mathrm{on} P_{2.7\sigma }$	f(x) = (p1x + p2)/(x2+ q1*x + q2)	p1 = −6.162 × 105, p2 = 1.077 × 105, q1 = 2.843 × 105, q2 = 4501
$\Delta PSNR-HVS-M$$\mathrm{on} P_{2\sigma }$	f(x) = (p1x + p2)/(x2+ q1x + q2)	p1 = 2.895 × 105, p2 = −2.549 × 105, q1 = −1.72 × 104, q2 = 2.263 × 104
$\Delta PSNR-HVS-M$$\mathrm{on} P_{2.7\sigma }$	f(x) = (p1x + p2)/(x3+ q1x2+ q2x + q3)	p1 = −10.97, p2 = 0.558, q1 = −1.99, q2 = 1.82, q3 = 0.048

.

One might be interested whether or not other approximations are able to provide better results. The Fourier models, e.g., the Fourier model 3, are able to provide quite a good fitting in the sense of quantitative criteria (see Figure 12), but the obtained curves can be not monotonous, and that is not in agreement with our assumptions. The same relates to polynomial approximations. For polynomials of the order 2 and 3, the fitting accuracy parameters are worse than reported above, whilst for polynomials for the order 4 and 5, the curves become not monotonous. Finally, good approximations can be obtained for the weighted sums of two exponentials (see Figure 13 and compare it to Figure 9); thus, the use of rational functions or the sums of two exponentials can be considered an appropriate choice.

Having the fitted curves at hand, it is possible to predict $\Delta PSNR$ and $\Delta PSNR-HVS-M$ for any image to be compressed. For this purpose, one has to calculate the input parameter and to substitute it into the approximating curve defined in Table 1. For example, the estimated $P_{2\sigma }$ is equal to 0.6. Then, according to the data in Figure 7 (or Figure 12), the predicted $\Delta PSNR$ is about −0.8 dB, and the predicted $\Delta PSNR-HVS-M$ is about −6.5 dB according to the data in Figure 8 and about −6.1 dB according to the approximation in Figure 13. In any case, one can conclude that the OOP for this image is absent and that it is not worth compressing it with Q_OOP. An example of such a situation is given in Figure 14, where both the predicted and the true $\Delta PSNR$ and $\Delta PSNR-HVS-M$ are negative (the predicted values are equal to −0.71 dB and −6.34 dB whilst the true values are equal to −0.73 dB and −5.09 dB, respectively). As one can see, the compressed image is slightly smeared.

One can argue that the prediction is not accurate for $\Delta PSNR-HVS-M$ when $P_{2\sigma }$ is about 0.2. Consequently, in this case, $\Delta PSNR-HVS-M$ can be from −16 dB to −7 dB depending on the image at hand whilst the predicted values are about −10.5 dB (see Figure 13). Such an accuracy of prediction seems low since the errors can be up to 5.5 dB; however, in practice, this is not a problem for several reasons. First, such a small $P_{2\sigma }$ usually corresponds to images corrupted by low intensity noise which is invisible. Then, no positive effect of noise filtering can be seen anyway. Second, such $\Delta PSNR-HVS-M$ values usually correspond to $PSNR-HVS-M^n$ about 55…60 dB. Then, after compression, one has a $PSNR-HVS-M^{tc}$ about 45…50 dB, i.e., the introduced distortions are invisible.

Analyzing the scatter-plots for $\Delta PSNR-HVS-M$, one can expect that the prediction accuracy for $P_{2\sigma }>0.5,$ which is of more interest, is considerably better than for $P_{2\sigma }\le 0.5$ (see, e.g., the scatter-plot in Figure 9). To check this hypothesis, we have calculated the RMSE of the fitted curves in the considered intervals. The RMSE for $P_{2\sigma }\le 0.5$ equals to 2.34 whilst it is equal to 1.21 for $P_{2\sigma }>0.5$, i.e., our assumption is valid. Then, one can carry out quite an accurate prediction in the interval $P_{2\sigma }>0.5$ where it is especially important to undertake decisions on the OOP’s existence and Q setting.

4.3. Factors Affecting the Accuracy of Prediction

There are also other factors affecting the accuracy of prediction. A first factor is noise realization. It is clear that, for a given test image and AWGN variance, the $\Delta PSNR$, $\Delta PSNR-HVS-M$, and even Q_OOP can vary from one noise realization to another. To study this aspect, we have analyzed three images of different complexity corrupted by AWGN with three different values. The values of $\Delta PSNR$ and $\Delta PSNR-HVS-M$ have been measured for Q_OOP and the variances of these parameters have been calculated. It has been established that variances of these parameters are about 0.001; even in the worst case the maximal variance was equal to 0.0033 for $\Delta PSNR$ and 0.0046 for $\Delta PSNR-HVS-M$. This means that the influence of realization can be neglected (recall that the RMSE of fitting is about 0.4 for $\Delta PSNR$ and about 1.2 for $\Delta PSNR-HVS-M$ in the area under interest). If the Q_OOP was different for different realizations, the differences were equal to 1 (e.g., an OOP was observed for Q equal to either 33 or 34) and the difference in values of $\Delta PSNR$ or $\Delta PSNR-HVS-M$ in the neighbor points was negligible (e.g., analyze the dependences in Figure 3 near their peaks). Thus, in practice, the influence of noise realization on the parameters of the OOP can be ignored.

One more parameter that can influence the prediction is a possible variation of the input parameter. Having an approximation $\Delta Metr\left(Q\right)$ and variance of the input parameter ${\sigma }_P^2$, a variance of $\Delta Metr\left(Q\right)$ can be estimated as $Va{r}_{\Delta Metr}^2\approx {\sigma }_P^2{\left(\frac{d\Delta Metr\left(Q\right)}{dQ}\right)}^2$. Then,$\Delta Metr\left(Q\right)$ can be large in places where the absolute values of $\frac{d\Delta Metr\left(Q\right)}{dQ}$ are large and if ${\sigma }_P^2$ is quite large. The absolute values of $\frac{d\Delta Metr\left(Q\right)}{dQ}$ are large for a large $P_{2\sigma }$ or, equivalently, a small $P_{2.7\sigma }$; however, we are more interested in places where the approximation curves cross the zero level. The absolute values of the derivatives there are about 50; thus, we need to understand what the values are of ${\sigma }_P^2$.

It is possible to expect that ${\sigma }_P^2$ might depend on several factors, namely, the number of blocks M, the properties of images and noise variance, and how the blocks are positioned in a considered image. To avoid possible problems with the images with some regular structures, we propose to apply a random positioning of blocks where indices of their left upper corner are (rounded-off) random variables in the limits from i = 1 till i = I_Im − 7 and from j = 1 till j = J_Im − 7, with a uniform distribution. Within this approach, we have first analyzed three test images of different complexity with three different noise variance values. A set of realizations of the random block positions have been generated for M = 500. A variance ${\sigma }_P^2$ for the input parameter $P_{2\sigma }$ was in the limits from 5 × 10⁻⁶ till 8 × 10⁻⁵. There is no obvious dependence on the image complexity but there is a tendency of ${\sigma }_P^2$ decreasing if the AWGN variance increases (${\sigma }_P^2$ is about 0.00001 for an AWGN variance about 200 that usually corresponds to $P_{2\sigma }\ge 0.5$). The latter is a positive factor since ${\left(\frac{d\Delta Metr\left(Q\right)}{dQ}\right)}^2$ is larger just for a large $P_{2\sigma }$. Then, ${\sigma }_P^2{\left(\frac{d\Delta Metr\left(Q\right)}{dQ}\right)}^2$ is about 2.5 × 10⁻² (the standard deviation is about 0.16 and it is sufficiently smaller than the RMSE of prediction due to fitting).

We have also carried out a similar study for the input parameter $P_{2.7\sigma }$. The limits of its variance variation are similar—from 3 × 10⁻⁶ till 8 × 10⁻⁵. The ${\sigma }_P^2$ values are smaller for a larger AWGN variance.

In addition, we have also fixed the positions of the blocks and calculated the variances ${\sigma }_P^2$ for a set of noise realizations. This time the variances were even smaller (from 2 × 10⁻⁶ till 6 × 10⁻⁶) and, hence, we can neglect this factor.

4.4. Other Practical Aspects

Note that it is quite easy and fast to carry out a prediction. A calculation of the DCT in 8 × 8 blocks is a standard operation in image processing [48] that can be performed efficiently using both hardware and software. Then, easy comparisons and elementary arithmetic operations are needed to calculate the input and output parameters.

One can also be interested in whether or not the lossy compression by BPG in the OOP produces some benefits compared to other lossy compression techniques applied to noisy images.

Table 2. Obtained simulation data.

Test Image	AWGN Variance	AGU			BPG
		PSNR (dB)	CR	MS-SSIM	PSNR (dB)	CR	MS-SSIM
Frisco	64	36.05	31.50	0.972	36.83	32.84	0.977
	100	34.84	38.36	0.965	35.68	41.17	0.972
	196	33.09	54.93	0.949	33.83	56.63	0.960
Fr01	100	28.26	9.33	0.966	29.66	10.09	0.976
	196	26.66	14.3	0.950	27.78	14.3	0.963
Aerial	100	28.29	8.87	0.985	29.63	9.54	0.976
	196	26.57	13.03	0.946	27.67	13.04	0.962
Airfield	100	27.30 *	7.69	0.951 *	28.25	10.52	0.956
	196	25.79	12.35	0.927 *	26.69	12.34	0.938

* OOP is absent.

contains data that allow for comparing BPG to the coder AGU [49] for some test images and some noise variances. First, a comparison has been completed for two images used in obtaining the approximating dependences (fitted curves), namely, the images, Frisco and Fr01. As one can see, the BPG encoder provides benefits in three senses: (1) it produces about a 0.8 dB better $PSN{R}^{nc}$ in the OOP; (2) it provides a better visual quality according to the visual quality metric $MS-SSI{M}^{nc}$, and (3) a slightly (by a few percent) larger CR is usually ensured.

Meanwhile, it can also be interesting to check if this happens for images not used in obtaining the scatter-plots (for image processing approaches based on learning, a verification stage is obligatory). For this purpose, we have obtained simulation data for two test images—Aerial and Airfield—that have not been used in previous analysis. The results are presented in Table 2. Their analysis shows the following. First, again the results for the BPG coder are better for both metrics and, at least, not worse according to the CR. An interesting situation has been observed for the test image, Aerial, corrupted by AWGN with a variance equal to 64 and the image, Airfield, corrupted by AWGN with a noise variance equal to 100. For the encoder AGU, an OOP is absent whilst for the encoder BPG, an OOP exists for both the considered metrics.

Thus, we can state that the BPG encoder outperforms AGU and this takes place for both images that have been used in obtaining scatter-plots and images that have been employed for verification.

5. Decision-Making and Other Practical Cases

Supposing that prediction has been carried out for a given image, one obtains $\Delta PSNR$ or $\Delta PSNR-HVS-M$ or both. A question is what to do further? There can be different options—to rely only on one parameter (rather than one of two) or on both. The choice is very heuristic without an obvious preference (e.g., special studies with customers assessing the compressed image quality can be carried out to obtain a reliable answer). We propose the following practical algorithm:

• If $\Delta PSNR$ + $\Delta PSNR-HVS-M$ > 1 dB, consider that an OOP exists and compress an image using Q_OOP;
• If $-1 \mathrm{dB} \le \Delta PSNR$ + $\Delta PSNR-HVS-M$ ≤ 1 dB, consider that an OOP might exist and use Q_OOP-1 (but not less than 28) to avoid oversmoothing;
• If $\Delta PSNR$ + $\Delta PSNR-HVS-M$ ≤ −1 dB, use Q = 28 to provide invisibility of introduced distortions and the possibility of noise removal in it by applying post-filtering to the decompressed images.

An example in Figure 13 corresponds to case 3. An example for situation 1 is presented in Figure 15. The positive effect of a lossy compression is seen, especially in an image’s homogeneous regions where the noise removal is obvious. Finally, an example for situation 2 is demonstrated in Figure 16. In this case, the positive effect of noise suppression can be noticed as well; however, edge/detail smearing takes place too.

If situation 1 is falsely recognized as situation 2 or vice versa, practically nothing happens since the recommended Q values differ only by unity. If situation 2 is falsely recognized as situation 3, it is not a problem since a “careful” compression is carried out (by the expense of a smaller CR). If situation 3 is falsely recognized as situation 2, a slightly more smearing of the compressed image can be observed (however, the CR is larger). Finally, situations 1 and 3 are practically never misclassified.

Thus, the fully automatic procedure for a given noisy image is as follows:

• Estimate the noise variance by some blind method of a noise variance assessment (if the noise variance is not known in advance);
• Calculate Q_OOP according to (6); calculate $P_{2\sigma }$ according to (7);
• Calculate $\Delta PSNR$ and $\Delta PSNR-HVS-M$ using the expressions and parameters in Table 1;
• Make a decision on the recommended Q_OOP according to the algorithm described above;
• Carry out a compression using the recommended Q_OOP.

As it has been stated in Section 2, AWGN can be a simplified noise model for which the noise variance or standard deviation can be estimated in a blind manner (automatically) [50,51]. A more general case is the signal-dependent noise model [52]. Let us demonstrate that the OOP is also possible for images corrupted by Poisson noise compressed by BPG. Note that two approaches to such a compression are possible. The first one is to apply BPG directly. The second one is to apply a proper VST (Anscombe transform in the considered case [53]) and some pre-normalization before the compression with inverse operations at the decompression stage. In this case, the signal-dependent noise converts to an almost pure additive and we arrive at the image/noise model studied above; thus, let us concentrate on the first approach.

The analysis has been carried out for six RS test images. The obtained rate-distortion curves $PSNR-HVS-M^{tc}\left(Q\right)$ and $MS-SSI{M}^{tc}\left(Q\right)$ are given in Figure 17. As one can see, the OOP exists for the metric MS-SSIM for the images, Frisco and Fr04, and for the metric PSNR-HVS-M for the image, Frisco (similar situations have been earlier observed for the AWGN case, see the plots for the test image Fr02 in Figure 7). In all three cases, the OOP is observed for Q = 37. According to $PSN{R}^{tc}$, the OOP exists as well and this happens for Q = 37, while $\Delta PSNR$ can reach 3 dB; thus, we can expect that automatic procedures for lossy compression can be designed for BPG for images corrupted by signal-dependent noise, including the speckle typical for synthetic aperture radar images.

It might also be true that two, three or more image components are corrupted by the noise and these images have to be compressed in a lossy manner. Certainly, a component-wise compression for which all the results obtained above are valid is possible; however, the joint compression of several component noisy images is possible as well. For example, available BPG software allows for compressing color, i.e., three-channel images; thus, it is easily possible to test this practical situation. As an initial case, let us assume that all the component images are corrupted by AWGN with the same noise variance. Since the results can be of interest for both three-channel and color images, four test images have been considered: the RS three-channel images, Frisco and Diego, and the widely used color test images, Lena and Baboon. The original images of size 512 × 512 pixels were presented in RGB, an AWGN with variance equal to 100 was independently added to each component image, a 4:2:2 version of BPG was applied, and the metrics were calculated independently for each component image. The obtained dependences are presented in Figure 18.

It follows from the preliminary analysis that the OOP can also exist and it takes place practically for the same Q for all component images. Meanwhile, there are also some specific results, such as more obvious OOPs for the G component; thus, additional studies are needed.

6. Conclusions

The task of the lossy compression of images corrupted by AWGN by the BPG coder is considered. The main attention is paid to the single-channel image case. It is shown that an OOP might exist according to the standard criteria as PSNR (and MSE). Moreover, an OOP might exist for the visual quality metrics such as MS-SSIM and PSNR-HVS-M, although, this occurs more rarely. It is demonstrated that OOP existence depends on the noise variance and image complexity where an OOP exists with a higher probability for simpler structure images and/or a higher intensity noise. With a noise intensity increase, the Q_OOP increases as well, and the corresponding expression is obtained. The approach to predicting the OOP’s existence and the metric values in it is proposed. This is based on obtaining the approximating dependences in advance using scatter-plots and curve fitting. Then, for a given image, a simple statistical parameter has to be estimated, its value has to be used as an approximator input and the prediction with decision-making needs to be completed. Its accuracy is analyzed and shown to be appropriate for practice. The recommendations on an automatic Q setting for different practical situations are given and illustrated.

The possibility of OOP existence for signal-dependent noise and three-channel images corrupted by AWGN is shown. This can be the direction of the future research. Another direction deals with improving the prediction accuracy for visual quality metrics by the joint processing of several input parameters that can be performed by a trained neural network.