Compression Ratio as Picture-Wise Just Noticeable Difference Predictor

Abstract

This paper presents the interesting results of applying compression ratio (CR) in the prediction of the boundary between visually lossless and visually lossy compression, which is of particular importance in perceptual image compression. The prediction is carried out through the objective quality (peak signal-to-noise ratio, PSNR) and image representation in bits per pixel (bpp). In this analysis, the results of subjective tests from four publicly available databases are used as ground truth for comparison with the results obtained using the compression ratio as a predictor. Through a wide analysis of color and grayscale infrared JPEG and Better Portable Graphics (BPG) compressed images, the values of parameters that control these two types of compression and for which CR is calculated are proposed. It is shown that PSNR and bpp predictions can be significantly improved by using CR calculated using these proposed values, regardless of the type of compression and whether color or infrared images are used. In this paper, CR is used for the first time in predicting the boundary between visually lossless and visually lossy compression for images from the infrared part of the electromagnetic spectrum, as well as in the prediction of BPG compressed content. This paper indicates the great potential of CR so that in future research, it can be used in joint prediction based on several features or through the CR curve obtained for different values of the parameters controlling the compression.

1. Introduction

In the transmission and archiving of multimedia content, an unavoidable process is compression, which can be lossless and lossy [1,2]. Visually lossless compression, which is a type of lossy compression, enables a significant saving of communication and memory resources compared to the original and its lossless case while preserving the excellent quality of visual content for end users (humans) [3]. Thus, this type of compression often enables optimal use of resources. For example, in [4], it was stated that when archiving JPEG compressed images using visually lossless compression, at least 10 times greater saving of memory resources can be achieved compared to the original image. Recently introduced studies consider compression of visual signals for machine analysis (detection, classification, and recognition tasks) [5,6], and within this type of compression, instead of satisfied user, satisfied machine ratio modeling is considered.

Determining the boundary between visually lossless and visually lossy compression is gaining more and more importance. Confirmation of this observation is the growing number of publicly available databases with the results of subjective tests [7,8,9,10,11], as well as the growing number of papers in which the prediction of this boundary is made through the determination of the desired value of the peak signal-to-noise ratio (PSNR) [12,13], image representation in bits per pixel (bpp) [13,14], or by predicting the value of the parameter that controls the compression (for example, the quality factor QF in JPEG or the parameter Q in BPG compression) [15,16,17,18].

The concept of just noticeable difference (JND) refers to determining the maximum difference (visibility threshold or visual redundancy) between two visual signals that the observer will not detect [19,20,21] and which reflects the tolerance of the human visual system. JND can be defined at the pixel level (pixel-wise) [22,23], the block level (patch-wise) [24], and the global level (picture-wise) [25], which is the subject of research in this paper. The application of JND models is of particular interest for perceptual image/video coding, watermarking, and image quality assessment [26,27,28,29].

The boundary between lossless and lossy compression is called the first JND point (JND#1). This boundary corresponds to the maximum compression ratio (CR) so that the average observer does not notice the difference between the original, uncompressed signal and its lossy compressed version [30,31]. In this way, observers will be very satisfied with the quality of the compressed signal, and on the other hand, memory, and communication resources will be saved (compared to the raw signal or its lossless compressed version).

There are different approaches for the JND#1 point position prediction. The position can be determined using a fixed value of objective measures, which corresponds to the similarity between the original image and its compressed version (such as PSNR, SSIM, PSNR-HVS-M, FSIM, and similar measures [32]), assuming that the used metric (measure) adequately assesses visual quality. Applying a fixed threshold (known or determined in advance) guarantees visually lossless compression but does not guarantee that it will be with the maximal possible CR. Thus, the fixed threshold value is not the best solution since the boundary between visually lossless and lossy compression considerably depends on the content of the image to be compressed. Additionally, when using these techniques, it is necessary to develop an optimal iterative strategy for reaching the desired objective quality value [32,33]. Approaches that use deep learning techniques in prediction can also be found in the literature [13,15,16,17,18]. These approaches have achieved the best prediction results and can be used both for the prediction of the desired value of PSNR (PSNR JND#1) and bpp (bpp JND#1) or for the direct prediction of the parameter that controls the degree of compression. The success of these approaches depends on the training set, and typically, thousands of features derived from the source signal are used as input. This might make them impractical when the time for carrying out the compression is limited. Finally, recent research has shown that reliable prediction of the position of the JND#1 point can also be obtained using simple features derived from the original uncompressed image [12,14]. Spatial information, spatial frequency, contrast, and CR (for lossless or near-lossless compression) have been proven as reliable features.

CR is an important parameter in coding, and it significantly depends on image complexity, parameters that control compression, and noise presence in a given image to be compressed. In papers that use CR to predict the image complexity [34,35] or position of the first JND point [14], it is calculated as the ratio of the size of the raw source signal and its JPEG compressed version obtained for a quality factor QF equal to 100. The main goal of this paper is to analyze the success of PSNR and bpp prediction of the first JND point of color and infrared images if the CR is calculated for all possible values of the parameter that controls the compression (QF for JPEG or Q for BPG coders). The analysis is conducted on four publicly available JND databases [36,37,38,39], with subjective test results on JPEG and BPG compressed images. Although it was standardized more than 30 years ago, the JPEG image format is still widely used. It is embedded in various consumer devices and supported in numerous applications, operating systems, and browsers [40]. BPG, which is derived from the H.265/HEVC video compression standard, is a freely available codec and is one of the promising image compression coders [41,42].

It is shown that the prediction performance can be significantly improved by the proper choice of QF/Q. Although the optimal QF value for CR calculation depends on the chosen database, it can be concluded that the prediction results of PSNR JND#1 JPEG compressed color images can be improved by using a QF value between 80 and 90, while the prediction results of bpp JND#1 JPEG images can be improved by using a QF value between 20 and 40.

Additionally, it is shown that the performance of CR can be improved in the prediction of PSNR and bpp JND#1 points of BPG compressed color images. The compression ratio for this image format is calculated as the ratio of the size of the original uncompressed image to its BPG version. For the prediction of PSNR JND#1 points, it is preferable to use CR calculated for Q from 15 to 30, while for the prediction of bpp JND#1 points, it is preferable to use values of Q from 25 to 35.

A separate part of this paper is devoted to the analysis of the performance of CR in the prediction of PSNR and bpp JND#1 points of JPEG and BPG compressed images from the infrared part of the electromagnetic spectrum, where 25 images from the recently published JND database of infrared images are used in this analysis [43]. On this type of grayscale images specific for surveillance and monitoring, it is shown that in the prediction of PSNR JND#1 points of JPEG images, it is preferable to use CR calculated for higher values of QF, while in the prediction of bpp JND#1 points of JPEG images, it is preferable to use lower values of QF. For BPG compressed infrared images in PSNR prediction, it is preferable to use lower values of the parameter Q, while in the prediction of bpp JND#1 points, it is preferable to use CR calculated for Q values from the middle of its range.

A logistic curve with four parameters is used in the prediction, along with the four quantitative performance indicators. Apart from the dependence of these indicators on QF/Q, for the considered compressed images, scatter plots of the CR vs. PSNR/bpp through three values of the parameters used to control the compression are also shown.

Finally, in this paper, we solve a particular task of studying CR applicability as a complexity indicator and looking for a quasi-optimal parameter that controls JPEG/BPG compression that will provide good performance for JND#1 compressed images of different types. To accomplish this task, numerous tests are conducted on available JND data, where the success of CR as a predictor is analyzed for the first time on BPG compressed images and infrared images. The results of this are the recommended (quasi-optimal) values of the QF/Q parameters for which the CR is determined and used in the prediction of the desired compressed image quality or image representation. The reliability of CR as a predictor is confirmed for different types of compression, different types of images, and different image modalities.

2. Materials and Methods

In this paper, four well-known publicly available image databases containing the results of JND subjective tests on compressed images [36,37,38,39] are used. The original images from these databases are practically noise-free or contain invisible noise, which is in accordance with the requirement that the source material in such subjective tests is clean; that is, the use of noisy sources is not permitted.

Table 1. Basic characteristics of the used JND databases.

Database	Publication Year	Image Size (In Pixels)	Number of Reference Images	Distortion Type	Distortion Levels Per Reference	Number of Test Stimuli	Number of Observers	Subjective Test	Test Environment
MCL-JCI [36]	2016	1920 × 1080	50	JPEG	100	5000	30	side by side + binary search	lab
JND-Pano [37]	2018	5000 × 2500	40	JPEG	100	4000	42 (at least 25 per image)	side by side + binary search	lab
KonJND-1k [38]	2022	640 × 480	504	JPEG	100	50,400	503 (an average of 42 per image)	flicker test	crowdsourcing
			504	BPG	51	25,704
JND-MID-MA [39]	2024	variable	105 (35 color, 35 grayscale, and 35 infrared)	JPEG	100	3500 × 3	26 (color), 19 (grayscale) and 22 (infrared)	side by side + binary search	lab
				BPG	52	1820 × 3	29 (color), 27 (grayscale), and 27 (infrared)

contains basic information about these databases.

KonJND-1k [38] is the database with the largest number of reference images (1008). Additionally, subjective results were collected through crowdsourcing and using a flicker test in which observers answered the question of whether they perceived a flicker effect for the displayed flickering image (the reference and the compressed test image were displayed successively at a frequency of 8 Hz). The remaining three databases were created by conducting subjective tests in controlled (laboratory) conditions, where a double-stimulus test was used, in which the reference image and its compressed version were shown side by side, and the observers were required to say when they noticed a difference between them. MCL-JCI is the first publicly available JND database [36], which contains the results of subjective tests on JPEG compressed real-world images in full HD resolution, while the JND-Pano database contains the results of JND tests on JPEG compressed panoramic images of significantly higher resolution [37]. The recently introduced JND-MID-MA database was created through subjective tests on 35 original images obtained using sensors operating in the visible and infrared (invisible) part of the electromagnetic spectrum [39]. Images of these two modalities are spatially aligned, whereby images from the visible part of the electromagnetic spectrum were used in subjective tests in color and grayscale versions [44,45]. In this paper are used the results of JND tests obtained on 25 color [44] and infrared images [43], which originate from the same sensors of the well-known Multi-scenario Multi-Modality database (M³FD) [46]. In addition to subjective tests on JPEG, KonJND-1k and JND-MID-MA databases also consider tests on BPG compressed images.

Typical layouts of user interfaces used to collect the results of subjective picture-wise JND tests are shown in Figure 1. In the flicker test (Figure 1a), the slider is used to determine the critical flickering point, i.e., the lowest level of degradation, for which the observers perceive the flicker effect [38]. In the double-stimulus test (Figure 1b), observers compare pairs of images (original and compressed versions) and determine whether differences can be observed between them or not. This binary-forced choice is followed by a bisection procedure, which allows for determining the JND#1 position with a reduced number of comparisons over a wide range of possible degradations (100 for JPEG and 52 for BPG) [36]. Additional information can be displayed to observers, such as test progress and instructions for conducting this experiment.

The results of the individual observer tests are collected as values of QF for JPEG and Q for BPG compression type. From these results, the subjectively guided ground truth values of QF and Q are obtained by eliminating unreliable observers and integrating the remaining results. This procedure differs in databases, and for detailed considerations, the reader is referred to [36,37,38,39]. The available ground truth values of QF and Q are used in this research to generate appropriate compressed images and obtain ground truth values of PSNR and bpp (denoted as PSNR JND#1 and bpp JND#1).

The examples shown in Figure 1 are from the JND-MID-MA (infrared) database, and they illustrate the diversity of content present in this database (in this case, we consider countryside and urban semantic or property-specific categories). As the space of possible natural scene images is enormous, in the analysis of image quality, it is preferable to use different databases and a wide range of original content. Below, we use such quantitative indicators (descriptors, features) as spatial information (SI) and colorfulness (CF) to briefly analyze the variety of source content present in databases [36]. SI is used as an indicator of image complexity [47], while CF is used as an indicator of color variety and intensity [48]. Although there are different ways to calculate SI, this indicator is most often determined as the mean gradient magnitude (SI_mean) obtained using Sobel 3 × 3 spatial masks, and it has been proven to be the best predictor of image complexity in compression [47]. Instead of colorfulness, for grayscale infrared images, the mean brightness (BR) is adopted here as a descriptor.

Figure 2 shows SI_mean versus CF and SI_mean versus BR plots along with the convex hulls for the original images of the analyzed databases (each point corresponds to one source image). From this Figure, it can be concluded that the KonJND-1k database has the widest range of source content, both in terms of complexity and colors. Additionally, subsets of source content used for JPEG and BPG compression are highlighted for this database (Figure 2a). MCL-JCI and JND-Pano databases have approximately the same range of SI_mean and CF descriptors (Figure 2b,c). The source images of the JND-MID-MA color database cover a wider range of complexity than the MCL-JCI and JND-Pano databases, but this database also has the smallest variety of colors. Although the JND-MID-MA color and infrared databases contain spatially aligned images of the same scene obtained by sensors operating in the visible and infrared part of the electromagnetic spectrum, it can be concluded that the range of SI_mean scores of infrared images is significantly narrower than the corresponding range of images from the visible part of the spectrum. This is a consequence of the nature of infrared images, which have less pronounced textural information than visible images.

Thus, the used databases have slightly different properties, and due to this, it is possible to expect that the results obtained for them can be slightly different as well.

Figure 3 shows examples of JND color and infrared images from the JND-MID-MA database. It can be seen that although these images are compressed, they are of excellent visual quality (on the boundary between visually lossless and visually lossy). Furthermore, it is very difficult to see the differences between JPEG and BPG compressed color, i.e., infrared images. In both considered cases, color, and infrared, JND images are obtained with a lower number of bits using BPG compression, so it can be concluded that in this case, the BPG compression is more efficient than JPEG. Additionally, it can be concluded that the PSNR for infrared images is higher than the PSNR of the corresponding color images. The results of the subjective tests of the JND-MID-MA database allow for comparing the efficiency of BPG and JPEG on the boundary between visually lossless and visually lossy compressions for color, grayscale, and infrared images [39]. This is not the case with KonJND-1k because the subjective tests on half of the database were conducted with BPG and on the other half, with JPEG compression of color images.

To predict the positions of the first JND points considered through PSNR and bpp, the CR calculated for different values of the parameter that controls compression (QF for JPEG or Q for BPG) is used. The compression ratio is calculated in relation to the size of the original uncompressed images (24-bit color and 8-bit infrared).

Four simple features, earlier studied by us in predicting the position of the first JND point [12,14,32], are used in further comparative analysis. They proved to be reliable and quite accurate. Namely, in addition to the already mentioned gradient-based descriptor SI_mean, contrast, homogeneity, and edge density are also employed. Contrast and homogeneity are calculated based on the gray level co-occurrence matrix (GLCM), known in image texture analysis and classification [14,34]. The frequencies of occurrence of pairs of gray levels at the positions (m, n) and (m + 1, n + 1) of the original images are used to determine the GLCM. Edge density represents the percentage of edge pixels in the original image, and the well-known Canny detector is adopted to determine the edge pixels [34].

The relationship between simple image features and PSNR JND#1 and bpp JND#1 may not be linear for several reasons. In order to eliminate the non-linearities introduced by the subjective tests (here mapped to PSNR JND#1 and bpp JND#1), the relationships between PSNR and bpp predictions and simple image features are characterized using non-linear regression. The shape of the regression is not critical if it is monotonic, generally acceptable, and has as few free parameters as possible to facilitate data interpolation. The most common functions used in regression and recommended by the ITU for research of image/video quality assessment in psychophysical experiments are the third-order polynomial with four parameters and the logistic functions with four and five parameters [49,50].

A logistic curve with four parameters will be used in further analysis for the prediction of the PSNR/bpp values based on CR and other features:

$$p={\displaystyle \frac{{\beta }_1-{\beta }_2}{1+\exp \left(\frac{o-{\beta }_3}{{\beta }_4}\right)}}+{\beta }_2$$

(1)

where o is an objective value (CR, SI_mean, contrast, homogeneity, and edge density in our research), and p is a prediction of PSNR/bpp JND#1. The optimal parameters β₁–β₄ will be obtained through the minimization of the mean squared error between the prediction set p and the ground truth PSNR/bpp JND#1 values obtained in subjective tests (PSNR JND#1/bpp JND#1) [50]. After this non-linear mapping, four quantitative indicators (linear correlation coefficient (LCC), Spearman’s rank–order correlation coefficient (SROCC), mean absolute error (MAE), and root mean square error (RMSE)) will be used to determine the degree of agreement between ground truth PSNR/bpp values and their predictions obtained using CR and other features according to Equation (1). LCC, MAE, and RMSE are prediction accuracy attributes, while SROCC is an indicator of prediction monotonicity.

Expressions for the prediction of PSNR/bpp JND#1 using image complexity indicator CR, obtained using the results of subjective tests of publicly available databases, will be determined once and in advance. Predictions based on this (as well as any other) indicator should be as accurate as possible. For JND#1 compression of an image at hand, it is necessary to first determine the CR based on the recommended value of the parameter that controls the compression (QF/Q) and, based on it, using Equation (1), obtain the PSNR/bpp value that needs to be achieved. This process should be quite fast, and in our case, it really is because CR is obtained only for one image compression, while simple mapping is used in prediction. Finally, the desired value of PSNR/bpp should be reached using approaches that are reliable and fast enough, which is also possible by applying simple approaches described in our earlier research [32].

3. Results

The success of CR in predicting the position of the first JND points is presented in this section through the results for JPEG color, BPG color, and infrared images. This is followed by the analysis of the relationship between PSNR and bpp of the first JND points for four databases, while the last part of this section contains validation of the CR-based prediction of the JND#1 through real examples and cross-database validation. In addition to the success of CR, with the aim of comparative analysis, the results for other simple predictors of the position of the first JND point, SI_mean, contrast, homogeneity, and edge density are given.

3.1. JPEG Color Images

3.1.1. PSNR JND#1 Prediction for JPEG Color Images

Figure 4 and Figure 5 show the values of quantitative indicators LCC and MAE for four publicly available JND databases. These values are obtained by calculating the degree of agreement between ground truth PSNR values obtained in subjective tests and PSNR values obtained by prediction based on CR calculated for QF values from 1 to 100. From these Figures, it can be concluded that the success of CR in prediction depends on the selected database, as well as that it significantly depends on the QF value for which it is calculated. Instead of the so far recommended value for which CR is calculated (QF = 100) [14,34,35], it is better to use CR calculated for other values of QF. From these two groups of plots, it can be concluded that the performance of CR on KonJND-1k and MCL-JCI databases is stable (and good) in a wide range of QF values from the central part of the available range from 1 (large CR) to 100 (almost lossless compression with CR close to unity). In contrast to these two, with the JND-Pano and JND-MID-MA databases, the optimum (minimum MAE, i.e., maximum LCC) moves to the right toward higher QF values. With JND-Pano, the optimum is expressed, which is not the case with the results obtained on the JND-MID-MA database. As a compromise solution for all four databases, the CR calculated for QF values between 80 and 90 would be used. Therefore,

Table 2. Quantitative indicators of the degree of agreement between ground truth and PSNR JND#1 predictions obtained using CR (calculated for three quality factors), SI_mean, contrast, homogeneity, and edge density for four databases.

		LCC	SROCC	MAE	RMSE	LCC	SROCC	MAE	RMSE
Database		KonJND-1k				MCL-JCI
CR	QF = 1	0.7822	0.7504	1.6350	2.0652	0.7504	0.6648	1.5791	2.0265
	QF = 85	0.9243	0.9164	0.9826	1.2650	0.9589	0.9455	0.7539	0.8694
	QF = 100	0.8705	0.8518	1.2108	1.6312	0.9392	0.9349	0.8442	1.0524
SImean		0.9110	0.9021	1.0789	1.3668	0.9398	0.9090	0.8528	1.0476
contrast		0.8728	0.8750	1.2425	1.6176	0.8805	0.8250	1.1540	1.4534
homogeneity		0.7465	0.7073	1.7224	2.2055	0.7746	0.7055	1.6140	1.9391
edge density		0.7922	0.7851	1.5670	2.0228	0.8416	0.7387	1.3004	1.6561
Database		JND-Pano				JND-MID-MA (25 Color Images)
CR	QF = 1	0.8770	0.8514	1.2504	1.7408	0.7010	0.7213	1.3702	1.7015
	QF = 85	0.9159	0.9221	1.0699	1.4541	0.9811	0.9662	0.3800	0.4617
	QF = 100	0.8590	0.8552	1.3613	1.8551	0.9762	0.9592	0.4018	0.5174
SImean		0.9099	0.9205	1.1225	1.5032	0.9440	0.9446	0.6264	0.7874
contrast		0.9515	0.9467	0.8656	1.1152	0.8392	0.8331	1.0447	1.2975
homogeneity		0.7592	0.7452	1.8998	2.3581	0.9234	0.8885	0.7263	0.9156
edge density		0.6420	0.6516	2.1834	2.7779	0.7787	0.6638	1.0672	1.4968

provides the values of the quantitative indicators LCC, SROCC, MAE, and RMSE obtained for all four databases if CR is calculated for QF values of 1, 85, and 100. Additionally, Figure 6 provides scatter plots of CR vs. ground truth PSNR JND#1 (each point corresponds to one source image). Table 2 also provides performance predictions for the other four simple features of the source image, and the best score for each of the performance indicators is bolded.

From Table 2, it can be concluded that by applying the recommended (adopted) value QF = 85, the significantly better prediction results of PSNR JND#1 are obtained than by using the standard value QF = 100 proposed in the literature. The gain achieved on the KonJND-1k and JND-Pano databases is greater than the gain achieved on the MCL-JCI and JND-MID-MA databases. Scatter plots in Figure 6 confirm these observations, which can be seen through the reduction in spreading around logistic curves in 2D space CR vs. PSNR JND#1. The LCC values on all four databases for CR calculated for QF = 85 are greater than 91%, with the LCC value being the highest on the JND-MID-MA database (which contains the least number of source images). At the same time, there is also the smallest MAE between the ground truth PSNR values and their predictions (0.38 dB). MAE is somewhat larger on the MCL-JCI database (0.75 dB), while on the largest picture-wise KonJND-1k and high-resolution JND-Pano databases, it is around 1 dB. The benefits, according to LCC and MAE, due to choosing the quasi-optimal QF/Q, are confirmed by SROCC and RMSE values for all considered databases. Furthermore, it can be seen that on the JND-Pano database, the best prediction results are obtained using contrast, and the results for CR and SI_mean are slightly worse. Meanwhile, for the remaining three databases, the contrast has a significantly worse performance than CR. In other words, CR provides the most stable performance. Additionally, it can be concluded that SI_mean has high performance on all four databases; although, on average, it is worse than CR.

3.1.2. bpp JND#1 Prediction for JPEG Color Images

Figure 7 and Figure 8 show the quantitative values of the degree of agreement between ground truth and CR-based predictions of image representation in bpp (LCC and MAE). Unlike the curves obtained for PSNR, here, the optimum shifts toward lower values of QF. A compromise solution for all four databases would be to use the CR calculated for QF values between 20 and 40. Therefore, in

Table 3. Quantitative indicators of the degree of agreement between ground truth and bpp JND#1 predictions obtained using CR (calculated for three quality factors), SI_mean, contrast, homogeneity, and edge density for four databases.

		LCC	SROCC	MAE	RMSE	LCC	SROCC	MAE	RMSE
Database		KonJND-1k				MCL-JCI
CR	QF = 1	0.7871	0.7926	0.1394	0.1913	0.7258	0.7551	0.1094	0.1510
	QF = 30	0.9347	0.9535	0.0727	0.1102	0.8420	0.8836	0.0770	0.1184
	QF = 100	0.9023	0.8832	0.0975	0.1337	0.7925	0.8306	0.0946	0.1338
SImean		0.9051	0.9202	0.0890	0.1318	0.8399	0.9095	0.0770	0.1191
contrast		0.8311	0.8229	0.1233	0.1724	0.8507	0.8971	0.0803	0.1153
homogeneity		0.7954	0.7688	0.1367	0.1879	0.5573	0.6002	0.1268	0.1822
edge density		0.8061	0.8137	0.1307	0.1835	0.6591	0.6319	0.1068	0.1650
Database		JND-Pano				JND-MID-MA (25 Color Images)
CR	QF = 1	0.8547	0.8501	0.0803	0.0994	0.7737	0.6917	0.0842	0.1023
	QF = 30	0.9255	0.9034	0.0567	0.0725	0.9160	0.8585	0.0506	0.0648
	QF = 100	0.8191	0.7891	0.0878	0.1098	0.8508	0.8200	0.0652	0.0849
SImean		0.9188	0.8991	0.0598	0.0756	0.9222	0.8585	0.0488	0.0624
contrast		0.8458	0.8368	0.0804	0.1021	0.8527	0.7531	0.0655	0.0844
homogeneity		0.7489	0.7473	0.1042	0.1269	0.6388	0.6354	0.0851	0.1242
edge density		0.7308	0.6795	0.1037	0.1307	0.4938	0.5400	0.1024	0.1404

, the values of objective indicators obtained for three values of QF (1, 30, and 100) are given, while Figure 9 shows the corresponding scatter plots of CR vs. bpp JND#1. For all four databases, better prediction results of bpp JND#1 are obtained if the CR value is calculated for QF = 30 than for QF = 100. Also, the scatter plots in Figure 9 show the reduction in scores spreading around the logistic curves for the CR-based case of QF = 30. From Table 3, it can be concluded that, except for the MCL-JCI database, the degree of agreement between ground truth values and CR-based predictions of bpp measured through LCC is greater than 91%. The corresponding mean absolute differences (MAE) are between 0.05 and 0.08 bpp. In addition to CR, it can be seen that SI_mean also has high prediction performance on all four databases, with the best results on the JND-MID-MA database.

3.2. BPG Color Images

3.2.1. PSNR JND#1 Prediction for BPG Color Images

Apart from determining the position of PSNR and bpp of the first JND points of images with JPEG compression, in this paper, the predictive capabilities of CR are also examined in determining the JND threshold for BPG images. Two databases containing the results of JND tests on BPG images are used.

The predictive abilities of CR on BPG images have not been investigated in detail so far. Unlike paper [14], in which the CR is calculated as the ratio of the size of the uncompressed image and its JPEG compressed version for QF = 100, in this paper, the CR is determined based on the sizes of the BPG compressed images obtained for different values of the parameter Q.

Quantitative values of LCC and MAE depending on the parameter Q for two BPG JND databases are shown in Figure 10 and Figure 11. In these Figures, the capabilities of CR in predicting PSNR JND#1 are analyzed, and it can be concluded that the performance of CR in the prediction of PSNR JND#1 significantly depends on the parameter Q. The optimal value of the parameter Q for PSNR prediction on the KonJND-1k database is 20, while for JND-MID-MA, the optimal value is Q = 30. Quantitative indicators of the degree of agreement between ground truth PSNR and their predictions obtained using CR calculated for Q values of 1, 25, and 51 are given in

Table 4. Quantitative indicators of the degree of agreement between ground truth and PSNR JND#1 predictions obtained using CR (calculated for three parameter Q values), SI_mean, contrast, homogeneity, and edge density for two databases.

		LCC	SROCC	MAE	RMSE	LCC	SROCC	MAE	RMSE
Database		KonJND-1k				JND-MID-MA (25 Color Images)
CR	Q = 1	0.8771	0.8793	0.9700	1.2931	0.7981	0.7292	0.9388	1.2421
	Q = 25	0.9253	0.9265	0.8018	1.0210	0.8451	0.7723	0.7928	1.1021
	Q = 51	0.6687	0.6516	1.5781	2.0017	0.7496	0.6954	1.1328	1.3647
SImean		0.8655	0.8599	1.0565	1.3484	0.8388	0.7892	0.9013	1.1226
contrast		0.7406	0.7415	1.4123	1.8090	0.8126	0.7731	0.9791	1.2017
homogeneity		0.7879	0.7834	1.2817	1.6580	0.7451	0.6485	0.9462	1.3750
edge density		0.8242	0.8324	1.2098	1.5273	0.6091	0.5062	1.1631	1.6351

while corresponding visual illustrations through scatter plots and the fitted curves are shown in Figure 12.

From Table 4, it can be concluded that the best prediction results among all predictors are obtained if the CR is calculated for Q = 25. The predictive ability is better for KonJND-1k than for the JND-MID-MA database. Mean absolute differences between ground truth PSNR values and predictions obtained for Q = 25 are approximately equal on both databases and are around 0.8 dB. Scatter plots in Figure 12 further confirm the success of CR in predicting PSNR JND#1 using the proposed value of Q = 25. Furthermore, according to almost all performance indicators, the SI_mean-based predictions are ranked second, and for the JND-MID-MA database, it can be concluded that the performance of this predictor is close to the performance of the best CR-based predictor.

If we compare the dynamic ranges of CR for JND-MID-MA JPEG images at QF = 100 (Figure 9d) and CR for JND-MID-MA BPG images with Q = 1 (Figure 12b), it can be concluded that they are approximately equal, from 2 to 6. However, by comparing CR for JPEG images at QF = 1 (Figure 9d) and CR of BPG images at Q = 51 (Figure 12b), it can be concluded that the degree of compression is significantly higher by applying BPG compression (up to several thousand times) compared to the CR for JPEG images (which is several hundred). Thus, it can be concluded that by applying BPG coding, a significantly higher degree of compression can be achieved than by applying JPEG coding. Although a similar conclusion is valid when comparing CR for KonJND-1k JPEG (Figure 9a) and BPG (Figure 12a) images, one should be fair to JPEG compression and emphasize that, unlike JND-MID-MA, JPEG and BPG, subsets of KonJND-1k database are not created from the same set of original images (see Table 1 and Figure 2a).

3.2.2. bpp JND#1 Prediction for BPG Color Images

The success of CR in predicting bpp JND#1 is in Figure 13 and Figure 14, illustrated through the dependencies of LCC and MAE on the parameter Q for two databases. In this case, for the KonJND-1k database, it is best to use the CR calculated for Q = 30, so the optimal value has shifted a little to the right in relation to the corresponding PSNR prediction. In contrast to this, CR performance decreases with increasing value of the Q parameter on the JND-MID-MA database. However, even on this database, the performance is satisfactory for the value of Q = 30. Quantitative indicators of the degree of agreement between ground truth bpp and their predictions for three selected values of Q, namely, 1, 30, and 51, are given in

Table 5. Quantitative indicators of the degree of agreement between ground truth and bpp JND#1 predictions obtained using CR (calculated for three parameter Q values), SI_mean, contrast, homogeneity, and edge density for two databases.

		LCC	SROCC	MAE	RMSE	LCC	SROCC	MAE	RMSE
Database		KonJND-1k				JND-MID-MA (25 Color Images)
CR	Q = 1	0.8933	0.8661	0.0978	0.1342	0.8942	0.8523	0.0852	0.1098
	Q = 30	0.9558	0.9722	0.0611	0.0877	0.8756	0.8385	0.0912	0.1185
	Q = 51	0.6717	0.6594	0.1655	0.2212	0.5065	0.4531	0.1729	0.2115
SImean		0.9061	0.9043	0.0885	0.1263	0.8336	0.7762	0.1029	0.1355
contrast		0.8231	0.8211	0.1226	0.1695	0.6530	0.5962	0.1491	0.1857
homogeneity		0.7906	0.7697	0.1347	0.1828	0.8294	0.8023	0.1093	0.1370
edge density		0.7970	0.8333	0.1271	0.1804	0.7493	0.6608	0.1146	0.1624

, while the corresponding scatter plots are shown in Figure 15. From Table 5, it can be concluded that the performance of CR at Q = 30 is significantly better on the KonJND-1k database, which contains a significantly larger number of images, than the performance on the JND-MID-MA database with only 25 images. CR-based predictions obtained for Q = 30 are significantly better than other predictors.

3.3. Infrared Images

A limited study by the authors of this paper was conducted to determine the positions of JND points of images from the infrared part of the electromagnetic spectrum compressed using JPEG and BPG codecs [39,43]. This study is conducted on 25 original images, which were spatially aligned with JND-MID-MA color images from the visible part of the spectrum used in the previous part of this paper. The success of CR in prediction is analyzed through quantitative indicators LCC, SROCC, MAE, and RMSE for JPEG and BPG compressed images.

3.3.1. JPEG Infrared Images

Figure 16 and Figure 17 show the values of LCC and MAE calculated between the ground truth values of PSNR/bpp and their CR-based predictions. The maximum LCC for the prediction of PSNR JND#1 moves toward the maximum values of QF, while for the prediction of bpp JND#1, it is best to use CR calculated in the range of QF from 20 to 40. Similar results were obtained on color images from the visible range. These conclusions are additionally confirmed in

Table 6. Quantitative indicators of the degree of agreement between ground truth and PSNR JND#1 predictions obtained using CR (calculated for three QF values), SI_mean, contrast, homogeneity, and edge density for JND-MID-MA database.

		LCC	SROCC	MAE	RMSE
CR	QF = 1	0.6798	0.5323	1.8707	2.3667
	QF = 85	0.9736	0.9015	0.5766	0.7363
	QF = 100	0.9780	0.9300	0.5610	0.6726
SImean		0.9310	0.6992	0.8909	1.1781
contrast		0.9126	0.6154	1.0479	1.3193
homogeneity		0.9678	0.8854	0.6692	0.8122
edge density		0.9186	0.7223	1.1100	1.2755

and

Table 7. Quantitative indicators of the degree of agreement between ground truth and bpp JND#1 predictions obtained using CR (calculated for three QF values), SI_mean, contrast, homogeneity, and edge density for JND-MID-MA database.

		LCC	SROCC	MAE	RMSE
CR	QF = 1	0.7173	0.7146	0.0456	0.0566
	QF = 30	0.8888	0.8408	0.0296	0.0372
	QF = 100	0.6222	0.6331	0.0483	0.0636
SImean		0.9046	0.8885	0.0277	0.0346
contrast		0.8562	0.8046	0.0335	0.0420
homogeneity		0.6876	0.6823	0.0444	0.0590
edge density		0.5638	0.6077	0.0501	0.0671

, as well as in Figure 18 and Figure 19. In the illustrations, the recommended values from the previous part of this paper, which correspond to images from the visible range, are used.

The results given in Table 6 and Table 7 show the universality of CR in predicting the position of the first JND point. Namely, the CR-based predictions of PSNR JND#1 obtained for QF = 85 and QF = 100 (Table 6) have the best performance among the analyzed features, while the performance of CR-based predictions of bpp JND#1 obtained for QF = 30 is ranked second (Table 7).

From Figure 18 and Figure 19, one can see the grouping of points around the logistic curves in 2D space for the recommended QF values (85 for PSNR, i.e., 30 for bpp). If we compare the values of CR within Figure 18 and Figure 19 for infrared images with CR values from Figure 6 and Figure 9 for the corresponding color images, it can be concluded that CR values are lower on infrared images for the same QF value, which could be expected considering that they are grayscale images.

3.3.2. BPG Infrared Images

The performance of predicting the positions of JND#1 points using CR is also analyzed for BPG compressed infrared images. Figure 20 and Figure 21 show the dependencies of LCC and MAE on the parameter Q. Variations in LCC exist in Figure 20 when predicting PSNR; however, if MAE is taken into consideration, it can be concluded that in prediction, it is better to use CR calculated for lower values of the parameter Q. From Figure 21, it can be concluded that in the prediction of bpp JND#1 for BPG compressed infrared images, it is better to use CR calculated for Q values from the central part, for instance, between 20 and 30. In

Table 8. Quantitative indicators of the degree of agreement between ground truth and PSNR JND#1 predictions obtained using CR (calculated for three parameter Q values), SI_mean, contrast, homogeneity, and edge density for JND-MID-MA database.

		LCC	SROCC	MAE	RMSE
CR	Q = 1	0.9113	0.8692	0.8079	1.1404
	Q = 25	0.8935	0.7669	0.9613	1.2439
	Q = 51	0.7690	0.5092	1.4317	1.7705
SImean		0.9062	0.7038	0.9053	1.1709
contrast		0.9073	0.6715	0.9167	1.1646
homogeneity		0.9056	0.8746	0.8770	1.1748
edge density		0.7517	0.7108	1.3542	1.8266

and

Table 9. Quantitative indicators of the degree of agreement between ground truth and bpp JND#1 predictions obtained using CR (calculated for three parameter Q values), SI_mean, contrast, homogeneity, and edge density for JND-MID-MA database.

		LCC	SROCC	MAE	RMSE
CR	Q = 1	0.7747	0.6854	0.0419	0.0568
	Q = 30	0.8490	0.8623	0.0328	0.0475
	Q = 51	0.5390	0.6654	0.0531	0.0757
SImean		0.8029	0.8462	0.0359	0.0536
SFmean		0.8150	0.8508	0.0345	0.0521
contrast		0.7196	0.7631	0.0412	0.0624
homogeneity		0.8053	0.6792	0.0407	0.0533
edge density		0.8181	0.5623	0.0417	0.0517

and Figure 22 and Figure 23, CR performance is additionally presented through corresponding performance quantitative indicators and scatter plots for three values of the Q parameter, which are used for color images.

By comparing the compression ratio in Figure 22 and Figure 23 calculated for images from the infrared part of the electromagnetic spectrum with CR values from Figure 12 and Figure 15 for the corresponding color images, it can be concluded that the degree of compression is lower for values Q = 1 and Q = 51 for grayscale infrared images, while it is approximately the same for Q = 30.

By analyzing the results within Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21, Figure 22 and Figure 23 and from Table 6, Table 7, Table 8 and Table 9, it can be concluded that the success of CR-based prediction is high, both for the prediction of PSNR JND#1 and for the prediction of bpp JND#1. Meanwhile, it can also be concluded that the prediction performance of PSNR JND#1 is somewhat better.

Finally, we can state the following. A comprehensive study conducted for four publicly available databases with a wide range of source content for two types of compression and PSNR JND#1 and bpp JND#1 predictions demonstrated the universality of the CR-based JND prediction application. In most cases, CR-based JND prediction proved to be the best among other simple predictors. Also, compared to other predictors, this approach showed the most stable results. This can be interpreted by the fact that in the background of determining CR, compressed images are obtained by applying quantization matrices that are derived from the results of subjective tests and take into account some of the masking effects characteristic of the human visual system. Although it has been suggested and confirmed here that excellent prediction results are obtained using the single quasi-optimal value of QF/Q, in future work, the prediction can be analyzed using several values of CR obtained for different values of QF/Q or using the complete dependence profile of CR versus the parameter controlling the compression.

3.4. Relationship Between PSNR and bpp

Based on the obtained results, it can be concluded that the recommended QF/Q values differ depending on whether PSNR or bpp prediction is performed. Figure 24, Figure 25 and Figure 26 show the relationship between bpp JND#1 and PSNR JND#1 for the four analyzed databases as an attempt to explain the mentioned observation.

Figure 24, Figure 25 and Figure 26 show a global trend that higher PSNR values correspond to lower bpp values of the first JND points, which is particularly pronounced for the KonJND-1k database. Moreover, this trend is more pronounced for JPEG than for BPG compression. Additionally, it can be seen from Figure 26 that the JND-MID-MA subset with infrared images is not so pronounced due to the large spread of points in the 2D bpp vs. PSNR plane. This may be due to the variety and quality of source content (infrared images are slightly blurred by nature [51]) and the inexperience of the observers in analyzing the quality of this specific type of grayscale image. What is additionally impressive is the spreading in 2D space, which is the smallest for the KonJND-1k database. This can be explained by the fact that using the flicker protocol (accepted for the KonJND-1k database) provides much more stable results than using the double-stimulus test (accepted for MCL-JCI, JND-Pano, and JND-MID-MA databases) for visually lossless image compression subjective quality assessment [52].

This relationship between bpp and PSNR of the first JND points is one of the reasons for shifting the optimal values of the parameters that control JPEG and BPG compression. Namely, as for a specific image, higher PSNR values can be obtained with higher QF values for JPEG, i.e., lower values of the parameter Q for BPG images; on the contrary, lower bpp values are obtained with lower QF values, i.e., higher values of the parameter Q. Therefore, optimal CR values for PSNR prediction are obtained if it is determined for images of better quality, and for bpp prediction if CR is determined for images of poorer quality.

3.5. Validation of the CR-Based Prediction of JND#1

Since the KonJND-1k database has the largest number of source content, the largest range of their complexity and colors, as well as because it contains the results of the JND tests for both JPEG and BPG compression, it was chosen as an anchor for the validation of CR-based predictions.

Table 10. Optimum values of the logistic function coefficients (β₁–β₄) for the prediction of the JND#1 position obtained for the KonJND-1k database.

		β1	β2	β3	β4
JPEG	PSNR (QF = 85)	−1162.4108	39.7012	−60.7780	15.2039
	bpp (QF = 30)	213.2326	0.3234	−87.9736	20.9052
BPG	PSNR (Q = 25)	−980.9471	40.7885	−221.1085	50.0275
	bpp (Q = 30)	0.1960	586.1767	−138.8726	−24.5626

contains the optimal values of the logistic function coefficients, Equation (1), of this database, which are used to predict PSNR JND#1/bpp JND #1 based on CR, both for JPEG and BPG compression of color images. These coefficients correspond to the interpolation laws shown in Figure 6, Figure 9, Figure 12 and Figure 15.

Figure 27 and Figure 28 show examples of images corresponding to the first JND points obtained using CR-based predictions of PSNR JND#1 (the coefficients given in Table 10 are used). Three examples are selected for JPEG and BPG compression types with low, medium, and high amounts of details (complexity) and colors. As the KonJND-1k database JND tests were performed on different subsets for JPEG and BPG compression, the examples used for these predictions were not used to determine the optimal values of the logistic function coefficients. Below the examples, we present the CF and SI_mean values of the corresponding original images, the prediction values (PR) of PSNR JND#1, as well as the values of the parameters used to reach the predicted (desired) value of PSNR.

From Figure 27, it can be concluded that JPEG compressed images have excellent visual quality. By a careful inspection of the image in Figure 27a, weak blocking effects typical for JPEG compression can be observed in the background (wall).

From Figure 28, it can be observed that the BPG compressed images have excellent quality. By significantly enlarging the image in Figure 28a, blurring can be seen as an effect typical for this type of compression. Thus, the examples shown in Figure 27 and Figure 28 confirm the usability of CR as a picture-wise predictor of the boundary between visually lossless and visually lossy compressions.

The usefulness of CR as a predictor is also analyzed through cross-database validation. The optimal laws of mapping CR to JND#1 obtained for the KonJND-1k database are applied to the original images of other databases (MCL-JCI, JND-Pano, and JND-MID-MA). As the results of subjective JND tests are available in this case, it is possible to compare the PSNR and bpp predictions with ground truth (subjective) scores. The relationships between predictions and ground truth values are shown in Figure 29, Figure 30 and Figure 31. The degrees of agreement between predictions and ground truth values through LCC and MAE are indicated below the images. Also, the best values of LCC and MAE for CR-based predictions obtained in previous parts of this paper are listed here (these values are marked with * and can be found in Table 2, Table 3, Table 4 and Table 5).

By comparing the values of LCC and LCC*, it can be concluded that KonJND-1k guided predictions follow the best CR-based performance obtained on individual databases. However, from these Figures, it can be seen that by applying the KonJND-1k guided PSNR prediction, obtained scores are generally lower than the ground truth PSNR scores (see Figure 29b,c and Figure 31a). This underestimation can be corrected by using a linear PSNR-dependent offset, which is particularly striking in Figure 29c. The performance can be improved by introducing an offset in bpp predictions as well (see Figure 30 and Figure 31b), which can bring the MAE values closer to the MAE* values. The gap that exists between predictions and ground truth scores can be explained here mainly by the use of different methodologies in subjective tests: in the case of the KonJND-1k database, it is the flicker test, and in the case of the rest, it is the double-stimulus test. As the discrepancy depends on the database used in the cross-database validation, other possible causes cannot be excluded, such as the experience of the observer, the settings of the display, the distance from the observer to the stimulus, the variety of content, etc. This conclusion opens the possibility for future research on the relationship between these two methodologies used in subjective JND tests, which would consist of conducting flicker and double-stimulus tests with all other settings fixed, especially because the standardization of subjective JND#1 detection is still not fully complete [52].

Compared to the computationally efficient and reliable approach described in [32], which provides the desired PSNR/bpp value, the proposed method requires only one additional compression of the source image for a quasi-optimal value of the parameter that controls the compression (known in advance). After that, CR is determined and used as input for the CR into the PSNR/bpp JND#1 mapping law. This process itself is fast enough to be used in real applications.

4. Discussion and Conclusions

Compression ratio, a well-known feature, has been used as a reliable predictor of image complexity for compression purposes, as a reliable picture-wise predictor of the first JND point, and for evaluating noise presence in the image to be compressed.

In this paper, it is shown how the prediction capabilities of compression ratio for picture-wise JND prediction can be further improved by the appropriate selection of the value of the parameter that controls the compression and for which the compression ratio is determined. Namely, in papers where the compression ratio is used to predict the position of the first JND point as the boundary between visually lossless and visually lossy compression, the degree of compression was calculated in relation to the size of the JPEG compressed image obtained for the quality factor QF = 100. A wide range of tests conducted in this paper on four publicly available JND databases showed that the performance of the compression ratio in predicting the position of the first JND point of JPEG compressed images significantly depended on the QF parameter for which it was calculated. After a detailed analysis of the results, it is recommended that the first JND point of these images be determined using the compression ratio calculated for QF = 85 if PSNR prediction is performed or for QF = 30 if prediction is performed in bpp.

For the first time, the success of the compression ratio as a picture-wise JND predictor is calculated in this paper for promising BPG compression. The analysis of two publicly available databases showed that the performance of the compression ratio as a predictor depends on the value of the parameter Q that controls the BPG compression and for which it is calculated. It is recommended that in the prediction of PSNR of the first JND points of BPG compressed color images, the compression ratio is calculated for the parameter Q = 25, and in the prediction of bpp, CR is calculated for Q = 30.

Regardless of the specific visual appearance of infrared images, for which JND tests are available in a recently published database, it is shown that the recommended QF/Q-values for color JPEG and BPG compressed images can also be used in the prediction of the positions of the first JND points in compression of this type of images.

The general trend is to use a compression ratio calculated for better-quality compressed images in PSNR prediction, and to use a compression ratio calculated for lower-quality compressed images in bpp prediction. This result is a consequence of the inverse relationship between bpp and PSNR of the first JND points, illustrated by the four used databases.

A wide range of tests showed the universality of compression ratio as a predictor of the boundary between visually lossless and visually lossy compression. The results of these tests are used for predictions in real examples, which illustrate the reliability of CR-based prediction. The conducted cross-database validation indicated the need for further analysis of the relationship between methodologies for conducting JND subjective tests because the results for different databases may be different. Although we do not yet know the true reasons for this, we hope that this validation will help further standardize the detection of JND#1 through subjective tests.

In this paper, the effectiveness of the compression ratio calculated for one value of the parameter that controls JPEG/BPG compression in the prediction of the first JND point is analyzed. In future work, the dependence of the compression ratio on the possible values of QF/Q will be used, and based on it, the position of the first JND point will be predicted, either directly or through the satisfied user ratio curve. Additionally, the possibility of prediction through a combined set of simple features will be analyzed, with compression ratio as a member of the set.

Finally, this research shows that quite an accurate prediction of JND is possible, at least for JPEG and BPG coders, especially by using the compression ratio, which proved to be a more universal predictor than the others; meanwhile, the provided results can serve for choosing and using other predictors for particular situations (databases, coders, etc.).