3.1. Effects of Integrating DWT with Prediction
In
Table 2, we present the average bitrate of unmodified JPEG 2000 (
rDWT) and average bitrate changes (
∆rvariant) due to applying various variants of DWT. A variant is described by a transform name followed by a method used to obtain the transform parameters. NH(
f,
n,
p) or RH(
f,
n,
p) denote the heuristic employed, where
f is the number of denoising filters;
n, number of iterations of the step B of the heuristic (0 means that only step A was performed); and
p, number of predictors. We used either all six denoising filters described in
Section 2.3 or None and Null, the latter resulting in SS-DWT instead of RDLS-SS-DWT. If we used only one predictor, then it was the NOP predictor that effectively disables prediction; otherwise we either employed all five predictors from
Table 1 or only the NOP and MED that proved to be the most efficient. By DWT+Pred(5) we denote applying prediction to the subbands of the unmodified DWT (a predictor, for each subband independently, was selected out of all five predictors). RDLS filters were selected by a heuristic based on actual bitrate of the transformed image, but the predictor for each subband was selected based on the memoryless entropy of prediction errors calculated using Equation (7)—both in DWT+Pred(5) and during operation of the heuristic. In the column labeled “Time rel.”, we report the total time of compression (including the heuristic) relative to the time of unmodified JPEG 2000. Thus, this time has no unit of measurement and, for instance, Time rel. of 1.69 (see the last row in
Table 2) means that the total compression time of the SS-DWT+Pred, NH(2,0,2) variant was by 69% higher than the time of unmodified JPEG 2000. Time rel. was estimated based on the variant’s time complexity and the actual execution times of elements of JPEG 2000 and the heuristic reported in
Table 3; execution times were measured on a computer system used in this research and averaged for several large images from the test set. We report the prediction time for the most complex MED predictor (we assumed that NOP was costless and others were equally complex as MED) and average denoising time per single denoising filter other than the costless None and Null (calculated assuming that all such denoising filters were computed because some operations are common for them).
To assess the effects of the hybrid transform, let us compare the best results obtained with and without prediction, i.e., the ones obtained using two iterations of the heuristic’s step B, all denoising filters, and, in the case of RDLS-SS-DWT+Pred, all predictors (rows 3 and 4 in
Table 2). Compared with using RDLS-SS-DWT in JPEG 2000, by exploiting prediction we almost doubled the average compression ratio improvements of both kinds of images. Bitrates of Photo and No-photo images were improved by over 2% and 31%, respectively, which is a very good result, in contrast to applying prediction to the unmodified DWT that did not lead to significant bitrate improvements. Comparing the effects of the above variants for individual images (shown in
Figure 3), we see that bitrates of almost all No-photo images were improved noticeably by employing prediction and especially large improvements (e.g., by over 50%) happened much more often than without prediction. When employing prediction, just like without prediction, greater improvements of Photo and No-photo images were observed for those images whose bitrate of unmodified JPEG 2000 was smaller. For instance, using RDLS-SS-DWT+Pred for images whose bitrate of unmodified JPEG 2000 was below 2.2 bpp resulted in bitrate improvement of at least 7.5%, with a single exception of a 5.5% improvement.
Figure 3.
Improvements of bitrates of individual images obtained using RDLS-SS-DWT and RDLS-SS-DWT+Pred plotted against the bitrate of unmodified JPEG 2000.
Figure 3.
Improvements of bitrates of individual images obtained using RDLS-SS-DWT and RDLS-SS-DWT+Pred plotted against the bitrate of unmodified JPEG 2000.
The bitrate improvements were very good but came at the expense of a big deterioration in the speed of compression. Fortunately, the extra cost of applying prediction in the hybrid transform is relatively low compared to the extra bitrate improvement due to prediction. We expected to reduce this cost similarly to reducing the cost of RDLS-SS-DWT and SS-DWT in [
16,
17], i.e., by adjusting the heuristic’s parameters, altering the heuristic itself, and employing entropy estimation for selection of denoising filters. For NH it seemed reasonable to perform step B only once because the improvement of the next iteration was small. Performing only step A (row 6) caused a drop in bitrate improvement of Photo images by a quarter. We may avoid this drop and improve the compression speed by using only one iteration of step B and adjusting other parameters of NH—the number of filters or predictors. It is better to reduce the number of filters because this number has less impact on the bitrate improvement and the cost of using several predictors is much smaller than the cost of using several filters (rows 7–9). The difference in speed of filter and predictor selection is due to employing estimation for selecting predictors and actual coding for filters; also the filters other than Null and None are much slower than predictors (see
Table 3). These filters appeared less useful for RDLS-SS-DWT+Pred than they were for RDLS-SS-DWT without prediction. In [
17], employing them resulted in increasing bitrate improvements of No-photo images by about 2 percentage points (compared with using only Null and None), whereas here they increased bitrate improvements by below 0.5 percentage points.
The variants discussed so far that use only None and Null filters (rows 8 and 9) seemed the most promising from a practical standpoint. However, their cost is several times greater than the cost of unmodified JPEG 2000 and may be too high for certain practical applications. To decrease the cost, in the next section we investigate an estimation-based filter selection. The compression speed may be also improved by employing RH instead of NH. So far, we evaluated the effects of combining RDLS-SS-DWT with prediction using NH because this heuristic imposes fewer constraints on the resulting transform. However, the negative impact of RH constraints on the bitrate (compare rows 8 and 10) is small as opposed to their positive impact on the compression speed or time complexity, so the practical variant of the hybrid transform should rather be based on RH.
In the last three rows of
Table 2 we report results obtained by performing only step A of the heuristic (which is denoted as NH but actually may be NH or RH since they differ only in step B) and using a reduced number of filters or predictors. Naturally, these variants obtained worse improvements for Photo images. Again, the most interesting were variants that used only 2 filters (last 2 rows). Their speed seems acceptable, but employing estimation for filter selection and the heuristic’s step B may result in better overall performance, which is investigated in the next section. Interestingly, because the same filter in the heuristic’s step A was applied to all RDLS-SS-DWT steps, these two variants resulted in either performing unmodified DWT followed by prediction (if the None filter was selected) or effectively skipping DWT (for the Null filter) and applying prediction to untransformed image data. Bitrate improvements of these variants are, especially for No-photo images, close to improvements obtained using one iteration of step B (rows 8 and 9). Considering the poor performance of DWT+Pred(5) (row 2), the question arose as to whether the RDLS-SS-DWT+Pred transform obtained using both steps of the heuristic was indeed a hybrid transform that integrates prediction with DWT or perhaps RDLS-SS-DWT+Pred gets reduced to either applying prediction instead of DWT or using unmodified DWT. Thus, we checked if prediction was actually applied to subbands of RDLS-SS-DWT transformed image data.
Figure 4 shows, for each RDLS-SS-DWT subband (denoted by the subband name followed by transform level) and on average for all subbands (denoted Avg.), in how many images the actual prediction was applied depending on the subband origin. We recognized three origins of subbands. By “All None” we denote that all the RDLS-SS-DWT steps used to obtain given subband employed the None filter, so the subband was identical to subband obtained with the unmodified DWT. “All Null” means that the Null filter was used in all steps (and all the reorder steps were skipped); in this case the subband samples were untransformed. “Mixed” means that at least two different filters were used, so the subband was created using RDLS-SS-DWT not reduced to DWT or to transform skipping. Origin name followed by “+ NOP” denotes that the NOP predictor was selected for the subband, so effectively prediction was not applied to the subband. Results for NOP are plotted in a dark color. Otherwise, an actual prediction was applied—which is plotted in a light color and denoted by “+ Pred”. Results for Photo and No-photo images are reported in the case of NH(2, 1, 2) and NH(6, 1, 5) heuristics.
It can be observed that prediction was actually applied to the subbands obtained after RDLS-SS-DWT, which had not been reduced to DWT or to transform skipping. It was used more often when we employed more filters and predictors. The share of subbands with Mixed origin was much larger in the case of Photo images (it is the most common origin of their subbands) than for No-photo. If all transform steps were skipped for a given subband (“All Null”), then it almost always was better to apply prediction to such a subband. Skipping all steps was the most common subband origin in No-photo images. If a subband other than LL was obtained as in the unmodified DWT (“All None”), which was generally the least common case for both kinds of images, prediction was almost never used. In contrast, for the LL subband, it was almost always used, which can be explained by the fact that the characteristic of this subband was the closest to the characteristic of the original image. On average for all images and NH(6, 1, 5), prediction was used for approximately 61% of subbands. This value was roughly similar in all subbands except for LL, where it was used much more often (92%).
On the one hand, the above results show that DWT was effectively combined with prediction by the use of RDLS and step skipping—RDLS-SS-DWT+Pred is a hybrid transform. On the other hand, motivated by noticing how common is the use of prediction for the
LL subband as well as in the case of “All Null” origin for all subbands and recalling effects of the heuristic’s step A (
Table 2), we proposed two simple transform variants, described below. They exploit prediction, but do not use RDLS or step skipping; we investigate them in
Section 3.3:
Skipping the entire DWT and applying prediction to an untransformed image, which has not been decomposed into subbands, this variant, compared to three-level DWT with all the steps skipped and actual prediction applied to all subbands, is simpler and should obtain similar bitrates so it may be effective for many images.
Application of the unmodified typical-level DWT with prediction applied to the LL subband only will almost always be better than DWT without prediction, although the difference will be small because of the small size of the LL subband.
3.3. Schemes Extending the Heuristic with Prediction Applied to Untransformed Image
In
Section 3.1 we suspected that applying prediction to an unmodified image, instead of applying multiple-level DWT, may be beneficial for some images—because RDLS-SS-DWT+Pred subbands of most No-photo images were actually transformed with three-level DWT with all the steps skipped and actual prediction applied. We also suspected that applying prediction to the
LL subband of unmodified multiple-level DWT would result in small bitrate improvements. Effects of the above variants, denoted NO-DWT+PredMED and DWT+PredMED_LL, respectively, are reported in
Table 5. We use the MED predictor and did not allow choosing any other one (including NOP), as using other predictors appeared less effective in the case of untransformed images or the
LL subband of the DWT-transformed image. Our simple entropy estimation-based method of selecting the predictor works better for high-pass filtered DWT subbands than for an untransformed image or the
LL subband. The estimation may probably be improved by using, instead of
H0, a conditional entropy that would better match the actual JPEG 2000 context entropy coder.
Compression ratio improvement of NO-DWT+PredMED in the case of No-photo images was little better than the results obtained using the most complex variant of the hybrid RDLS-SS-DWT+Pred (−31.49% vs. −31.19%). Obviously, DWT is not the best transform for preparing the No-photo images for entropy coding. When we used the unmodified standard JPEG 2000 coder, for about one-third of this image set (81 of 247), it was better to use JPEG 2000 with the DWT stage skipped than to apply the unmodified DWT. No-photo images contained artificially inserted sharp lines. In [
16] we noticed that, due to the way 2D-DWT is built by using 1D-DWTs, sharp lines affect the unmodified DWT similarly to noisy pixels. Information on them may get propagated to the filtered subbands even in the case of the low-pass filtering of the update LS (that generally is supposed to remove the high-frequency information from the subband). The DWT efficiency can be vastly improved for these images by using RDLS, step skipping, and prediction, but using simple prediction instead of modified DWT is a better solution. NO-DWT+PredMED is not as effective for Photo images as the hybrid transform; however, we can combine the advantages of both approaches by adaptively selecting a better one for each image. Before investigating such a scheme, we noted that DWT+PredMED_LL is little better than unmodified DWT for both kinds of images without a noticeable increase in compression time.
In
Table 5 we also report bitrate improvements obtained by compression schemes that select, for each image, between NO-DWT+PredMED and some other earlier investigated transform variant.
H0_sel (rows 3–5) denotes a scheme in which the selection was based on a very quick entropy estimation. We compared the entropy of the image transformed using NO-DWT+PredMED and DWT+PredMED_LL and selected NO-DWT+PredMED if it resulted in a smaller entropy. Otherwise, the other transform was selected (DWT+PredMED_LL or a variant of RDLS-SS-DWT+Pred). If NO-DWT+PredMED was selected, then the selection did not involve executing the heuristic and, thus, the compression time was shorter. Hence, for
H0_sel we reported compression time computed by taking into account that NO-DWT+PredMED was selected for 42% of images from our test set. By min_bpp (rows 6–10), we denoted schemes in which both transform variants were performed and the one resulting in a smaller bitrate was selected. Generally, the proposed schemes, compared to RDLS-SS-DWT+Pred alone, allow further bitrate improvements that in some cases can be achieved at a reduced cost (see also
Figure 5).
The effects of the two
H0_sel schemes involving the hybrid transform (rows 4 and 5 in
Table 5) are very good from a practical standpoint when we want to significantly improve compression ratios without a significant increase in compression time. By extending RDLS-SS-DWT+Pred variants obtained using
H0_RH (recall results from
Table 4) with an option of using NO-DWT+PredMED instead of RDLS-SS-DWT+Pred, we decreased the average compression time and improved compression ratios of No-photo images by over 1.5 percentage points. The faster such scheme, compared to unmodified JPEG 2000, improved bitrates of Photo and No-photo images by about 1.6% and 31.4%, respectively, at the expense of increasing the compression time on average by 21%.
The two schemes that did not use RDLS-SS-DWT+Pred and obtained lower bitrate improvements of Photo images than others (rows 3 and 6) are also interesting. These improvements were of about 1.2% to 1.4%, which, on the other hand, is better than the best improvements obtained using RDLS-SS-DWT without prediction [
17]. Based on bitrate improvements and the compression time, other schemes may seem more practical but the advantage of these schemes is in their simplicity. They neither employ a heuristic nor integrate filters into transform steps. They require only minimal changes to JPEG 2000, i.e., the unconditional application of the MED predictor to the
LL subband after performing DWT or to an untransformed image if DWT is not used. They even may be seen as a single alteration of JPEG 2000 if we consider an untransformed image to be an
LL subband of an image transformed using the 0-level DWT.
For greater bitrate improvements than the abovementioned H0_sel schemes, we may select a transform based on the actual bitrate (min_bpp), use the actual bitrate instead of an estimated one for selecting filters in the heuristic (RH instead of H0_RH), and use a more complex heuristic (NH instead of RH). This way we attain bitrate improvements of over 2% for Photo images and over 32% for No-photo, but at the cost of compression time increased several times that may be too high for certain practical applications.